Big data applications represent a fast-growing category of high-value applications that are increasingly employed by business and technical computing users. However, they have exposed an inconvenient dichotomy in the way resources are utilized in data centers. A new white paper that focuses on these issues is available here on insideBIGDATA – “Big Workflow: More than Just Intelligent Workflow Management for Big Data” written by Intersect360 Research and sponsored by Adaptive Computing.
Conventional enterprise and web-based applications can be deployed efficiently in virtualized server environments, where resource management and scheduling is generally confined to a single server. By contrast, data-intensive analytics and technical simulations demand large aggregated resources, necessitating intelligent scheduling and resource management that spans a computer cluster, cloud, or entire data center. Although these tools exist in isolation, they are not available in a general-purpose framework that allows them to inter-operate easily and automatically within existing IT infrastructure.
A new approach, known as “Big Workflow,” is being created by Adaptive Computing to address the needs of these applications. It is designed to unify public clouds, private clouds, Map Reduce-type clusters, and technical computing clusters. Specifically Big Workflow will:
- Schedule, optimize and enforce policies across the data center
- Enable data-aware workflow coordination across storage and compute silos
- Integrate with external workflow automation tools
Such a solution will provide a much-needed tool-set for managing big data applications,shortening timelines, simplifying operations, and maximizing resource utilization, and preserving existing investments.
Click HERE to download the whitepaper to better understand Adaptive Computing’s vision for meeting the unique workflow requirements of big data applications.
Sign up for the free insideBIGDATA newsletter.