Big Data Fabric Can Surmount Big Data Problems

In this special guest feature, Ravi Shankar, Chief Marketing Officer at Denodo – a provider of data virtualization software – reviews the 6 layers of big data fabric and its benefit in terms of agility to undergo profound transformations without impacting daily operations. Ravi brings to his role more than 25 years of proven marketing leadership and product management, business development, and software development expertise with enterprise software leaders such as Oracle, Informatica, and Siperian.

In the technology world, the solution to one problem often unearths additional problems. However, after a necessary period of course correction, we often take technologies for granted. Big Data offers a solution to the problem of storing vast amounts of data and processing in a short amount of time, but after almost a half-decade, Big Data is creating problems of its own. But, can we surmount them so that Big Data can truly live up to its promise?

Big Data makes it easy to store data, yet it has not evolved into a single source for all enterprise data, which resurfaces the age-old problem of data silos. Also, because Big Data enables companies to store the data as an unstructured component, delivering data in the right format requires preparation, curation, orchestration, and integration. Finally, organizations do not run mission-critical operations on Hadoop or other Big Data systems, simply because the security is questionable.

Enter Big Data Fabric

Forrester has been aware of these issues for a number of years, and proposed “Big Data fabric” as a solution in March 2016. Big Data fabric is composed of six-layers:

1. First, the Data Ingestion layer takes in all kinds of data including: structured data; unstructured data; data from devices, sensors, logs, clickstreams, and applications; and data from both cloud and on premises sources.
2. Next, the Processing and Persistence layer is performed by cloud based systems such as Hadoop and Spark.
3. The Orchestration layer handles transformation and cleansing.
4. The Data Discovery layer is the critical next step, because it solves the silo problem, and it does that using a mixture of data modeling, data preparation, data curation, and data virtualization. Data virtualization creates a combined, virtual view of the data across two or more silos, which can be accessed by consumers in real-time as if the disparate silos were part of the same dataset.
5. The Data Management and Intelligence layer provides security and governance across the other five layers.
6. Finally, the Data Access layer delivers the data directly to analysts or to applications, tools, and dashboards.

Enabling Seamless Cloud Migrations

One powerful benefit of a Big Data fabric is the agility to undergo profound transformations without impacting daily operations. Let’s take the case of Logitech, a global provider of mice, trackballs, and other accessories.

For several years, Logitech had been developing and delivering data services for analytics using on-premises systems. But provisioning data services for business users has been reactive, time consuming, and inefficient, so the company wanted to move IT operations to the cloud. Because this transition would result in data being fragmented across on premises and cloud systems, Logitech wanted to implement a Big Data fabric to unify the disparate data sources, and minimize the impact of this transition on day-to-day operations.

Logitech implemented a Big Data fabric on Amazon AWS. Through the data virtualization capabilities of the data discovery layer, data from on-premises Excel files, machine generated data, social media data, and other Internet data is seamlessly integrated. The integrated data is now the single source of truth at Logitech, which feeds analytics and reporting applications such as Tableau, Pentaho BA, and web services.

The Big Data fabric made Logitech’s cloud journey not only possible, but possible as a live migration with minimal impact on business operations.

The Many Benefits of Big Data Fabric

With its many layers, Big Data fabric offers many potential benefits and enables companies to:

• Integrate Big Data systems with on-premises and cloud data sources for a complete view of data across the enterprise.
• Access data in real-time, using the data virtualization capabilities of the data discovery layer.
• Easily migrate to cloud infrastructure, while keeping business systems running continuously.
• Save resources when integrating data, since very little data needs to be replicated with data virtualization.

Big Data implementations, on their own, will create as many problems as they solve. With Big Data fabric, however, Big Data implementations can surmount these problems and fulfill their promises.

Sign up for the free insideBIGDATA newsletter.