Sign up for our newsletter and get the latest big data news and analysis.

Combining Data Sources to Drive Smarter Decisions

David-Abramson-headshotIn this special guest feature, David Abramson, Director of Product Management at Logi Analytics, discusses how the advent of new and disparate data sources means we must rethink the way we manage data. David Abramson has more than 10 years experience in full lifecycle product development and management, from product inception through general availability. He has shepherded multiple analytics and business intelligence products, and has worked with hundreds of customers, both enterprises and ISVs, to support data-driven application implementations.

Data fuels us. Whether you are a consumer with a FitBit or a business user looking to analyze performance, it all comes back to data. In fact, IDC’s Digital Universe study predicts the amount of data on the planet will grow tenfold by 2020 – from around 4.4 zettabytes to 44 zettabytes.

But harnessing that data for analysis has been a major hurdle for businesses everywhere.

Traditionally, IT departments have managed data for their businesses in a centralized manner – and that data came from only a few applications. Now, BI is a completely different ballgame. Not only are companies collecting ever-increasing volumes of data, they’re collecting that data from a dizzying number of sources – applications, sensors, machines, and more – and in multiple formats.

Data is no longer centrally located, and continuing to operate on a strictly IT-centric model is unrealistic. Complicating matters more is the fact that tech-savvy users are clamoring to get their hands on all this data so they can make more-informed business decisions.

Companies can no longer flow all their data through a master system, store it in a data warehouse, and call it a day. Today, we face the complex task of tracing the origins of our data, blending it, and getting it into the hands of users for analysis – all at the speed of business.

Following the Data

The landscape of where data originates is vastly different than it was a decade ago. In many companies, individual departments manage their own applications and generate their own data. And everyone in the organization has to not only figure out how to access that information, but also decide what to do with it.

On top of that, some of this data isn’t “owned” by any single department. For instance, think about data from social platforms like Twitter and Facebook– or, in the case of manufacturing companies, data that comes from machines and sensors.

Some companies also use public data – in other words, data they’re not even generating themselves. For example, weather data is useful to logistics or manufacturing companies, while census or demographic-based data is useful in the retail sector. These are all new classes of data that can be very valuable – but only if you have a plan for combining data sources, storing them, and building analytics on top.

Combining the Data

There’s no sense in simply storing your data and then doing nothing with it. Once you’ve found and secured your data, you need to combine it and prepare it for analysis. In the past, a “data mashup” meant displaying all the data from different sources on a single dashboard screen.

Today, we call it “data blending,” and it’s a much more sophisticated process. Not only do we display the data, we join (or combine) it to find common values, and then we query it to gain insights.

The key here, however, is to get these capabilities into the hands of your users. The idea of a central data warehouse where you store blended data is just too difficult to manage: Data today is updated constantly, and different people will want to combine data in different ways. For instance, your marketing team may want to blend its Marketo data with Salesforce, as well as data from Unbounce or Google Analytics. This will be vastly different from the finance team, who will want to blend data from Salesforce with Quickbooks and some internal data.

Fortunately, BI technology has evolved to simplify data preparation. Now, the everyday user can connect, acquire, and blend data from nearly any type of source; cache it in a high-performance, self-tuning repository; and prepare it using smart profiling, joining, and intuitive data enrichment.

When users are empowered with all their data and don’t have to constantly request access from IT, they’re able to make smarter decisions, because they will see a complete picture – not just pieces that they have to mash together in Excel.

Rethinking Data Management

The advent of new and disparate data sources means we must rethink the way we manage data. We cannot simply modernize the traditional model of a centralized system. Instead, we must focus on a higher degree of self-service – empowering our users to make better decisions around the data wherever they sit in the organization.

 

Sign up for the free insideBIGDATA newsletter.

Leave a Comment

*

Resource Links: