Getting Results from Big Data is Like Finding Needles in Haystacks

In this special guest feature, Praful Saklani, CEO and founder at Pramata, discusses how companies attempting to leverage the benefits of big data are oftentimes realizing it’s like finding needles in the haystack. An increasing number of companies apply big data technologies in the form of AI and machine learning, but don’t see the results they want. A different approach is needed to leverage smaller data sets at high precision. Praful has deep expertise in the artificial intelligence technologies core to Pramata, and experience in delivering enterprise process solutions to large corporate customers. Prior to co-founding Pramata, he founded and served as CEO of Yatra Corporation, which leveraged artificial intelligence and Internet technologies to optimize travel management processes. He graduated from Swarthmore College, where he received High Honors in economics and political science.

In May, I attended the CFO Forum in New York City, where I led a thought-leadership session about how organizations can tame data chaos. I had some great discussions on this topic with finance leaders. One theme I heard over and over was that, for CFOs today, identifying the information that can really make a difference to business performance is like “trying to find needles in haystacks.” And with the popularity of big data initiatives, it feels like their teams are just recommending more hay.

At an increasing number of companies, the strategy for locating those crucial nuggets of information has been to apply big data technologies plus a mixture of predictive analytics and artificial intelligence (AI) in the form of machine learning (ML). Inspired by the success of e-commerce organizations, many enterprises have launched broad initiatives along these lines. But the finance execs I talked to seemed lukewarm, even a bit frustrated by these efforts because so far they haven’t delivered the decisive results that were expected.

Big data is a big investment, and one that calls for a generous amount of time as well as cash. No question, it will prove its worth in a range of future use cases. But CFOs and other enterprise leaders don’t have to wait for big data initiatives to mature if they want to exploit their data in ways that can have a significant impact on revenue. There’s a different approach that can deliver fast and impressive results: leveraging smaller data sets at high precision.

What B2B orgs can learn from the e-commerce giants

You can work wonders with the right data. We’ve all seen this in our consumer lives; for example, the success of B2C companies in fine tuning ad spend to predict and influence consumer behavior. Or the online travel portals that crunch staggering amounts of data to tailor offers and suggestions precisely to our needs. But here’s the thing: It has to be clean, accurate data – not just lots of data.

Consider the inputs that B2C retailers and e-commerce firms are working with. Nearly all of it comes directly from their own platforms, so it’s highly standardized. They accumulate vast amounts of data very quickly, and they can curate it with great specificity. They can train their ML engines on it with a high degree of accuracy.

Contrast that to the situation in large B2B organizations. They’re working with relatively smaller volumes of data (though still very large, in most cases). But more importantly, the data is much less orderly and standardized. Companies too often manage their critical commercial information using a combination of tribal knowledge, ad hoc CRM data, and spreadsheets containing data pulled from disparate systems.

And the systems tend to proliferate. As a result of M&A activity, business reorgs, and successive technology initiatives, it’s not unusual for a large B2B company to be running as many as four CRM tools and ten or more billing systems and contract repositories, all across multiple pricing structures. Ensuring that commercial agreements are accurately recorded in the first place is a challenge, and so is keeping that information up-to-date.

Given this fragmentation of commercial relationship data, large organizations often strain to find answers to even straightforward questions like “What have our five biggest customers bought from us over the past five years?” or “Where do we have opportunities for price increases based on CPI?” So it’s not surprising that this non-standardized, often inaccurate material delivers disappointing results when it’s fed into ML algorithms.

The important lesson to draw from the success of B2C predictive is this: Mobilizing huge volumes of data is great, but it’s not enough. Quantity must be accompanied by quality.

The power of small data at high precision

In fact, if you have high-quality data, and it’s tightly focused around the goals you want to achieve, the quantity of that data becomes a lot less critical. Instead of analyzing, say, a thousand items of information across a million customers, you can look at five or six items across 100,000 customers, and still get stellar results.

It makes all kinds of sense to focus your data initiatives where they can have the biggest bottom-line impact. Now, I’m aware that every new tech solution provider or best-practices advocate you’ve ever met advises the same thing – “consider the business benefits” first and last. But in data projects, it’s an absolutely critical step because it bears on the choice of data sets you’ll want to work with. An analogy I like to use is this: it’s like the difference between saying “I’m going to get healthy” vs. “I’m going to lose ten pounds in three months.” The first is too wide to be useful; the second gives you immediate insight into what you need to do.

You might want to start by considering your most valuable commercial relationships and specific business challenges. Do you have any gaps in your customer value life cycle where you’re leaking revenue? Do you want to drive up customer retention? Do a better job of seizing cross-sell opportunities?

Sign up for the free insideBIGDATA newsletter.