In this special guest feature, Susan Cook, CEO of Zaloni, discusses her definition of “data marketplaces,” a slightly different concept to the large industry-wide data exchanges and marketplaces in financial services, pharma and healthcare. These data marketplaces have grown up within enterprises (even though the data itself could be internally or externally sourced) to service all their various consumers of data. Zaloni is an award-winning leader in data management and operations. Susan brings to her position a wealth of enterprise software and solution sales experience in Data and Analytics gained over two decades of global leadership roles at IBM, MicroStrategy, and Oracle.
A year ago at one of the world’s largest banks, it wasn’t uncommon for a data scientist or business user to have to wait three months for IT to fulfill a specific data request. This was understandable, in large part because of the complexity of the bank’s environment, with data siloed across more than 5,000 transactional systems.
Put another way, when you’ve got 4.5 petabytes of data scattered throughout the enterprise, of course it’s going to take time for the IT team to cobble together terabytes of disparate data for an analyst.
Fortunately, it doesn’t have to be this way anymore. Today, consumers of analytics at that same bank can self-source the data they need, skipping IT altogether, in six days or less. Even at much smaller enterprises, the same math holds true, driving the same transformational improvements in the speed-to-value: data requests that once took days or weeks are now fulfilled in hours or days.
The difference is what I am calling internal “data marketplaces,” a slightly different concept to the large industry-wide data exchanges and marketplaces in financial services, pharma and healthcare. These data marketplaces have grown up within enterprises (even though the data itself could be internally or externally sourced) to service all their various consumers of data. This marketplace concept is the latest manifestation of the Amazonification of the enterprise and the latest practical evolution in turning disorganized data sprawl into targeted data shopping.
True Self-Service Data Discovery, Not Just a Sexy Visualization
Long envisioned as a holy grail of data management, the idea of an analyst or data scientist “shopping” for the data they need (and are allowed to have), adding it to their “cart,” and bringing it to their analytics sandbox is here and working today.
So what has changed? For veterans of the data campaigns, it will come as no surprise that the secret to enterprise data marketplace success goes far beyond a sleek UI or Amazon-like user friendliness. True success lies just as much in the back-end “supply chain” capabilities that streamline data collection, automate governance and demonstrate provenance that allow the end consumer to have faith they are getting the right product from their shopping experience.
Today’s analysts and data scientists are no longer just “consumers” of data, they are sophisticated professionals. Data literacy is now an essential job skill, and the discipline is growing in large, mature enterprises. As such, the data these consumers provision through these marketplaces has to pass these three essential tests:
- They have to trust it. The governance must be bulletproof and automated.
- They have to see it. The lineage or provenance of the data must be transparent and credible.
- They have to be able to use it. Self-service provisioning into an analytics-ready sandbox is essential to take action quickly.
Passing the Test through Innovation
A number of key technological developments – underscored by a big shift to a DataOps mindset – are helping pass the test by making data more accessible and productive than ever before, positioning the enterprise to reap data’s ultimate potential as this decade’s most valuable business currency.
First, extensible platforms are helping enterprises manage data sprawl. Enterprise data can and does live anywhere – on prem, in the cloud, across multiple clouds, or a combination of each. This used to seem like an intractable problem that blunted the speed and utility of data. Typically, data management tools were just as siloed as the data – “here’s our cloud tool, here’s our Hadoop tool, etc.”
Second, augmented metadata catalogs are now harnessing the power of machine learning to tag, annotate, and enrich data, serving as a virtual repository of data at scale. Augmented data management technologies can now automate and make recommendations to secure, mask and control data, making it much easier to comply with regulatory mandates like GDPR, CCPA, etc. This automation and intelligence enables companies to orchestrate their current data management systems and turbocharge their performance, all while greatly reducing administrative costs.
These technological building blocks of today’s forward-thinking organizations enable data marketplaces to perform seamlessly, delivering trusted, governed, and useful data quickly and easily to those throughout the enterprise who need it. The benefits include greater productivity, faster speed-to-value, and a new level of data agility that is urgently needed. I was recently talking to a good friend, a CDO of a global hospitality company, who stated that the benefits of getting the right data into the right hands at the right time to impact a customer’s experience can be directly attributed to a significant increase in revenue.
The three-month, or even three-week, data “request-to-delivery” cycles of the 2010’s seem dangerously outdated. Across industries, from medical testing to financial services, new COVID-19 demands, regulations, and government programs, are creating a data tsunami that must be managed with an efficiency worthy of the people depending on them.
The stakes have been raised. Data marketplaces, and the enabling technologies upon which they are built, will be vital in meeting not only the enterprise’s, but also the world’s, biggest challenges.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind