Survey: Enterprise IT Professionals Share Views on the Maturity of Their Big Data Deployments

Kyvos Insights, a big data analytics company, shared the results of a recent survey of information technology (IT) professionals at enterprises in the U.S. that explores their thoughts on, and experiences with, big data and business intelligence (BI) on big data adoption.

This survey reveals that most organizations have yet to derive the value they expected from their investments in big data infrastructure,” said Ajay Anand, vice president of products at Kyvos Insights and a co-presenter of the webinar. “Despite the fact that business intelligence (BI) is the most important use case for big data, even fewer are effectively using it to uncover actionable insights within the information they’ve gathered.”

When it comes to putting big data to work and justifying investments in the required infrastructure, respondents overwhelmingly pointed to the effectiveness of BI on big data efforts as the deciding factor. Seventy percent indicated that BI is the most important use case for big data, but only four percent indicated that they have managed to bring BI on big data to the core of their business processes. Only 25 percent feel they are deriving substantial value from their big data investments.

For most enterprises, BI is what ultimately enables them to use their data to make better decisions, work in new ways, and act with a new understanding of their worlds,” said business intelligence and data expert Claudia Imhoff of the Boulder BI Braintrust. “That’s why it’s crucial that enterprises bring BI into the big data world to ensure access to ALL data.”

Participants in the survey, conducted throughout the spring of 2018, included individuals from 300 companies – among them BI analysts, CTOs, data architects, data scientists, developers, directors and other IT professionals. Additional findings from the survey include:

• Organizations with an effective BI on big data platform in place are already deriving great value from their big data infrastructure: Although still a minority, more than half of these respondents, 54 percent, point to operational efficiency as a key return, while 43 percent say it resulted in a reduction of costs. Other notable benefits to date included increased customer satisfaction, 30 percent, and more targeted marketing, 29 percent.
• Data growth, new data sources and cost savings were the top drivers for moving to a big data infrastructure: Half of respondents pointed to scale-out needs as the reason for deploying big data infrastructure and more than half, 54 percent, used it to incorporate new data sources.
• The BI on big data skills shortage is real and is having a chilling effect on user adoption: More than half of respondents, 52 percent, say their organization lacks the skill set required to effectively implement BI on big data projects and 57 percent of organizations report that fewer than 50 employees use BI on big data in their daily work.
• Big data is already pushing traditional BI tools to the breaking point: Only 40 percent believe their BI infrastructure measures up in terms of response times for large amounts of data and analysis at the fine granularity needed to provide actionable business insights.
• Looking to the future, organizations will demand real-time capabilities and the ability to scale to massive amounts of data: More than half of survey participants view the speed of response times as a key requirement of their BI on big data infrastructure, while half already view the ability to scale out to massive amounts of data – datasets with hundreds of billions of rows – as necessary.
• Advanced analytics and data warehouse offload remain singular use cases: Although 70 percent of respondents point to BI as the most important use case and workload supported by their big data infrastructure, 59 percent also use it for advanced analytics and just over half, 52 percent, utilize it to offload legacy data warehouses.
• The race to the cloud continues: More than half of all survey participants, 54 percent, expect to move their big data infrastructure to the cloud over the next three years.

The real revolution, when big data lives up to the hype, will occur when organizations deploy self-service BI on big data using solutions that are intuitive enough for business users to master and deliver interactive response times on massive data sets for thousands of concurrent users,” added Anand. “Those solutions are available now.”

Sign up for the free insideBIGDATA newsletter.