Big Data in 2016: Cloudy, with a Chance of Disappointment, Disillusionment, and Disruption

Print Friendly, PDF & Email

Like last year, I thought that I’d wrap up my writing calendar with some prognostications on Big Data in 2016. I doubt any of these six will come as a surprise to most readers, but what may be a surprise is how emphatically our worlds will have changed twelve months from now, when I take a crack at predicting the world of 2017. Happy New Year!

1. Welcome to the Trough

As Big Data moves through the Gartner hype cycle for technology adoption, we will naturally progress into the “trough of Disillusionment.” Organizations have been whipped into a frenzied pitch by the promise of Big Data, and nearly all organizations have been attempting to use Big Data to transform their business, or at least the results that they produce.

Because Big Data has been the latest “Big Thing” and “Shiny, New Object” in the business world, it has been ever so slightly over-sold; particularly over the last year or so. Organizations have been told that all they need to do is buy and implement this or that new technology and magically, they’ll have amazing new results from their businesses. Unfortunately, like every technology innovation that preceded it, Big Data is merely an enabler, enhancer and amplifier. If your business processes or management approaches are garbage, Big Data will make them much more so.

Expect to see many organizations become deeply disillusioned by Big Data in 2016 because they had hoped to get different results from their business, without using Big Data to actually change how they operated. Those who used Big Data to make substantive changes to how they operate will dramatically out-compete those who used Big Data to produce merely-more-detailed reports, but little actual change.

2. The Cloudy Future of Analytics

For years, Big Data has been too big, too expensive and too complicated for anyone outside of the Fortune 500. After all, the technologies were new, unproven and not even close to ready for prime time, and “real” data scientists were tied up in Universities, large companies, government agencies or any number of tiny, disruptive startup companies. Hence, many small- and mid-sized companies were left on the sidelines of this revolution.

This year, you will see an explosion of cloud-based analytics solutions designed to embrace the mid-market. Some may merely provide storage and compute capacity while others will provide full-blown analytics platforms, complete with DIY training. The best will also provide on-demand expertise from data-Jedi-for-hire, which will explain why such a large number of big company data scientists will change jobs in the next 12 months.

3. Open Warfare Online

Unfortunately, issues related to information security will escalate beyond data breaches, hacking attacks, and identity theft. In 2016, we will see open warfare on the internet between digital have’s and have-not’s. Whether it is nations attacking one another for state secrets and political leverage, Anonymous escalating their fight with ISIS, or cyber criminals holding people and organizations hostage for millions of dollars in ransom, you can expect an ever-increasing amount of online conflict in the coming year.

Not only will the volume of attacks grow, the techniques, the numbers of victims and the consequences to all of us will also grow; probably dramatically. Last year’s attacks against Ashley Madison, Sony, United Airlines, Excellus BCBS, Experian and the IRS will seem trivial compared to those that will likely come in 2016. Don’t be surprised by attacks against the power grid, the global financial infrastructure, the military, mass-media and other “pillars of our society.”

This may sound rather dystopian, but the trends are all pointing in this direction. While their techniques, technologies, and approaches will become increasingly sophisticated, the goals of the attackers will be rather simple: social disruption, political change, and good old fashioned profit motive. In an increasingly-interconnected and automated world, brought on by Big Data, you’re as likely to have your power or water cut off for a week as you are in having your credit card number stolen.

4. Persuasive Analytics Becomes Normal and Expected

If, in 2015, you haven’t had a creepy experience with persuasive analytics, you either live in a cave, or you likely weren’t pay attention. Whether it’s instant coupons delivered one click after shopping for something online, getting an invite to a “flash sale” on a favored app, or having a friend or family member receive a notice of your browsing history or physical location, persuasive analytics is the big news in Big Data.

No doubt your organization played around with predictive analytics over the last couple of years; nearly everyone has. But, you probably also came to the same conclusion as everyone else: predictive analytics is a waste of time and money. Knowing what MIGHT happen in the future has no value if you don’t benefit from the insight. CHANGING the future, so that you CAN benefit from it is how you monetize Big Data. This is the distinction between predictive and persuasive analytics. In the former you spend money, in the latter you make money.

The revolution of predictive analytics is driven by the Digital Trinity of mobility, social media, and data analytics. Leverage this trio correctly and your business will thrive. Do so incorrectly, and you’ll wonder why your business is dying before your eyes.

5. Privacy Comes to the Fore

While personal privacy has been all but surrendered in the United States, there has been a growing trend towards personal privacy and commercial restraint in other countries. The last two years have seen major moves in the privacy arena by the European Union, including the judgment against Google in the right to be forgotten and the nullification of the Safe Harbor provision between the EU and the US.

Similar actions in jurisdictions around the globe demonstrate a growing awareness of just how valuable our individual information has become and how important it is that we take an active role in managing our data.

You should expect to see greater governmental action against the unfair, undisclosed, uncontrolled collection and use of end-user data, even as the use of such information becomes a commercial and governmental imperative. As both consumers and citizens, we will expect organizations to meet our needs predictively, while at the same time we will want to be able to control the unfettered access that these organizations have to our most intimate details. This is a huge privacy paradox, and all organizations pursuing a Big Data strategy should have information governance and privacy as central themes in all of their efforts.

6. Introducing the iPresident

While many people may not realize it, the last two presidential elections in the United States were heavily influenced by the Digital Trinity. In next-year’s election, the White House will be won by whomever uses Digital Trinity most effectively. In the past, the use of the Trinity to sway voters was fairly rudimentary, and not obvious to the public at-large.

Next year, the impact of the Trinity won’t be nearly so subtle, or passive. Persuasive analytics will be used to drive new voters to the polls, push very targeted and specific political agendas to the fore and drive the mass media at least as much as the mass media tries to drive society. As events in Syria, Libya, France, Greece, Ferguson Missouri, Baltimore and Hong Kong have shown us, the Digital Trinity is an enabler of dramatic social change. Many of these changes will be positive, others will be decidedly less so. Either way, expect significant disruption to the same old same old in our society.

American politics is about to be fundamentally, comprehensively and permanently changed by the full application of Big Data and many of those who have held power in our country for a very long time will no longer have a seat at the table. This process will be front and center in 2016 as the presidential election unfolds before our eyes. If you’ve paid any attention to the run up to the election in the second-half of 2015 you’ve noted the degree to which things seem different this time. Trust me you haven’t seen anything yet! Next year, pop some popcorn, tune into the election coverage, and settle in for some great entertainment, because this will be the year that the real power of the Digital Trinity will take center-stage.

Chris_Surdak_NEWContributed by: By Christopher Surdak, JD. Chris is Global Subject Matter Expert on Information Governance, analytics and eDiscovery for HP Autonomy. He provides guidance and expertise to C-Level executives in how Autonomy’s unique and revolutionary technology can transform how their businesses operate.




Sign up for the free insideBIGDATA newsletter.

Speak Your Mind



  1. This is nice information. SMAC, with its key components namely, social media, mobile technologies, analytics and cloud computing, is a globally renowned concept. It is on the path to becoming a big rage and is expected to be fully evolved by 2020.