Big data is on the rise, and that will continue throughout all of 2017. Everyone is talking about it. Everyone is implementing it in some way, shape or form. Consumers have even embraced it and are generating data by the server load.
In fact, more data has been created in the last two years than throughout the entire history of the human race. By 2020, the total amount of data in existence will have risen from the 4.4 zettabytes of today to about 44 zettabytes (approximately 44 trillion gigabytes).
That’s insane.
But as massive and popular as the concept is, it’s not without faults. It stands to reason that anything so powerful and so versatile can be error-prone. Big data is certainly not an exception to that rule. All the charts, graphs and analytical data being collected by companies across the world have the potential to cause some consequential decisions. Not to mention, the data can be used in many ways, and not all of them are good. This explains why there are so many pessimistic attitudes surrounding big data. Even as it continues to grow, many feel we’re misplacing our faith in technology.
Susan Etlinger, an industry analyst for Altimeter Group, discussed this during a 2014 TED Talk.
“[Today] we can process exabytes of data at lightning speed, which also means we have the potential to make bad decisions far more quickly, efficiently and with far greater impact than we did in the past.”
But is big data really that much of a problem? Is it something we should be avoiding like the plague, instead of embracing with open arms? How can we be sure the problems won’t get worse?
We Must Establish Controls and Limitations
The first thing we must do is establish a series of controls and limitations that specify how data can be used. It’s no secret that certain decisions and activities require humans pulling the strings, as opposed to an algorithm or a computer. If this had been a priority, an effective teacher like Sarah Wysocki would never have been fired under the guise of poor performance. An algorithm decided she was a poor teacher, and so she was chosen for the cut. This is despite the fact that many students and parents ranked her teaching abilities highly. There should be restrictions on what these systems and their collected data can be used for.
This ties into how the concept is being used, too. In the past, businesses have largely relied on operational reporting when it came to making decisions. Now that’s flip-flopped, and companies are starting to realize that data analytics is great for decision-making and planning. Many companies may think they’re doing analytical reporting, but are doing operational reporting instead, which is completely different. To know the difference, you must first understand both concepts and know when each should be applied. Big data and data analytics are not always the answer.
Transparency Is Necessary
There are certain forms of data — mostly personal — that you don’t want floating around, freely available to the public. But that’s a far cry from the kind of data that companies like Amazon.com or Facebook are collecting. They hold the private data pretty close to their vest and hardly ever discuss or share the nature of what they’re collecting.
This is not a good thing.
The Open Algorithms Project aims to take private data and make it available for safe consumption. Now, keep in mind, we’re not talking about passwords or social security numbers here. The kind of data that should be public relates to behavior, algorithms and analytics. For example, if Amazon knows a particular demographic is in need of an item, they should be sharing that information, not keeping it to themselves. The exclusivity isn’t contained to these big companies, either. It’s for everyone. All business, big or small, should be willing to share the data they have collected, within reason. Wouldn’t you like to know the information these companies are collecting, and what it implies about you?
We Should Have the Final Say
Finally, big data systems can help influence decision-making, and it can reveal a lot of information. But it’s not always right. The algorithms and systems that big companies are using must be transparent enough that people can challenge decisions that have been made. There must be a balance between these systems, the companies using them and the individuals they represent.
In the case of Sarah Wysocki, she should have been able to challenge the decision that led to her firing. With the proper evidence, she should have even been able to overrule the move. Better yet, she should have been able to influence future analytics and decisions. They should have taken into account the problems inherent with her testing, and fixed them so other teachers aren’t fired for the same reason.
This can be applied to any big data system, not just the one that handled employment decisions for the school board in Wysocki’s area.
Ultimately, humans (that’s all of us!) should have the final say.
Contributed by: Kayla Matthews, a technology writer and blogger covering big data topics for websites like Productivity Bytes, CloudTweaks, SandHill and VMblog.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind