In this special guest feature, Nikhil Bhatia, Director of Product Management at Riversand Technologies, addresses some of the common myths and misconceptions around the areas of Big Data, Analytics & AI and presents a pragmatic approach and some best practices to apply these technologies in today’s competitive world. Nikhil is one of the key leads for conceptualization, design and development of the Riversand Data Platform and Apps which enable customers to discover, manage, analyze and govern Master Data in their organization using Big Data, Cloud, Analytics and AI technologies. In addition, he is also involved in strategy, business development, sales and marketing functions. His educational background includes an MBA from one of India’s leading business schools and degrees in Information Technology and IPR Law.
With the confluence of growth in data, computing power to process that data and the democratization of AI technologies in the cloud, any organization can avail the benefits of Big Data, Analytics and AI to improve their business outcomes. But this should not be considered as a “magic” solution which can solve any business problem that an organization might have. This article addresses some of the common myths and misconceptions around these areas and presents a pragmatic approach and some best practices to apply Analytics & AI in today’s competitive world.
Myth: BigData Technologies are better and cheaper than traditional technologies
Although the allure of BigData is that even though data is too large or complex it can be managed using commodity hardware and open source (hence cheaper) technologies, the reality is far from this myth. It takes tremendous effort, skill and resources to really operationalize open-source BigData technologies to solve real world problems. Organizations should also understand that BigData technologies are not for solving all kinds of problems. Because they are built for large scale, these technologies can’t really handle smaller data sets. For some problems, a smaller data set is enough so applying BigData technologies wouldn’t be appropriate or necessary.
Myth: Every data problem can be solved just by using Analytics or AI
The best value from Analytics and AI can be realized after framing the right problem. The business value of the problem has to be understood and directly related to cost or revenue for the organization. Typically, a problem which requires a significant amount of time and effort by the organization interpreting information to gain knowledge is a prime candidate to generate value using Analytics and AI. That being said, sometimes the simple answer may still be to change a process or way of working which reduces the information itself rather than automate the interpretation of it. Say an organization is collecting invoices or negotiating promotional terms through email and wants to automate the reconciliation process. It might actually be better and easier to implement a new collaboration tool to raise and manage invoices or promotions rather than to implement an AI solution to comb through the emails and automatically interpret this information.
Myth: The better technology you use, the better the value of AI you will realize
There are three major components to AI: the data, typically a mathematical model and the software to generate and run the model. The way AI works is that by running data through the software a model needs to be discovered and evolved. The software pieces for AI today are not as packaged as some of the traditional software and hence there is a plethora of tools and frameworks, both open source vs. paid as well as developed by software giants such as Google and Microsoft vs. startups to be used to develop the AI models. Hence the main goal in sight should always be to create a transferable, packaged model to solve a specific business problem and not the technology to be used.
Myth: AI and Analytics in and of themselves generate value
It is always AI and Analytics plus “something” which provides this value to an organization. Say a retail organization discovers bottlenecks in the approval process of products they want to sell online by using Analytics and AI. That’s useful information and insights, but the final solution is using a better workflow or increasing resources or increasing automation in the approval process – this is what is going to provide the actual value. In addition, value from AI and Analytics is not created on day 1 and may feel underwhelming at the beginning. Any value that is generated will typically be greater than what the organization is doing today, so it’s a start. These solutions get better over time and usage, and hence such initiatives do require patience and executive sponsorship. AI is basically math done a different way, so the right problem and right expectation is important. Instead of finding a breakthrough the organization should focus on solving practical day-to-day problems.
In conclusion, there is a misconception in the market that anybody and everybody can take a bunch of open-source tools and create an AI solution which will provide immense value and completely change the way an organization operates. It’s great that people are talking about and getting excited about the potential for AI. However, the reality is that operationalizing AI without having a comprehensive data platform is going to be nearly impossible. Organizations must have a data platform foundation which can scale, is hybrid in nature and has the ability to consume all kinds and volumes of data. One of the biggest issues is that even though the core processing technologies for AI and Analytics may be fast and scalable, the rest of the pipeline to consume and move data remains slow and this bottleneck does not allow us to achieve the results in real time. 80% of the work in Analytics and AI is around collecting, cleansing, preparing and munging data. Unless the same data stores and pipelines are being used by everybody in the organization, the multiple, separate efforts being done by different teams for different purposes is just a waste of time and resources. Not to mention that the organization will never realize the true benefit and value that AI can provide when done properly. Organizations need to map out their data strategy first (and ensure they have a solid data foundation) before they try to embark on the promises of BigData, AI and Analytics.
Sign up for the free insideBIGDATA newsletter.
Speak Your Mind