Why Do Businesses Find Big Data and Advanced Analytics a Good Match?

Print Friendly, PDF & Email

According to the research analyzing how companies use big data, businesses do believe in a sweet “big data & advanced analytics” couple. Judge for yourselves: in 2015-2017, companies named data warehouse optimization as #1 big data use case, while in 2018, the focus shifted to advanced analytics. Another stat proves this trend: 84% of enterprises invest in advanced analytics to support improved business decision-making.

Naturally, a question arises: what is so special about the alliance of big data and advanced analytics in business practice? To answer this question, let’s consider a couple of practical examples where customer and operational big data is complemented with different advanced analytics techniques, for example, machine learning and its subset – neural networks.

Customer big data & advanced analytics

Customer big data + clustering algorithms = behavioral customer segmentation

In the time of traditional data, companies applied a RFM (Recency, Frequency, Monetary) model to get behavior-based customer segmentation. With the advent of big data, businesses got the opportunity to enrich their traditional models with new, more detailed data, such as individual customer’s website (or a mobile app) surfing history, the entire transaction history, as well as the response to incentives. To get the patterns out of this data, companies apply machine learning algorithms.

The identified patterns serve as a foundation for clustering customers to one of such segments as loyal customers, potential loyalists, hibernating and at risk customers. With customer segments accurately defined, a business will be able to better target their efforts, for example, to manage their marketing communication more efficiently by sending messages with a special discount to the segment of hibernating customers.

Customer big data + machine learning = recommendation engines

Customer big data is the ‘fuel’ for recommendation engines. Apart from search and purchasing histories, comments and product reviews left, businesses collect the details on a customer’s engagement with their websites or applications, for example, product views, time spent on a web page, products put to cart, products removed from there.

There are two approaches to creating a recommendation engine – a customer-based (it’s called ‘collaborative filtering’) and an item-based one – and both of them rely on machine learning algorithms that sift through all the available data and check billions of possible data combinations.

Here’s how a customer-based approach works: say, we have two customers, the first bought products A, B and C, while the second customer bought products B, C and D. The algorithm analyzes their purchasing histories and finds similarities in the customers’ behavior. Assuming that if customers agreed in the past, they will also agree in the future, the algorithm predicts what other products each customer may like and can recommend product D to Customer 1 and product A to Customer 2.

If we have Product A in our catalog and the ML algorithm sees that thousands of customers often buy Product B with it, this dependence can also become the ground for recommendations. This approach is an item-based one.

Can this mix of big data and advanced analytics be beneficial to businesses? We believe that it can, for example, 35% of the revenue that Amazon generates is attributed to their recommendation engine.

Operational big data & advanced analytics

Operational big data + deep neural networks = inventory optimization

In order to forecast demand, a company should have the most detailed sales data. The required degree of detail, in this case, is at least a two-year history of daily sales per SKU (if the predictions are expected per SKU) split by stores, with the promotions reflected. While this is the minimal requirement, a business can add more data, for example, weather observations, as this can potentially increase the accuracy of the predictions.

A deep neural network treats this detailed historical data as an input and learns to identify the dependencies between the input and the observed optimal inventory level. Under the careful guidance of professional data scientists, the network can understand the mistakes it makes in the process of learning (for example, when it predicts 5,400 items while actually 6,500 were required). Moreover, it considers these mistakes and reconsiders the dependencies so that a mistake is reduced. After the deep neural network gets trained, it can consume new data (not the historical data that it has already seen), apply the coefficients that it found best suited during the training and predict the optimal inventory level.  

Operational big data + convolutional neural networks = supplier risk assessment

To assess supplier risks, a business should have two sets of operational data at their disposal: supplier profiles (with the info such as supplier name, size, price level, and more) and the details on deliveries made by these suppliers (delivery criticality, timeliness, completeness, etc.)

Convolutional neural networks (CNNs) deal with patterns in ordered data that influence predictions (they are called ‘features’ in the CNN language). A CNN’s task can be split into three subtasks: extracting features from the bulk of historical data, performing a number of convolution and pooling operations and actually making predictions of a supplier’s possible failure.

However, CNNs won’t show any good results without data scientists, who design its architecture, define the parameters and filters, monitor whether the CNN is trained as intended and contribute in many other ways to the accuracy of predictions.

The other side of the coin

The sweet “big data & advanced analytics” couple can become a competitive advantage for business, as it enables companies to make data-driven decisions, which, in turn, lead to increased customer satisfaction and loyalty, as well as optimized internal business processes.

However, apart from the benefits they bring, big data and advanced analytics also require substantial efforts. To design and implement a solution that would provide good enough accuracy, the team of professional data scientists is required. Plus, companies should be prepared to invest serious efforts in the solution’s adoption as their employees are likely to resist the new practices.

About the Author

Alex Bekker is the Head of Data Analytics Department at ScienceSoft, an IT consulting and software development company headquartered in McKinney, Texas. Combining 20+ years of expertise in delivering data analytics solutions with 10+ years in project management, Alex has been leading both business intelligence and big data projects, as well as helping companies embrace the advantages that data science and machine learning can bring. Among his largest projects are: big data analytics revealing media consumption patterns in 10+ countries, private labels product analysis for 18,500+ manufacturers, BI for 200 healthcare centers.

Sign up for the free insideBIGDATA newsletter.

Speak Your Mind



  1. Yeah with Big data & analytics, one can find deep insights to make wise decisions (be it individuals or big corporations) where to invest money & time, and where businesses are lagging, etc…! If you can’t invest in analytics, you are loosing your core business.

    Great tips & insights! Appreciated for your efforts! Thanks for sharing!