Sign up for our newsletter and get the latest big data news and analysis.

Intel Case Study: Speeding Up a Big Data Platform

Sponsored Post

In this technology cast study, we’ll examine MeritData, Inc., a leading big data analysis technology and service provider in China. The company’s product is called Tempo, a big data platform that has been widely used by well-known power, manufacturing, financial, and global enterprises and by cloud service providers. MeritData helps its customers explore and  exploit data value―and ultimately creates value through data processing, data mining, and data visualization solutions. This is achieved through the fusion of high-performance computing (HPC) technology,  leading data analysis algorithms, high-dimensional visualization, and creative data visualization language.

Intel worked with MeritData’s algorithm engineers to optimize the company’s multiple data mining algorithms in order to analyze data quickly and precisely. MeritData needed to optimize its algorithms from the perspectives of algorithm principles, computing, and  programming. The extreme learning  machines (ELM), in-house L1/2 sparse iterative algorithm, and linear regression (LR) were all optimized by Intel® Data Analytics Acceleration  Library (Intel® DAAL) and Intel® Math Kernel Library (Intel® MKL). The result was average performance improvements ranging from 3x all the  way to 14x.

Increasing Performance in Big Data Analysis Platforms

In order to analyze the data effectively, and to process more data more quickly, the MeritData needed to improve their algorithm’s performance to the extreme. Algorithm modeling can help with a high computation load that needs to perform repeated computation iterations on input data. When the data volume is modest, operating time is usually acceptable. But as the data volume surges, the operating time of some algorithms increases  exponentially―until they can no longer meet customer requirements.

To meet the continually growing demand for data mining, MeritData worked closely with Intel. Using Intel MKL and Intel DAAL, MeritData was able  to accelerate Tempo’s core algorithm library, giving its customers a powerful data analysis solution. Compared with the original hardware- independent implementation, the new scheme can quickly and accurately analyze huge amounts of data processing and modeling, enabling customers  to quickly explore and realize the full value of their data―with an average performance improvement of 3x and a speedup of up to 14x.

Tempo Big Data Analysis Platform Architecture

Based on Intel MKL and Intel DAAL, Intel worked with MeritData to build a Tempo big data analysis platform to accelerate the core algorithm  libraries on Intel® architecture. Through the cloud computing architecture, the team implemented a big data analysis solution for fast modeling and analysis. At the same time, it provided integrated services to satisfy its customers with different fields and levels of data analysis to achieve the value  of their data, data visualization exploration, and in-depth analysis.

Based on Intel MKL and Intel DAAL, MeritData built the Tempo big data analysis platform. Click To Tweet

The system architecture of the Tempo platform includes the following layers, providing  unified cloud service access, cloud resource scheduling, and cloud platform management.

  • Data Access Layer – needed to be able to accept different sources of data, including intelligently adapting to both SQL and non-SQL databases, docking with Kafka, flume streaming data sources, and unstructured text data sources.
  • Analysis and Modeling Layer -based on Intel MKL and Intel DAAL, it accelerates the core analysis algorithms of the Tempo data analysis platform.
  • Result Display and Access Layer – after analysis modeling is completed, the modeling results are displayed as statistics and can generate a result  report to improve the utilization rate of resources and help make decision-making more agile. It also allows the customer to build a new model  based on pre-trained results, directly accepting the new data sources and making accurate predictions. This layer provides an access interface in  the form of an API call, making it convenient for the customer’s future development.

Tempo big data analysis platform for data-mining system architecture

 

Tempo’s Performance Benchmark Improvement

MeritData’s customers have an insatiable need for increased computing performance to let them model and analyze bigger data with faster speed and  greater fidelity. Based on Intel MKL and Intel DAAL, Intel worked with MeritData to accelerate core algorithms of the Tempo large data analysis platform over the original method. Performance was boosted dramatically, effectively helping customers maximize their data value, meet their  business needs, and quickly analyze huge amounts of new data. Specifically, the team was able to optimize Tempo’s ELM and in-house L1/2 sparse iterative algorithms. Time comparison before and after the optimization with Intel® MKL are shown in the data visualization below.

Time comparison before and after the optimization with Intel® MKL

Summary

Through close collaboration with Intel engineers, MeritData adopted both Intel DAAL and Intel MKL for algorithm optimization in the Tempo big  data analysis platform. Now the company can analyze big data fast, model accurately, and satisfy customers from different fields with different levels  of data analysis. The solution greatly improved MeritData’s data analysis processing capacity and significantly improved the performance and user experience.

 

Download your free 30-day trial of Intel® DAAL

 

Leave a Comment

*

Resource Links: