Sign up for our newsletter and get the latest big data news and analysis.

News, reviews and feature articles on companies that are changing how organizations leverage massive volumes of data

insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads – Part 3

Artificial Intelligence (AI) and Deep Learning (DL) represent some of the most demanding workloads in modern computing history as they present unique challenges to compute, storage and network resources. In this technology guide, insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads, we’ll see how traditional file storage technologies and protocols like NFS restrict AI workloads of data, thus reducing the performance of applications and impeding business innovation. A state-of-the-art AI-enabled data center should work to concurrently and efficiently service the entire spectrum of activities involved in DL workflows, including data ingest, data transformation, training, inference, and model evaluation.

insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads – Part 2

Artificial Intelligence (AI) and Deep Learning (DL) represent some of the most demanding workloads in modern computing history as they present unique challenges to compute, storage and network resources. In this technology guide, insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads, we’ll see how traditional file storage technologies and protocols like NFS restrict AI workloads of data, thus reducing the performance of applications and impeding business innovation. A state-of-the-art AI-enabled data center should work to concurrently and efficiently service the entire spectrum of activities involved in DL workflows, including data ingest, data transformation, training, inference, and model evaluation.

DAOS Delivers Exascale Performance Using HPC Storage So Fast It Requires New Units of Measurement

Forget what you previously knew about high-performance storage and file systems. New I/O models for HPC such as Distributed Asynchronous Object Storage (DAOS) have been architected from the ground up to make use of new NVM technologies such as Intel® Optane™ DC Persistent Memory Modules (Intel Optane DCPMMs). With latencies measured in nanoseconds and bandwidth measured in tens of GB/s, new storage devices such as Intel DCPMMs redefine the measures used to describe high-performance nonvolatile storage.

insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads

Artificial Intelligence (AI) and Deep Learning (DL) represent some of the most demanding workloads in modern computing history as they present unique challenges to compute, storage and network resources. In this technology guide, insideBIGDATA Guide to Optimized Storage for AI and Deep Learning Workloads, we’ll see how traditional file storage technologies and protocols like NFS restrict AI workloads of data, thus reducing the performance of applications and impeding business innovation. A state-of-the-art AI-enabled data center should work to concurrently and efficiently service the entire spectrum of activities involved in DL workflows, including data ingest, data transformation, training, inference, and model evaluation.

StreamSets Launches StreamSets Transformer

StreamSets, Inc., provider of the DataOps platform for modern data integration, released StreamSets® Transformer, a simple-to-use, drag-and-drop UI tool to create native Apache Spark applications. Designed for a wide range of users — even those without specialized skills — StreamSets Transformer enables the creation of pipelines for performing ETL, stream processing and machine-learning operations. Now, data engineers, scientists, architects and operators gain deep visibility into the execution of Apache Spark while broadening usage across the business.

Interview: Terry Deem and David Liu at Intel

I recently caught up with Terry Deem, Product Marketing Manager for Data Science, Machine Learning and Intel® Distribution for Python, and David Liu, Software Technical Consultant Engineer for the Intel® Distribution for Python*, both from Intel, to discuss the Intel® Distribution for Python (IDP): targeted classes of developers, use with commonly used Python packages for data science, benchmark comparisons, the solution’s use in scientific computing, and a look to the future with respect to IPD.

Develop Multiplatform Computer Vision Solutions with Intel® Distribution of OpenVINO™ Toolkit

Realize your computer vision deployment needs on Intel® platforms—from smart cameras and video surveillance to robotics, transportation, and much more. The Intel® Distribution of OpenVINO™ Toolkit (includes the Intel® Deep Learning Deployment Toolkit) allows for the development of deep learning inference solutions for multiple platforms.

The AI Opportunity

The tremendous growth in compute power and explosion of data is leading every industry to seek AI-based solutions. In this Tech.Decoded video, “The AI Opportunity – Episode 1: The Compute Power Difference,” Vice President of Intel Architecture and AI expert Wei Li shares his views on the opportunities and challenges in AI for software developers, how Intel is supporting their efforts, and where we’re heading next.

Fast-track Application Performance and Development with Intel® Performance Libraries

Intel continues its strident efforts to refine libraries optimized to yield the utmost performance from Intel® processors. The Intel® Performance Libraries provide a large collection of prebuilt and tested, performance-optimized functions to developers. By utilizing these libraries, developers can reduce the costs and time associated with software development and maintenance, and focus efforts on their own application code.

What Happened to Hadoop? And Where Do We Go from Here?

Apache Hadoop emerged on the IT scene in 2006 with the promise to provide organizations with the capability to store an unprecedented volume of data using cheap, commodity hardware. Hadoop facilitated data lakes were accompanied by a number of independent open source compute engines – and on top of that, “open source” meant free! What could go wrong?