Sign up for our newsletter and get the latest big data news and analysis.

How to Ensure an Effective Data Pipeline Process

In this contributed article, Rajkumar Sen, Founder and CTO at Arcion, discusses how the business data in a modern enterprise is spread across various platforms and formats. Data could belong to an operational database, cloud warehouses, data lakes and lakehouses, or even external public sources. Data pipelines connecting this variety of sources need to establish some best practices so that the data consumers get high-quality data delivered to where the data apps are being built.

Video Highlights: Modernize your IBM Mainframe & Netezza With Databricks Lakehouse

In the video presentation below, learn from experts how to architect modern data pipelines to consolidate data from multiple IBM data sources into Databricks Lakehouse, using the state-of-the-art replication technique—Change Data Capture (CDC).

eBook: Unlock Complex and Streaming Data with Declarative Data Pipelines 

Our friend, Ori Rafael, CEO of Upsolver and advocate for engineers everywhere, released his new book “Unlock Complex and Streaming Data with Declarative Data Pipelines.” Ori discusses why declarative pipelines are necessary for data-driven businesses and how they help with engineering productivity, and the ability for businesses to unlock more potential from their raw data. Data pipelines are essential to unleashing the potential of data and can successfully pull from multiple sources.

Optimizing Data Integration to Enable Cloud Data Warehouse Success

In this contributed article, Mark Gibbs, Vice President of Products at SnapLogic, looks at best practices for data integration success, shares advice on how to optimize your CDW investments, and reviews common issues to avoid during the process. Data integration comes enables the CDW by mobilizing your data and automating the business processes that drive your business to deliver deep data insights and increase time to value.

Databricks Launches Data Lakehouse for Retail and Consumer Goods Customers

Databricks, the Data and AI company and pioneer of the data lakehouse architecture, announced the Databricks Lakehouse for Retail, the company’s first industry-specific data lakehouse for retailers and consumer goods (CG) customers. With Databricks’ Lakehouse for Retail, data teams are enabled with a centralized data and AI platform that is tailored to help solve the most critical data challenges that retailers, partners, and their suppliers are facing.

From Data Warehouses and Data Lakes to Data Fabrics for Analytics

In this contributed article, Kendall Clark, Founder and CEO of Stardog, discusses how data fabric is fast-becoming the data architecture foundation for analytics and how it is revolutionizing the $50 billion data lakes/warehouse market. Supported by real-word examples, the article explores how technologies such as expressive semantic modeling, knowledge graph, and data virtualization are connecting disparate data lakes to streamline data pipelines, reduce dataops costs and improve analytics insight.

Data Warehouse Costs Soar, ROI Still Not Realized

Enterprises are pouring money into data management software – to the tune of $73 billion in 2020 – but are seeing very little return on their data investments.  According to a new study out from Dremio, the SQL Lakehouse company, and produced by Wakefield Research, only 22% of the data leaders surveyed have fully realized ROI in the past two years, with most data leaders (56%) having no consistent way of measuring it. 

How to Overcome Obstacles in Data Lake and Warehouse Strategies: 3 Best Practices for Enterprise Architects

In this special guest feature, Kimberly Read, the enterprise architect at Faction, suggests that to support the business case for multi-cloud, enterprise architects can benefit by addressing three primary considerations. Multi-cloud initiatives—drawing on services from public and private clouds—can help organizations stay ahead of the curve.

Data Evolution in the Cloud: The lynchpin of competitive advantage

This report from our friends over at Snowflake reveals the extent to which the data sharing economy is powering business growth and how organizations are leveraging data from a range of sources to drive innovation, create better customer experiences, and meet regulatory requirements.