Sign up for our newsletter and get the latest big data news and analysis.

EnterpriseDB Announces New Apache Spark Connecter to Speed Postgres Big Data Processing

EnterpriseDB® (EDB™), the database platform company for digital business, announced the general availability of a new version of the EDB Postgres Data Adapter for Hadoop with compatibility for the Apache Spark cluster computing framework. The new version gives organizations the ability to combine analytic workloads based on the Hadoop Distributed File System (HDFS) with operational data in Postgres, using an Apache Spark interface.

Percipient Launches SparkPLUS to Solve Apache Spark’s Out-of-memory Problems

Percipient, a Singapore-based startup, is launching a revolutionary solution to address the memory issues incurred by users of open source platform, Apache Spark. By delivering unified data a priori to the Spark platform, Percipient’s SparkPLUS solution is able to multiply the platform’s computing space, thereby greatly enhancing its utility for real time and analytical applications.

Monte Carlo Simulations in Ad-Lift Measurement Using Spark

In this talk from Spark Summit East 2016, Prasad Chalasani explores some of the challenges that arise in setting up scalable simulations in a specific application, and share some solutions and lessons learned along the way, in the realms of mathematics and programming.

Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming

In the talk below, Michael Armbrust, gives an overview of some of the exciting new API’s available in Spark 2.0, namely Datasets and Structured Streaming. Together, these APIs are bringing the power of Catalyst, Spark SQL’s query optimizer, to all users of Spark.

IBM Unleashes the Power of Machine Learning with Watson-enabled Data Platform

IBM (NYSE:IBM) announced IBM Watson Data Platform to help companies gain more valuable insights from data. The platform delivers the world’s fastest data ingestion engine and cognitive-powered decision-making to data professionals, allowing them to collaborate in the IBM Cloud, with the services they prefer. IBM is also making IBM Watson Machine Learning Service available – making machine learning simple with an intuitive, self-service interface.

Databricks Sets New World Record for CloudSort Benchmark Using Apache Spark at $1.44 Per Terabyte

Databricks®, the company founded by the the team that created the popular Apache® Spark™ project, announced that in collaboration with industry partners, it has broken the world record in the CloudSort Benchmark, a third-party industry benchmarking competition for processing large datasets.

Apache Spark Survey Reveals Increased Growth in Users and New Workloads Including Exploratory Data Science and Machine Learning

In order to better understand Apache Spark’s growing role in big data, Taneja Group conducted a major market research project, surveying approximately 7,000 people. The sample was made up of technical and managerial job roles from around the world directly involved in big data.

Splice Machine Announces Native PL/SQL Support to Accelerate Migrations from Oracle to Hadoop

Splice Machine, provider of the open-source SQL RDBMS powered by Hadoop and Spark, announced that it now supports native PL/SQL on Splice Machine.

Bigstep Launches High-Performance, Low-Latency Spark-as-a-Service for Real-Time Streaming Applications

Bigstep, the big data cloud provider, today launched a bare-metal Spark-as-a-Service offering.

Databricks Adds Deep Learning Support to Cloud-Based Apache Spark Platform

Databricks®, the company founded by the creators of the Apache® Spark™ project, today announced the addition of deep learning support to its cloud-based Apache Spark platform.