“The Hadoop framework has become the most popular open-source solution for Big Data processing. Traditionally, Hadoop communication calls are implemented over sockets and do not deliver best performance on modern clusters with high-performance interconnects. This talk will examine opportunities and challenges in optimizing performance of Hadoop with Remote DMA (RDMA) support, as available with InfiniBand, RoCE (RDMA over Converged Enhanced Ethernet) and other modern interconnects.”
The panel discussion video below comes from the Los Angeles Spark Users Group. The talk fosters a lively discussion on Spark’s initial goals, where it came from and what the future holds for Spark. Many leading Big Data vendors are responding by introducing Spark’s capabilities into their architectures. The panel discussion is between the top Hadoop distribution vendors – Cloudera, MapR, and Pivotal.
“When organizations operate both Lustre and Apache Hadoop within a shared HPC infrastructure, there is a compelling use case for using Lustre as the file system for Hadoop analytics, as well as HPC storage. Intel Enterprise Edition for Lustre includes an Intel-developed adapter which allows users to run MapReduce applications directly on Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.”
Datameer, an end-to-end big data analytics application for the Hadoop ecosystem, today introduced Datameer 5.0 with Smart Execution, a new patent-pending technology that intelligently and dynamically selects the best-of-breed compute framework at each step in the big data analytics process.
“The vision for the Internet of Things is very powerful – a world in which assets, devices, machines, and cloud-based applications seamlessly interoperate, enabling new business models and services; with big data analytics as a foundation to support intelligent decision making in this connected world. As with every vision, the question is how to make it happen. This presentation provides key success factors for IoT, as well as a detailed overview of concrete IoT uses cases in the areas of automotive and transport, manufacturing and supply chain, as well as energy. Finally, a framework for IoT implementation is presented, which helps making your IoT projects a success.”
Continuent, Inc., a leading provider of open source database clustering and replication solutions, today announced Continuent Tungsten 3.0, a powerful solution that combines advanced clustering and replication technologies to meet the transaction processing and analytic needs of the entire business.
“Fortissimo Foundation is a clustered, pervasive, global direct-remote I/O access system that linearly scales I/O bandwidth, memory, Flash and hard disk storage capacity and server performance to provide an “in-memory” scale-out solution that intelligently aggregates all resources of a data center cluster into a massive global name space, bridging all remote compute and storage resources to look and act as if they were local.”