Optimizing Hadoop for Intel Architecture

Print Friendly, PDF & Email

Over at The Data Stack, Intel’s Tim Allen writes that the key to optimizing Hadoop on x86 is to tune the underlying Java so that it takes advantage of capabilities in Intel hardware. When you do that, you can expect to see up to 70 percent faster performance on Hadoop sort operations.

Hadoop spawns a new Java Virtual Machine* (JVM) for each MapReduce function on each slave node. This means that a large analytics job can result in the creation of thousands of individual JVMs. Because Hadoop does not share memory resources across nodes, each JVM and Java service must perform optimally. Reduced performance on any single node can hamper data analytics performance across the cluster.

Read the Full Story.

Speak Your Mind