InsideBIGDATA Guide to In-Memory Computing

In-memory computing is an emerging field of importance in the big data industry. It is a quickly evolving technology, seen by many as an effective way to address the proverbial
3 V’s of big data—volume, velocity, and variety. Big data requires ever more powerful means to process and analyze growing stores of data, being collected at more rapid rates, and with increasing diversity in the types of data being sought—both structured and unstructured. In-memory computing’s rapid rise in the marketplace has the big data community on alert. In fact, Gartner picked
in-memory computing as one of the Top Ten Strategic Initiatives.  But what is In-memory computing? Put simply, in-memory computing primarily relies on keeping data in a server’s RAM instead of much slower spinning disk or flash devices and massive parallelization as a means of processing at faster speeds. In-memory computing especially applies to processing problems that require extensive access to data analytics, reporting, data warehousing, high-speed transactions and big data applications. There are two primary ways for how IMC solves the problems of big data—lowering latency time for analytics applications and yielding higher throughput for transactional applications. By removing high-latency devices like disk drives out of the equation, the time it takes to process a request and deliver a response (latency) is dramatically reduced. Estimates vary depending on disk speed and available input/output (I/O) bandwidth, but by one measure, RAM latency is estimated at 83 nanoseconds and disk latency at 13 milliseconds, a whopping speedup of 6 orders of magnitude. You can’t own this entire speed advantage because there are CPU processing time and other constraints in the mix, but disk I/O has long throttled performance. In-memory performance improvements vary by application, data volume, data complexity, and concurrent-user loads, but by any measure the speedup with IMC can be dramatic.  The case for IMC and analytics applications is clear, but what about mainstream transactional systems?
IMC represents a significant opportunity when you can run a query, process, or transaction 10x, 20x, even 100x faster than you used to. What will that difference do for your business? The answer will determine whether IMC is a specialty tool for only a few unique use cases, or a platform upon which your entire enterprise IT runs. The purpose of this Guide is to provide the information that enterprise thought-leaders need to make strategic decisions about the use of IMC.