Over at Enterprise Storage Forum, Dr. Jeff Layton from Dell discusses why it’s important to break down what we mean by “Big Data” and begin by looking at how the bits are accessed and stored.
After considering how to access the data, primarily to get it into our storage system, we really need to think about how the data is actually stored. For example, is the data stored in a flat text file and if so, how is the data organized and how do you access it? Is the data stored in some sort of database? Is the data stored in some standard format such as hdf5? Or is the data stored in a proprietary format? Answering this question is more critical than people think because you will have to write methods to get the data you want and retrieve it in virtually any tool. In many cases, the choice of how and where the data is written and stored depends on the actual applications being developed and how the results are created and stored. But be sure to pay close attention to this aspect of Big Data — in what form will the data be stored?
Read the Full Story.