Writing for Tuesday, October 22

Writing for Tuesday, October 22

by Garriott Jones -
Number of replies: 0

 Hadoop has completely reinvented the way businesses crunch their data. Hadoop got its start from yahoo. Hadoop was built to harness the power of thousands of ordinary computer servers. Facebook then started using Hadoop because it was the only database system that could store and process the massive amounts of data that was being generated by Facebook. Hadoop would go on to reinvent data analysis, this led to other programs trying to play catch up. Now, EMC is releasing Pivotal HD. "this tool can store the massive amounts of information Hadoop was created to store, but it's designed to ask questions of this data significantly faster than you can with the existing open source platform." The trouble with Hadoop has always been that it takes a very long time to analyze data. Along with Pivotal HD, it will have a new  "'query engine" that can more quickly run SQL queries on data stored across a massive cluster of systems using the Hadoop File System.' "The query extensions of Pivotal HD behave slightly differently in that they require a restart of the query when a machine is lost, an individual query needs to be restarted but the integrity, accessibility and functionality of the system is guaranteed to continue. We consider this a small price to pay for several orders of magnitude performance enhancement as we do not materialize any results during processing." Pivotal HD and the new query engine is yet another sign that Hadoop will continue to reinvent the way businesses crunch their data.