I’ve never had children, and so have never had to supervise squabbling siblings, each accusing the other of selfishness and insufficient sharing. Perhaps the MapReduce vendors are a form of karmic payback. Be that as it may, my client Cloudera has organized Hadoop World on October 2 in New York, and my other client Aster Data is hosting a MapReduce-centric Big Data Summit the night before, at the same venue. Even if you don’t go, both conference’s agenda pages offer a peek into what’s going on in MapReduce applications. I’m not going either, but even so I hope to post an overview of MapReduce uses after the conferences serve to publicize some of them.
Even better, I plan to hold a couple of webinars on MapReduce, the first at 10 am (blech) and 1 pm Eastern time on October 15. They’re sponsored by Aster Data, and so will have a strong SQL/MapReduce orientation.
In connection with its conference, Aster is introducing an nCluster-Hadoop connector — i.e., a loader from HDFS (Hadoop Distributed File System) implemented in SQL/MapReduce. In particular:
- While Aster nCluster has a solid parallel load capability from SQL sources, I believe this is the first time Aster is doing parallel load from a source that doesn’t talk to it in SQL. (Presumably, an alternative would be for the Hadoop cluster to run Hive.) I don’t know how this compares to, say, Greenplum’s implementation of Scatter/Gather.
- Unlike other parallel loading in Aster nCluster, the nCluster-Hadoop connector bypasses the loader nodes and goes straight to the worker nodes.
- This is not a load utility; it’s just a SQL function.
Meanwhile, each of SenSage and Splunk told me last week that they’ve been doing what amounts to MapReduce under the covers since their respective Day 1s. Who knew? (More on each company later.)
And as previously noted, Netezza and Teradata are doing MapReduce too. One of the exhibit-hall videos at Netezza’s Enzee Universe conference tour mentioned MapReduce, but I’ll confess to never having stopped to check what it actually was saying.