May 12, 2011

Data integration vendors and Hadoop

There have been many recent announcements about how data integration/ETL (Extract/Transform/Load) vendors are going to work with MapReduce.  Most of what they say boils down to one or more of a few things:

Some additional twists include:

Finally, my former clients at Pervasive, who haven’t briefed me for a while, seem to have told Doug Henschen that they have pointed DataRush at MapReduce.* However, I couldn’t find evidence of same on the Pervasive DataRush website beyond some help in using all the cores on any one Hadoop node.

*Also see that article because it names a bunch of ETL vendors doing Hadoop-related things.

Comments

One Response to “Data integration vendors and Hadoop”

  1. Keith Kohl on May 16th, 2011 7:42 am

    Thanks for the BLOG Curt, very interesting on what you’re seeing in the market.

    As way of an update, our sort “plug-in” contribution to the Hadoop community is progressing. We have received several suggestions from the community which we working to incorporate. The community also asked that we use the new MR framework that is being proposed and we are investigating this. As you note, we’ll be providing Hive-like functionality with users creating MR processes graphically with DMExpress. This should significantly simplify MR development. As we’ve discussed DMExpress biggest strengths are large file processing throughput. The DMExpress engine will execute the data processing within the Hadoop framework. The net result is much easier development in a graphical environment and extremely fast throughput.

    In addition, Syncsort is working to benchmark higher volumes on a larger cluster than what we announced on May 4 to demonstrate what we strongly believe to be higher scalability (this could be worded better). We are working with comScore to do this. Stay tuned…

    –Keith Kohl

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.