August 26, 2008

Known applications of MapReduce

Most of the actual MapReduce applications I’ve heard of fall into a few areas:

That covers all MapReduce apps I recall hearing about via commercial companies and users, and also includes most of what’s in the two big sources I found online. To wit:

1. In a slide presentation, Google offers the following applications of MapReduce:

2. The Hadoop applications page offers a rich trove of applications. Excerpts include:

There also were some research apps and some general processing speed-up apps I found harder to excerpt.

Some of our recent links about MapReduce

Comments

16 Responses to “Known applications of MapReduce”

  1. Winning with Data: Aster Data Systems Blog » Blog Archive » Leveraging In-Database MapReduce on August 26th, 2008 1:44 am

    [...] the sky is really the limit for anyone to build powerful analytic apps. Curt Monash has posted an excellent compendium of applications that are successfully leveraging the MapReduce paradigm [...]

  2. MapReduce links | DBMS2 -- DataBase Management System Services on August 27th, 2008 5:19 am

    [...] Three major applications of MapReduce [...]

  3. Three approaches to parallelizing data transformation | DBMS2 -- DataBase Management System Services on August 27th, 2008 5:21 am

    [...] third approach is my Subject Of The Week: MapReduce. When I posted a list of canonical MapReduce applications, my friends at Aster Data offered one pushback — I left out the area of data transformation. [...]

  4. Why MapReduce matters to SQL data warehousing | DBMS2 -- DataBase Management System Services on August 27th, 2008 5:22 am

    [...] Three major applications of MapReduce [...]

  5. The Great MapReduce Debate | DBMS2 -- DataBase Management System Services on August 27th, 2008 10:46 pm

    [...] Three major applications of MapReduce [...]

  6. Luke Lonergan on August 28th, 2008 1:13 pm

    There is a coding tutorial available at this link in the middle of the page: http://www.greenplum.com/resources/mapreduce/

    Key things to note about Greenplum’s MR implementation:
    - It’s very similar in form and expression to Google and Hadoop
    - Extensions for Joins and Pipelined task execution
    - Native parallel file access
    - Parallelism is full and transparent to the programmer

    In summary: we have implemented MapReduce within which you can write SQL, Perl, Python and many more languages. It is straightforward to use MR programs written for Hadoop or Google and port them to Greenplum.

  7. MapReduce sound bites | DBMS2 -- DataBase Management System Services on August 28th, 2008 2:47 pm

    [...] Three major applications of MapReduce [...]

  8. Winning with Data: Aster Data Systems Blog » Blog Archive » MapReduce Educational Resources on September 10th, 2008 11:45 am

    [...] If you are unable to attend, or eager to understand, here are some MapReduce resources you may find informative: Aster’s whitepaper on In-Database MapReduce; Google Labs’ MapReduce research paper; Curt Monash’s post on Known Applications of MapReduce. [...]

  9. Infology.Ru » Blog Archive » Почему MapReduce так важен для хранилищ данных? on October 5th, 2008 2:59 am

    [...] По существу, вы можете сделать все, что угодно с одной записью* – это шаг map. Но вы сильно ограничены в том, как вы можете объединить информацию о многих (часто промежуточных) записях – это шаг reduce. Тем не менее, шаг reduce позволяет вам выполнять подсчет, суммирование и другие операции агрегирования. Сей факт, вкупе с универсальной мощью шагов map, делает MapReduce полезным, по меньшей мере, для трех важных классов приложений: [...]

  10. Infology.Ru » Blog Archive » Три подхода к распараллеливанию процесса преобразования данных on October 9th, 2008 4:36 pm

    [...] «Темой Недели»: MapReduce. Когда я опубликовал список канонических приложений MapReduce, мои друзья из компании Aster Data предложили мне еще одно [...]

  11. Infology.Ru » Blog Archive » Несколько тезисов о MapReduce on November 3rd, 2008 2:31 pm

    [...] Три основных области применения MapReduce [...]

  12. Winning with Data: Aster Data Systems Blog » Blog Archive » The Importance of Visibility Across Rows on April 22nd, 2009 2:49 pm

    [...] Examples abound. Consider a SQL/MR function which applies a complex model to score the data in the database, whether it’s scoring a customer for insurance risk, scoring an internet user for an ad’s effectiveness, or scoring a snippet of text for its sentiment. These functions often construct a data structure in memory to accelerate scoring, which works very well with the SQL/MR API: build the data structure once and reuse it across a large number of rows. [...]

  13. Confluence: Philip Zeyliger on April 26th, 2009 5:49 pm

    Hadoop-MR Use Cases…

    I’m trying to college known uses of Hadoop/GFS/MapReduce, and categorize them somewhat.  When possible, citations are great…….

  14. Introduction to Datameer | DBMS2 -- DataBase Management System Services on April 16th, 2010 11:50 pm

    [...] Datameer seems to be designed for the classic MapReduce use cases of ETL and heavy data [...]

  15. MapReduce replacing complex SQL queries - Enterprise IT Consultant Views on Technologies and Trends on October 11th, 2010 2:54 am

    [...] Google points out that MapReduce is a powerful tool that can be applied for a variety of purposes including distributed grep, distributed sort, web link-graph reversal, term-vector per host, web access log stats, inverted index construction, document clustering, machine learning and statistical machine translation. A much longer list of MapReduce applications is available at http://www.dbms2.com/2008/08/26/known-applications-of-mapreduce/. [...]

  16. エド ハーディー 通販 on October 25th, 2013 1:15 am

    Hi there just wanted to give you a quick heads up. The text in your post seem to be running off the
    screen in Ie. I’m not sure if this is a formatting issue or something
    to do with web browser compatibility but I thought I’d post to
    let you know. The design look great though! Hope you get the issue solved soon.

    Thanks

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.