*I probably have it worse than most people in that regard, because my clients run tentative feature lists and announcement schedules by me well in advance, which may get changed multiple times before the final dates roll around. I also occasionally miss some detail, if it wasn’t in a pre-briefing but gets added at the end.

Anyhow, the three big themes of this month’s announcements are probably:

• Integrating different kinds of analytic processing into databases and DBMS.
• Taking advantage of hardware advances.
• Playing catchup in areas where small vendors’ products weren’t mature yet.

For example, the three biggest data warehouse DBMS product announcements this month are probably:

• Aster Data nCluster 4.5. Much like Aster’s prior release — Aster Data nCluster 4.0 – Aster Data nCluster 4.5 has a major focus on integrating analytics and database processing. This time, the emphasis is on application development tools and pre-built analytic packages. In addition, Aster’s management tool GUIs have been upgraded, building on catch-up functionality in the Aster Data nCluster 4.0.
• Netezza’s “i” add-on to its existing TwinFin products. With Netezza TwinFin(i), Netezza becomes the second MPP RDBMS vendor with a comprehensive “Big Data Analytic Platform” kind of strategy. (Netezza would surely argue that it was the first, but that depends on how seriously one took Netezza’s prior attempt.) Many of the details are different from Aster’s, of course, but the general philosophy is similar. So far, Netezza has announced one interesting proprietary library of analytic packages (for linear/matrix algebra), plus the port of 4,000 or so functions in open source libraries.
• Vertica 4.0. Vertica has had a highly innovative columnar DBMS architecture from the getgo, but at the cost of some restrictions or awkwardness in the relationship between data layout and SQL processing. Vertica says that Vertica 4.0 fixes all that. In addition, it has some analytic processing enhancements, especially in the time series area, where Vertica doesn’t vigorously dispute that Sybase IQ previously had an advantage.

In addition,

• Teradata is announcing its Data Warehouse Appliance 2580, the successor to the Teradata 2550. This is purely a hardware refresh; Teradata’s hardware and software upgrades are not generally synced. The Teradata 2580 upgrades CPUs from Harpertown to Nehalem, includes 3X the RAM of its predecessor, and offers an option for 1 TB disks (thus lowering the bottom price/TB a lot, to $31K list).
• Aster, Vertica, and ParAccel have all called attention to the fact that, if solid-state drives have interfaces like those of disk drives, and if a DBMS supports disk drives, then a DBMS also supports solid-state drives as well. At least Aster and ParAccel have signaled that they have at least one customer or prospect each interested in Fusion I/O’s solid-state technology, especially in the retail sector. This is basically a hardware matter as well, and a big deal only for those who were somehow unaware of the impending dominance of solid-state memory technology.
• Sybase announced its Aleri acquisition earlier this month.
• Various vendors have bragged about various rankings, awards, or benchmarks, or – sometimes less tediously — about last year’s sales results.

• Netezza’s software runs on well-designed proprietary hardware. Aster runs on hardware that’s more off-the-shelf.
• Aster was first to ship, and will also be first to ship an IDE (Integrated Development Environment).
• MapReduce is central to Aster’s approach. Netezza TwinFin(i) supports MapReduce too, specifically a Hadoop implementation, but I don’t get the sense that everything Netezza does is built on MapReduce underpinnings.
• Both Aster and Netezza try to provide rich functionality for creating in-memory data structures parallel analytic programs can use. Both seem to let you escape from the pure relational-table paradigm more easily than, say, Teradata’s new persistent memory capabilities do.
• Aster and Netezza have made different choices about what kinds of prebuilt analytic packages to offer. Netezza could actually leapfrog Aster in this regard, but let’s see where each vendor is by, say, mid-year. If you care about the details of built-in analytic functions, you really should consider executing non-disclosure agreements with both those companies.
• Both Aster and Netezza stress that you can run analytic functions out-of-process, greatly reducing the chance that they crash the database. Netezza and I’m pretty sure also Aster also retain the option of running in-process, which provides maximum performance. (In Netezza’s case C++ is the only in-process language supported, and I think Aster has a similar limitation.)
• Like Aster, Netezza is integrating SQL queries and other analytic processing under the same workload management rubric.
• Much like Aster, Netezza is tap-dancing by implying much richer forthcoming SAS support than anything currently announced. (The crunch-per-paragraph ratio in either vendor’s SAS-related press releases to date is distressingly low.)

More specifically, here are some highlights of what I know, am guessing, and/or am allowed to say about Netezza TwinFin(i) at this time.

• The foundation for the analytic add-ons in Netezza TwinFin(i) is some sort of low-level “analytic executables.” Not understanding exactly what these are is my biggest area of confusion in the whole TwinFin(i) stack. Are they all C++, with everything translated into same? Is there Java all the way down as an alternative? (E.g., Hadoop is written in Java.) Anyhow, whatever it is, it’s surely a big improvement on Netezza’s prior Verilog-based generation of analytic extensibility technology.
• The announced list of languages supported in Netezza TwinFin(i) is Java, Python, Fortran, R, and C/C++. More are coming.
• Netezza has named a lot of analytic functions it is adding, and hinting about more to come. It has named CRAN/R and GNU libraries, saying those have 1900 or more functions each. Netezza has also built its own linear algebra library for TwinFin(i), called nzMatrix. And as previously noted, TwinFin(i) also boasts a Hadoop implementation.
• I haven’t heard about much in the way of TwinFin(i)-specific IDE support.
• I don’t really have details as to what kinds of in-memory data structures Netezza TwinFin(i) does or doesn’t support.

• Aster Data Analytic Foundation, a set of analytic packages prebuilt in Aster’s SQL-MapReduce
• Aster Data Developer Express, an Eclipse-based IDE (Integrated Development Environment) for developing and testing applications built on Aster nCluster, Aster SQL-MapReduce, and Aster Data Analytic Foundation

And in other Aster news:

• Along with the development GUI in Aster nCluster 4.5, there is also a new administrative GUI.
• Aster has certified that nCluster works with Fusion I/O boards, because at least one retail industry prospect cares. However, that in no way means that arm’s-length Fusion I/O certification is Aster’s ultimate solid-state memory strategy.
• I had the wrong impression about how far Aster/SAS integration has gotten. So far, it’s just at the connector level.

Aster Data Developer Express evidently does some cool stuff, like providing some sort of parallelism testing right on your desktop. It also generates lots of stub code, saving humans from the tedium of doing that. Useful, obviously.

But mainly, I want to write about the analytic packages. I’m not convinced that they’re a big deal in themselves yet, or that a whole lot of person-months have gone into their combined development. Still, I think they provide a great indication of one direction in which analytic functionality is going. And by the way, Aster promises to release a lot more of that kind of thing over the next 12 months.

Aster’s flagship analytic package is nPath, which is like a regular expression matcher, but for (time) series of data rather than for character strings. The main use for nPath is in pulling specific kinds of event sequences out of web or network event logs. However, one could imagine uses in other sectors that focus on temporal or sequential data (e.g., trading, intelligence, other sensor analysis), should existing SQL- and/or CEP-based technologies not prove sufficiently flexible. Aster 4.5 adds some new aggregation capabilities around nPath.

Other not-wholly-new packages in the Aster Data Analytic Foundation announcement are for sessionization (of clickstream data and the like) and tokenization (of text/character string data). While sessionization can be done in SQL, Aster thinks its MapReduce-based version is faster, since it doesn’t require self-joins. Makes sense. Aster’s tokenization sounds lame, however – text analytics in MapReduce tends to reinvent simplistic wheels for no clear reason, and Aster doesn’t seem to be an exception. (Aster would argue, however, that anything it does in SQL-MapReduce is more flexible than pure SQL or pure MapReduce alternatives.)

Another example of better-living-without-self-joins is Aster’s new market basket package. This lets you look at a set of point-of-sale data, pick a small integer N, and pull out all the sets of N things that were bought by the same person at the same time. I haven’t probed the claim in detail, but Aster implies there’s less combinatorial explosion in its approach than it is in the self-join alternative.

Note: Gartner highlighted self joins as a performance challenge in its recent Data Warehouse Magic Quadrant.

Aster is also releasing a few statistical and general analytic functions — specifically (and I quote a slide):

• exponential moving average
• weighted moving average
• simple moving average
• volume-weighted average price
• correlation
• linear regression
• logistic regression
• approximate_percentile
• approximate_count_distinct

The point of the last two items on the list is that if you set a non-zero tolerance for error, you can you can count things or order them into bins very efficiently – especially in terms of RAM — while being guaranteed not to exceed your error tolerance.

Note: One obvious inference from this list — which Aster gladly confirms — is that Aster has high hopes of selling to the financial services industry.

Finally, Aster is releasing its first pure graph-analytic function, for finding the shortest path between a given pair of nodes.

While I had the Aster folks on the phone anyway, I also took the opportunity to ask about the Aster nCluster 4.0 capability to create fairly persistent non-relational in-memory data structures. Specifically, I asked whether different users could access the same in-memory structure, and was told that this is a little klugey but not too horrendous. That suggests Aster’s capability may be a strict superset of UDF-based (User-Defined Function) approaches to meeting the same need, at least from a functionality standpoint. However, ease of creating those in-memory structures may still be better in the more SQL/UDF-centric approach favored by Teradata.

• Personally selected annual lists of 12 “Most influential” companies and 36 “Companies to watch” in analytics- and database-related sectors.
• Made it clear that these are his personal selections.
• Nonetheless has called it an Editors’ Choice list, rather than Editor’s Choice.

(Actually, he’s really called it an “award.”)

People advising Doug — who come to think of it actually are Contributing Editors to Intelligent Enterprise or something like that — included Cindi Howson, Seth Grimes, three others, and me.

And if past is prologue, I will now get a flood of PR emails calling my attention to this award that I already have both participated in and blogged about.

As usual, the sense:nonsense ratio on these lists was pleasingly high. Analytic DBMS vendors cited included IBM, Microsoft, Netezza, Oracle, Sybase, and Teradata in the “Most influential” group, with Aster, Greenplum, HP, Infobright, and Vertica among the “To watch” crowd. It’s tough to argue with those selections, whose most questionable element is probably the not-ridiculous supposition that HP could do something interesting over the coming year. Cloudera and Intersystems also made the list, deservedly.

All three of QlikTech, Tableau, and TIBCO made the list, which is appropriate given the potential for and interest in interactive data exploration technology.  The BI majors, independent or otherwise, were all on as well. In text mining, Doug included Attensity and Clarabridge, which I think is exactly right. (Plus OpenCalais.)  Upon reflection, I probably should have nominated Mark Logic, even though most of its business is non-enterprise; but hey, nobody’s perfect, and the same goes for lists. Open source was well represented, with Apache, Actuate, Jaspersoft, Eclipse, Infobright, Nuxeo and R all being cited (but not Ingres or Pentaho). Kalido made the list, with my endorsement, their silly I-CASE like marketing messaging notwithstanding.

Speaking of imperfections — there only are a few category names, and so category assignments can be pretty bizarre. (In an ideal world, middleware wouldn’t be included under “enterprise applications”.) Greenplum hasn’t really “extended” its DBMS with a “cloud” option. As much as I’d like Netezza to be more influential than SAP, that’s probably not the best way to rank them. And there are a number of “This company is on a roll!” kinds of comments that I wouldn’t necessarily endorse.

But those are all nitpicks. On the whole, it’s another nice job.

*Links to Gartner Magic Quadrants commonly break, but that one worked at the time of this posting.

• Gartner thinks that data warehouse appliances are on the rise, due to their simplicity.
• Gartner correctly says that Teradata has been a data warehouse appliance vendor from the getgo.
• Gartner characterizes IBM as being an appliance vendor as well.
• Gartner suggests that HP is having trouble living up to its technical promises for Neoview.
• Gartner further suggests — no surprise here — that HP Neoview has had very few new customers past its initial wave.
• Gartner notes IBM’s difficulties in selling data warehouse installations of DB2, despite what on paper is great-sounding technology.
• Gartner says — also no surprise — that illuminate “has seen little success in North America since opening its first office in the U.S. over two years ago.”
• Ingres has evidently gotten a few BI-centric “appliance” deals, e.g. with Jaspersoft. But basically Ingres isn’t doing well in data warehousing.
• Gartner does say Ingres has “the strongest open-source DBMS offering for data warehousing.” Being very literal about “open source,” that’s a defensible claim — but it’s pretty irrelevant in a world where Greenplum Single-Node Edition can be had for free. It also waves away all the data mart use cases in which Infobright Community Edition shines.
• Gartner says that Netezza is working out as a “complex workload” enterprise data warehouse provider, according to reference checks, in addition to its established success in data mart scenarios.
• Gartner says Oracle’s offering has finally become “accepted” in the market for databases >50 TB. I guess I can live with that fairly weak claim, but I wouldn’t go much further than that.
• Gartner asserts that, unlike software-only Oracle, Oracle Exadata isn’t significantly harder to administer than “other mixed OLTP/OLAP DBMS vendors,” because Exadata is fast enough you don’t need to jump through all those hoops any more to get tolerable performance. The money quote is “one reference reported reducing the number of indexes by a factor of 100 to fewer than five.” Note, however, that Gartner does not seem to assert that Exadata’s ease of use rivals that of the newer analytic DBMS specialists.
• Gartner confirms Oracle’s reluctance to do onsite Exadata POCs, but says it is not absolute. This is roughly compatible with what I’m hearing elsewhere, and indeed with Oracle own claims to be ramping up availability of Exadata POC hardware.
• Gartner’s criteria for inclusion include at least 10 different organizations having a product “in production.” Thus, the big surprise was ParAccel being included. The money quote there is “With approximately 20 customers in the pharmaceutical, retail, financial and media/advertising analytics sectors, ParAccel has a good reference base.” That assessment is difficult to reconcile with other information, but I’ve been told Gartner is sticking to its guns. That assessment would be even harder to believe if those 20 references were all alleged to be true production customers.
• Gartner notes that you basically can’t run a 1 TB+ MySQL data warehouse without sharding. (Of course, Infobright has an alternative, and up to a small number of terabytes so does Kickfire.)
• Gartner reports that at least some customers are pleased with Sybase IQ’s mixed workload/enterprise data warehouse capabilities.
• Gartner correctly notes that Oracle Exadata is a price-competition challenge for Teradata.
• Gartner notes that 20% of Vertica’s customers are outside the US. While not shocking, that’s more than I realized.
• Gartner notes something I don’t think I’ve posted yet, which is that Vertica has a customer with 300 TB of data. (The identity is a deep dark secret, but if I told you you probably wouldn’t recognize the name anyway.)

As does any such piece, the Gartner Data Warehouse DBMS Magic Quadrant also has outright errors.  For example:

• Aster Data isn’t really “the newest entrant to the DBMS data warehouse world.”
• Aster’s SQL/MapReduce was not new in Release 4.0.
• Greenplum isn’t yet pushing down code to the storage tier.
• I’m not sure what kind of database-tier parallelism Gartner is claiming is new in Oracle in 11g Release 2 — but I doubt it’s really new. Rather, what Oracle has done recently is make parallelism less administratively cumbersome.
• Vertica wasn’t really the first DBMS in the cloud. At most it was the first pure-play analytic DBMS to get there.

• MapReduce was named and popularized — but not invented — by Google.
• “MapReduce” variously refers to:
  • A programming paradigm
  • Execution engines that implement the programming paradigm
  • Distributed file systems that work with the execution engines
• In particular, Hadoop is a MapReduce execution engine that includes or is closely associated with HDFS (Hadoop Distributed File System).
• MapReduce and analytic DBMS can interact in a number of different ways, including:
  • Tight integration between a DBMS and exposed MapReduce functionality, e.g. Aster Data’s SQL/MapReduce or Greenplum.
  • Integrated MapReduce “under the covers”, e.g. SenSage or Oracle. This may or may not follow all the rules Google laid out for MapReduce, but it’s at least similar in spirit.
  • Looser coupling between DBMS and a MapReduce system, e.g. Vertica/Hadoop, in which MapReduce may or may not run on a different cluster than the DBMS.
  • Not at all, except perhaps insofar as a quasi-DBMS such as Hive is implemented over a MapReduce system such as Hadoop/HDFS.
• As predicted by Monash’s First Law of Commercial Semantics, different vendors have individual variants on those themes. For example, as per a registration-required white paper, Splunk is moving to publicly expose a not-quite-complete form of MapReduce.
• MapReduce implementations such as Hadoop are sometimes regarded as part of the NoSQL “movement”. When they are, many generalities about NoSQL — such as that it doesn’t deal with analytics — are falsified.
• So far as I can tell, mainstream enterprise (as opposed to web, scientific, investment, etc.) data mining folks may be looking at MapReduce for data mining, but they haven’t done much to adopt it yet. Probably that’s because the outfits who have the greatest need are the same ones that have the largest sunk investments in more traditional ways of doing data mining.
• Cloudera != Hadoop. On the other hand, if you want to use Hadoop, it makes a lot of sense to do business with Cloudera.
• Non-DBMS MapReduce != Hadoop. On the other hand, Hadoop is the default choice for non-DBMS MapReduce.
• MapReduce != Hadoop, period. DBMS-based MapReduce is also a legitimate technical strategy.

• Registration for tomorrow’s webinars
• Replay of the first webinar
• My slides from the first webinar

The main subjects of the webinar will be:

• Some review of material from the first webinar (all three presenters)
• Discussion of how MapReduce can help with three kinds of analytics:
  • Pattern matching (Jonathan will give detail)
  • Number-crunching (I’ll cover that, and it will be short)
  • Graph analytics (I haven’t written the slides yet, but my starting point will be some of the relationship analytics ideas we discussed in August)

Arguably, aspects of data transformation fit into each of those three categories, which may help explain why data transformation has been so prominent among the early applications of MapReduce.

As you can see from Aster’s title for the webinar (which they picked while I was on vacation), at least their portion will be focused on customer analytics, e.g. web analytics.

Aster Data is officially announcing the 4.0 release of nCluster. There are two big pieces to this announcement:

• Aster is offering a slick vision for integrating big-database management and general analytic processing on the same MPP cluster, under the not-so-slick name “Data-Application Server.”
• Aster is also offering a sophisticated vision for workload management.

In addition, Aster has matured nCluster in various ways, for example cleaning up a performance problem with single-row updates.

Highlights of the Aster “Data-Application Server” story include:

• At its core, the Aster “Data-Application Server” is the Aster nCluster MPP analytic DBMS, enhanced with basic application server functionality (I didn’t ask for details of that part), running on the same nCluster worker nodes that answer SQL queries.
• Thus, Aster is eliminating a lot of the data movement that plagues three-tier architectures and other less-integrated approaches.
• The Aster “Data-Application Server” further offers integrated workload management for applications and queries; more on that below.
• The Aster “Data-Application Server” requires applications to be parallelized and invoked via Aster’s SQL/MapReduce.
• As befits a MapReduce-based system, the Aster “Data-Application Server” lets you write your applications in lots of different languages (the usual suspects, and it also does .NET).
• The Aster “Data-Application Server” runs applications in their own process spaces, protecting the DBMS server from crashes and other damaging behavior.
• The Aster “Data-Application Server” allows applications to manage memory themselves, persistently, and not just via relational constructs. Thus, if you want your application to maintain a graph, mini rules engine, and/or finite state machine, you can, without doing SQL contortions.

In a compelling proof point for the Aster Data-Application Server’s slickness, Aster has leapfrogged Teradata and Netezza in the extent to which SAS functionality is integrated into Aster’s DBMS. (Aster and SAS both say that you can do full SAS modeling in parallel on Aster, but even so I wouldn’t be surprised to discover there were some parts of SAS’ system that turned out to be exceptions.) Of course, Aster is hardly the only analytic DBMS vendor to have the idea of explicitly enhancing general analytic processing; that’s why we see lots of MapReduce announcements, and it’s also why Teradata enhanced its UDFs (User-Defined Functions) to have some kind of persistent memory.* But I don’t know of anybody else whose approach is quite so elegant and general at this time.

*Unfortunately, I don’t yet know much about Teradata’s UDF enhancements. I neglected to drill down on Global Persistent Memory when it was mentioned a couple of times at Teradata Partners last week, and Teradata was unable to accommodate my request this week for a rapid follow-up briefing on the subject.

Aster’s approach to workload management is similarly stylish. The idea is:

• Lots of variables are available to be taken into account (e.g., user role, expected query duration, actual duration of a running query, etc.)
• SQL statements can be written against any of these variables.
• The SQL statements serve as rules to set query/task priorities.
• There seem to be a few different ways to measure priority, including explicit allocation of CPU or I/O resources, as well as more conventional “This group of queries gets higher priority than that one” kinds of metrics.
• The whole thing provides integrated workload management for queries, applications, load jobs, data redistribution, and so on.

Right now the interface is – well, you’re manipulating a SQL table. A more conventional workload management GUI is slated for the second quarter of 2010.

Discussing subjects such as mirroring and ILM (Information Lifecycle Management) with Aster can be tricky, as Aster uses the word “partition” in confusing ways. Anyhow, Aster has a few different levels of compression, and the ability to apply different levels of compression to different partitions, to change compression levels via ALTER TABLE, and to alter (presumably increase) compression on the fly when doing online backup. Aster is also part of a growing trend to eschew RAID, instead doing mirroring in its own software. (Other examples of this strategy would be Vertica, Oracle Exadata/ASM, and Teradata Fallback.) Prior to nCluster 4.0, this caused a problem, in that the block sizes for mirroring were so large as to create a lag in transactional updating. But Aster says this problem is now solved, and indeed claims that nCluster 4.0 is superior to most rivals in transactional efficiency.

And finally, while I was talking w/ Aster Data anyway, I checked up on cloud and MapReduce customer penetration. The answers were:

• Aster has two serious production cloud users, both of which have been disclosed for a while, namely:
  • ShareThis, which runs Aster nCluster on Amazon EC2
  • Didit, which runs Aster nCluster on AppNexus
• Outside of those two, Aster sees some cloud use for test, development, prototyping, etc.
• Every single Aster customer uses SQL/MapReduce — i.e., they invoke MapReduce via Aster nCluster SQL queries.
• Some of those customers use MapReduce for ETL, some use it for actual analytics.

]]> http://www.dbms2.com/2009/10/30/aster-data-application-server-ncluster/feed/ 6 http://www.dbms2.com/2009/10/18/three-big-myths-about-mapreduce/ http://www.dbms2.com/2009/10/18/three-big-myths-about-mapreduce/#comments Sun, 18 Oct 2009 16:14:37 +0000 Curt Monash http://www.dbms2.com/?p=1135 Once again, I find myself writing and talking a lot about MapReduce. But I suspect that MapReduce-related conversations would go better if we overcame three fairly common MapReduce myths:

• MapReduce is something very new
• MapReduce involves strict adherence to the Map-Reduce programming paradigm
• MapReduce is a single technology

So let’s give it a try.

When Dave DeWitt and Mike Stonebraker leveled their famous blast at MapReduce, many people thought they overstated their case. But one part of their story – one that both Mike and Dave say was most central to their case – was never effectively refuted, namely the claim that these ideas aren’t particularly new. I haven’t actually read enough computer science literature to have an independent opinion on that issue. But I’ll say this – claims from companies such as SenSage, Oracle, or Splunk that “We’ve been doing MapReduce all along” seem pretty credible to me.

True, what those companies were doing things may not have looked exactly like the instant-classic MapReduce programming paradigm. But the same is true of many things almost everybody would agree count as MapReduce. In particular, it is often not the case that you alternate Map and Reduce steps, each of whose outputs is a set of simple <Key, Value> pairs, with data redistributed based on Key at every step.

Here are some examples of what I mean, drawn from my recent MapReduce webinar.

• If you do text indexing in MapReduce, your goal is to wind up with a text index. So at some point you Reduce to a pair <WordName, {all the (DocumentID, offset) pairs for the whole corpus, suitably ordered}>. That’s a heckuva compound “Value”.
• The goal of data mining is usually to estimate a rather small number of parameters based on a large overall data set, often – depending on algorithm – in the form of a single vector. When you do that in MapReduce. you partition data among nodes, calculate something on each node that is structured more or less like your final vector. So when it comes time for the reduce, you just ship all of your vectors – one per node – to a single Reduce node, and do the appropriate math. Redistribution based on Key would be quite pointless.
• When you sessionize clickstream logs in MapReduce, you may have just as many output records as input records. However, they now are reformatted, and might have a SessionID appended. In those cases, Reduce isn’t doing much by the way of reduction.
• And as I happens in some Vertica-Hadoop use cases around mortgage trading, sometimes MapReduce can even make data sets vastly larger.

By no means do I think this is a weakness of the MapReduce programming paradigm. Rather, I think it’s a MapReduce strength. But it’s not quite the way MapReduce has been promoted and explained to the IT public.

Finally: MapReduce, as commonly conceived, spans two different – albeit closely related – technology domains:

• Parallel programming
• Distributed data management

For example, I imagine Greenplum’s and Vertica’s MapReduce/SQL combined syntaxes are very similar to each others. But Vertica’s data management implementation of MapReduce, which relies on Hadoop, is very different from Greenplum’s, which is tied into the Greenplum DBMS. Similary, non-DBMS MapReduce implementations are commonly associated with distributed file systems – notably HDFS (Hadoop Distributed File Systems) or Google’s internal GFS (Google File System). In those systems, the parallel language execution part should be aware of how the distributed file management part works – but perhaps that awareness can be pretty lightweight.

Right now, this is a distinction pretty much without a difference. If you choose an implementation of MapReduce — like pure Hadoop (say in the Cloudera distribution) or Hadoop-Vertica or Aster Data’s SQL/MapReduce – you’re basically picking an entire technology stack. But those stacks are going to do a whole lot of changing and maturing in the near future – and as they do, it’s likely that projects will interact or even combine in all sorts of interesting ways.

Bottom line: There are a lot of different ways to exploit MapReduce-related technology.

• The subject is MapReduce.
• The sponsor is Aster Data.
• Part of the webinar will be an explanation of MapReduce basics, especially the conflict between theory/propaganda and reality.
• As you might guess from the identity of the sponsor, there will be an emphasis on how MapReduce and SQL play nicely with each other.
• You can register for the webinar on Aster’s site.
• (Edit) The webinar replay can be found here.
• I’ve already uploaded the slides from which I will present. (But not the ones from which Aster folks will be talking. I’ve seen those, and there’s some good technical crunch in some of them.) The “Notes” under the slides have a number of relevant URLs for follow-up, as well as a small number of explanatory comments (e.g., as to why one slide simply has a quote from and corresponding picture of Shakespeare).