Greenplum Single-Node Edition:

• Is free of charge, although you can buy support.
• Has no restrictions on use, production or otherwise.
• Has no restrictions on database size.
• Is closed-source.

For those who want free, terabyte-scale data warehousing software, Greenplum Single-Node Edition may be quite appealing, considering that the main available alternatives are:

• General-purpose open-source DBMS, such as PostgreSQL and MySQL (lacking analytic DBMS performance and features)
• Infobright Community Edition (the other best choice – Infobright’s commercial sales success indicates the solidity of Infobright’s technology)
• Rough research-project code and other other questionable open source offerings
• Crippleware from other commercial analytic DBMS vendors (e.g., Teradata)

For example, comparing PostgreSQL-based Greenplum with PostgreSQL itself, Greenplum offers:

• The ability to scale out queries across all cores in your box (and no, pgpool is not a serious alternative)
• Storage alternatives such as columnar (I am told that EnterpriseDB recently stopped funding a project for a PostgreSQL columnar option)

Greenplum would surely also argue that its software is superior to PostgreSQL in parallel load, compression, MapReduce integration, and general fit-and-finish. I imagine that in some (perhaps not all) cases it would be right. PostgreSQL’s main technical advantages over Greenplum would probably lie in the area of datatype extensibility.

The main target users for Greenplum’s Single-Node Edition are obviously individual enterprise power users or very small analytic teams. I.e., it’s people with a data mart need that a central data warehouse isn’t meeting. Potential benefits to Greenplum include:

• Adding value to its Enterprise Data Cloud story
• Seeding the market for future enterprise sales
• Depriving competitors of revenue, perhaps at enterprises too small to ever be paying Greenplum customers

In addition, I see free Greenplum as a charity offering that could be appealing to scientists who face PostgreSQL performance limitations.

Related links

• Greenplum Free Single-Node Edition press release (I’m quoted)
• MySQL Performance blog on MonetDB and Infobright community edition
• PostgreSQL’s restriction to one core per query
• Infobright’s restriction to one core per query

The basic idea of HadoopDB is to put copies of a DBMS at different nodes of a grid, and use Hadoop to parcel work among them. Major benefits when compared with massively parallel DBMS are said to be:

• Open/cheap/free
• Query fault-tolerance
• The related concept of tolerating node degradation that isn’t an outright node failure.

HadoopDB has actually been built with PostgreSQL. That version achieved performance well below that of a commercial DBMS “DBX”, where X=2. Column-store guru Abadi has repeatedly signaled his intention to try out HadoopDB with VectorWise at the nodes instead. (Recall that VectorWise is shared-everything.) It will be interesting to see how that configuration performs.

The real opportunity for HadoopDB, however, in my opinion may lie elsewhere. Rather than trying to compete with parallel relational DBMS, HadoopDB might do more good parallelizing more specialized kinds of database engines. How about, for example, a massively parallel XML manager to compete with MarkLogic? Or a massively parallel array processor other than the still-nascent SciDB? Or, even more to the point, something that parallelizes a yet-more-specialized scientific data management engine? That kind of area is where I suspect the potential for HadoopDB really lives.

]]> http://www.dbms2.com/2009/09/13/hadoopdb/feed/ 3 http://www.dbms2.com/2009/09/10/what-could-or-should-make-oraclemysql-antitrust-concerns-go-away/ http://www.dbms2.com/2009/09/10/what-could-or-should-make-oraclemysql-antitrust-concerns-go-away/#comments Thu, 10 Sep 2009 14:53:23 +0000 Curt Monash http://www.dbms2.com/?p=879 When the Oracle/MySQL deal was first announced, I wrote:

I can probably come up with business practices that could make things very hard on Oracle/MySQL competitors … but I haven’t found a compelling antitrust trigger on my first pass over the subject.

Subsequently, there’s been a lot of discussion about whether or not Oracle can use control of MySQL to make life difficult for third-party MySQL storage engine vendors.

Now that the European Commission is delaying the Oracle/Sun deal, explicitly because of Oracle/MySQL antitrust fears.  That is, the European Commission wants to be reassured that an Oracle takeover of MySQL won’t unduly impinge upon the future availability of open source/low cost DBMS alternatives.  This raises that natural question:

What could Oracle do to assure concerned parties that its ownership of MySQL won’t unduly hamper open-source-based DBMS competition?

I think that’s indeed the crucial question. The Oracle/Sun deal has enough momentum at this point that it both should and will be allowed to happen — perhaps with safeguards — rather than banned outright. If  you have concerns about Oracle’s pending acquisition of MySQL, you should speak up and outline what kinds of regulatory safeguards would alleviate the problems you foresee.

More or less obvious possibilities include:

• Divest MySQL. This is obviously an extreme measure, but it surely would work.
• Provide some money and trademark rights to MySQL forkers. If MariaDB and Drizzle were put into strong competitive positions with MySQL today, it’s hard to argue how regulators could object to any future Oracle maneuverings Oracle might envision with the GPLed side of MySQL.
• Offer a standard, attractive, long-term deal to MySQL bundlers. The commercial/non-GPL version of MySQL is a requirement for appliance vendors (surely), OEM vendors (probably), and storage engine vendors (maybe — I disagree, but I’m evidently in the minority).
• Strengthen PostgreSQL. Realistically, that’s not going to be part of any Oracle/MySQL resolution, so I’ll leave it as a subject for another time.

The downside to Continuent Tungsten ’s stripped down architecture is that it doesn’t solve the most extreme performance scale-out problems. Instead, Continuent focuses on the other big benefits of keeping your data in more than one place, namely high availability and data loss prevention (i.e., backup).

Continuent has been around for a number of years, starting out in Finland but now being based in Silicon Valley. For most purposes, however, it’s reasonable to think of Continuent and Tungsten as start-up efforts.

As you might guess from the references to Finland and MySQL, Continuent’s products are open source, or at least have open source versions. I’m still a little fuzzy as to which features are open sourced and which are not. For that matter, I’m still unclear as to Tungsten’s feature list overall …

My client actually signed a deal for EnterpriseDB’s Postgres Plus Advanced Server and GridSQL, but unwound the transaction quickly. (They say EnterpriseDB was very gracious about the reversal.) There seem to have been two main reasons for the flip-flop.  First, it seems that EnterpriseDB’s version of Postgres isn’t up to PostgreSQL’s 8.4 feature set yet, although EnterpriseDB’s timetable for catching up might have tolerable. But GridSQL apparently is further behind yet, with no timetable for up-to-date PostgreSQL compatibility.  That was the dealbreaker.

The current base-case plan is to use generic open source PostgreSQL, with scale-out achieved via hand sharding, Hibernate, or … ??? Experience and thoughts along those lines would be much appreciated.

Another option for OLTP performance and scale-out is of course memory-centric options such as VoltDB or the Groovy SQL Switch.  But this client’s database is terabyte-scale, so hardware costs could be an issue, as of course could be product maturity.

By the way, a large fraction of these updates will be actual changes, as opposed to new records, in case that matters.  I expect that the schema being updated will be very simple — i.e., clearly simpler than in a classic order entry scenario.

• After much prodding, Greenplum finally gave me clear list pricing. Greenplum perpetual licenses list at $16K/core or $70K/terabyte. Annual maintenance is 22% of purchase price. Alternatively, one can buy an annual subscription on either basis, at 50% of the perpetual license purchase price. Of course, that’s just list. Quantity discounts are de rigeur.
• Greenplum had about 65 paying customers at the end of Q1. I’ve forgotten how that jibes with a figure of 50 customers last August.
• Greenplum claims rich functionality in standard SQL. In particular, Greenplum says “lots” of customers are using SQL 2003 OLAP. Greenplum further says it has “comprehensive” SQL-92 and –99 support.
• Greenplum Release 3.3 has “more flexible” compression, which Greenplum bravely asserts is now fairly close to columnar compression in effectiveness. (Aster Data and other row-based vendors make similar claims.)
• Greenplum Release 3.3 contains few performance enhancements for analytics, fixing an OLAP edge case that wasn’t previously parallelized — relevant buzzwords include grouping, aggregates, and DISTINCT, apparently in combination with each other — and speeding up sorts.
• Greenplum’s data loading story goes something like this:
  • Greenplum has an external tables facility that, in principle, could be used to index and query on tables outside Greenplum. It’s almost never actually used for that. However, external tables is the main way to load data into Greenplum from another relational DBMS.
  • A huge benefit of loading Greenplum via external tables is that you can load in parallel without passing the data through the master node.
  • Another benefit is that you can do ETL by building a view on the foreign database, then loading that view verbatim into Greenplum. (I guess this is an exception to Greenplum’s ELT orientation.)
  • In addition, Greenplum has something called Scatter/Gather which puts daemons on the hosts for flat files, allow the files to be loaded into Greenplum in parallel.
  • Like many data warehouse DBMS vendors, Greenplum tells you that if update volumes are high enough, you should bang them into something else and then feed the data warehouse in microbatches. Greenplum’s recommendations for the “something else” are PostgreSQL or file systems. Apparently, this is happening at some Greenplum telco customers. In one case, latency is only 15 seconds.
• In general, Greenplum asserts that very little work is done at the Greenplum master node, and the Greenplum master node isn’t a bottleneck.
• Greenplum proudly promises that its customers will never have to do dump/restore for any release, even the big more-than-point ones that only come around every few years.
• Greenplum added some Greenplum-awareness features to the pgAdmin III Postgres administration tool, which seems to be the most widely used tool with Greenplum today.
• Greenplum says it has 10-gigabit switches running in the lab, but doesn’t need them. For now it’s sticking with its “handful of commodity 1-gigabit switches” strategy.
• Greenplum MapReduce news and commentary include:
  • I’ve only ever gotten a single clear example of Greenplum MapReduce production use. But multiple Greenplum users are actively developing in MapReduce, judging by their dialog with the company.
  • Greenplum 3.3 has some MapReduce ease-of-use/programming upgrades, in low-glitz areas such as error-handling.
  • Greenplum’s current MapReduce language support is: Perl, Python, R, and C. Java didn’t make it into Release 3.3
  • Greenplum agrees with the MapReduce skeptics’ claim that you can in principle do anything in UDFs (User-Defined Functions) you can in MapReduce, but believes that sometimes doing it in MapReduce turns out to be easier.

]]> http://www.dbms2.com/2009/06/05/greenplum-update-release-3-3/feed/ 10 http://www.dbms2.com/2009/05/22/yet-more-on-mysql-forks-and-storage-engines/ http://www.dbms2.com/2009/05/22/yet-more-on-mysql-forks-and-storage-engines/#comments Fri, 22 May 2009 06:15:34 +0000 Curt Monash http://www.dbms2.com/?p=790 The issue of MySQL forks and their possible effect on closed-source storage engine vendors continues to get attention.  The underlying question is:

Suppose Oracle wants to make life difficult for third-party storage engine vendors via its incipient control of MySQL?  Can the storage engine vendors insulate themselves from this risk by working with a MySQL fork?

As laid out most clearly in a comment thread to a previous post*, Mike Hogan (CEO of ScaleDB) believes closed-source storage engine vendors can use a MySQL fork without running afoul of the GPL. In a nutshell, what he proposes is an inbetween layer of software, itself open-sourced, that on one side interfaces with MySQL, and on the other side talks cleanly enough to storage engines that it doesn’t infect them with the GPL.

*For some reason, the identical comments have also appeared one by one on a Stephen O’Grady blog post, with links back to the original thread. I did not actually post the comments attributed to me, so I presume there’s some automated process going on.

The most natural way for such software to be created would be obviously be in connection with the new Open Database Alliance.  So Matthew Aslett of the 451 Group asked the ODA’s two founding CEOs — Peter Zaitsev of Percona and Monty Widenius of Monty Program AB — what they thought on the subject.  He got rather different-sounding answers.  Zaitsev in effect said “Yes, Mike Hogan’s idea probably works, but one never knows for sure when there are lawyers involved. ” Widenius in effect said “Nope. A license would have to be purchased from MySQL.”

On a first reading that all looks discouraging, but let’s probe further. In particular, let’s invoke the open source community’s famous distinction between two kinds of freedoms: “Free as in speech and free as in beer.” “Free as in speech” takes care of most technical fears — there’s no way Oracle can directly stop forkers from creating their own version of MySQL.

True, third-party storage vendors might have to compete with Oracle’s own storage engines, where Oracle has four kinds of competitive advantage:

• General business clout
• A whole lot of database development expertise
• The opportunity to build tight hooks between MySQL’s generic front end and Oracle’s own preferred back end(s)
• Alternatively, the opportunity to foot-drag on MySQL development and thus sabotage the third-party storage engine market altogether

But the first two of those points are exactly what independent DBMS vendors already have to deal with when they compete with Oracle, many of them quite successfully (especially in the analytic DBMS market). Ditto, really, for the third one. And the fourth is exactly what forking takes care of.

The situation for “Free as in beer” is not quite as clean.  Could Oracle could successfully charge a financial “tax” on every closed-source MySQL storage engine sale, prohibitive or otherwise — even the ones running with forked MySQL rather than Oracle’s code line?  Mike Hogan says No, Monty Widenius says Yes, and Peter Zaitsev isn’t sure. I find Hogan’s argument fairly persuasive, but he and I are probably in the minority.

So let’s suppose Widenius’ pessimistic view is correct. Right now it seems that MySQL charges a non-prohibitive tax those engine vendors are perfectly happy to pay.  If Oracle made drastic increases in those charges, it could face all sorts of PR, business, and even legal adverse consequences. So the risk that Oracle cripples the MySQL storage engine vendors strictly through licensing fees seems relatively low.

One last question complicates all this even further — why would the storage engine vendors want to rely on MySQL-compatible front ends indefinitely anyway? The whole point of specialized storage engines is to do things very differently from generic MySQL, so I’m not clear on what kind of user technical skills argument there is.  For most uses, the Postgres interface is as good or better than MySQL’s. Perhaps a third open source front-end flavor could eventually become popular.  And by the way, Monty Widenius himself wrote:

The reason we decided on the name “Open Database Alliance” was to be able to include companies and people working on all other open source database in the Alliance.

This is work in progress. We will make this clear on the Alliance web site ASAP.

Interesting times indeed.

More later.  I have a radio interview in a few minutes on a very different subject.

• Ingres is much of the way toward SQL 92 compatibility.
• Ingres also has added some industry-common proprietary SQL extensions. E.g., it appears multiple vendors have adopted Oracle’s way of doing substrings along with the standard way.
• Development chief Bill Maimone points out that when Ingres was still owned by CA, a lot of work was done to increase availability. (Arguably, Ingres’ main use at CA was as the underpinning for CA Unicenter.)
• I’m sure Ingres would disagree with this phrasing, but it sounds as if Ingres’ appliance strategy is more bait-and-switch than anything else. E.g., Ingres appliances don’t have the Red Hat version of Linux many users want.
• That said, Ingres’ BI appliance has enjoyed a “fair” number of small sales. Ingres claims – not unreasonably — that with a mature optimizer and parallel query, its product is suitable for at least lightweight reporting despite a lack of more advanced data warehousing features.
• Ingres does say, using the appliances as an example, that it built an installer much easier than what it had before.
• Although Ingres said something last August to Seth Grimes that sounded like it was going to integrate MonetDB, that’s not actually happening. Ingres does seem to have some kind of data warehousing speed-up project in the works, but was coy as to details.
• Ingres believes it will soon leapfrog closed-source vendors in its geospatial support, helped by significant community contributions. (Geospatial was the only part of Ingres it didn’t have the licensing rights to open-source, which is a bit part of the reason it’s being replaced.)
• However, it didn’t sound as if Ingres was doing much else in datatype extension.
• Speaking of data-warehousing-oriented enhancements, another example of nontrivial community contribution is partition-compatible hash aggregates.
• Another example is column encryption.
• Generally, Ingres echoes what other open source vendors say: The community will work on what it wants to work on, and many of the vendor’s priorities have to be implemented by the vendor itself.
• I didn’t ask directly, but I get the sense Ingres has concentrated some of it sales efforts in the financial industry, for three reasons. First, Ingres says it is doing a lot of Sybase replacements. Second, when Ingres said it hasn’t had as much success replacing Oracle, but has some hopes that this will change, it used financial companies as an example. Third, that’s the market Ingres used to say it was focusing on.
• Ingres says it also is doing OK in Microsoft replacements. But MySQL replacements are harder because MySQL is so non-standard that porting applications is more challenging.
• In another slap at MySQL, Ingres suggested that it’s hard to reference a BLOB in MySQL without pulling the whole thing down to the client.
• While conceding Postgres is good technology, Ingres asserted having a support organization is a competitive advantage. I asked a couple of times “What about EnterpriseDB?”, but didn’t get a response.

]]> http://www.dbms2.com/2009/04/02/ingres-update/feed/ 6 http://www.dbms2.com/2009/03/18/database-implications-if-ibm-acquires-sun/ http://www.dbms2.com/2009/03/18/database-implications-if-ibm-acquires-sun/#comments Wed, 18 Mar 2009 14:48:13 +0000 Curt Monash http://www.dbms2.com/?p=722 Reported or rumored merger discussions between IBM and Sun are generating huge amounts of discussion today (some links below). Here are some quick thoughts around the subject of how the IBM/Sun deal — if it happens — might affect the database management system industry.

• IBM is already serious about supporting multiple database management systems. DB2 on open systems is IBM’s flagship DBMS. But DB2 on mainframes and at least one flavor of Informix seem to be getting maintained and enhanced fairly seriously as well. And IBM has further DBMS products as well (e.g., DB/2 on the AS/400). There’s little reason to think IBM would orphan MySQL or any other DBMS product.
• IBM is very open-source-friendly. For a company that grew up for decades on proprietary software — and still is a huge software products vendor — IBM is very serious about open source. If you doubt that, I have two words for you: “Linux” and “Eclipse”.
• MySQL might finally get its industrial-strength act together. IBM is good at database management and good at open source. MySQL becoming a no-apologies transactional DBMS would obviously put pressure on Ingres, PostgreSQL, and EnterpriseDB, although there surely would be lots of happy talk about the open source DBMS market being validated, lifting all the vendors and so on. Also, a better MySQL could be bad news for Microsoft SQL Server too.
• Sun has a lot of DBMS partnerships right now. Obviously, Sun owns MySQL, and has partnerships with MySQL storage engine vendors such as Infobright and Kickfire. Sun also has a substantial partnership with Greenplum, and a Barneyesque* one with ParAccel. And of course Sun has strong working relationships with major database vendors such as Oracle and Sybase. What’s more, on a case-by-case basis, Sun may cooperate in the field with yet other DBMS sellers. E.g., I’ve confirmed at least one instance of a Sun sales rep recommending a Kognitio DBMS.
• IBM partners with outside DBMS vendors too. You’d think IBM’s gazillion DBMS product lines would be enough. But nooooo. I frequently hear rumblings of IBM’s hardware or services operations working with other DBMS products as well. (This is, of course, actually to their credit.)
• Short-term, there probably would be little effect on partnerships. Greenplum runs on Sun’s Thumper/Thor line of boxes. DB2 doesn’t, and certainly isn’t optimized for same. In the short term, to sell Thors, Sun would presumably continue to sell Greenplum.
• Longer-term, there could be a DBMS rationalization. DB2, Informix, MySQL + storage engines, and big independent vendors such as Oracle and Sybase would surely always get attention. That’s a lot. There might not be room for much mind share for many database products and vendors beyond that list.

*A Barney partnership is one in which two or more vendors get on stage and do a song and dance about how much they love each other, with little substance beyond that.

Related links

• Larry Dignan thinks the IBM/Sun deal is sensible and ripe to happen.
• Dana Gardner thinks otherwise.
• Matt Asay seems to agree that IBM understands the open source business.
• Before IBM acquired it, solidDB was scheduled to provide a serious MySQL transaction processing engine.