• The Hadapt 1.0 product is going “Early Access” today.
• General availability of Hadapt 1.0 is targeted for an officially unspecified time frame, but it’s soon.
• Hadapt raised a nice round of venture capital.
• Hadapt added Sharmila Mulligan to the board.
• Dave Kellogg is in the picture too, albeit not as involved as Sharmila.
• Hadapt has moved the company to Cambridge, which is preferable to Yale environs for obvious reasons. (First location = space they’re borrowing from their investors at Bessemer.)
• Headcount is in the low teens, with a target of doubling fast.

The Hadapt product story hasn’t changed significantly from what it was before. Specific points I can add include:  

• With one exception to date, Hadapt beta customers have used PostgreSQL as the underlying DBMS, rather than some faster columnar system.
• Sure, you want to process data on the nodes where it resides on the cluster. But if each copy is replicated 3X or so, that gives you good flexibility to be adaptive by deciding which of the three copies you’ll operate against.
• In Hadapt Version 1.0, scheduling and workload management are pretty much Hadoop’s. However …
• … an improvement in scheduling is being actively researched.
• In general, Hadapt’s design philosophy for executing SQL is to use MapReduce to get data to the proper nodes, while using the underlying DBMS for node-specific operations such as:
  • Initial retrieval from disk.
  • Joins and aggregations on data residing at (or visiting) a specific node.

A very busy Daniel Abadi also took the time to walk me through how Hadapt does joins. More precisely, what we discussed about joins includes some of the last features being added to Hadapt 1.0; many of the pieces are still missing from early-access Hadapt 1.0, and some may even slip out of the Hadapt 1.0 GA version. As Dan tells it, there are five kinds of joins in Hadapt:

• Co-partitioned join. Both tables being joined happen to be partitioned on the join key. Happy happy joy joy. The tables are joined locally on each node, with the results aggregated via MapReduce.
• Directed join. One of the tables being joined happens to be partitioned on the join key. MapReduce distributes the other table along the join key, joins happen locally, and MapReduce does the rest.
• Broadcast join. One of the tables is broadcast in its entirety to every node. Joins then happen locally, and MapReduce does the rest.
• Split semijoin. One of the tables is projected to the join key and a row ID, and then distributed via MapReduce. Joins then happen locally. Later on, the joined rows are completed with the help of a second projection on the first table. MapReduce does the rest.
• Distributed/parallel hash join. Sometimes, Hadapt indeed joins just as Hadoop/Hive would.

Highlight’s of Hadapt’s performance story include:

• Dan contends that using a DBMS rather than HDFS (Hadoop Distributed File System) for I/O always gives a performance advantage.
• DBMS local-node join performance can be presumed to be superior as well.
• Of course, Dan also thinks that using a columnar DBMS would extend Hadapt’s performance advantage further, but most of the specifics of what Hadapt has told me about why they don’t routinely use a columnar DBMS yet are NDA.
• Even beta Hadapt/PostgreSQL outperforms Hadoop/Hive by almost 10X at Hadapt’s relatively small number of beta customer sites.

• Hadapt is additional software on a cluster that also runs fully functional Hadoop/HDFS. (Cloudera Hadoop more than straight-from-Apache Hadoop to date, but that’s not a requirement.)
• The cluster also runs a DBMS on every node, such as PostgreSQL or one of Infobright/Vectorwise.
• Hadapt’s software manages parallel SQL queries by distributing them to the DBMS living on each node. Hadapt says that the resulting query performance far outshines Hive’s.
• Hadapt further says that, by exploiting the partner DBMS, its SQL functionality outpaces Hive’s as well.
• Target Hadapt use cases are centered around keeping machine-generated or other poly-structured data in Hadoop, and extracting, enhancing, or otherwise deriving some of it to live in the relational store.
• In particular, Hadapt seems like an interesting choice when you want to use that relational data as you work on other data that’s still in HDFS, or if you want to keep using the relational data in other kinds of MapReduce jobs.
• That all fits well with my thoughts about the importance of derived data.

Other evolution from what I wrote about Hadapt a few months ago includes:

• Hadapt is in beta now.
• Hadapt has added adult supervision in the form of Philip Wickline, late of Endeca.

In other news, Hadapt is our newest client.

*At least if the underlying DBMS is a fast one. Hadapt likes VectorWise for that purpose, and is showing performance comparisons that assume VectorWise is underneath.

**It seems that Hadapt in the future is assured of having more SQL coverage than Hive does today.

It’s still early days for the Hadapt company. Funding is on the angel level. There seem to be six employees — Yale professor Daniel Abadi, CEO Justin Borgman, Chief Scientist Kamil Bajda-Pawlikowski,* and three other coders. The Hadapt product will go into beta at an unspecified future time; there currently are a couple of alpha users/design partners. The Hadapt company, a Yale spin-off, obviously needs to move from Connecticut soon. I wasn’t able to detect any particular outside experience in the form of directors or advisors. And Hadapt’s marketing efforts are still somewhat ragged. So basically, the reasons for believing in Hadapt pretty much boil down to:

• Daniel Abadi is a star.**
• Hadapt’s own tests show that Hadapt is a whole lot faster than Hive.

*Bajda-Pawlikowski is one of the two Abadi students who did the HadoopDB work. It turns out he had numerous years of coding experience before entering graduate school. (The other student, Azza Abouzeid, is pursuing an academic career.)

**Vertica was built around Daniel’s C-Store Ph.D. thesis. He was involved in H-Store as well. He has a really good blog. He’s a really nice guy. Etc.

As you might have guessed from the name, the Hadapt guys are proud that their technology is “adaptive,” which communicates their fond belief that Hadapt’s query optimization and planning are more modern and cool than other folks’ query planning and optimization. In particular, Daniel suggested that Hadapt is more thoughtful than most DBMS are about looking at the size of intermediate result sets and  then replanning queries accordingly.

However, the really cool adaptivity point is that Hadapt watches the performance of individual nodes, and takes that into account in query replanning. Daniel asserts, credibly, that this is a Really Good Feature to have in cloud and/or virtualized environments, where Hadapt might not have full control and use of its nodes. I’d add that it could also give Hadapt a lot of flexibility to be run on clusters of non-identical machines.

On the negative side, Hadapt will not at first have any awareness of how its underlying DBMS are optimized; it will plan for VectorWise the same way it does for PostgreSQL. In that regard, this is a DATAllegro 1.0 story. If I understood correctly, Hadapt has specific connectors for a couple of DBMS (probably exactly those two), and can also talk JDBC to anything. PostgreSQL was apparently 5X faster than MySQL when tested (with either ISAM or InnoDB); Daniel snorted about, for example, MySQL’s apparent fondness for nested-loop joins over hybrid hash. On the other hand, he was more circumspect about his reasons for favoring VectorWise over, to name another open source columnar DBMS, Infobright.

And finally, a couple of other points:

• Hadapt will be closed source, although it will of course rely on large amounts of other people’s open source software. Pay no attention to the importance Daniel previously ascribed to HadoopDB’s open source nature.
• Hadapt decompresses data before moving it from node to node, and also before doing non-SQL MapReduce operations on it. Pay no attention to the years Daniel spent insisting columnar DBMS absolutely must operate on data in compressed form.

First and foremost, the fourth annual New England Database Summit (nee “Day”) is next week, specifically Friday, January 28. As per my posts in previous years, I think well of the event, which has a friendly, gathering-of-the-clan flavor. Registration is free, but the organizers would prefer that you register online by the end of this week, if you would be so kind.

The two things potentially wrong with the New England Database Summit are parking and the rush hour drive home afterwards. I would listen with interest to any suggestions about dinner plans.

One thing I hope to figure out at the Summit or before is what the hell is going on on Vertica’s blog or, for that matter, at Vertica. The recent Mike Stonebraker post that spawned a lot of discussion and commentary has disappeared. Meanwhile, Vertica has had three consecutive heads of marketing leave the company since June, and I don’t know who to talk to there any more. 

Speaking of blog problems, we’ve had performance/reliability glitches here again. Melissa Bradshaw determined that the problem was an apparently activated WP Super Cache not actually caching anything. We should be OK now, so please let me know if there are further difficulties. One interesting step — it turns out that there’s a WordPress plug-in that does automatic EXPLAINs (if you’re the blog administrator).

Another interesting Mike Stonebraker post can be found (at least for now) over on the VoltDB blog. He continued his assault on the CAP Theorem, arguing that availability is an exaggerated concern when there are bug- or other human-error-driven kinds of outages, and also arguing that the concept of “partition tolerance” is misguided. Commenters pushed back, pointing out that in geographically distributed scenarios, the CAP Theorem sense of partitioning is quite a legitimate concern.

When I posted an expansive definition of machine-generated data a few weeks ago, Daniel Abadi shot back advocating a narrower one (see the comment thread, which includes a link to his thoughtful post). The disagreement boils down to conflicting intuitions as to whether the machine-data/true-human-data ratio will keep growing rapidly, in hybrid cases such as web logs or social gaming.

Dave McClure recently offered a survey of hot startup investing themes. High on his list were location-based services, which is a reminder to us all that geo-spatial data is becoming much more important. Ray Wang is savvy enough to understand the privacy dangers location-based services cause, but influential though Ray is, his view will probably remain in the minority. Machine-generated data and video each also make appearances on Dave’s list.

And wait! I have even more links for you!  Several are taken from Thomas Houston’s choices for The Best Tech Writing of 2010. He chose well. I recommend sampling his list further.

• In an article about new electronic exchanges, the New York Times shared some numbers — 56% of trading volume “high speed” in stocks, 1/3 or so when looking at domestic futures, .1 milliseconds to do a NASDAQ trade, 13 milliseconds for a trade that involves Chicago/NYC communication, 60 milliseconds for NYC/Frankfurt. Slashdot offers photos and other context.
• James Taylor caught up with once-hot KXEN, and evidently got the impression KXEN was focusing a lot of its efforts on the tedious, time-consuming data-preparation side of modeling.
• Richard Tibbetts is being pretty funny on his blog.
• (Slashdot) The Russian government seems to be getting into open source software in a big way. Well, PostgreSQL is already big in Russia (close to 1 million installations, I was once told), so this might conceivably add some energy to its development.
• In Drupal 7, Drupal now has “a built-in test environment, version upgrade manager, and a database abstraction layer for use with MariaDB, SQL Server, MongoDB, Oracle, MySQL, PostgreSQL, and SQLite.” That may explain how MongoDB can hope to further penetrate the Drupal market.
• The Boston Phoenix argues that government lacks the manpower, budget, and expertise to keep up with its responsibilities in preserving and exposing information. Fixing that problem sounds like a pretty worthy open source development effort to me.

Finally:

• Clay Shirky reminded us that modern machine learning is what replaced old-style AI.
• Nominally reviewing a book he obviously disdains, Garry Kasparov — in my opinion the most admirable world chess champion ever — surveyed computer chess in quick, nontechnical way. The whole thing is a bit wordy even so, so I’ll quote one part:

In 2005, the online chess-playing site Playchess.com hosted what it called a “freestyle” chess tournament in which anyone could compete in teams with other players or computers. … The surprise came at the conclusion of the event. The winner was revealed to be not a grandmaster with a state-of-the-art PC but a pair of amateur American chess players using three computers at the same time. Their skill at manipulating and “coaching” their computers to look very deeply into positions effectively counteracted the superior chess understanding of their grandmaster opponents and the greater computational power of other participants. Weak human + machine + better process was superior to a strong computer alone and, more remarkably, superior to a strong human + machine + inferior process.

• Despite heavy emphasis on the word “sharding,” dbShards’s scale-out is transparent to the application programmer. E.g., in dbShards + MySQL, the APIs are more or less the same ones you’d expect for MySQL (JDBC, etc.)
• If the DBMS underneath is ACID-compliant (e.g., MySQL + InnoDB), then the dbShards version is ACID-compliant too.
• Beyond those basics, I forgot to check the fine details of dbShards’ MySQL (or PostgreSQL) syntax support. Todd Hoff, however, did not forget.
• dbShards keeps copies of each shard on two different servers, via asynchronous log-shipping. This allows for failover in both planned and unplanned outages.
• dbShards wants you to distribute big tables among shards via a “shard key,” which is a lot like the distribution key in MPP analytic DBMS. You’re encouraged to replicate small, low-update-volume tables across each shard.
• Cory says that dbShards has good join performance when – you guessed it! – everything being joined is co-located shard-by-shard, because the tables were distributed on the same shard key and/or replicated across each shard. Cory can’t imagine why you’d want to do an inner join under any other circumstances.
• The basic dbShards query execution model is: A query comes in; it’s parsed; a shard key is automagically detected (one hopes); the “global configuration file” is checked to see which shard to ship the work off too. I forgot to ask whether lookup was done via a hash table (the obvious guess) or something else. The programmer can put hints in the code comments to direct the sharding, but Cory asserts those aren’t needed very often.
• Cory says that insert performance with dbShards + MySQL + InnoDB is 1500-3000 inserts per shard per second, scaling almost linearly with the number of shards. I forgot to ask how many shards this had been tested for.
• If you want blazing dbShards performance, Cory’s base-case figure is 25 gigabytes of data per node, so that the most commonly used indexes can camp out in memory. (I forgot to ask what kind of hardware he was assuming per node.) This is if you’re going to be doing joins or aggregrations. If it’s just single-row inserts and updates, or if your performance requirements are lower, you can go with 10X that figure.
• Cory tells stories wherein going from an unsharded database to 4 or so shards took database re-indexing time down 50X or more. Apparently, such tasks can be exponential or even super-exponential with database size over InnoDB. (That said, I’d be surprised if all large InnoDB users suffered from that problem to the same degree.)
• dbShards’ customer workloads are all >= 50% reads. This is reflective of dbShards’ design priorities.
• As long as it can be in charge, dbShards is happy to interface to whatever kind of database backup software you want to use on a node by node basis. (dbShards wants to drive your backup software for you so that it can be sure the replicas are handled properly.)
• It’s “fairly common” for dbShards to be paired with memcached. I forgot to ask whether memcached typically lived on its own pool of servers, or on the same pool that runs dbShards.
• Future DBMS options under consideration for dbShards include Oracle and (unspecified) in-memory.

Business highlights for CodeFutures and dbShards include:

• dbShards’ price is $5000/server/year, including support and OEMed MySQL, with stated quantity discounts up to 40%.
• dbShards cloud pricing is different (on a usage basis).
• dbShards has 6 or so customers, half each on-premises and in the cloud. One of them is Facebook. (Those “100s” of customers mentioned on the dbShards website are for a fairly unrelated product.)
• CodeFutures has been at this 2 ½ years or so. There is no venture capital in the company.
• Early deals dbShards deals have evidently involved a fair amount of professional services.
• Counting contractors, Code Futures has 10-12 people, which has been as high as 15.
• Target dbShards customers are as you’d expect. Cory says he’s actually been more successful getting early-adopter money out of Web companies than Wall Street firms.
• There are a couple of dbShards PostgreSQL customers for greenfield applications. Most dbShards customers and prospects, however, are looking to scale out existing apps.
• Despite its connection to open source DBMS, there’s nothing open source about dbShards itself.

Analytic technology isn’t only for you. It’s also for your customers, citizens, and other stakeholders.

I am not referring here to what is well understood to be an important, fast-growing activity — providing data and its analysis to customers as your primary or only business — nor to the related business of taking people’s data, crunching it for them, and giving them results. That combined sector — which I am pretty alone in aggregating into one and calling data mart outsourcing — is one of the top several vertical markets for a lot of the analytic DBMS vendors I write about. Rather, I’m talking about enterprises that gather data for some primary purpose, and have discovered that a good secondary use of the data is to reflect it back to stakeholders, often the same ones who provided or created it in the first place.

For now I’ll call this category stakeholder-facing analytics, as the shorter phrase “stakeholder analytics” would be ambiguous.* I first picked up the idea early this decade from Information Builders, for whom it had become something of a specialty. I’ve been asking analytics vendors for examples of stakeholder-facing analytics ever since, and a number have been able to comply. But the whole thing is in its early days even so; almost any sufficiently large enterprise should be more active in stakeholder-facing analytics than it currently is.

*Comments as to what the category should be called are welcome below.

Examples of stakeholder-facing analytics include:

• Enterprises report back on the business customers do with them. For example:
  • Credit card companies provide reports on spending back to their credit card holders, especially small businesses.
  • So do office supply retailers.
  • Brokerage firms provide reporting back to their small-institution customers.
• Governments expose information to their citizens online.
  • In an early example, New York City restaurant ratings were put online.
  • Putting SEC filings online has has been a huge success.
  • The Obama Administration has committed to putting large amounts of information online.
• Regulated companies (such as utilities) could be required to put data online directly, without even using the government as an intermediary.
• Some part of Fox — perhaps MySpace Music? — offers free access to a PostgreSQL extract from its Greenplum database to each of its largest advertisers.
• Google Analytics offers some basic BI for free to website owners everywhere.
• Anybody from web hosting companies to public utilities could open their kimonos and allow their customers to track adherence to actual or implied SLAs (Service Level Agreements) in areas such as uptime, length of outage, responsiveness, and the like.

So what cool examples do you have of stakeholder-facing analytics?*

*Yes, this is an invitation to drop links to case studies into the comment thread below.

Nonetheless, OLTP/general-purpose data management startup activity has recently picked up, targeting what I see as some very real opportunities and needs. So as a jumping-off point for further writing, I thought it might be interesting to collect a few observations about the market in one place.  These include:

• Big-brand OLTP/general-purpose DBMS have more “stickiness” than analytic DBMS.
• By number, most of an enterprise’s OLTP/general-purpose databases are low-volume and low-value.
• Most interesting new OLTP/general-purpose data management products are either MySQL-based or NoSQL.
• It’s not yet clear whether MySQL will prevail over MySQL forks, or vice-versa, or whether they will co-exist.
• The era of silicon-centric relational DBMS is coming.
• The emphasis on scale-out and reducing the cost of joins spans the NoSQL and SQL-based worlds.
• Users’ instance on “free” could be a major problem for OLTP DBMS innovation.

I shall explain.

Big-brand OLTP/general-purpose DBMS have more “stickiness” than analytic DBMS.

• OLTP applications are more complex than analytic ones, and hence more tightly wired into particular brands of DBMS. For example, third-party packaged OLTP applications are typically portable among only a few brands of DBMS. But third-party business intelligence tools, and the BI “applications” built in them, are more easily and widely portable.
• Specific technical observations such as “OLTP apps tend to use stored procedures, which are DBMS-specific” or “OLTP apps tend to have lots and lots of tables” serve to underscore the first point.
• An enterprise’s highest-value data is commonly the financial stuff handled by its core OLTP systems, so those are the last things they want to mess around with just to get some cost savings. Security, high availability, and so on are major considerations that can outweigh cost.

By number, most of an enterprise’s OLTP/general-purpose databases are low-volume and low-value. Indeed, “OLTP” is often a misnomer, which is why I tend to go with “general-purpose” or some similarly wishy-washy phrase instead.

• In theory, this is a ripe area for what I’ve called mid-range DBMS.
• The big brand vendors try hard to keep as many of those databases for themselves as they can. Enterprise-wide license pricing helps. Going forward, so will virtualization/consolidation strategies, such as Oracle’s Exadata-centric approach.
• A variety of mid-range DBMS alternatives beyond the big brands have technical merit, at least in some cases and configurations – MySQL, PostgreSQL, Intersystems Cache’, and so on.
• The only such mid-range DBMS alternative with much large enterprise business momentum, however, appears to be MySQL.

“General-purpose” might be a better term than “OLTP” anyway.

• I don’t have a link, but it’s widely agreed that over half of the processing on an “OLTP” enterprise app is commonly reporting and so on.
• “Operational BI” is progressing by fits and starts, but it is progressing.
• Anything customer-facing — web-based, call center, or otherwise — is likely to include a heavy dose of “real-time” analytic optimization.

Most interesting new OLTP/general-purpose data management products are either MySQL-based or NoSQL.

• VoltDB is the main exception that jumps to mind.
• This isn’t true in the analytic DBMS area, where Netezza, Greenplum, Aster, Vertica and others started from PostgreSQL’s code, APIs, or both.

It’s not yet clear whether MySQL will prevail over MySQL forks, or vice-versa, or whether they will co-exist.

• MySQL is a limited product without all the third-party storage engines that are being developed.
• Oracle’s promise of MySQL good behavior has an expiration date.
• None of the MySQL front-end alternatives are remotely mature yet.

The era of silicon-centric relational DBMS is coming.

• I think “silicon” means “solid-state memory” as much as or more than it means “RAM,” but that’s not yet certain.
• What is pretty certain is that, thanks to Moore’s Law, some kind of silicon will increasingly replace disk.
• Oracle’s increasingly Flash-centric story is a challenge to everybody.
• RAM-centric VoltDB will launch fairly soon. (By the way, while VoltDB still has a lot in common with H-Store, they’re not exactly the same thing. And H-Store research is progressing too.)
• RethinkDB is being developed, focused directly on solid-state memory. Based on the sparse information available online, RethinkDB sounds somewhat like a dumbed-down H-Store.
• New disk-based vendors may never optimize their use of disk, instead targeting a solid-state future. (E.g., I think Akiban should and quite well might follow this path.)

The emphasis on scale-out and reducing the cost of joins spans the NoSQL and SQL-based worlds. We hear that from the NoSQL guys all the time. But I also just heard it from Akiban.

Users’ instance on “free” could be a major problem for OLTP DBMS innovation. Vendors of new OLTP data management technologies often feel obligated to open source their products, notwithstanding the historical lack of revenue in the open source OLTP DBMS market. As just one of many examples, Nova Spivack wrote:

I have recently seen some new graph data storage products that may provide the levels of scale and performance needed, but pricing has not been determined yet. In short, storage and retrieval of semantic graph datasets is a big unsolved challenge that is holding back the entire industry. We need federated database systems that can handle hundreds of billions to trillions of triples under high load conditions, in the cloud, on commodity hardware and open source software. Only then will it be affordable to make semantic applications and services at Web-scale.

I hear similar things from other startups, who evidently believe they need and/or are entitled to enjoy sophisticated, high-performance, zero-cost, specialized database management technology.

Greenplum Single-Node Edition:

• Is free of charge, although you can buy support.
• Has no restrictions on use, production or otherwise.
• Has no restrictions on database size.
• Is closed-source.

For those who want free, terabyte-scale data warehousing software, Greenplum Single-Node Edition may be quite appealing, considering that the main available alternatives are:

• General-purpose open-source DBMS, such as PostgreSQL and MySQL (lacking analytic DBMS performance and features)
• Infobright Community Edition (the other best choice – Infobright’s commercial sales success indicates the solidity of Infobright’s technology)
• Rough research-project code and other other questionable open source offerings
• Crippleware from other commercial analytic DBMS vendors (e.g., Teradata)

For example, comparing PostgreSQL-based Greenplum with PostgreSQL itself, Greenplum offers:

• The ability to scale out queries across all cores in your box (and no, pgpool is not a serious alternative)
• Storage alternatives such as columnar (I am told that EnterpriseDB recently stopped funding a project for a PostgreSQL columnar option)

Greenplum would surely also argue that its software is superior to PostgreSQL in parallel load, compression, MapReduce integration, and general fit-and-finish. I imagine that in some (perhaps not all) cases it would be right. PostgreSQL’s main technical advantages over Greenplum would probably lie in the area of datatype extensibility.

The main target users for Greenplum’s Single-Node Edition are obviously individual enterprise power users or very small analytic teams. I.e., it’s people with a data mart need that a central data warehouse isn’t meeting. Potential benefits to Greenplum include:

• Adding value to its Enterprise Data Cloud story
• Seeding the market for future enterprise sales
• Depriving competitors of revenue, perhaps at enterprises too small to ever be paying Greenplum customers

In addition, I see free Greenplum as a charity offering that could be appealing to scientists who face PostgreSQL performance limitations.

Related links

• Greenplum Free Single-Node Edition press release (I’m quoted)
• MySQL Performance blog on MonetDB and Infobright community edition
• PostgreSQL’s restriction to one core per query
• Infobright’s restriction to one core per query

The basic idea of HadoopDB is to put copies of a DBMS at different nodes of a grid, and use Hadoop to parcel work among them. Major benefits when compared with massively parallel DBMS are said to be:

• Open/cheap/free
• Query fault-tolerance
• The related concept of tolerating node degradation that isn’t an outright node failure.

HadoopDB has actually been built with PostgreSQL. That version achieved performance well below that of a commercial DBMS “DBX”, where X=2. Column-store guru Abadi has repeatedly signaled his intention to try out HadoopDB with VectorWise at the nodes instead. (Recall that VectorWise is shared-everything.) It will be interesting to see how that configuration performs.

The real opportunity for HadoopDB, however, in my opinion may lie elsewhere. Rather than trying to compete with parallel relational DBMS, HadoopDB might do more good parallelizing more specialized kinds of database engines. How about, for example, a massively parallel XML manager to compete with MarkLogic? Or a massively parallel array processor other than the still-nascent SciDB? Or, even more to the point, something that parallelizes a yet-more-specialized scientific data management engine? That kind of area is where I suspect the potential for HadoopDB really lives.

]]> http://www.dbms2.com/2009/09/13/hadoopdb/feed/ 4 http://www.dbms2.com/2009/09/10/what-could-or-should-make-oraclemysql-antitrust-concerns-go-away/ http://www.dbms2.com/2009/09/10/what-could-or-should-make-oraclemysql-antitrust-concerns-go-away/#comments Thu, 10 Sep 2009 14:53:23 +0000 Curt Monash http://www.dbms2.com/?p=879 When the Oracle/MySQL deal was first announced, I wrote:

I can probably come up with business practices that could make things very hard on Oracle/MySQL competitors … but I haven’t found a compelling antitrust trigger on my first pass over the subject.

Subsequently, there’s been a lot of discussion about whether or not Oracle can use control of MySQL to make life difficult for third-party MySQL storage engine vendors.

Now that the European Commission is delaying the Oracle/Sun deal, explicitly because of Oracle/MySQL antitrust fears.  That is, the European Commission wants to be reassured that an Oracle takeover of MySQL won’t unduly impinge upon the future availability of open source/low cost DBMS alternatives.  This raises that natural question:

What could Oracle do to assure concerned parties that its ownership of MySQL won’t unduly hamper open-source-based DBMS competition?

I think that’s indeed the crucial question. The Oracle/Sun deal has enough momentum at this point that it both should and will be allowed to happen — perhaps with safeguards — rather than banned outright. If  you have concerns about Oracle’s pending acquisition of MySQL, you should speak up and outline what kinds of regulatory safeguards would alleviate the problems you foresee.

More or less obvious possibilities include:

• Divest MySQL. This is obviously an extreme measure, but it surely would work.
• Provide some money and trademark rights to MySQL forkers. If MariaDB and Drizzle were put into strong competitive positions with MySQL today, it’s hard to argue how regulators could object to any future Oracle maneuverings Oracle might envision with the GPLed side of MySQL.
• Offer a standard, attractive, long-term deal to MySQL bundlers. The commercial/non-GPL version of MySQL is a requirement for appliance vendors (surely), OEM vendors (probably), and storage engine vendors (maybe — I disagree, but I’m evidently in the minority).
• Strengthen PostgreSQL. Realistically, that’s not going to be part of any Oracle/MySQL resolution, so I’ll leave it as a subject for another time.