April 5, 2010

Notes on the evolution of OLTP database management systems

The past few years have seen a spate of startups in the analytic DBMS business. Netezza, Vertica, Greenplum, Aster Data and others are all reasonably prosperous, alongside older specialty product vendors Teradata and Sybase (the Sybase IQ part).  OLTP (OnLine Transaction Processing) and general purpose DBMS startups, however, have not yet done as well, with such success as there has been (MySQL, Intersystems Cache’, solidDB’s exit, etc.) generally accruing to products that originated in the 20th Century.

Nonetheless, OLTP/general-purpose data management startup activity has recently picked up, targeting what I see as some very real opportunities and needs. So as a jumping-off point for further writing, I thought it might be interesting to collect a few observations about the market in one place.  These include:

I shall explain.

Big-brand OLTP/general-purpose DBMS have more “stickiness” than analytic DBMS.

By number, most of an enterprise’s OLTP/general-purpose databases are low-volume and low-value. Indeed, “OLTP” is often a misnomer, which is why I tend to go with “general-purpose” or some similarly wishy-washy phrase instead.

“General-purpose” might be a better term than “OLTP” anyway.

Most interesting new OLTP/general-purpose data management products are either MySQL-based or NoSQL.

It’s not yet clear whether MySQL will prevail over MySQL forks, or vice-versa, or whether they will co-exist.

The era of silicon-centric relational DBMS is coming.

The emphasis on scale-out and reducing the cost of joins spans the NoSQL and SQL-based worlds. We hear that from the NoSQL guys all the time. But I also just heard it from Akiban.

Users’ instance on “free” could be a major problem for OLTP DBMS innovation. Vendors of new OLTP data management technologies often feel obligated to open source their products, notwithstanding the historical lack of revenue in the open source OLTP DBMS market. As just one of many examples, Nova Spivack wrote:

I have recently seen some new graph data storage products that may provide the levels of scale and performance needed, but pricing has not been determined yet. In short, storage and retrieval of semantic graph datasets is a big unsolved challenge that is holding back the entire industry. We need federated database systems that can handle hundreds of billions to trillions of triples under high load conditions, in the cloud, on commodity hardware and open source software. Only then will it be affordable to make semantic applications and services at Web-scale.

I hear similar things from other startups, who evidently believe they need and/or are entitled to enjoy sophisticated, high-performance, zero-cost, specialized database management technology.

Comments

8 Responses to “Notes on the evolution of OLTP database management systems”

  1. Mark Callaghan on April 5th, 2010 10:16 am

    Many DBMS products are free to use but they aren’t free to operate. Lots of people make a living keeping them running and the open-source products frequently require a bit more care than the closed- source ones. It would be interesting to read estimates on the point at which the open-source products have better price/performance. How big must a deployment be before the license cost from closed-source is much larger than the extra people cost from open-source?

    VoltDB and TokuDB (http://www.tokutek.com) are two of my favorites in the OLTP space. I wish they would write more about their products. Thanks for the link to the new H-Store paper.

  2. J. Andrew Rogers on April 5th, 2010 2:51 pm

    Nova’s comment likely refers to a new theoretical architecture for relational algebra implementations that is uniquely suited for massively parallelizing graph analytic databases. It will be difficult for an open source implementation to emerge because the core algorithms are buried in IP. There are at least two companies developing products based on it, one of which is a major database vendor.

    The middle ground for cases like this is to go straight to a cloud offering for cost-sensitive customers. Not ideal but workable and it allows the vendor to stay in control of their product. It is very difficult to make money open sourcing a product when core development is being funded entirely by the company.

  3. Curt Monash on April 6th, 2010 3:44 am

    Andrew,

    Can you say more about that, online or off?

    CAM

  4. Curt Monash on April 6th, 2010 3:45 am

    Mark,

    You are very right to focus on TCO, as opposed to specific types of cost.

    CAM

  5. Andy Ellicott on April 6th, 2010 8:51 am

    Hi Mark,
    Thanks for the kudos. BTW–we’ve opened up the VoltDB beta program and are providing more info about it.

    For those who want to try it or read the product documentation, just request access to the early release web site at http://www.voltdb.com.

  6. J. Andrew Rogers on April 7th, 2010 12:01 pm

    Curt, I would be happy to say more offline, just drop me an email.

  7. Joe Adinolfi on April 9th, 2010 6:57 am

    Curt, i defintely agree that solid state memory is the future direction for analytical DBMS’s. I have seen the slide ware on the Teradata Extreme Performance Appliance and if reality is even close then this technology is a game changer. There are questions on longevity and availability but since SSD’s have no moving parts i would expect them to be more reliable than HD’s. Pretty exciting stuff.

  8. Notes on short-request scale-out MySQL | DBMS 2 : DataBase Management System Services on January 5th, 2013 8:53 am

    [...] notes on all this from April, 2010 are already badly outdated, but may be interesting anyway. Categories: Akiban, dbShards and [...]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.