September 8, 2013

Layering of database technology & DBMS with multiple DMLs

Two subjects in one post, because they were too hard to separate from each other

Any sufficiently complex software is developed in modules and subsystems. DBMS are no exception; the core trinity of parser, optimizer/planner, and execution engine merely starts the discussion. But increasingly, database technology is layered in a more fundamental way as well, to the extent that different parts of what would seem to be an integrated DBMS can sometimes be developed by separate vendors.

Major examples of this trend — where by “major” I mean “spanning a lot of different vendors or projects” — include:

Other examples on my mind include:

And there are several others I hope to blog about soon, e.g. current-day PostgreSQL.

In an overlapping trend, DBMS increasingly have multiple data manipulation APIs. Examples include: 

So will these trends take over the DBMS world?

Developing a multi-purpose DBMS is extremely difficult, and even harder if it’s layered.

But on the plus side, it can be great to have one DBMS handle multiple kinds of data.

And by the way — the more different functions a DBMS performs, the more they may need to be walled off from each other. In particular, I’ve long argued that it’s a best practice for e-commerce sites to manage access control, transactions, and interaction data in at least two separate databases, and preferably in three. General interaction logs do not need the security or durability that access control and transactions do, and there can be considerable costs to giving them what they don’t need. A classic example is the 2010 Chase fiasco, in which recovery from an Oracle outage was delayed by database clutter that would have fit better into a NoSQL system anyway. Building a single DBMS that refutes my argument would not be easy.

So will these trends succeed? The forgoing caveats notwithstanding, my answers are more Yes than No.

Related links


7 Responses to “Layering of database technology & DBMS with multiple DMLs”

  1. DB2 Hub | Layering of database technology & DBMS with multiple DMLs on September 8th, 2013 9:12 am

    […] Layering of database technology & DBMS with multiple DMLs Tags: database, engine, geospatial, hadoop, live, smart, text search, tokutek and tokudbYou might also like ✎ ✎ ✎ ✎ ✎ ✎ ✎ ✎ /* […]

  2. Ofir on September 12th, 2013 5:15 am

    regarding layering of database technology, I agree.
    I recently wrote something similar on the future of databases on Hadoop – the new starting point for databases on Hadoop could be to leverage HDFS for storage, ORC/Parquet for efficient file format, YARN for resource management, HCatalog for metadata management and maybe even build the execution engine on top of Tez.
    So, the missing pieces are mostly decent optimizer and decent glue 🙂 Of course, depending on the database goals and architecture, such re-use might not always make sense.
    This could make new databases achieve maturity faster, but potentially deliver less value compared to existing solutions…

  3. Curt Monash on September 12th, 2013 1:34 pm

    Just remember, Ofir:

    A “small matter of programming” rarely is. 🙂

  4. Up next: software-defined caching on your processors | whatsweb on September 14th, 2013 12:51 pm

    […] Layering of database technology & DBMS with multiple DMLs ( […]

  5. JSON in DB2 | DBMS 2 : DataBase Management System Services on September 24th, 2013 4:37 am

    […] a growing trend for DBMS to beef up their support for multiple data manipulation languages (DMLs) or APIs — and there’s a special boom in JSON support, MongoDB-compatible or otherwise. So I […]

  6. Multi-model database managers | DBMS 2 : DataBase Management System Services on May 28th, 2016 9:50 am

    […] … single-model systems will become increasingly obsolete. […]

  7. Introduction to SequoiaDB and SequoiaCM | DBMS 2 : DataBase Management System Services on March 13th, 2017 8:06 am

    […] is a layered […]

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.