February 15, 2008

Database management system choices — overview

This is the first in a 5-part series of posts on data management product choices. By pre-arrangement, Mike Stonebraker is responding on The Database Column, starting with his own taxonomy of DBMS types.

In the 1990s, most database management experts believed that a single general-purpose DBMS could meet substantially all needs. If you just kept adding in enough datatypes and data access methods (e.g., specialized indexes), your DBMS could eventually do a good job of meeting almost any requirement. And so, from the late 1990s into the beginning of this decade, it seemed that technology was supporting business trends, and the DBMS industry was inexorably consolidating. There was an oligopoly of high-end vendors, who sold increasingly similar super-sophisticated database management systems. Nothing else in database management seemed to matter.

Well, we were wrong. The big thing we overlooked is that database optimizations go down to the level of actual storage. It makes a huge difference how you arrange the data, and even what kinds of devices you store it on. High-end data warehouses run best on shared-nothing massively multi-parallel (MPP) systems. Smaller ones may in the future do best with solid-state disks. Classic online transaction processing (OLTP) systems still do well in a shared-everything architecture.* And that’s just for the relational systems; some kinds of data shouldn’t be arranged in rows and columns at all.

*Even that may be over-generous to traditional shared-everything. Oracle RAC, high availability wide-area replication, and the H-Store research project all suggest that shared-everything’s dominance of high-end OLTP is at road’s end.

The plot thickens further. Most of these technical categories are populated by small companies, with relatively immature products – and in immaturity there is diversity. Thus, there is a broad range of viable data management products, each the best choice in at least some specific application and deployment scenarios.

Recently, one more alternative has emerged – create your own DBMS, or don’t use one at all. There also are half-and-half solutions, in which (commonly) MySQL is used to manage a variety of metadata, but media files might be left just in the file system. This underlies much of the buzz around Amazon and Google services or technologies such as EC2, SimpleDB, and MapReduce. But in most cases, especially for enterprise uses, the best way to go is with a DBMS, or else some DBMS-like data management technology such as a search engine or complex event/stream processing tool.

The database diversity series so far

Comments

14 Responses to “Database management system choices — overview”

  1. Daniel Abadi on February 17th, 2008 11:46 am

    Hi Curt,

    I noticed you mentioned H-Store a couple of times in your “Database
    management system choices” series. Your readers might want to find out more about this project. The H-Store Website can be found at:
    http://db.cs.yale.edu/hstore/

  2. Curt Monash on February 17th, 2008 12:10 pm

    Thanks, Daniel. I have a post coming on the subject of H-Store. Links will be edited in accordingly. 🙂

    CAM

  3. Curt Monash on February 17th, 2008 12:11 pm

    Or if you’re available to talk, I’d be happy to talk with you first and THEN write the post. So far I’ve only discussed the project with Mike.

    CAM

  4. Curt Monash on February 18th, 2008 2:36 am

    Thanks. H-Store post is now up, with links inserted into older posts as appropriate.

    http://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-the-complete-destruction-of-the-old-dbms-order/

    Talk with you soon!

    CAM

  5. Between the Lines mobile edition on February 20th, 2008 4:25 pm

    […] Where can I learn more? Academically, H-Store is described in a paper and slide presentation. I examined H-Store and its assumptions at some length over on DBMS2, where you’ll also find an extensive analysis of other specialized database technologies. The H-Store team writes for the Database Column blog, and H-Store discussion is expected over there soon. Also related is a series of posts Stonebraker and I have done on general database diversity. […]

  6. DBNews 2007 #6 | PettiNix on March 1st, 2008 11:12 am

    […] l’articolo di Diamond Notes. Se siete indecisi nella scelta del DBMS adatto a voi un post di DBMS2 potrebbe esservi molto utile. Gli ultimi movimenti di SUN vi lasciano perplessi? Non sapete se la […]

  7. james on March 18th, 2008 10:11 am

    could u publish some few research projects done in dbms
    probably with ms access

  8. Curt Monash on March 18th, 2008 1:53 pm

    James,

    I’m not sure what you’re asking.

    CAM

  9. DBMS2 — DataBase Management System Services » Blog Archive » Database management system choices — mid-range-relational on April 25th, 2008 12:10 am

    […] This is the fourth of a five-part series on database management system choices. For the first post in the series, please click here. […]

  10. DBMS2 — DataBase Management System Services » Blog Archive » Mike Stonebraker calls for the complete destruction of the old DBMS order on May 31st, 2008 12:30 am

    […] Part 1: Database management system choices – overview […]

  11. Mike Stonebraker’s DBMS taxonomy | DBMS2 -- DataBase Management System Services on June 26th, 2008 3:40 am

    […] Part 1: Database management system choices – overview […]

  12. Database management system choices – beyond relational | DBMS2 -- DataBase Management System Services on June 26th, 2008 3:41 am

    […] This is the fifth of a five-part series on database management system choices. For the first post in the series, please click here. […]

  13. Database management system choices – relational data warehouse | DBMS2 -- DataBase Management System Services on June 26th, 2008 3:44 am

    […] This is the third of a five-part series on database management system choices. For the first post in the series, please click here. […]

  14. Database management system choices – 4 categories of relational | DBMS2 -- DataBase Management System Services on June 26th, 2008 3:44 am

    […] This is the second of a five-part series on database management system choices. For the first post in the series, please click here. […]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.