• Facebook et al. are in effect Software as a Service (SaaS) vendors, not enterprise technology users. In particular:
  • They have the technical chops to rewrite their code as  needed.
  • Unlike packaged software vendors, they’re not answerable to anybody for keeping legacy code alive after a rewrite. That makes migration a lot easier.
  • If they want to write different parts of their system on different technical underpinnings, nobody can stop them. For example …
  • …  Facebook innovated Cassandra, and is now heavily committed to HBase.
• It makes little sense to talk of Facebook’s use of “MySQL.” Better to talk of Facebook’s use of “MySQL +  memcached  + non-transparent sharding.” That said:
  • It’s hard to see why somebody today would use MySQL +  memcached  + non-transparent sharding for a new project. At least one of Couchbase or transparently-sharded MySQL is very likely a superior alternative. Other alternatives might be better yet.
  • As noted above in the example of Facebook, the many major web businesses that are using MySQL +  memcached  + non-transparent sharding for existing projects can be presumed able to migrate away from that stack as the need arises.

Continuing with that discussion of DBMS alternatives:

• If you just want to write to the memcached API anyway, why not go with Couchbase?
• If you want to go relational, why not go with MySQL? There are many alternatives for scaling or accelerating MySQL — dbShards, Schooner, Akiban, Tokutek, ScaleBase, ScaleDB, Clustrix, and Xeround come to mind quickly, so there’s a great chance that one or more will fit your use case. (And if you don’t get the choice of MySQL flavor right the first time, porting to another one shouldn’t be all THAT awful.)
• If you really, really want to go in-memory, and don’t mind writing Java stored procedures, and don’t need to do the kinds of joins it isn’t good at, but do need to do the kinds of joins it is, VoltDB could indeed be a good alternative.

And while we’re at it — going schema-free often makes a whole lot of sense. I need to write much more about the point, but for now let’s just say that I look favorably on the Big Four schema-free/NoSQL options of MongoDB, Couchbase, HBase, and Cassandra.

… start-ups that are building technologies to enable MySQL and other SQL databases to get over some of the problems they have in scaling past a certain size. … I’d like to get a sense as to whether or not the problems are as severe and wide spread as these companies are telling me? If so, why wouldn’t a customer just move to a new database?

While that sounds as if he was asking about scale-out relational DBMS in general, MySQL or otherwise, short-request or analytic, it turned out that he was asking just about short-request scale-out MySQL. My thoughts and comments on that narrower subject include(d) but are not limited to: 

• The biggest web companies had to go to non-transparently sharded MySQL years ago. The NoSQL movement is, in no small part, an attempt to improve upon that. Ditto for scale-out short-request MySQL.
• Some overlapping categories of companies or projects who need scale-out short-request database processing are:
  • The aforementioned big companies who have other applications they haven’t hand-sharded yet.
  • Other web companies whose applications are getting that big.
  • Conventional enterprises whose web efforts happen to be very big.
  • Sensor networks and other massive sources of machine-generated data.
  • Certain specialized areas (e.g., financial trading).

• Relatively few of these applications are totally impossible to do in Oracle. But the Oracle approach might be very expensive.
• In particular, there’s a break point when companies — often SaaS vendors — outgrow Oracle Standard Edition.
• Yes, the alternatives usually are one of MySQL or Oracle.
• InnoDB isn’t an alternative to these newer technologies; it’s just a piece of the puzzle and indeed of default MySQL now. Several of them — e.g. dbShards — are meant to be used in conjunction with InnoDB.
• Merging his list and mine, the high-performance/scale-out MySQL alternatives look like dbShards, Schooner, ScaleBase, ScaleDB, Tokutek, Akiban, Xeround, and Clustrix. The first two are to my knowledge more proven than the rest.
• Proprietary hardware and the associated hardware/appliance pricing aren’t very appealing for these applications. That speaks against Oracle Exadata and Clustrix, and is the reason Schooner switched to a software-only strategy despite some initial appliance sales.
• However, hardware band-aids such as solid-state drives or even RAM-based solid-state storage could make more sense:
  • If, for performance, you’ve scaling out your database so that it fits in RAM on each box, you don’t really have a disk-based architecture anyway, now do you?
  • Even if you’re not doing that yet — if your problem is throughput rather than storage capacity, silicon-based storage could be a big help.
  • In principle, devices of that kind can be moved from one application to another, after the first one is rearchitected not to need them. (In practice, however, I don’t know of anybody who is doing that. I also don’t believe that Kaminario et al. are marketing that kind of idea, more’s the pity.)
• My notes on all this from April, 2010 are already badly outdated, but may be interesting anyway.

It also is understood that, realistically, I can’t be expected to know and mention the very latest news for all the many products in the categories. Even so, I think this would be fine time to check just where NoSQL and HVSP adoption stand. Here is most of what I know, or links to same; it would be great if you guys would contribute additional data in the comment thread.

In the NoSQL area: 

• Back in April, the VoltDB guys told me they thought Cassandra and HBase were the two NoSQL systems with the most momentum.
• I know distressingly little about HBase adoption, but a source who may or may not wish to remain anonymous was kind enough to alert me that Twitter and StumbleUpon each have ~30 node deployments, for analytics and analytics/HVSP respectively.
• I wrote in detail on Cassandra adoption last month. News since then includes:
  • Facebook is rumored to have dropped Cassandra completely.
  • Twitter clarified that it may not be quite as lovestruck by Cassandra as before, but they’re still very close friends.
  • It’s not obvious that the Cassandra Summit unveiled a lot of new adoption stories.
• Northscale’s Membase is still in its early days.  Zynga is bought in, however, as is something called NHN Korea. (Edit: I subsequently saw NHN Korea on a prominent SEO expert’s list of the top half dozen or so search engines in the world. Who knew?)
• Basho has listed a few Riak customers. If memory serves (I haven’t spoken with Basho for a while, and some of my notes are misplaced due to some computer sloppiness), Basho has a few dozen customers in total.
• Mozilla has a 4 machine, 64 core Riak cluster in production.
• Hypertable has a few users/project sponsors, Baidu being the biggest name among them.
• I don’t really know how the MongoDB/10gen guys are doing. I think this is at least as much my fault as theirs. Anyhow, they seem to have links to a couple of folks who have written about MongoDB usage.
• NimbusDB is still in stealth mode. I’d be surprised if they had users for a while yet, since in January they didn’t yet sound as if development was very far underway. (Actually, I forget whether NimbusDB is supposed to be SQL-based or not.)

Among the SQL or SQL-friendly guys:

• Clustrix says it has a few production users, some big-name, but is not disclosing them yet.
• dbShards has around 6 customers, including Facebook. (Facebook may outpace even Twitter and Zynga in using the most products mentioned in this post.)
• As of May, VoltDB had one paying customer, plus 150 beta customers who weren’t in production yet.
• Akiban says they’ll get me up to speed on Thursday.
• ScaleDB seems to be pedaling along in perennial beta. Whether ScaleDB has any actual beta users is less clear. On the plus side, checking that out uncovered a pretty funny April Fool blog post.
• Groovy Corporation seems to have disappeared, or morphed into something called uCirrus, or something like that.

In a nutshell, ScaleDB’s proposition is:

• Innovative approach to indexing relational DBMS, providing performance advantages.

• Shared-everything scale-up that ScaleDB believes will leapfrog the MySQL engine competition already in Release 1. (In my opinion, this is the least plausible part of the ScaleDB story.)

• State-of-the-art me-too facilities for locking, logging, replication/fail-over, etc., also already in Release 1.

Like many software companies with non-US roots, ScaleDB seems to have started with a single custom project, using a Patricia trie indexing system. Then they decided Patricia tries might be really useful for relational OLTP as well. The ScaleDB team now features four developers, plus half-time or so “Chief Architect” involvement from Vern Watts. Watts seems to pretty much have been Mr. IMS for the past four decades, and thus surely knows a whole lot about pointer-based database management systems; presumably, he’s responsible for the generic DBMS design features that are being added to the innovative indexing scheme. On ScaleDB’s advisory board is PeopleSoft veteran Rick Berquist, about whom I’ve had fond thoughts ever since he talked me into focusing on consulting as the core of my business.*

*More precisely, Rick pretty much tricked me into doing a day of consulting for $15K, then revealed that’s what he’d done, expressing the thought that he’d very much gotten his money’s worth. But I digress …

ScaleDB has no customers to date, but hopes to be in beta by the end of this year. Angels and a small VC firm have provided bridge loans; otherwise, ScaleDB has no outside investment. ScaleDB’s business model thoughts include:

• $1,000/server/year license fee, or something in that range.

• Early focus on Web 2.0 kinds of customers (e.g., social networking companies may enjoy the join performance ScaleDB plans to offer).

• Early focus on MySQL OLTP (but, like proud parents everywhere, they think the technology is so wonderful that it could eventually be pretty much all things to all people).

The company is based in Menlo Park, CA.

Probably I should explain what Patricia tries actually are, and how they can help relational DBMS. An ordinary trie* is a way of indexing data that looks a lot like – unsurprisingly – a tree. For example, suppose you need to index a lot of character strings, each consisting of lower-case Latin letters. From the root node you point to the 26 possibilities for starting letter. From those you point to the next possible letter, and so on. Combinatorial explosion is averted because you only have edges if there’s actually a string with that letter combination. Thus, when indexing a corpus of classic novels, there might be a path i-t-i-s-a-t-r-u-t-h-u-n-… and so on, but none that starts i-a-u-z-z-z.

*”Trie” is sometimes pronounced like “tree”, sometimes like “try.”

Patricia tries add a now-obvious compression technique. Namely, if there’s only one branch from a node, just collapse it. Thus, the example I gave above would become something more like i-t-i-s-a-truth-universally-acknowledged-…, or perhaps something even more compact.

While these ideas were evidently invented with text documents in mind, there’s no reason they can’t be applied to other kinds of strings – specifically, to those stored in relational databases. (And numbers can just be treated as strings of bits.) As I wrote last year in discussing solidDB, which uses a similar approach:

The canonical index structure in a disk-centric OLTP RDBMS is a tree of blocks. The record sought is in a block somewhere. There are index blocks whose entries are pointers to the correct block based on values in the index column. There are index blocks of pointers to other index blocks. And so on. One can traverse these trees in very few steps, but each step is costly, because each step involves examining the whole block.

SolidDB, by way of contrast, uses a core index structure called the trie. The key value on which the record search is based is divided into chunks of bits. Each chunk leads to a tree node with a small number of choices for the next chunk. There are more steps, but each step is much cheaper.

Benefits of this strategy include compression and in-memory performance. But a naive implementation would, as in other pointer-based systems, lead to unacceptable disk thrashing. ScaleDB’s answer is to layer the index, essentially creating a “trie of tries.” The company confidently claims that, in almost all cases, data can be found via a single disk read. Part of that story is the assertion that their indexing scheme achieves tremendous compression vs. conventional b-trees.

So far, that all sounds like a performance win, of unclear magnitude. (ScaleDB says it’s hoping for a 3X or better performance advantage versus traditional b-tree-based approaches.) But there’s another cool part as well. The ScaleDB trie doesn’t necessarily end with the first row it finds; it also reaches through to capture foreign-key relationships. E.g., if customer FOO123 places an order with OrderID BAR456, the BAR456 isn’t just found via the path B-A-R-4-5-6. It also can be found via FOO-1-2-3-BAR-456. Thus, referential integrity and updatable views are baked into the core database management architecture.

I look forward to seeing how this all works out, in Release 1 and beyond.

Edit: One way to think of this as the integration of the network and relational data models, ala IDMS/R, but with more compact linked lists. And I believe Predrag Dizdarevic when he tells me IDMS/R did wind up working pretty well, in a rare instance of a DBMS technology success post acquisition by CA.