• Technology it already has released*, but has not published any reference architectures for
• A Barney partnership**

In other words, Vertica has succumbed to the common delusion that it’s a good idea to put out half-baked press releases the week of TDWI conferences. But if we look past that kind of all-too-common nonsense, Vertica is highlighting an interesting technical story, about how the analytic DBMS industry can exploit solid-state memory technology.

*Upgrades to Vertica FlexStore to handle Flash memory, actually released as part of Vertica 4.0

** With Fusion I/O

To set the context, let’s recall a few points I’ve noted in the past:

• Solid-state memory’s price/throughput tradeoffs obviously make it the future of database storage.
• The Flash future is coming soon, in part because Flash’s propensity to wear out is overstated. This is especially true in the case of modern analytic DBMS, which tend to write to blocks all at once, and most particularly the case for append-only systems such as Vertica.
• Being able to intelligently split databases among various cost tiers of storage – e.g. Flash and disk – makes a whole lot of sense.

Taken together, those points tell us:

For optimal price/performance, analytic DBMS should support databases that run part on Flash, part on disk.

While all this is a future for some other analytic DBMS vendors, Vertica is shipping it today.* What’s more, three aspects of Vertica’s architecture make it particularly well-suited for hybrid Flash/disk storage, in each case for a similar reason – you can get most of the performance benefit of all-Flash for a relatively low actual investment in Flash chips: 

• Vertica lets you split tables by column, and Vertica FlexStore is versatile enough to let you put only the most-used columns in Flash. (Vertica offers a figure that 85% of usage calls on only 15% of columns, but I don’t know how rigorously grounded those numbers are.)
• To the extent that Vertica data is more compressed than many of Vertica’s competitors’ (which it probably is, debates over the magnitude of Vertica’s advantage notwithstanding), the total storage-hardware cost of sticking stuff in Flash is less when you use Vertica than with other systems.
• Vertica has relatively less need for temp space than some other systems. (Vertica uses figures of <20% of total storage, vs. 30%+ for some other systems.) If you want to use Flash for temp space, so as to accelerate your toughest queries, that can save you some cash …
• … and by the way, temp space is an especially good use of Flash, because temp space is accessed in a less sequential manner than data storage is.

The least obvious of those points are about temp space; I only understood the particulars when Vertica development chief Shilpa Lawande explained them to me Thursday.

* At least in theory; customer adoption may be a different matter.

But before drilling down on temp space, let me first note that there’s one offsetting factor to all those “We need somewhat less Flash than the other guys” Vertica advantages. Like all serious databases, a Vertica installation keeps two or more copies of all data, to that there’s no storage single point of failure. In a flexible system like Vertica, you can put one copy on Flash and one on disk. But if you do that in Vertica, you forgo fully exploiting one possible benefit of Vertica’s architecture – the ability to store different copies of a column in different orders, which are beneficial for accelerating different groups of queries.*

*More precisely, you don’t get the full benefits of Flash acceleration for every query touching those columns.

OK. Back to temp space. There are four kinds of things you can put in storage if you’re running a database management system:

• The software itself.
• Persistent data. (I.e., tables, if the DBMS you’re running is relational.)
• Metadata, especially the kind that lets you find data – indexes, zone maps, catalogs, etc.
• Temporary data constructs built as part of, say, a sort-merge join. These, by definition, are what populate temp space.

Just to be clear, those constructs are NOT temporary tables of the sort created by, say, Microstrategy; such tables are handled like any other data. Rather, they are ephemeral creations and, so far as I can tell, not tables at all.

Vertica offered two theories as to why its DBMS requires less temp space than competitors do:

• To the extent data is decompressed before being operated on in memory by the DBMS, that decompression would of course also apply to temp space as well. Vertica prides itself on keeping data compressed all the way through, and seems to get away with smaller temp space allocations as a benefit.
• Since Vertica can store columns in expedient sort orders, it does less sorting overall, and sorting is a big use of temp space.

Obviously, no matter which DBMS you use, the amount of temp space you need is surely workload-dependent. Even so, Vertica’s claim to something of an advantage seems legit.

Truth be told, I’m not convinced the savings involved are great enough to matter a whole lot – but it’s a fun subject to think through.

And finally: One of my biggest surprises since starting to look at analytic-DBMS-on-Flash has been the centrality of temp space. Talking to Vertica Thursday, I finally uncovered a key reason why: Temp space tends to be accessed via multiple streams of data at once. I’m still struggling with WHY that is true, with two reasons suggested being:

• Temp space can be accessed by multiple operations at once. (But isn’t that also true of the rest of storage?)
• Merge sorts, a common use of temp space, read multiple streams of data. (Couldn’t you tweak your software to make that not be true?)

But if we grant that temp space naturally is accessed in multiple places at once – well, that’s a lot like random I/O, and if you’re doing a lot of random reads, you’d love to use something other than spinning disk.

]]> http://www.dbms2.com/2010/08/16/vertica-flash-temp-space/feed/ 10 http://www.dbms2.com/2010/08/12/teradata-future-product-strategy/ http://www.dbms2.com/2010/08/12/teradata-future-product-strategy/#comments Thu, 12 Aug 2010 10:37:14 +0000 Curt Monash http://www.dbms2.com/?p=2769 I think Teradata’s future product strategy is coming into focus. I’ll start by outlining some particular aspects, and then show how I think it all ties together.

The immediate hook here is that I had a short conversation with Scott Gnau of Teradata yesterday, triggered by Teradata’s acquisition of Kickfire’s assets. Takeaways from that part included:

• The acquisition is all about Kickfire’s data pipelining technology.
• Scott (in my opinion rightly) thinks that isn’t particularly tied to Kickfire’s choice of particular DBMS architecture (fairly vanilla columnar).
• No decision has been made about whether the right vehicle for this technology is an FPGA (Field Programmable Gate Array), conventional Intel CPU, RAM, etc.

If you want to handicap Teradata’s future data pipelining strategy, you might note that:

• Kickfire’s own choice – and hence its existing implementation – is an FPGA.
• VectorWise’s approach to pipelining is Intel-based, apparently at the cost of being closely tied to specific generations of Intel CPUs.
• XtremeData’s approach to pipelining is FPGA-based.
• Teradata has a lot more development resources than any of those other companies, as well as important existing products, and hence has both means and motive to shoehorn new technology into older system designs.

While I had Scott on the phone, I brought up a few other subjects too. Highlights included:

• Teradata’s Flash-based appliance is doing just fine in beta test and customer POCs (Proofs of Concept).
• Other kinds of Teradata appliance are not inconceivable.
• Scott thinks Michael McIntire’s condemnation of Active-Active architectures is overstated. That said,
  • Scott does acknowledge a need for greater Active-Active scalability, and suggests that the reason Xkoto’s current products are being discontinued is their lack of scaling.
  • Scott seems quietly confident the scaling will get done.
• Scott is emphatic that Teradata is not going to go to a two-tier architecture. In particular, the point of splitting storage/lightweight database processing and heavyweight database processing on separate tiers is generally to save bandwidth, and Teradata’s BYNET is typically less than 10% loaded.
• Scott didn’t dispute my claim that this all suggests Teradata Virtual Storage is the future, at the expense of a rigid delineation among specific use-case-focused product lines.
• Unlike Netezza or Aster, Teradata doesn’t seem to plan analytic capability that works outside the UDF (User Defined Function) framework. However, Scott noted that Teradata has long had the capability that Aster and Netezza now also have of letting you run analytic code either in “protected mode” (if the process fails the whole database doesn’t crash) or in the database kernel (best performance, if you’re sufficiently confident in the code’s stability to take the risk). Scott also spoke of the release later this quarter of Teradata FastPath, which will offer yet better performance (however, there’s a gotcha to Teradata FastPath that’s still NDA).

Putting all that together with the rest of what we know about Teradata, I’m going to call out three pillars of Teradata’s long-term product strategy:

• Same fundamentals as always. Teradata’s core product strategy is:
  • Single DBMS, capable of meeting all analytic needs while running in a single instance, usually running on …
  • … proprietary hardware …
  • … built from conservatively-chosen parts.
• Selective vertical application stack. No matter how horizontally-oriented they are, many companies that have been in the analytic technology business for a while wind up with some vertical applications. It sort of just happens. Teradata is no exception. Teradata also likes to sell services to its product customers, and some of those are quite vertical-aware.
• Mutable, modular platform. This is what I highlighted above. Note that it’s philosophically attuned with the one-system-does-everything approach Teradata prefers. More subtly, please also note that it goes well with customer-by-customer price customization, which is almost a must for Teradata given the Innovator’s Dilemma kind of pricing box it finds itself in.

So far, that’s not too exciting, except in the details of how Teradata’s engineers make that all work. But there’s a fourth pillar to Teradata’s technical strategy as well, and it’s a wild card: tight partnerships. Every time I talk with Teradata hardware chief Carson Schmidt, he seems excited about some particular version of a part or other – sometimes from a reasonably established vendor (once it was LSI Logic), sometimes from a tiny one (notably the “stealth” start-up on which Teradata bet its first solid-state product.) In the future, I expect tight business intelligence partnerships as well. Cognos BI will be increasingly integrated with IBM’s DBMS and hardware; Business Objects’ BI will increasingly be integrated with SAP’s applications; and Oracle’s BI will eventually be integrated with everything. How do you compete with that if you‘re Microstrategy? Well, you try to have superior product, of course – but you also partner as closely with DBMS vendors as you can, an approach Microstrategy has already started. Predictive analytics stalwart SAS, of course, is on a partnership binge as well.

Teradata has a larger installed base than almost all its competitors, and enjoys richer third-party software and service support as a result. But I suspect that going forward, for Teradata to remain a leading competitor at price points it is willing to accept, Teradata’s “ecosystem” advantages will need to ratchet up one or several notches.

• Teradata is discontinuing Xkoto’s existing product Gridscale, which Scott characterized as being too OLTP-focused to be a good fit for Teradata. Teradata hopes and expects that existing Xkoto Gridscale customers won’t renew maintenance. (I’m not sure that they’ll even get the option to do so.)
• The point of Teradata’s technology + engineers acquisition of Xkoto is to enhance Teradata’s active-active or multi-active data warehousing capabilities, which it has had in some form for several years.
• In particular, Teradata wants to tie together different products in the Teradata product line. (Note: Those typically all run pretty much the same Teradata database management software, except insofar as they might be on different releases.)
• Scott rattled off all the plausible areas of enhancement, with multiple phrasings – performance, manageability, ease of use, tools, features, etc.
• Teradata plans to have one or two releases based on Xkoto technology in 2011.

Frankly, I’m disappointed at the struggles of clustering efforts such as Xkoto Gridscale or Continuent’s pre-Tungsten products, but if the DBMS vendors meet the same needs themselves, that’s OK too.

The logic behind active-active database implementations actually seems pretty compelling: 

• You may well be keeping a second copy of your database for high availability/hot standby.
• You might even be keeping a third copy for off-site disaster recovery.
• In some cases, you might have reasons beyond disaster recovery to distribute a database around the world.
• So why not allow queries to be run against all the copies?
• And by the way, splitting the workload up a bit by kinds (e.g., long-running vs. short query) might let you optimize the implementation of each copy of the database. (This last point becomes even more important with the rise of solid-state memory.)

Analytic DBMS vendors pretty much all need to offer this. (Possible exception: If they have a data-mart-only positioning so extreme that customers will never care about any form of failover.) That said, I must confess to not having done a good job of tracking who does or doesn’t have which features in this area to date; informative comments to this post in that regard would be much appreciated!

I think that even software-only analytic DBMS vendors should design their systems in an increasingly storage-aware manner

promising to explain what I meant later on. So here goes. 

There always have been good technical reasons to tailor hardware to analytic database software. Data moves through disk controller, network, RAM, CPU and more, each with its own data rate. Getting different kinds of parts into the right balance doesn’t completely eliminate bottlenecks – the Wonderful One-Hoss Shay is poetic fiction – but it certainly can help. As a result, every analytic DBMS vendor of any size offers at least one of:

• A Type 0 appliance
• A Type 1 appliance
• A “recommended hardware configuration”

And beyond performance, appliances and pre-specified hardware configurations offer at least the possibility of easing installation, administration, and support.

There also are marketing reasons to offer an appliance or something appliance-like.

• To various extents, Oracle, Teradata, Microsoft, IBM, Netezza, and EMC are all telling the world that your hardware should be optimized for your analytic DBMS.
• Smaller vendors such as Vertica and Aster Data also tend to cobble together some sort of appliance, in part so they don’t have to say they disagree.
• Thus, a “We don’t see any point in special hardware assembly at all” story would leave an analytic DBMS vendor pretty far out on a limb.

Finally, there are three overlapping technical trends that increase the need for storage-awareness in analytic DBMS. First and foremost is the rise of solid-state memory. For starters, I believe:

• Flash will be important for analytic DBMS soon.
• There are good technical reasons for this.
• Oracle’s marketing will make a big deal out of the Flash aspects of Exadata, so other analytic DBMS vendors will need a response. And of course, if Netezza or Teradata preemptively make a big deal of their Flash-based offerings, that just adds to the pressure for Flash adoption on everybody else.
• But it’s not just Flash – Flash, other solid-state memory, and disk will be combined in various ways.

But this move to Flash will require analytic DBMS vendors to be increasingly storage-aware for at least three reasons:

• It just adds another level of complexity to their hardware-balancing challenges.
• Flash overturns some of the fundamental assumptions of modern analytic DBMS, in particular:
  • Sequential reads are hugely better than random
  • The worst bottleneck is at the point where data comes out of storage.
• The Flash technology stack is still immature, and you have to pick your poison in how to deal with it. Vendors are making very different choices in this regard – and they do have to choose.

Another trend that could naturally lead analytic DBMS vendors to be more storage-aware is their incorporation of what could be viewed as hierarchical storage/ILM technologies. Different data is stored in different ways and/or on different kinds of storage hardware. (Vendors pursuing – you guessed it – different approaches to this include Teradata, Greenplum, Vertica, and Sybase.) The more automatic that process is, the more storage-aware the DBMS will need to be.

Finally, there are reasons to think that DBMS should be split between conventional servers and smart storage. This is, of course, the Exadata strategy. Netezza’s two-processor approach, while rather different, also somewhat validates the idea.

The acquisition of Greenplum will be an all-cash transaction and is expected to be completed in the third quarter of 2010, subject to customary closing conditions and regulatory approvals. The acquisition is not expected to have a material impact to EMC GAAP and non-GAAP EPS for the full 2010 fiscal year. Upon close, Bill Cook will lead the new data computing product division and report to Pat Gelsinger. EMC will continue to offer Greenplum’s full product portfolio to customers and plans to deliver new EMC Proven reference architectures as well as an integrated hardware and software offering designed to improve performance and drive down implementation costs.

Greenplum is one of my biggest vendor clients, and EMC is just becoming one, but of course neither side gave me a heads-up before the deal happened, nor have I yet been briefed subsequently. With those disclaimers out of the way, some of my early thoughts include:

• I wish my clients would never buy each other, but it’s inevitable.
• I don’t think anybody evaluating Greenplum should be much influenced by this deal one way or the other. (Whether they will be is of course a different matter.)
  • EMC tends to run its bigger software acquisitions in a fairly hands-off manner. There’s no particular FUD (Fear/Uncertainty/Doubt) reason why this deal should stop anybody from buying Greenplum software.
  • I also don’t think adding a rich parent adds much of a reason to buy from Greenplum. But if you’re the type who’s nervous about smaller vendors — well, Greenplum now isn’t so small.
• Greenplum Chorus could, in principle, work with non-Greenplum DBMS. That possibility suddenly looks a lot more realistic.
• The list of analytic DBMS vendors with an appliance orientation is pretty impressive, including:
  • Oracle, with Exadata
  • Microsoft, partially
  • Teradata
  • Netezza
  • Now EMC/Greenplum, at least partially
  • Weaker players such as:
    • The ailing Kickfire, which a client (not Kickfire itself) tells me is being shopped around
    • The reeling HP Neoview
    • XtremeData, but I’m still waiting to hear of XtremeData’s first real sale
• Greenplum is something of a specialist in large databases. EMC has to love that.
• Greenplum’s weakness is concurrency.
• Greenplum’s “polymorphic storage” is a good fit for a storage vendor with appliance-y ideas.
• And finally — I think that even software-only analytic DBMS vendors should design their systems in an increasingly storage-aware manner, and have been advising my vendor clients of same. I’ll blog that line of reasoning separately when I get a chance, and edit in a link here after I do.

Related links (edit)

• Here’s the promised post as to why analytic DBMS need to be ever more storage-aware.
• Dave Kellogg crunched the EMC/Greenplum numbers, coming up with an estimated valuation range of $3-400 million, the high end of which is rumored to be correct.
• Merv Adrian suggests the big EMC/Greenplum loser is ParAccel, a viewpoint which presumably presupposes that the EMC/ParAccel partnership was significant in the first place.
• I talked with Ben Werther and posted more about Greenplum and EMC.

• Netezza now intends to come out with a Flash-based appliance earlier than it originally expected.
• Indeed, Netezza has suspended — by which I mean “scrapped” — prior plans for a RAM-heavy disk-based appliance. It will use a RAM/Flash combo instead.*
• Tim Vincent of IBM told me that customers seem ready to adopt solid-state memory. One interesting comment he made is that Flash isn’t really all that much more expensive than high-end storage area networks.

Uptake of solid-state memory (i.e. Flash) for analytic database processing will probably stay pretty low in 2010, but in 2011 it should be a notable (b)leading-edge technology, and it should get mainstreamed pretty quickly after that. 

*So far as I can tell, that’s one of the two significant roadmap changes between the 2009 and 2010 editions of Enzee Universe. The other one is that the robust form of appliance-to-appliance replication technology is coming out later than Netezza had originally planned and hoped.

There also is increasing reason to think that the issues with Flash memory wearing out are overwrought.  And by the way, the entire history of enterprise solid-state memory use is basically shorter than the time in which these products supposedly will wear out, so it’s not as if there have been a lot of real-life failures out there.)

• First, clever things are being done in the area of error correction codes, although for the most part I defer that part of the discussion to Petascan’s Camuel Gilyadov. E.g., this seems to be the idea behind Anobit.
• Second, analytic DBMS are pretty much an ideal use case for Flash reliability. Suppose, as is the case for many products and implementations, you only write things in big blocks. Then you are, ipso facto, resetting the Flash bits only in big blocks. Thus, at least in theory, you automatically have pretty perfect wear leveling.

• VoltDB is based on the H-Store technology, which I wrote about in February, 2009. Most of what I said about H-Store then applies to VoltDB today.
• VoltDB is a no-apologies ACID relational DBMS, which runs entirely in RAM.
• VoltDB has rather limited SQL. (One example: VoltDB can’t do SUMs in SQL.) However, VoltDB guy Tim Callaghan (Mark Callaghan’s lesser-known but nonetheless smart brother) asserts that if you code up the missing functionality, it’s almost as fast as if it were present in the DBMS to begin with, because there’s no added I/O from the handoff between the DBMS and the procedural code. (The data’s in RAM one way or the other.)
• VoltDB’s Big Conceptual Performance Story is that it does away with most locks, latches, logs, etc., and also most context switching.
• In particular, you’re supposed to partition your data and architect your application so that most transactions execute on a single core. When you can do that, you get VoltDB’s performance benefits. To the extent you can’t, you’re in two-phase-commit performance land. (More precisely, you’re doing 2PC for multi-core writes, which is surely a major reason that multi-core reads are a lot faster in VoltDB than multi-core writes.)
• VoltDB has a little less than one DBMS thread per core. When the data partitioning works as it should, you execute a complete transaction in that single thread. Poof. No context switching.
• A transaction in VoltDB is a Java stored procedure. (The early idea of Ruby on Rails in lieu of the Java/SQL combo didn’t hold up performance-wise.)
• Solid-state memory is not a viable alternative to RAM for VoltDB. Too slow.
• Instead, VoltDB lets you snapshot data to disk at tunable intervals. “Continuous” is one of the options, wherein a new snapshot starts being made as soon as the last one completes.
• In addition, VoltDB will also spool a kind of transaction log to the target of your choice. (Obvious choice: An analytic DBMS such as Vertica, but there’s no such connectivity partnership actually in place at this time.)

I should also note that when Tim Callaghan described architectural options to get around 2PC performance issues, they sounded a lot like eventual consistency. Maybe tunable RYW consistency isn’t in the cards, but at least there’s a NoSQL-like possibility with VoltDB.

VoltDB’s open source strategy is:

• VoltDB will be open sourced.
• Community VoltDB will be GPLed. Professional Edition VoltDB has a non-GPL license.
• The VoltDB Professional Edition won’t start out with features beyond the Community Edition ones, but will gain such later on. I didn’t get the sense the plans for those features were completely baked yet, but ideas mentioned included:
  • Management/monitoring tools.
  • Integration with expense closed-source enterprise software products, such as ones in the management/monitoring area.
  • Yet more “extreme”/edge-case performance.
• Before VoltDB decided for sure that it wasn’t selling licenses, it sold a license to Getco, which also seems to be an investor in the company.

VoltDB had a beta test with about 150 participants. None is in production yet, although at least a few are clearly headed there. Most VoltDB beta testers are in some kind of online business, with a particular concentration in everybody’s new favorite market, online gaming. Most of the rest are in investment/trading — a major target market for at least three different Mike Stonebraker companies — and a few are in telecom. VoltDB assures me that some of the beta users are companies one actually has heard of before, but VoltDB is not in a position to name any of those.

VoltDB is not ideally suited for a classic order management system, since you’d want to partition both on CustomerID and SKU, the latter because you’d constantly updating inventory stock levels. However, this argument doesn’t apply in the case of virtual goods. Virtual goods that are sold for real money — and hence need ACID levels of transaction integrity — are thus a clear target market for VoltDB. (The example that came up was in, you guessed it, online gaming.) The other interesting use case that Tim highlighted was low-latency analytics/ELT. For reasons I didn’t totally grasp, Tim likes to call this “Stateful ELT.” (Given that the data goes into the VoltDB database before much else happens to it, I’m pretty sure I heard “ELT” correctly. But I guess I might have been mishearing “ETL”.)

VoltDB company highlights include:

• VoltDB has about a dozen employees, all but two of whom are technical. (However, I’m not sure they’re counting Andy Ellicott against the two. But then, last I heard he wasn’t full time at VoltDB.)
• VoltDB’s venture funding status is, if I may paraphrase, “Mumble mumble.”
• Although long separate from Vertica, VoltDB is still located in Vertica’s offices.

• Nothing in my original short post about Clustrix was actually incorrect.
• Clustrix plans to reveal actual production “name-brand” customers soon.
• The name of Clustrix’s software, or at least the guts thereof, is Sierra.
• Clustrix’s products have actually been in general availability since last quarter, with some versions at customer sites for 2 years. Development started 3 ½ years ago.
• Clustrix says its technology is for OLTP systems, which it calls “non-batch/non-analytic,” with mixed read/write workloads. All Clustrix’s example target markets are “internet verticals,” such as photo sharing, gaming, social media, e-commerce, etc.
• Clustrix’s heart is in SQL, as is most of its customer base. Clustrix Sierra’s key-value-store option has little or no performance advantage over Clustrix Sierra’s SQL option, nor any other advantage over SQL that came up in discussion.
• Clustrix Sierra is “wire-compatible” with MySQL, but doesn’t use MySQL code; Clustrix wrote all the code itself.
• Clustrix asserts that Clustrix Sierra supports the “vast majority” of MySQL features. Examples of MySQL features Clustrix doesn’t support at this time are full-text search and geospatial indexing.
• Indeed, Clustrix claims Clustrix Sierra can be used to replace MySQL with few or zero changes to existing applications.
• I specifically asked about referential integrity, which has a poor performance reputation in MySQL. Besides saying they supported it, Clustrix said that some customers actually use referential integrity in some of their less active tables.
• Clustrix Sierra is fully ACID-compliant, with no eventual consistency or RYW consistency story. The default number of copies of each datum is two, and they’re kept consistent via two-phase commit.
• Clustrix Sierra is fully parallel, with no “head” node. I forgot to ask how it was determined which queries would be addressed to and/or controlled by which nodes, but I presume there’s some sort of a load-balancing scheme.
• Clustrix says that because Clustrix Sierra uses MVCC (Multi-Version Concurrency Control), and thus reads and writes don’t block each other, global locks aren’t a major issue. (They’re rare or short or something – I have trouble seeing why they would be non-existent.)
• Clustrix says there’s a second class of locks and latches that are purely local and short-lived, for B-tree indexes and the like. (I didn’t drill down into those either.) I guess this means Clustrix Sierra is B-tree-centric, which makes sense for an OLTP-oriented system.
• Clustrix Sierra distributes data among nodes via consistent hashing (default), range partitioning, or “full distribution”(i.e., coping a – presumably small – table to each node). The choice of distribution plans is manual now; more automation is a future feature.
• Clustrix Sierra’s CBO (Cost-Based Optimizer) is, as one would hope, distribution-aware.
• Clustrix Sierra compiles query fragments and ships them off to the relevant nodes. A fragment might contain both instructions for SQL to be executed locally and for where data is to be sent next.
• Clustrix says that Clustrix Sierra does data migration and redistribution (e.g., when you add a node) transparently online, and further says that in practice this doesn’t cause a performance hit.
• As for Clustrix hardware:
  • Clustrix makes Type I appliances.
  • A Clustrix node contains 2 quad-core chips, 32 gigs of RAM, and 7 160 GB solid-state drives.
  • Specifically, Clustrix is using Intel SSDs, with a SAS interface.
  • Clustrix says solid-state memory isn’t really essential to the product design; it’s just cheap in terms of $/IOPS (I/O Per Second).
• A minimum Clustrix configuration is 3 nodes, for redundancy. After that you can add nodes one at a time. Clustrix says it built a 20-node system in-house, leading me to suspect that customers don’t have anything bigger than 20 nodes either.
• That 20-node Clustrix system was tested to show near-linear scalability. (In discussing this, Clustrix tends to forget to use the word “near”.)
• Clustrix has partnered with somebody to provide global 4-hour-response support. As of now Clustrix seems to be active mainly in North America and Europe.
• Clustrix is formed from the combination of two startups, which I’ve heard elsewhere were called Clustrix and Sprout. Exactly when the combination happened sounds a little different depending on who’s telling the story (one version has the predecessors still being separate well into 2008, but Clustrix implies the combination happened pretty much on Day 1).

Now Slashdot and ZDnet have weighed in, although for the most part they only are suggesting 50-100% performance hits. One good quote is:

It’s a lot easier to interfere with a moving head arm than it is to mess one up that’s locked on a track, so this isn’t surprising in the least for vibration to affect reads that require numerous long seeks. I’m surprised it’s not worse than they’ve found.

Moving the head requires accelerated head stepping to top speed, stepping to close to the track, slowing down, stopping at the destination track, waiting for the head to settle, and reading an address block to find out where you managed to land. If you find you missed the track, you have to go through the whole seek process again. (usually only once more, those short adjustment hops are pretty reliable because they’re lower speed) But that really hurts your single block read time.

Add to that the fact that the “high performance” drives are making more risky higher speed track changes, which increase the odds of missing your target and make the operation more sensitive to vibration. I’ve written direct HDD io code before, and sure, you can up the step speed to get very nice seek time boosts, but then you start missing your track and start getting reseeks. Usually you go with the fastest that’s acceptably reliable, and that puts you on the bleeding edge of having problems, where things like vibration can run you off the deep end of the bell curve.

• Clustrix is making OLTP DBMS.
• The core problem Clustrix tries to solve is scale-out, without necessarily giving up SQL. (I couldn’t immediately tell whether Clustrix supports NoSQL-style key-value interfaces enthusiastically, grudgingly, or not at all.)
• Unlike Akiban or VoltDB, Clustrix makes database appliances. The Clustrix software seems to assume a Clustrix appliance.
• A key feature of Clustrix’s database appliances is that they rely on solid-state memory. I’m guessing that Clustrix appliances don’t even have disks, or that if they do the disks store some software or something, not actual data. (As previously noted, I agree with Oracle in thinking that much of the progress in database technology this decade will come from proper design for solid-state memory.)
• Clustrix talks of things that sound like compiled queries and attempts to avoid locks. However, it doesn’t sound as extreme in these regards as VoltDB.
• Clustrix also talks of things that sound like consistent hashing.
• The brand name “Sierra” also shows up along with the brand name “Clustrix.”