Vertica’s innovative architecture for flash, plus more about temp space than you perhaps wanted to know
Vertica is announcing:
- Technology it already has released*, but has not published any reference architectures for.
- A Barney partnership.**
In other words, Vertica has succumbed to the common delusion that it’s a good idea to put out half-baked press releases the week of TDWI conferences. But if we look past that kind of all-too-common nonsense, Vertica is highlighting an interesting technical story, about how the analytic DBMS industry can exploit solid-state memory technology.
** With Fusion I/O
To set the context, let’s recall a few points I’ve noted in the past:
- Solid-state memory’s price/throughput tradeoffs obviously make it the future of database storage.
- The flash future is coming soon, in part because flash’s propensity to wear out is overstated. This is especially true in the case of modern analytic DBMS, which tend to write to blocks all at once, and most particularly the case for append-only systems such as Vertica.
- Being able to intelligently split databases among various cost tiers of storage – e.g. flash and disk – makes a whole lot of sense.
Taken together, those points tell us:
For optimal price/performance, analytic DBMS should support databases that run part on flash, part on disk.
While all this is a future for some other analytic DBMS vendors, Vertica is shipping it today.* What’s more, three aspects of Vertica’s architecture make it particularly well-suited for hybrid flash/disk storage, in each case for a similar reason – you can get most of the performance benefit of all-flash for a relatively low actual investment in flash chips:
- Vertica lets you split tables by column, and Vertica FlexStore is versatile enough to let you put only the most-used columns in flash. (Vertica offers a figure that 85% of usage calls on only 15% of columns, but I don’t know how rigorously grounded those numbers are.)
- To the extent that Vertica data is more compressed than many of Vertica’s competitors’ (which it probably is, debates over the magnitude of Vertica’s advantage notwithstanding), the total storage-hardware cost of sticking stuff in flash is less when you use Vertica than with other systems.
- Vertica has relatively less need for temp space than some other systems. (Vertica uses figures of <20% of total storage, vs. 30%+ for some other systems.) If you want to use flash for temp space, so as to accelerate your toughest queries, that can save you some cash …
- … and by the way, temp space is an especially good use of flash, because temp space is accessed in a less sequential manner than data storage is.
The least obvious of those points are about temp space; I only understood the particulars when Vertica development chief Shilpa Lawande explained them to me Thursday.
* At least in theory; customer adoption may be a different matter.
But before drilling down on temp space, let me first note that there’s one offsetting factor to all those “We need somewhat less flash than the other guys” Vertica advantages. Like all serious databases, a Vertica installation keeps two or more copies of all data, to that there’s no storage single point of failure. In a flexible system like Vertica, you can put one copy on flash and one on disk. But if you do that in Vertica, you forgo fully exploiting one possible benefit of Vertica’s architecture – the ability to store different copies of a column in different orders, which are beneficial for accelerating different groups of queries.*
*More precisely, you don’t get the full benefits of flash acceleration for every query touching those columns.
OK. Back to temp space. There are four kinds of things you can put in storage if you’re running a database management system:
- The software itself.
- Persistent data. (I.e., tables, if the DBMS you’re running is relational.)
- Metadata, especially the kind that lets you find data – indexes, zone maps, catalogs, etc.
- Temporary data constructs built as part of, say, a sort-merge join. These, by definition, are what populate temp space.
Just to be clear, those constructs are NOT temporary tables of the sort created by, say, Microstrategy; such tables are handled like any other data. Rather, they are ephemeral creations and, so far as I can tell, not tables at all.
Vertica offered two theories as to why its DBMS requires less temp space than competitors do:
- To the extent data is decompressed before being operated on in memory by the DBMS, that decompression would of course also apply to temp space as well. Vertica prides itself on keeping data compressed all the way through, and seems to get away with smaller temp space allocations as a benefit.
- Since Vertica can store columns in expedient sort orders, it does less sorting overall, and sorting is a big use of temp space.
Obviously, no matter which DBMS you use, the amount of temp space you need is surely workload-dependent. Even so, Vertica’s claim to something of an advantage seems legit.
Truth be told, I’m not convinced the savings involved are great enough to matter a whole lot – but it’s a fun subject to think through.
And finally: One of my biggest surprises since starting to look at analytic-DBMS-on-flash has been the centrality of temp space. Talking to Vertica Thursday, I finally uncovered a key reason why: Temp space tends to be accessed via multiple streams of data at once. I’m still struggling with WHY that is true, with two reasons suggested being:
- Temp space can be accessed by multiple operations at once. (But isn’t that also true of the rest of storage?)
- Merge sorts, a common use of temp space, read multiple streams of data. (Couldn’t you tweak your software to make that not be true?)
But if we grant that temp space naturally is accessed in multiple places at once – well, that’s a lot like random I/O, and if you’re doing a lot of random reads, you’d love to use something other than spinning disk.