• You really, really want to have multiple data streams coming out of temp space, as close to simultaneously as possible.
• The storage performance characteristics of such a workload are more reminiscent of “random” than “sequential” I/O.

If everybody else is cool with it too, I can live with that.

Meanwhile, I talked again with Tim Vincent of IBM this afternoon. Tim endorsed the temp space/Flash fit, but with a different emphasis, which upon review I find I don’t really understand. The idea is:

• Analytic DBMS processing generally stresses reads over writes.
• Temp space is an exception — read and write use of temp space is pretty balanced. (You spool data out once, you read it back in once, and that’s the end of that; next time it will be overwritten.)

My problem with that is: Flash typically has lower write than read IOPS (I/O per second), so being (relatively) write-intensive would, to a first approximation, seem if anything to disfavor a workload for Flash.

On the plus side, I was reminded of something I should have noted when I wrote about DB2 compression before:

Much like Vertica, DB2 operates on compressed data all the way through, including in temp space.

• Technology it already has released*, but has not published any reference architectures for
• A Barney partnership**

In other words, Vertica has succumbed to the common delusion that it’s a good idea to put out half-baked press releases the week of TDWI conferences. But if we look past that kind of all-too-common nonsense, Vertica is highlighting an interesting technical story, about how the analytic DBMS industry can exploit solid-state memory technology.

*Upgrades to Vertica FlexStore to handle Flash memory, actually released as part of Vertica 4.0

** With Fusion I/O

To set the context, let’s recall a few points I’ve noted in the past:

• Solid-state memory’s price/throughput tradeoffs obviously make it the future of database storage.
• The Flash future is coming soon, in part because Flash’s propensity to wear out is overstated. This is especially true in the case of modern analytic DBMS, which tend to write to blocks all at once, and most particularly the case for append-only systems such as Vertica.
• Being able to intelligently split databases among various cost tiers of storage – e.g. Flash and disk – makes a whole lot of sense.

Taken together, those points tell us:

For optimal price/performance, analytic DBMS should support databases that run part on Flash, part on disk.

While all this is a future for some other analytic DBMS vendors, Vertica is shipping it today.* What’s more, three aspects of Vertica’s architecture make it particularly well-suited for hybrid Flash/disk storage, in each case for a similar reason – you can get most of the performance benefit of all-Flash for a relatively low actual investment in Flash chips: 

• Vertica lets you split tables by column, and Vertica FlexStore is versatile enough to let you put only the most-used columns in Flash. (Vertica offers a figure that 85% of usage calls on only 15% of columns, but I don’t know how rigorously grounded those numbers are.)
• To the extent that Vertica data is more compressed than many of Vertica’s competitors’ (which it probably is, debates over the magnitude of Vertica’s advantage notwithstanding), the total storage-hardware cost of sticking stuff in Flash is less when you use Vertica than with other systems.
• Vertica has relatively less need for temp space than some other systems. (Vertica uses figures of <20% of total storage, vs. 30%+ for some other systems.) If you want to use Flash for temp space, so as to accelerate your toughest queries, that can save you some cash …
• … and by the way, temp space is an especially good use of Flash, because temp space is accessed in a less sequential manner than data storage is.

The least obvious of those points are about temp space; I only understood the particulars when Vertica development chief Shilpa Lawande explained them to me Thursday.

* At least in theory; customer adoption may be a different matter.

But before drilling down on temp space, let me first note that there’s one offsetting factor to all those “We need somewhat less Flash than the other guys” Vertica advantages. Like all serious databases, a Vertica installation keeps two or more copies of all data, to that there’s no storage single point of failure. In a flexible system like Vertica, you can put one copy on Flash and one on disk. But if you do that in Vertica, you forgo fully exploiting one possible benefit of Vertica’s architecture – the ability to store different copies of a column in different orders, which are beneficial for accelerating different groups of queries.*

*More precisely, you don’t get the full benefits of Flash acceleration for every query touching those columns.

OK. Back to temp space. There are four kinds of things you can put in storage if you’re running a database management system:

• The software itself.
• Persistent data. (I.e., tables, if the DBMS you’re running is relational.)
• Metadata, especially the kind that lets you find data – indexes, zone maps, catalogs, etc.
• Temporary data constructs built as part of, say, a sort-merge join. These, by definition, are what populate temp space.

Just to be clear, those constructs are NOT temporary tables of the sort created by, say, Microstrategy; such tables are handled like any other data. Rather, they are ephemeral creations and, so far as I can tell, not tables at all.

Vertica offered two theories as to why its DBMS requires less temp space than competitors do:

• To the extent data is decompressed before being operated on in memory by the DBMS, that decompression would of course also apply to temp space as well. Vertica prides itself on keeping data compressed all the way through, and seems to get away with smaller temp space allocations as a benefit.
• Since Vertica can store columns in expedient sort orders, it does less sorting overall, and sorting is a big use of temp space.

Obviously, no matter which DBMS you use, the amount of temp space you need is surely workload-dependent. Even so, Vertica’s claim to something of an advantage seems legit.

Truth be told, I’m not convinced the savings involved are great enough to matter a whole lot – but it’s a fun subject to think through.

And finally: One of my biggest surprises since starting to look at analytic-DBMS-on-Flash has been the centrality of temp space. Talking to Vertica Thursday, I finally uncovered a key reason why: Temp space tends to be accessed via multiple streams of data at once. I’m still struggling with WHY that is true, with two reasons suggested being:

• Temp space can be accessed by multiple operations at once. (But isn’t that also true of the rest of storage?)
• Merge sorts, a common use of temp space, read multiple streams of data. (Couldn’t you tweak your software to make that not be true?)

But if we grant that temp space naturally is accessed in multiple places at once – well, that’s a lot like random I/O, and if you’re doing a lot of random reads, you’d love to use something other than spinning disk.

]]> http://www.dbms2.com/2010/08/16/vertica-flash-temp-space/feed/ 10 http://www.dbms2.com/2010/06/21/data-warehouse-load-latency/ http://www.dbms2.com/2010/06/21/data-warehouse-load-latency/#comments Mon, 21 Jun 2010 12:15:17 +0000 Curt Monash http://www.dbms2.com/?p=2319 I took advantage of my recent conversations with Netezza and IBM to discuss what kinds of data warehouse load latency were practical. In both cases I got the impression:

• Subsecond load latency is substantially impossible. Doing that amounts to OLTP.
• 5 seconds or so is doable with aggressive investment and tuning.
• Several minute load latency is pretty easy.
• 10-15 minute latency or longer is now very routine.

There’s generally a throughput/latency tradeoff, so if you want very low latency with good throughput, you may have to throw a lot of hardware at the problem.

I’d expect to hear similar things from any other vendor with reasonably mature analytic DBMS technology. Low-latency load is a problem for columnar systems, but both Vertica and ParAccel designed in workarounds from the getgo. Aster Data probably didn’t meet these criteria until Version 4.0, its old “frontline” positioning notwithstanding, but I think it does now.

Related link

• Just what is your need for speed anyway?

• Sybase’s main businesses are:
  • Classic OLTP DBMS (Sybase ASE, for Adapative Server Enterprise, unless I’ve missed yet another name change).
  • Analytic technology — mainly Sybase IQ, but more generally Sybase RAP.
  • Mobile technology. (The frequently renamed small DBMS SQL Anywhere was the foundational product of and still is included in the mobile division.)
• SAP’s thoughts on in-memory database management are interesting. However, I think SAP’s oft-repeated claim that it has a lot of important in-memory database technology to bring to Sybase (or for that matter SAP customers) is mainly smoke and mirrors. Cool data access methods, good niche database products, and broadly applicable multi-domain DBMS innovations are three different things. Granting that SAP probably has the first and thinks it has the second is not the same as giving it much credence for having the third.
• SAP claims that, 15 years after its refusal to support Sybase turned Sybase into a DBMS also-ran, it by now is “relatively simple” to port SAP’s apps to Sybase ASE, and that they will make that happen. I actually believe that SAP’s apps will soon run on Sybase ASE, where by “soon” I mean “in a couple of years for no-apologies general availability.” (Certifying a DBMS for SAP is a long process.) The main missing features — e.g., row-level locking — were already put into Sybase back in the last millenium. Nor could there be fundamental architectural problems that keep SAP from supporting Sybase ASE, or else SAP couldn’t have supported Microsoft SQL Server (which, long ago, was a Sybase fork).
• I don’t see any market or competitive dynamics that would lead the SAP acquisition to hurt Sybase’s ASE or mobile businesses. General merger management mishegas is, of course, always a possibility.
• SAP Business Objects partners with Sybase IQ’s competitors. That could be a problem. However, coopetition is pretty strong in the business intelligence market. I don’t think any of SAP Business Objects, IBM Cognos, or Oracle Business Intelligence are much held back from partnering by competitive dislike of their parent companies.
• The rest of SAP might be able to drum up some extra business for Sybase IQ.
• It would be natural for IBM/Cognos to now buy a columnar DBMS of its own. Vertica is an obvious first choice. ParAccel would surely come much cheaper. Since ParAccel has little chance of surviving as an independent company — too immature and too little differentiation to overcome that — I’d expect ParAccel’s board to jump at the chance to sell out.
• It would be interesting if SAP Business Objects would revive the CEP-based BI idea.
• I gather Sybase’s AnswersAnywhere concept network/object model-based natural language/speech recognition technology never went anywhere. Unsurprising (it seemed like it needed too much hand-building to scale semantically), but regrettable even so.
• I don’t see anything in this acquisition that would revive PowerBuilder (Sybase’s Visual Basic competitor), Sybase’s CASE (Computer-Aided Software Engineering) tools, and so on.
• And on the personal side — I’ll probably lose Sybase as a customer due to this merger, but it could have been worse. A lot of vendors smaller than Sybase are bigger customers for Monash Research.

Edit: Right after I posted this, I saw email from Sybase clarifying that Sybase’s in-memory technology, while slightly influenced by some ANTs IP Sybase bought non-exclusive rights to, is essentially home-grown. That’s what I thought, but the call sounded like it was saying something different.

Further coverage of SAP/Sybase:

• SAP believes in database proliferation
• More quick reactions to SAP/Sybase
• Technical basics of Sybase IQ
  

• Vertica plans to build a massive, world-conquering sales force.
• If Vertica dips back into negative cash flow to do that and has to raise more venture capital, so be it.
• “Triple-digit” revenue growth is expected for this year.

*I’ve since heard more both from Ralph and his former colleagues, and I’m comfortable taking the move more or less at face value — for some reasons he doesn’t want to spell out, Ralph really wanted to move back home to South Africa.

While they were at it, Vertica also put out a press release reporting very good success in the social gaming market. The biggest and best known of the bunch is Zynga. Three months ago, Wayne Eckerson had figures of 3 TB/day added to the database, 200 nodes, and >40 million users. Now Zynga is using a figure of >65 million daily users and 230 nodes. More precisely, at Zynga:

• There are two Vertica databases with identical data.
• Each Zynga Vertica database runs on 115 nodes.
• Zynga’s two Vertica database clusters are used for different applications.
• It’s undisclosed exactly what Zynga runs on what Vertica cluster. But best practice would be to put mission-critical, fast-response stuff on one cluster, and use the other for longer-running or less-critical queries — plus have it be available as hot standby — given that I don’t see much reason to put data geographically close to users around the world for reasons of latency or whatever.
• An undisclosed amount of data, amounting to all of what Wayne earlier estimated at 3 TB, is added to each of Zynga’s Vertica databases daily.

In other news, Vertica now states its customer count as being >130.

• The X Company (XCo) has a <1 TB database.
• 100s of XCo’s customers log in at once to run reports. 50-200 concurrent queries is a good target number.
• XCo had been “suffering” with Oracle and wanted to upgrade.
• XCo didn’t have a lot of money to spend. Netezza pulled out of the sales cycle early due to budget (and this was recently enough that Netezza Skimmer could have been bid).
• Greenplum didn’t offer any references that approached the desired number of concurrent users.
• Ultimately the evaluation came down to Vertica and ParAccel.
• Vertica won.

Notes on the Vertica vs. ParAccel selection include:

• ParAccel sent an engineer on-site to do a proof-of-concept (POC), and generally competed very hard for the deal.
• Vertica dropped by for a sales call once, and let XCo do the Vertica POC itself.
• Not surprisingly, XCo got the impression that Vertica was easier to set up and administer than ParAccel.
• Also, when ParAccel emphasized architectural features such as custom “backplane” and compiled queries, XCo got the impression – right or wrong – that ParAccel’s performance was more brittle or situational than Vertica’s.
• ParAccel was modestly faster than Vertica in the POC. (I think — Vertica’s numbers were described as being “very competitive.”)
• In multiple ways, Vertica gave the impression of greater product and vendor maturity than ParAccel.

My contact continues to be interested in all things Greenplum, and has recommended Greenplum Single-Node Edition to his analyst colleagues.

• Vertica had a “killer” Q4 and is doing very well in Q1 again.
• Vertica burned hardly any cash last year; i.e., it was close to cash-flow neutral in 2009.
• Vertica is hiring aggressively, e.g., in sales.
• Vertica is well down the path with several CEO candidates who Jerry regards as outstanding. He is hopeful there will be a new CEO in April. (But I bet that would be late April, given what Jerry mentioned about his own travel plans.)
• Absent a full-time CEO, Jerry and Andy Palmer are spending a lot more time with Vertica.
• One Vertica customer is approaching a petabyte of user data. The last time Vertica had checked, that customer had been more in the ¼ petabyte range.
• Other multi-hundred terabyte Vertica databases were mentioned, including one where Vertica claims to have beaten Teradata and perhaps other competitors in a head-to-head competition (it sounds like that one’s too recent to be deployed yet).
• Vertica sees Aster and Greenplum competitively more often than it sees ParAccel.
• Vertica sees Sybase IQ competitively a lot in financial services (in new-name accounts for Sybase as well as where some kind of Sybase DBMS is an incumbent), and more occasionally in other sectors.

NDA parts of the conversation also gave me the impression that Vertica is moving forward just as eagerly as its peers. I.e., I didn’t uncover any reason to think that Ralph’s departure is a sign of trouble, of the company being shopped, etc.

On the other hand, I didn’t uncover any other clear reason for Ralph’s departure either. The party line is that Ralph left for “personal reasons”. It was his decision to leave. He did a great job while at Vertica. Basically, Vertica is saying that, even though all was going swimmingly, Ralph just up and quit, leaving some very valuable unvested stock options on the table at what had been his first CEO gig ever.

Contacting Ralph didn’t add any immediate insight. He responded quickly via a Twitter DM, but was in the airport for a trip to his home country of South Africa, and deferred discussion until after his return.

Incidentally, Infobright and ParAccel have both also had recent CEO turnover. Stated reasons in each case were of the “Right person to lead the next stage of the company’s growth” variety.

*I probably have it worse than most people in that regard, because my clients run tentative feature lists and announcement schedules by me well in advance, which may get changed multiple times before the final dates roll around. I also occasionally miss some detail, if it wasn’t in a pre-briefing but gets added at the end.

Anyhow, the three big themes of this month’s announcements are probably:

• Integrating different kinds of analytic processing into databases and DBMS.
• Taking advantage of hardware advances.
• Playing catchup in areas where small vendors’ products weren’t mature yet.

For example, the three biggest data warehouse DBMS product announcements this month are probably:

• Aster Data nCluster 4.5. Much like Aster’s prior release — Aster Data nCluster 4.0 – Aster Data nCluster 4.5 has a major focus on integrating analytics and database processing. This time, the emphasis is on application development tools and pre-built analytic packages. In addition, Aster’s management tool GUIs have been upgraded, building on catch-up functionality in the Aster Data nCluster 4.0.
• Netezza’s “i” add-on to its existing TwinFin products. With Netezza TwinFin(i), Netezza becomes the second MPP RDBMS vendor with a comprehensive “Big Data Analytic Platform” kind of strategy. (Netezza would surely argue that it was the first, but that depends on how seriously one took Netezza’s prior attempt.) Many of the details are different from Aster’s, of course, but the general philosophy is similar. So far, Netezza has announced one interesting proprietary library of analytic packages (for linear/matrix algebra), plus the port of 4,000 or so functions in open source libraries.
• Vertica 4.0. Vertica has had a highly innovative columnar DBMS architecture from the getgo, but at the cost of some restrictions or awkwardness in the relationship between data layout and SQL processing. Vertica says that Vertica 4.0 fixes all that. In addition, it has some analytic processing enhancements, especially in the time series area, where Vertica doesn’t vigorously dispute that Sybase IQ previously had an advantage.

In addition,

• Teradata is announcing its Data Warehouse Appliance 2580, the successor to the Teradata 2550. This is purely a hardware refresh; Teradata’s hardware and software upgrades are not generally synced. The Teradata 2580 upgrades CPUs from Harpertown to Nehalem, includes 3X the RAM of its predecessor, and offers an option for 1 TB disks (thus lowering the bottom price/TB a lot, to $31K list).
• Aster, Vertica, and ParAccel have all called attention to the fact that, if solid-state drives have interfaces like those of disk drives, and if a DBMS supports disk drives, then a DBMS also supports solid-state drives as well. At least Aster and ParAccel have signaled that they have at least one customer or prospect each interested in Fusion I/O’s solid-state technology, especially in the retail sector. This is basically a hardware matter as well, and a big deal only for those who were somehow unaware of the impending dominance of solid-state memory technology.
• Sybase announced its Aleri acquisition earlier this month.
• Various vendors have bragged about various rankings, awards, or benchmarks, or – sometimes less tediously — about last year’s sales results.

For starters, there’s a lot of new analytic functionality. This isn’t Aster/Netezza-style ambitious. Rather, there’s a lot more SQL-99 functionality, plus some time series extensions of the sort that financial services firms – an important market for Vertica – need and love. Vertica did suggest a couple of these time series extensions are innovative, but I haven’t yet gotten detail about those.

Perhaps even more important, Vertica is cleaning up a lot of its previous SQL optimization and execution weirdnesses. In no particular order, I was told:

• Vertica’s delete performance is up “literally” 30-100X, at least in the case of “large” deletes. Performance for “large” updates has been enhanced as well.
• Vertica has finally cleaned up all vestiges of its prior bias to star schemas. For example, Vertica concedes that its product previously would sometimes force a star execution plan that wasn’t really appropriate.
• It is no longer the case that you need to define projections before you load a table into Vertica. This is now fully automatic.
• Vertica 4.0 automatically redesigns the database when new nodes are added to the system.
• When a database designer does hand-tune projections – and there’s no shame in this still being a possibility in Vertica 4.0 – that hand-tuning is now pulled back into the automatic generation/recommendation/whatever wizards for further projections. I.e., there’s a kind of DBA round-trip engineering going on.
• Vertica used to require that tables being joined be identically “segmented” (I think this means distributed across joins). That is no longer the case in 4.0.
• In connection with this new-found flexibility, Vertica now supports full outer joins directly, rather than requiring the left outer join/right outer join/UNION kluge.
• The Vertica 4.0 optimizer is smarter than its predecessor about things like predicate pushdown into subqueries, or exploiting commonality between predicates and partition keys.
• There’s a fundamental change that I don’t understand very well in the Vertica execution engine basic unit of work. It sounds as if in the past all the disk-based data containers the query needed got opened at once and read into memory, whether or not there was enough RAM and CPU cores to handle them, and this problem has now been fixed.
• Vertica always seemed to say that you could query immediately on new data, because even if it hadn’t hit disk yet – the ROS (Read-Optimized Store) – it was available in memory – the WOS (Write-Optimized Store). And queries were in essence federated between the ROS and WOS. But apparently it’s a new feature in Vertica 4.0 that you can read totally fresh data without locking. I confess to not understanding this very well either. (It has something to do with what  Vertica calls “Epochs”.)
• Temporary tables can now be created in Vertica on a local/session basis without any DDL. Make temporary tables easier and more performant is important for a variety of reasons:
  • Microstrategy, Company V* et al. use lots of temp tables. E.g,, Company V on Vertica has 3000 permanent tables and 5-7000 temporary ones.
  • Vertica rightly points out that temporary tables are also important for ELT (Extract/Load/Transform).
  • Vertica further says that single-node OEMs such as security appliance vendors use lots of temp tables.

*Company V = one of the more prominent vertical-market application providers.

In other Vertica highlights:

• It sounds as if 4.0 is the first Vertica release with what I would regard as serious workload management.
• While Vertica has stored and retrieved Unicode since Vertica 3.5 or so, 4.0 will be the first Vertica release in which Unicode is sorted and collated properly.
• Stored-procedure-like functionality is still a future for Vertica.

• Personally selected annual lists of 12 “Most influential” companies and 36 “Companies to watch” in analytics- and database-related sectors.
• Made it clear that these are his personal selections.
• Nonetheless has called it an Editors’ Choice list, rather than Editor’s Choice.

(Actually, he’s really called it an “award.”)

People advising Doug — who come to think of it actually are Contributing Editors to Intelligent Enterprise or something like that — included Cindi Howson, Seth Grimes, three others, and me.

And if past is prologue, I will now get a flood of PR emails calling my attention to this award that I already have both participated in and blogged about.

As usual, the sense:nonsense ratio on these lists was pleasingly high. Analytic DBMS vendors cited included IBM, Microsoft, Netezza, Oracle, Sybase, and Teradata in the “Most influential” group, with Aster, Greenplum, HP, Infobright, and Vertica among the “To watch” crowd. It’s tough to argue with those selections, whose most questionable element is probably the not-ridiculous supposition that HP could do something interesting over the coming year. Cloudera and Intersystems also made the list, deservedly.

All three of QlikTech, Tableau, and TIBCO made the list, which is appropriate given the potential for and interest in interactive data exploration technology.  The BI majors, independent or otherwise, were all on as well. In text mining, Doug included Attensity and Clarabridge, which I think is exactly right. (Plus OpenCalais.)  Upon reflection, I probably should have nominated Mark Logic, even though most of its business is non-enterprise; but hey, nobody’s perfect, and the same goes for lists. Open source was well represented, with Apache, Actuate, Jaspersoft, Eclipse, Infobright, Nuxeo and R all being cited (but not Ingres or Pentaho). Kalido made the list, with my endorsement, their silly I-CASE like marketing messaging notwithstanding.

Speaking of imperfections — there only are a few category names, and so category assignments can be pretty bizarre. (In an ideal world, middleware wouldn’t be included under “enterprise applications”.) Greenplum hasn’t really “extended” its DBMS with a “cloud” option. As much as I’d like Netezza to be more influential than SAP, that’s probably not the best way to rank them. And there are a number of “This company is on a roll!” kinds of comments that I wouldn’t necessarily endorse.

But those are all nitpicks. On the whole, it’s another nice job.