Let’s try eight categories instead. While no categorization is ever perfect, these each have at least some degree of technical homogeneity. Figuring out which types of analytic database you have or need — and in most cases you’ll need several — is a great early step in your analytic technology planning. 

Enterprise data warehouse (Full or partial)

• Kinds of data likely to be included: All, but especially operational
• Likely use styles: All
• Canonical example: Central EDW for a big enterprise
• Stresses: Concurrency, reliability, workload management

The enterprise data warehouse (EDW) ideal says that you copy all your data into one place, and drive all decision-making from there. Full EDWs are pipedreams. Still, a partial EDW makes sense for most large enterprises, and many indeed already have one. The first product lines to consider for classical EDWs are Teradata, DB2, Exadata, and maybe Microsoft SQL Server, especially if you’re going to stress concurrency and/or operational use cases.

Traditional data mart

• Kinds of data likely to be included: All
• Likely use styles: Business intelligence, budgeting/consolidation, investigative
• Examples: Reporting servers, planning/consolidation servers, anything MOLAP, etc.
• Stresses: Performance, concurrency, TCO

Whether or not you have something like an enterprise data warehouse, it’s common to have lighter-weight data marts as well. A traditional data mart might drive reports and dashboards. Or it might be specialized for budgeting, planning, and/or consolidation.  Some investigative analytics may be in the mix as well.

Any DBMS that can support an EDW can also support a data mart, but it may not be the most cost-effective way to do so. Columnar DBMS might have more attractive performance and TCO (Total Cost of Ownership); the same goes for Netezza. Some of them — e.g. Sybase IQ and Vertica — have excellent track records in concurrent usage as well. Ted Codd pushed what amounts to MOLAP (Multidimensional OnLine Analytic Processing) systems for these use cases. But relational DBMS commonly do a better job, which is one reason most major MOLAP products have wound up at RDBMS companies.

Investigative data mart — agile

• Kinds of data likely to be included: All, especially customer-centric
• Likely use styles: Investigative
• Canonical example: A few analysts getting a few TB to examine
• Stresses: Ease of setup/load, ease of admin, price/performance

Besides the traditional data mart, there are at least two other kinds. Both are focused on investigative analytics, but they’re differentiated by database size.

If you have just a few analysts,* looking at no more than a few terabytes of data (perhaps even just some gigabytes) — and if that data is “single-subject” and fairly homogenous — your watchwords should be “cheap”, “easy”, and “fast”. You don’t need to invest in much hardware, in expensive software, in much administrative effort (the analysts can be their own DBAs),  nor should you endure much set-up time. Just grab a product, grab some data, and start running queries (or extracts into the statistical tool of your choice).

*If you have dozens or even hundreds of analysts hitting the same database, you’re probably back to the more concurrency-oriented scenarios outlined above.

Infobright is often cost-effective among columnar analytic DBMS. Other vendors might cut you a price break as well. If you have multiple terabytes of data, don’t rule out Netezza’s lowest-end products (even if they’d really rather sell you something bigger). Or, if you’re in the sub-terabyte range, maybe you can get by with an in-memory BI tool such as QlikView, and not do anything special on the DBMS side at all.

Investigative data mart — big

• Kinds of data likely to be included: All, especially customer-centric, logs, financial trade, scientific
• Likely use styles: Investigative
• Canonical example: Single-subject 20 TB – 20 PB relational database
• Stresses: Performance, scale-out, analytic functionality

But if you’re looking at tens of terabytes of relational data, or even more, you really do have a “big data” problem. Performance and scalability are major challenges, usually best addressed by MPP (Massively Parallel Processing) systems, such as Netezza, Vertica, Aster Data, ParAccel, Teradata, or Greenplum. Performance POCs (Proofs Of Concept) are a big part of the buying process. Vendor price negotiations are crucial too.

Actually, in the low tens of terabytes you might be able to get away with a shared-disk system that has excellent compression — e.g., columnar products like Sybase IQ, Infobright, or SAND, rather than just Vertica and ParAccel.

Assuming you have affordable, scalable query performance, the competitive differentiator can switch to additional analytic functionality. Aster, Netezza, ParAccel, Vertica, and Greenplum either offer full analytic platforms, or seem to be on the path to doing so. Teradata, which now owns Aster Data, offers substantial built-in analytic capability in its traditional products as well, and the same goes for Sybase IQ.

Continued in Part 2, where we cover some of the more difficult use cases.

• Vertica has a bunch of customers, including seven with 1 or more petabytes of data each.
• Vertica has progressed down the analytic platform path, with Monday’s release of Vertica 5.0.

And of course you know:

• Vertica (the product) is columnar, MPP, and fast.*
• Vertica (the company) was recently acquired by HP.**

*Similar things seem true of ParAccel, but most of the other serious columnar analytic DBMS aren’t actually MPP (Massively Parallel Processing) yet. More precisely, they have  shared-everything architectures, especially on the storage level.

** Vertica says it has a “staggering” pipeline now that it’s been with HP for a few months.  I also gather that the post-merger HP/Vertica appliance product line formally rolled out last week.

As for product maturity:

• Vertica 4.0 cleaned up a lot of stuff.
• Vertica 5.0 goes further in a variety of areas, notably clustering administration and database tuning/design.

But here’s something I hadn’t fully realized — Vertica claims concurrent usage as a competitive strength. By this I mean:

• Vertica says that it has some customers with 1000s of users, in BI/dashboarding kinds of applications.
• Vertica asserts it can support 1000 users on a single appliance rack.
• Vertica tries to drive POCs (Proofs Of Concept) towards testing concurrency.

This is all consistent with a user example I blogged about last year.

That said, while Vertica introduced respectable workload management features in Vertica 4.0, its main claim to concurrency is simply speed — if each query ends quickly, you never have to execute all that many of them at once.

Anyhow, there will (or at least should be) articles written about Vertica 5.0, and I may not be that easy to find for comment, what with Enzee Universe and all. So here are a few Vertica soundbites:

• Having seven petabyte-level commercial users is an impressive testament to Vertica’s scalability. I think only Teradata could best that number among analytic DBMS, unless you want to count Hadoop/Hive.
• Vertica’s analytic platform capabilities are new, and initially not as rich as Aster Data’s or Netezza’s, especially in the area of language support. But they’re a good first step.
• Judging by the examples of EMC/Greenplum and IBM/Netezza, Vertica’s honeymoon period at HP is likely to last for a while. (Edit: That said, not all is peachy at EMC/Greenplum.)

Sybase said that Sybase IQ had > 2000 direct customers and >500 indirect customers (i.e., end customers of OEMs). That’s counting by customers; I know from prior discussions that Sybase IQ is running at close to two installations per customer. I also believe that Sybase counts different divisions of the same large enterprise as separate customers.

Vertica cited a figure of 500 customers as of April (end Q1?), which is close to 600 now, about 40% or a little more direct. The difference between this and a 2010 year-end figure of 328 is not only new sales, but also slow reporting by OEMs.  One cool figure — a single OEM reported 82 end sales in a single (quarterly?) report. And a number of those direct customers are substantial; Vertica’s customer logo page features lots of telcos, lots of internet companies, and the national operation of Blue Cross/Blue Shield.

Pay no attention to small inconsistencies in the number of Vertica direct customers (250 at year-end, no more than that now); Colin Mahony just estimates these numbers for me from memory, and minor inaccuracies are quite excusable.

Even cooler — Vertica reports 7 customers with a petabyte or more of user data each. About 5 of the 7 are obvious-suspect big-name firms; but unsurprisingly, those big names are NDA. I did secure permission to say that there are 2 telecom companies, a mobile gaming vendor, another internet company, and 3 financial services outfits of various kinds.

SAND Technology reported >600 total customers, including >100 direct. Since SAND has been around since the 1990s, those aren’t great average annual figures, but they’re probably more than many people (including me) thought.

Infobright reported around 200 total paying customers, 130 direct. There are surely a lot more users of open source Infobright, but precise numbers are of course hard to come by.

If I asked ParAccel in the April go-round, I’ve misplaced their answer, but back in October the figure was >30 customers, 2 of them over 100 terabytes. I’ve seen published figures of 40+ for ParAccel since.

• This is a list of Monash Advantage members.
• All our vendor clients are Monash Advantage members, unless …
• … we work with them primarily in their capacity as technology users. (A large fraction of our user clients happen to be SaaS vendors.)
• We do not usually disclose our user clients.
• We do not usually disclose our venture capital clients, nor those who invest in publicly-traded securities.
• Included in the list below are two expired Monash Advantage members who haven’t said they will renew, as mentioned in my recent post on analyst bias. (You can probably imagine a couple of reasons for that obfuscation.)

With that said, our vendor client disclosures at this time are:

• Aster Data
• Cloudera
• CodeFutures/dbShards
• Couchbase
• EMC/Greenplum
• Endeca
• IBM/Netezza
• Infobright
• Intel
• MarkLogic
• ParAccel
• QlikTech
• salesforce.com/database.com
• SAND Technology
• SAP/Sybase
• Schooner Information Technology
• Skytide
• Splunk
• Teradata
• Vertica

That list includes the two I’m obfuscating, plus one more who just emailed to say a signed renewal contract is arriving this week. It does not include others who, less concretely, have said they will sign up soon.

Also, I guess there’s a bit of a gray area for Tableau. As far as I’m concerned, I’m doing an upcoming co-sponsored webinar just for Monash Advantage member Aster Data. Indeed, I declined to contract with or bill Tableau directly for its share,  because I had no good way to do that paperwork. But even so, Tableau is a cosponsor, was involved in the planning discussions and, behind the scenes, is surely footing part of the bill.

Note: Links to Gartner Magic Quadrants tend to be unstable. Please alert me if any problems arise; I’ll edit accordingly.

In my comments on the 2008 Gartner Data Warehouse Database Management Systems Magic Quadrant, I observed that Gartner’s “completeness of vision” scores were generally pretty reasonable, but their “ability to execute” rankings were somewhat bizarre; the same remains true this year. For example, Gartner ranks Ingres higher by that metric than Vertica, Aster Data, ParAccel, or Infobright. Yet each of those companies is growing nicely and delivering products that meet serious cutting-edge analytic DBMS needs, neither of which has been true of Ingres since about 1987. 

The general list of “market forces, end-user expectations and vendors’ resulting solution approaches” at the top of the 2010 Gartner Data Warehouse Database Management System Magic Quadrant article is a mixed bag. Following Gartner’s order, I’ll address those first, and particular companies cited afterwards. Specific items and comments include:

• “Increased demand for optimization techniques and performance enhancement.“ Gartner seems to be saying that data warehouse DBMS buyers want lists of specific, esoteric performance features. Well, buyers always want their DBMS to run fast, and they’d like the products to be mature enough to have been through a few rounds of Bottleneck Whack-A-Mole, but otherwise I’m not sure I’d put that at the top of my list.
• “The argument made by purchasing departments that buying power increases when dealing with a single, incumbent vendor.“ I agree that vendor consolidation and account control are a huge part of the Oracle, Microsoft, IBM and even Teradata stories. (Vertica can prove it’s 10X more price-performant than Oracle and still not get the business.) But it’s not just about price negotiations; once annual maintenance is included, one has to squint pretty hard to see Oracle as a low-cost alternative. Also important is reducing the number of total product-specific skill-sets needed on the IT staff.
• “Prepackaged, prebalanced warehouse environments delivered using data warehouse appliances.“ Yep. To varying extents, Oracle, Microsoft, Teradata, and IBM are all committed to designed-hardware strategies.
• “Expectations for the delivery of on-site POCs.“ Honestly, not as many buyers insist on on-site Proofs of Concept as should. Still, Oracle is shameful in its reluctance to do them. (Teradata tries to avoid them too, for obvious reasons of expense, but is much more gracious about capitulating when the buyer insists.)
• “Cost controls and data warehouse performance management.“ See next comment.
• “Demands for delivering a fully mixed workload.“ I’d have phrased the workload management and administrative tools points rather differently than this, but so be it.
• “Demands for departmental analytics delivered quickly via data marts.“ Agreed. Data-mart-only installations are a huge part of the market of the analytic DBMS market. Data mart spin-out is also important.
• “Wider indexing and fast performance within clusters of data, delivered via column-based solutions.“ This bizarrely seems to conflate column stores and parallel processing (both of which are of course highly important).
• “A wave of new data warehouse implementers seeking fast-track, low-risk delivery.“ Well, yes. Netezza noticed that quite some years ago. And by now the long-gestation EDW (Enterprise Data Warehouse) is widely disliked.
• “Global organizations seeking distributed solutions as potential architecture.“ If this is the MPP point, it’s oddly phrased. If this is a suggestion that data warehouses should be partitioned across wide-area networks, it’s just plain odd. If it’s a reiteration that departments like to control their own data marts, I agree. And if it’s a comment on keep-data-in-the-country privacy laws, it could be the most prescient thing Donald Feinberg has said in many years.

Long though it is, that list of general items and issues for the 2010 Gartner Data Warehouse Database Management System Magic Quadrant has some gaps. Most glaringly, I don’t see any references to advanced analytics in general, or even to the specific case of integrated predictive analytics. There’s also nothing about solid-state memory or other storage-technology considerations, although in fairness it’s still early days for much of what vendors conceive of as competitive differentiation in those respects.

Here are some vendor-specific comments on the 2010 Gartner Data Warehouse Database Management System Magic Quadrant:

• It’s pretty bizarre to compare 1010data to database.com or Microsoft Azure. Kognitio would be a better choice. So would cloud-hosted instances of Vertica, Aster Data nCluster, or others.
• Gartner’s comments on Aster Data and nCluster are actually pretty reasonable.
• Gartner’s comments on EMC/Greenplum are a bit Kool-Aid-drinky, and don’t account for the inevitable flailing that occurs right after an acquisition. But otherwise they’re pretty reasonable.
• I don’t take IBM’s super-comprehensive-all-inclusive architectural stories as seriously as Gartner does.
• I don’t take Netezza’s small stable of OEM partners as seriously as Gartner does. I also don’t share Gartner’s optimism for the continuation of Netezza’s NEC partnership in the face of IBM’s Netezza ownership.
• I’m even more skeptical about illuminate than Gartner is.
• I’m delighted that Gartner has adopted my phrase machine-generated data (Infobright is one of several firms pushing that one).
• “Only open-source column-store DBMS” is a bit exaggerated, but Infobright is indeed the only one with serious traction, or offered by a serious analytic DBMS vendor.
• What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.
• While Gartner’s write-up of Kognitio is a bit confused, that’s excusable. Kognitio’s strategy changes often.
• I’m not persuaded by the claim of low Microsoft TCO. The days when Microsoft’s tools were vastly better than the competition’s are long gone. And using an OLTP DBMS for data warehousing generally takes more people effort than using something more purpose-built.
• Gartner is right to ding Oracle for high prices, high people costs, and unwillingness to do onsite POCs.
• Gartner is right that Exadata is a huge improvement over non-Exadata Oracle data warehousing.
• Gartner is right to suggest that Exadata can easily handle data warehouses over 20 terabytes in size, but wrong to suggest that software-only Oracle also can. Just because the pain is less than it was with earlier releases of Oracle doesn’t mean it isn’t still bad.
• Gartner’s comments on ParAccel are pretty reasonable.
• Gartner’s comments on compression in connection with SAND make no technical sense (tokenization is a key form of columnar compression, not an alternative to it). Also, SAP’s acquisition of Sybase is a business challenge for SAND, not a technical one.
• Unless I’m forgetting something, Sybase IQ has no more in-database data mining than any other Fuzzy Logix partner does.
• Gartner failed to note that, like other DBMS dating back to the 1990s and before, Sybase IQ is more complex to administer than some newer products are.
• Gartner’s take on Teradata is pretty reasonable.
• Gartner’s take on Vertica, while sloppy, is basically sensible. However, Gartner failed to note that Vertica is a laggard in non-query analytics. (I am sure those deficiencies are being addressed, but Vertica’s competitors are moving ahead as well.)

One big part of PADB 3.0 was an analytics extensibility framework. If we match PADB against my recent analytic computing system checklist, 

• ParAccel is proud of PADB’s coverage in analytics-oriented SQL standard capabilities.
• I’m not aware of any PADB SQL goodies that go beyond the ANSI standards.
• PADB has a pretty flexible framework for user-defined functions (UDFs). In particular, ParAccel asserts this framework is even better than MapReduce, because it lets you do more steps at once, although I have trouble convincing myself that that makes sense in an important way.
• Anyhow — like Aster Data, ParAccel asserts that the same framework on which its DBMS is built has now been exposed to people wanting to write other kinds of analytic processes. (But Aster Data describes its framework as being pretty straight MapReduce.)
• All of PADB’s analytic process execution capabilities are subsumed in the UDF framework.
• PADB does not yet contain much in the way of fully parallelized analytic libraries. Exception: Like many of its competitors, ParAccel has a Fuzzy Logix partnership.
• ParAccel hasn’t focused yet on analytic development ease of use. (And that’s putting it mildly.)
• The only language now supported for PADB analytics is C++. ParAccel promises more language support, with (at least) Java and R coming in the summer.
• In line with its extreme focus on speed, ParAccel for now offers only in-process analytics execution.
• In a near-future release (just heading into QA now), ParAccel promises that PADB UDFs will be very flexible in terms of what kinds of memory structures it manages. However, if you want a structure to persist past the end of a query, you need to map it to a row architecture.
• ParAccel’s workload management is still primitive — just a short-query bias, rather than any kind of explicit prioritization. Hence, the question as to whether workload management extends to analytic process execution is fairly moot.

In other news, ParAccel’s Bala Narasimhan wrote:

Historically, an analyst who wants to spin up a new data mart with all of this data will have to wait for a number of days for the data copy to be made available. Instead, if you deploy PADB with a SAN that has fast and efficient snapshot and cloning capabilities, you can spin up multi-TB data marts in seconds.

That turns out to be not quite as ridiculous as it sounds. The scenario is:

• You’re using storage-area network technology with a copy-on-write option.
• You use the SAN’s copy-on-write option to make a second virtual copy of the database in question (or of certain tables/files/blocks from it).
• You point a separate instance of PADB at it, either on a separate cluster (“in seconds” — yeah, right) or else via virtualization (e.g. VMware  — that sounds more plausible).

Hmm. I have no actual knowledge of this, but it sounds like a capability that EMC should also offer soon, given the historical Greenplum focus on data mart spin-out.

*Main examples of “conventional” = filtering, simple aggregrations.

The point of such functionality is generally twofold. First, it helps you execute analytic algorithms with high performance, due to reducing data movement and/or executing the analytics in parallel. Second, it helps you create and execute sophisticated analytic processes with (relatively) little effort.

For now, I’m going to refer to an analytic RDBMS that has been extended by advanced-analytics functionality as an analytic computing system, rather than as some kind of “platform,” although I suspect the latter term is more likely to wind up winning.  So far, there have been five major categories of subsystem or add-on module that contribute to making an analytic DBMS a more fully-fledged analytic computing system:

• SQL extensions. Examples include SQL-2003 analytics (notably windowing), or vendor-specific temporal functionality.
• A framework for UDFs (User-Defined Functions) to further extend SQL. At its core, a relational DBMS is a big SQL interpreter. SQL, while powerful, only does a limited number of things. User-Defined Functions are new predicates in the SQL language that do additional things.
• An execution engine for analytic processes that is less coupled to the SQL engine than a pure UDF framework might be. The two main approaches are MapReduce (e.g. Aster Data) and general C++ libraries (Netezza, ParAccel).
• Libraries of pre-built analytic processes. Commonly included are statistics, (other machine learning), general linear algebra, and Monte Carlo analysis. Some of these functions are fully parallelized (perhaps tens per vendor). Others just play nicely with the vendor’s execution framework, in that a separate copy can be run on each node (up to thousands per vendor, for those who bring in open source statistics libraries).
• Development tools such as integrated development environments (IDEs). Aster keeps trying to convince me that having built a nice Eclipse IDE is a major competitive differentiation.

The most structural or architectural are the UDF framework and the non-UDF analytic execution engine.  But even those are in essence add-on modules, which means that pretty much any vendor can do any part of them if they invest enough resources in the effort. So I expect considerable convergence over time as the industry and market discover which capabilities are or aren’t particularly useful.

When I’m being told about an analytic DBMS that supposedly has evolved into an analytic computing system, some of my top-of-mind questions are:

• How does the execution work? UDFs? MapReduce? Something else? What forms can the inputs and outputs of a UDF take? And by the way, what’s your complete list of MapReduce integration possibilities?
• What languages are currently supported? The obvious choices  are C++ (if that’s the style of execution engine); SAS, R, or other statistical languages; and anything that is commonly associated with MapReduce (Java, Python, et al.).
• In-process, out-of-process, or both? In-process runs faster; out-of-process is more stable, in that the advanced-analytics part can crash without bringing down the whole DBMS. Even if other languages are available out-of-process, C++ might be the only in-process choice. If you don’t have out-of-process execution, you may not be serious about offering really broad analytic capabilities.
• Is there anything special about your library of pre-built, fully parallel processes? For example, I like Netezza’s broad approach to linear algebra, Greenplum’s sparse vector manipulation, and a number of Aster Data’s packages. ParAccel’s list looks interesting too, although I haven’t grilled them about what is or isn’t fully parallel.
• How does the associated memory management work? Can you create temporary data structures that survive longer than the process that spawned them? (Those can be useful for various kinds of lookup table.) Do they have to be tabular? (Graphs and other alternatives can be useful.) And by the way, do UDFs or other processes have enough RAM under their control to run efficiently?
• How is workload management handled? Hopefully, everything that runs on the same cluster is handled by one integrated workload management system. Vendors for whom that isn’t true today should have a clear road map for getting there, because the alternative is something of a mess.

Please note what I’m not including in this discussion — the integration of DBMS and fairly ordinary business intelligence. That may have virtues, for reasons of price or performance, and the virtues may grow as in-memory BI and/or data management capabilities evolve. But for the foreseeable future, BI/DBMS integration is a fairly separate matter from the integration of analytic DBMS with sophisticated investigative analytics.

Evidently Mike Olson of Cloudera is still telling the machine-generated data story, exactly as he should be. The Information Arbitrage/IA Ventures folks said something similar, focusing specifically on “sensor data” …

… and, even better, went on to say: 

Privacy is dead.
What do we consider to be the boundaries of privacy, especially with respect to items like medical data? In a data privacy-free world, should we be regulating data usage instead? How do we deal with asymmetric access to our personal data, e.g., how is it that insurance companies claim the right to our personal information?

Obviously, my answer to the second question is Yes!!!!

Also from Hadoop World — Dave Menninger, now an analyst, reports on some Hadoop metrics:

How big is “big data”? In his opening remarks, Mike shared some statistics from a survey of attendees. The average Hadoop cluster among respondents was 66 nodes and 114 terabytes of data. However there is quite a range. The largest in the survey responses was a cluster of 1,300 nodes and more than 2 petabytes of data. (Presenters from eBay blew this away, describing their production cluster of  8,500 nodes and 16 petabytes of storage.) Over 60 percent of respondents had 10 terabytes or less, and half were running 10 nodes or less.

That eBay comment was particularly interesting.

A while back, Doug Henschen noted that Netezza flagship reference Catalina Marketing is now at 2.5 petabytes. Most of that is in one 600 billion row table. Oddly, the article talks of the Netezza/SAS partnership accelerating model-building via in-database scoring (not modeling) technology. Doug also wrote of a lot of analytic DBMS replacements, including:

• Microsoft by ParAccel
• Oracle by Aster Data, IBM, Oracle Exadata, probably Netezza, and probably Hadoop
• Netezza by Greenplum
• IBM by Teradata

Carl Olofson pointed out on Twitter that DataScaler was an in-memory database technology just bought by Oracle. This inspired me to google on them, and I found a sparse DataScaler CEO blog. I link it because of an amusing juxtaposition — the second-to-last post says, in effect, “We make appliances and we recommend all these awesome technology design partners who helped us design the hardware,” while the very last post says “Designing our own hardware was a mistake.”

Fred Holahan is now VP of Marketing at VoltDB, which is a lesson to me about giving free consulting … Anyhow, Fred tells me that VoltDB has about a dozen users on their way to production, some of whom are headed to being VoltDB paying customers, some of whom are not.

• ParAccel asserts that PADB is much faster than other analytic DBMS — even close competitors such as Vertica — on especially complex queries. “60-way joins” were mentioned. So was the flattening of correlated subqueries.
• ParAccel also claims industry-leading performance on simpler queries, but not by the same (or perhaps even particular large) margins.
• Mercifully, ParAccel no longer claims to have never, ever lost on performance in a customer evaluation. But it still says that is very close to being true.
• Major reasons ParAccel gives for PADB’s high performance include:
  • Like Vertica, Sybase IQ, and others, PADB uses a columnar architecture.
  • ParAccel thinks PADB’s newest query optimizer — fondly named Omne — is outstanding.
  • ParAccel’s PADB compiles its queries.
  • In general, ParAccel is just performance-obsessed.
• One could also mention:
  • ParAccel’s PADB runs smoothly in-memory, if that’s what you want.
  • ParAccel also offers a Flash option for PADB.
  • Like many other analytic DBMS vendors, ParAccel has created a custom networking protocol. (ParAccel has talked about that altogether too much in the past.)
  • Like Vertica, ParAccel’s PADB generally decompresses data as late as the particular compression scheme used allows. (Well, actually, that’s not one ParAccel mentions unless asked.)
  • ParAccel has long encouraged one to put part of one’s database on direct-attached storage as a kind of persistent cache, plus all of it on a storage-area network, because PADB can optimize its scans to go against both physical stores.
  • ParAccel’s PADB does encryption a block at a time, rather than a row at a time, so there’s very little overhead to using the encryption feature.
• ParAccel says that PADB has no indexes, materialized views, etc., notwithstanding that I heard something different from Barry Zane a few years ago. This is the basis for ParAccel’s claim that no tuning (or at least very little) is required, or indeed even possible …
• … and similarly, it is the reason ParAccel encourages prospects to do ad-hoc queries in their POCs (Proofs Of Concept), at least when Vertica is the competitor.
• However, ParAccel’s PADB has rather complex initial set-up. This has been the basis for widespread skepticism about ParAccel’s “no tuning” claim. ParAccel is working to automate that away, but admits to being only part-way through the process.
• Highlights of ParAccel’s data writing strategy include:
  • PADB sends data transactionally to disk.
  • PADB usually sends data to disk a block at a time, because it is coming in fast enough for that to work out (either due to bulk load or streaming).
  • PADB is append-only …
  • … so PADB has a garbage-collection mechanism called Vacuum. Right now Vacuum has to be started manually, but doesn’t block reads and writes; full background garbage collection is of course a roadmap feature.
  • As is natural for append-only systems, ParAccel’s PADB has MVCC (MultiVersion Concurrency Control) and snapshot isolation.
• Name a compression method, and PADB probably has it — 13 in all by ParAccel’s count, including dictionary/token, run-length encoding, Delta, LZ, and so on.

Tracking ParAccel’s customer success has long been difficult. The 2009 Gartner Magic Quadrant claim of ~20 ParAccel customers seems odd to everybody, including ParAccel. ParAccel’s own reporting of customer wins around then was quite confusing. And ParAccel’s customer count a year before that was extremely low. But ParAccel’s Michael Weir just rounded up some figures for me, namely:

• ParAccel has 30+ revenue-recognized customers, not counting OEMs, OEMs’ customers, or paid POCs.
• 2 ParAccel customers have > 100 TB of user data.
• 7 ParAccel customers have > 10 TB of user data.
• The largest ParAccel cluster is 28 nodes and growing.

Naturally, Michael went on to note that even relatively small databases can have high value.

One last note: ParAccel has approximately 78 employees.

• Aster Data showed me a lot of customer names and deal sizes, across a bunch of industries (mainly enterprise rather than web). Yes, Aster’s market success is for real. (But almost all those details are NDA.)
• Sybase’s product plans for IQ are pretty impressive. (But the most interesting parts are, you guessed it, NDA.)
• I’ve kissed and made up* with ParAccel, now that they’ve replaced their CEO, replaced their marketing chief, and stopped the worst of the marketing nonsense I used to complain about. ParAccel has some interesting plans for ParAccel 3.0 which are, naturally, NDA.
• The Peoplesoft guys are doing it over again at Workday. Only this time, their platform isn’t a relational DBMS. Rather, it’s an in-memory, completely object-oriented data model, with disk used only on a “Just in case the power ever goes out” basis. (Thankfully, nothing at all about our conversation was NDA.)
• I’m finally feeling good about Northscale’s memcached-compatible persistent store Membase. The main reason is that they showed me a near-term path to interfaces that are richer than key-value. Also, Todd Hoff reassured me that even pure persistent memcached has a place.
• Rumor says that even the one app for which Facebook was using Cassandra — in-box search — has been decommissioned. On the other hand, numerous other scale-0ut DBMS (SQL or otherwise) seem to have Facebook footholds. But details are — all together now! — NDA.

*If you know ParAccel’s new marketing exec Michael Weir, you surely guessed I mean that only in a figurative sense.

From elsewhere:

• Daniel Abadi offered his analysis of Kickfire’s demise. In general I agree, but Daniel neglected to mention one hugely important factor — the chicken-egg negative effect of Kickfire’s lack of market or marketing traction. Customers were extremely reluctant to buy from Kickfire because they perceived, correctly, that Kickfire’s survivability was far from assured.
• While the InfiniDB forums suggest that there are at least a couple of production users of Calpont’s free InfiniDB, Calpont seemingly has a long way to go to be even as successful as Kickfire. But Calpont does have a bit of money to spend on lead generation; maybe some day they’ll even have actual customers.
• In a response to a question I messaged over, XtremeData tells me they have actual customers now. Press releases to follow.
• The admiration for the job Mark Hurd did at HP is in my opinion overstated. Sure, the financial/operational management appeared to work, but HP did little on Hurd’s watch to strengthen its reputation or customers’ loyalty. In particular:
  • HP’s analytics efforts have accomplished little.
  • HP’s data warehouse appliance efforts have failed pathetically.
  • From what I hear, HP’s execution in its Exadata partnership was not good.
  • HP’s server business in general is distinguished mainly by HP being a big company.
  • HP’s EDS acquisition has been rocky, not that EDS was sailing so smoothly on its own beforehand.
  • HP’s success in PCs amounts to “arguably, HP sucks a little less than the other guys”.
  • HP’s elite reputation is long gone (admittedly, for the most part that predates Hurd).
• Doug Henschen evidently favors really strong intellectual property protection for software, even forbidding plug-compatible reverse engineering. I agree with Doug up to the point that it should be forbidden to copy proprietary software, but I don’t see why he (or a court) would view such behavior as copying.