• In general, I regard Gartner Magic Quadrants as a bad use of good research.
• Illustrating the uselessness of — or at least poor execution on — the overall quadrant metaphor, a large majority of the vendors covered are lined up near the line x = y, each outpacing the one below in both of the quadrant’s dimensions.
• I find fewer specifics to disagree with in this Gartner Magic Quadrant than in previous year’s versions. Two factors jump to mind as possible reasons:
  • This year’s Gartner Magic Quadrant for Data Warehouse Database Management Systems is somewhat less ambitious than others; while it gives as much company detail as its predecessors, it doesn’t add as much discussion of overall trends. So there’s less to (potentially) disagree with.
  • Merv Adrian is now at Gartner.
• Whatever the problems may be with Gartner’s approach, the whole thing comes out better than do Forrester’s failed imitations.

*At the time of this posting, I don’t yet have a link. However, I expect that to change quickly, and I plan to edit this paragraph accordingly. If nothing else, I hope people will drop links into the comment thread.

Specific company comments, roughly in line with Gartner’s rough single-dimensional rank ordering, include:

• The Gartner Magic Quadrant’s comments on Teradata seem pretty fair. I don’t think I’m much in disagreement when I say:
  • Teradata has the richest, most mature analytic DBMS offering.
  • Teradata has an outstanding track record both for managing large data volumes and for high-concurrency mixed workloads.
  • Aster Data was a cool Teradata acquisition, even if Teradata/Aster synergies or integration have been nominal to date.
  • Teradata still needs to get out of its own way in marketing, positioning, packaging, and/or defining its premium-priced system vs. its more moderately-priced alternatives. Indeed, as necessary as this approach may have been to fending off encroachments by Netezza and others, what Teradata really needs to do is evolve to a more pick-your-own-node-combination mix-match kind of offering.
• Gartner has talked with a lot of Oracle Exadata users who say that the product works; Gartner has also stopped beating Oracle up for its previous policy of almost never doing onsite POCs (Proofs of Concept); both parts of that ring true with me. But Gartner also rightly dings Oracle for various issues in cost and cumbersomeness. Overall, while I agree there are organizations for which Oracle should indeed be a top-ranked choice, there are many others who shouldn’t put Oracle on their short list.
• Third in the Gartner MQ rankings is IBM.
  • Gartner gets so caught up in reciting the names of various IBM product offerings that it neglects to say much good about DB2 itself. (I tend to have a similar problem.)
  • But Gartner does mention concurrency as a strength. I agree, especially if we presume that that was a reference to DB2 rather than Netezza.
  • Gartner cites Netezza’s post-acquisition annual growth rate as 30%. Gartner seems to think this is a good number. I disagree, but in Netezza’s defense, it has had to endure IBM’s post-acquisition on-boarding process.
• Arguably fourth in the Gartner Data Warehouse Magic Quadrant rankings is EMC/Greenplum.
  • In general, Gartner likes the taste of Greenplum Kool-Aid.
  • Gartner neglects to ding Greenplum for concurrency challenges, which I view as an oversight given Gartner’s general stress on that area.
  • Gartner does ding Greenplum for support challenges.
  • Gartner neglects to praise Greenplum for true hybrid row/columnar data management, a feature shared by Teradata and Vertica, among others, but not by Oracle, DB2, or Netezza.
  • Gartner located a half-petabyte Greenplum database. This doesn’t surprise me, even though Greenplum has frequently made exaggerated claims about large-size database successes in the past.
  • Gartner reports a >400 figure for Greenplum customers, which is plausible.
• In its first deviation from strict one-dimensional rank ordering, the Gartner Magic Quadrant ranks Sybase ahead of Greenplum in completeness of vision but behind in “ability to execute”.
  • If that were the other way around, it might make more sense. Greenplum promises anything and everything you might ever want for analytic data management or the associated analysis; but Sybase has vastly more analytic DBMS users than Greenplum does, running a variety of demanding workloads.
  • Gartner appears to think that Sybase IQ requires less database administration than I do.
  • Gartner seems concerned that SAP will position HANA and Sybase ASE as, between them, the only DBMS you’ll ever need, casting doubt on Sybase IQ’s future. I wouldn’t worry about that if you have a problem you want to solve today.
• The Gartner Magic Quadrant for Data Warehouse Database Management Systems ranks Microsoft sixth overall, despite noting that there isn’t a single production reference for Microsoft’s Parallel Data Warehouse. In support of this ranking, it for example cites the compression feature, which distinguishes Microsoft SQL Server from no other product on the list except Kognitio. If you have such an undemanding data warehousing problem that many different analytic DBMS could meet your needs, there’s a good chance Microsoft SQL Server can also do the job; and if you’ve bought into the Microsoft technology stack, you might as well keep going down that path. Otherwise, I don’t know why somebody should adopt Microsoft’s offering at this time.
• Seventh along the main diagonal path in the Gartner Magic Quadrant is HP Vertica. I’d rank Vertica higher than that, but in fairness I note two execution concerns. First, HP has a lousy track record, both in acquisitions and in data warehousing/analytics. Second, Vertica is bad about answering my email. Anyhow, Gartner doesn’t seem to have given Vertica credit either for its full customer count or for the multiple petabyte-scale databases Vertica runs.
• 1010data is an outlier, with Gartner noting that it only partly fits in with other “Data Warehousing Database Management” companies, and hence kind of confessing that 1010data on the Magic Quadrant is somewhat arbitrary. Stuff like that is bound to happen, given the inherent difficulties of defining market categories. Anyhow, my thoughts on 1010data include:
  • I’m nervous about the fact that 1010data doesn’t actually control its own DBMS technology, but rather relies on old code from the small private company KX Systems.
  • There are three main reasons to consider 1010data:
    • You want to enter the data mart outsourcing business in a casual way, and you like its SaaS offering.
    • You want to engage in stakeholder-facing analytics in a casual way, and you like its SaaS offering.
    • You love 1010data’s particular set of interactive analytic features and performance.
• Back to the main path winding along the Gartner Magic Quadrant main diagonal — next up is ParAccel. While I question some of the peripheral comments, I agree with Gartner’s core messages that:
  • ParAccel, the product, is blazingly fast in certain use cases.
  • ParAccel, the company, is dangerously small.
• Eighth on the Gartner MQ’s main path is Kognitio. This is too high. Kognitio positions itself as offering in-memory DBMS, yet stubbornly refuses to do any kind of data compression. That’s an awful combination of choices. As for using Kognitio’s data warehousing SaaS offering — why would you do that, when more modern products are available on a SaaS/cloud basis as well?
• Ninth in the Gartner Magic Quadrant main rankings is SAND.
  • The SAND section is not a triumph of Gartner accuracy. For example:
    • Gartner completely missed the errors in SAND’s reported customer counts.
    • Gartner refers to SAND as being “in existence for approximately nine years”, which is too low by at least a factor of 2.
    • Gartner says “SAND is a privately held company”, even though Merv knows better than that.
  • Otherwise, Gartner’s opinion on SAND seems to boil down to “Interesting technology and ideas, but dangerously small company.” I agree.
• Tenth and too low in the Gartner MQ main rankings is Infobright.
  • At least by some metrics (e.g. customer count), Infobright isn’t as dangerously small as ParAccel, SAND, Kognitio, et al.
  • That said, Infobright is small and focused on machine-generated data. So I wouldn’t be confident in Infobright’s future technology path for human-generated data use cases.
  • Infobright’s performance is uneven — blazing in cases where the Knowledge Grid helps, but not necessarily stellar by analytic DBMS standards when full table scans are called for.
  • I agree with Gartner that the possibility of Oracle/MySQL future shenanigans is a concern. But while the energy behind MySQL forking efforts doesn’t seem too great right now, I’d expect them to revive and offer a successful escape path if it seemed Oracle was going to indeed play hardball.
  • Also, given that it’s already an open source vendor, there are various kinds of assurances Infobright could give that would also help alleviate customer concerns.
• Actian, formerly Ingres, took a big tumble in Gartner’s rankings versus last year, when I simply wrote “What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.” I’m even a little harsher about Ingres/Actian’s DBMS products and prospects than Gartner is, but at least now we’re in the same ballpark.
• Along with Infobright, ParAccel, and SAND, Exasol appears to be another of the “good columnar technology/small company” crowd. As with other such products, one should be careful about fit-and-finish features that are missing today, as there is no assurance they’ll be added in a timely manner going forward.
• illuminate Solutions, which was on last year’s Gartner list, now appears to be an ex-company.

• Academic scientists like their software to be open source, for reasons that include both free-like-speech and free-like-beer.
• What’s more, they like their software to be dead-simple to administer and use, since they often lack the dedicated human resources for anything else.

Even so, I think that academic researchers, in the natural and social sciences alike, commonly overlook the wealth of commercial software that could help them in their efforts.

I further think that the commercial software industry could do a better job of exposing its work to academics, where by “expose” I mean:

• Give your stuff to academics for free.
• Call their attention to your free offering.

Reasons to do so include:

• Public benefit. Scientific research is important.
• Training future customers. There’s huge academic/commercial crossover, especially as students join the for-profit workforce.

The biggest issue is probably large-scale database management. There’s a feeling, permeating for example parts of the XLDB conference and the associated SciDB project, that data stores suitable for holding large amounts of data are either:

• Hadoop or
• Forbiddingly expensive.

I think that’s overstated. In particular:

• You can put >10 terabytes of machine-generated data (or any other kind) into Infobright and have it well taken care of; Infobright is open source.
• You can put >1 petabyte into [name redacted],* among others; [name redacted]* should be out soon with a generously free offering for academic users. Edit: That would be Vertica.
• Conventional relational queries, graph analysis, statistical analysis preparation and more can all be much faster in a good analytic DBMS than in alternative kinds of data stores.
• Integration between SQL and other analytic languages is ever improving, as analytic DBMS evolve into “analytic platforms“.

*My permission to use the name was yanked after this post was largely drafted. I’m sufficiently pleased with the forthcoming offering itself that I can’t get upset about the procedural confusion.

With a couple of exceptions, the statistics/predictive analytics situation seems more reasonable. Industry leaders such as SAS Institute and SPSS (now an IBM company) have engaged in varying degrees of academic outreach. R is in the process of crossing over from academia to business.

Unfortunately, I know next to nothing about Stata or, elsewhere in the technical languages area, Mathworks/Matlab. (Who knew that Mathworks was a $600 million company, local to my geographical area?)

One statistical tool that should perhaps be more present in academia is KXEN. KXEN seems to have some nice differentiation in not making you understand in advance which of your variables are most important. Econometricians and others with large numbers of independent variables might wish to take note.

If you think the true situation is nonlinear, and you’re trying to approximate it with linear models, you almost always have a large number of variables to consider. True, monomials in independent variables aren’t actually independent, but it might be interesting to pretend that they are and see if any insights fall out that could help in more rigorous analysis.

I’d further argue that, as part of neglecting commercial analytic DBMS, the scientific community in particular neglects the potential of integrated analytic platforms. Admittedly, the early leaders in that area — Aster Data, perhaps followed by Netezza (now an IBM company) — aren’t exactly priced in an academic-friendly way. But Vertica, EMC Greenplum, et al. are playing catch-up with analogous technology, and they’re more likely to offer appealing academic pricing.

There’s also the investigative analytics side of business intelligence, especially in the area of visualization/discovery. While Spotfire (now a TIBCO company) got much of its start in research-oriented areas, the otherwise more visible — no pun intended — QlikTech and Tableau don’t seem to have done much in academia. Datameer and yet-younger Hadoop-oriented business intelligence startups don’t seem to be doing much on the academic front either, more’s the pity.

Frankly, I think that most scientific analytic technology needs are also found in the business world.* That convergence will only get closer as businesses focus more on machine-generated data. Commercial software companies should pay more attention to scientists, and scientists should gaze out more often from their ramshackle, budget-constrained ivory towers.

*The converse isn’t as true. Businesses have issues not well reflected in science, derived (for example) from the complexity of their transactional schemas, or from office-politics considerations around “one version of the truth”.

Edit: Some links that seem relevant to this year’s XLDB program

• Zynga and LinkedIn
• Objectivity Infinite Graph
• eBay as of last year’s XLDB (the most expensive blog post I ever wrote, in light of Greenplum’s subsequent response)

• 100s of gigabytes of data at first, growing to >1 terabyte over time.
• High peak loads.
• Public cloud portability (but they have private data centers they can use today).
• Simple database design — not a lot of tables, not a lot of columns, not a lot of joins, and everything can be distributed on the same customer_ID key.
• Stream the data to a data warehouse, that will grow to a few terabytes. (Keeping only one year of OLTP data online actually makes sense in this application, but of course everything should go into the DW.)

So I’m leaning to saying:  

• They should go with a scalable, MySQL-based solution.
  • Lots of third-party software works with MySQL, in case that’s helpful.
  • Yes, any one vendor is small and not yet firmly established, but there are numerous vendors around with interesting MySQL scaling stories.
  • In a vendor emergency, just going with Oracle’s MySQL stuff would probably work …
  • … especially because there are these lovely things in the world called solid-state drives.
  • There’s also good escapability if one wants to move away from MySQL, because everybody knows how to handle MySQL data.
• The first product to look at is dbShards, because it meets all the topology needs:
  • Local scale-out (transparent sharding).
  • Local high availability.
  • Remote disaster recovery (details of that are underway).
• The first analytic DBMS to look at is Infobright.
  • Yes, I know Infobright is focused more on machine-generated data these days, but this client’s analytic needs are so straightforward Infobright should pass with flying colors.
  • The MySQL-to-MySQL aspect should make ETL dead simple.
  • Again, there’s escapability.

Mainly, this is all fine. But I’m getting pushback on the solid-state aspect, for fear that it will compromise public cloud portability.

Am I missing something here? As far as I’m concerned, if you’re planning an OLTP system with a many-year lifespan today, of course you should assume solid-state storage. Maybe you scale out just as far as you would with disk, striping indexes or entire databases across the RAM of multiple servers. It that case, having solid-state backing reduces the risk of bottlenecks. Maybe you don’t scale out as far as you would with disk. In that case, solid-state backing saves you money.

As for public-cloud support for solid-state storage, that’s coming fast, right? (Actually, I have data points in support of that theory, but they’re a bit tenuous.) A large fraction of web businesses with private data centers seem to be using solid-state storage — from Facebook on down — or so the NoSQL/NewSQL/short-request DBMS guys tell me. Surely a number of public cloud vendors are close behind.

• Hadapt is additional software on a cluster that also runs fully functional Hadoop/HDFS. (Cloudera Hadoop more than straight-from-Apache Hadoop to date, but that’s not a requirement.)
• The cluster also runs a DBMS on every node, such as PostgreSQL or one of Infobright/Vectorwise.
• Hadapt’s software manages parallel SQL queries by distributing them to the DBMS living on each node. Hadapt says that the resulting query performance far outshines Hive’s.
• Hadapt further says that, by exploiting the partner DBMS, its SQL functionality outpaces Hive’s as well.
• Target Hadapt use cases are centered around keeping machine-generated or other poly-structured data in Hadoop, and extracting, enhancing, or otherwise deriving some of it to live in the relational store.
• In particular, Hadapt seems like an interesting choice when you want to use that relational data as you work on other data that’s still in HDFS, or if you want to keep using the relational data in other kinds of MapReduce jobs.
• That all fits well with my thoughts about the importance of derived data.

Other evolution from what I wrote about Hadapt a few months ago includes:

• Hadapt is in beta now.
• Hadapt has added adult supervision in the form of Philip Wickline, late of Endeca.

In other news, Hadapt is our newest client.

Let’s try eight categories instead. While no categorization is ever perfect, these each have at least some degree of technical homogeneity. Figuring out which types of analytic database you have or need — and in most cases you’ll need several — is a great early step in your analytic technology planning. 

Enterprise data warehouse (Full or partial)

• Kinds of data likely to be included: All, but especially operational
• Likely use styles: All
• Canonical example: Central EDW for a big enterprise
• Stresses: Concurrency, reliability, workload management

The enterprise data warehouse (EDW) ideal says that you copy all your data into one place, and drive all decision-making from there. Full EDWs are pipedreams. Still, a partial EDW makes sense for most large enterprises, and many indeed already have one. The first product lines to consider for classical EDWs are Teradata, DB2, Exadata, and maybe Microsoft SQL Server, especially if you’re going to stress concurrency and/or operational use cases.

Traditional data mart

• Kinds of data likely to be included: All
• Likely use styles: Business intelligence, budgeting/consolidation, investigative
• Examples: Reporting servers, planning/consolidation servers, anything MOLAP, etc.
• Stresses: Performance, concurrency, TCO

Whether or not you have something like an enterprise data warehouse, it’s common to have lighter-weight data marts as well. A traditional data mart might drive reports and dashboards. Or it might be specialized for budgeting, planning, and/or consolidation.  Some investigative analytics may be in the mix as well.

Any DBMS that can support an EDW can also support a data mart, but it may not be the most cost-effective way to do so. Columnar DBMS might have more attractive performance and TCO (Total Cost of Ownership); the same goes for Netezza. Some of them — e.g. Sybase IQ and Vertica — have excellent track records in concurrent usage as well. Ted Codd pushed what amounts to MOLAP (Multidimensional OnLine Analytic Processing) systems for these use cases. But relational DBMS commonly do a better job, which is one reason most major MOLAP products have wound up at RDBMS companies.

Investigative data mart — agile

• Kinds of data likely to be included: All, especially customer-centric
• Likely use styles: Investigative
• Canonical example: A few analysts getting a few TB to examine
• Stresses: Ease of setup/load, ease of admin, price/performance

Besides the traditional data mart, there are at least two other kinds. Both are focused on investigative analytics, but they’re differentiated by database size.

If you have just a few analysts,* looking at no more than a few terabytes of data (perhaps even just some gigabytes) — and if that data is “single-subject” and fairly homogenous — your watchwords should be “cheap”, “easy”, and “fast”. You don’t need to invest in much hardware, in expensive software, in much administrative effort (the analysts can be their own DBAs),  nor should you endure much set-up time. Just grab a product, grab some data, and start running queries (or extracts into the statistical tool of your choice).

*If you have dozens or even hundreds of analysts hitting the same database, you’re probably back to the more concurrency-oriented scenarios outlined above.

Infobright is often cost-effective among columnar analytic DBMS. Other vendors might cut you a price break as well. If you have multiple terabytes of data, don’t rule out Netezza’s lowest-end products (even if they’d really rather sell you something bigger). Or, if you’re in the sub-terabyte range, maybe you can get by with an in-memory BI tool such as QlikView, and not do anything special on the DBMS side at all.

Investigative data mart — big

• Kinds of data likely to be included: All, especially customer-centric, logs, financial trade, scientific
• Likely use styles: Investigative
• Canonical example: Single-subject 20 TB – 20 PB relational database
• Stresses: Performance, scale-out, analytic functionality

But if you’re looking at tens of terabytes of relational data, or even more, you really do have a “big data” problem. Performance and scalability are major challenges, usually best addressed by MPP (Massively Parallel Processing) systems, such as Netezza, Vertica, Aster Data, ParAccel, Teradata, or Greenplum. Performance POCs (Proofs Of Concept) are a big part of the buying process. Vendor price negotiations are crucial too.

Actually, in the low tens of terabytes you might be able to get away with a shared-disk system that has excellent compression — e.g., columnar products like Sybase IQ, Infobright, or SAND, rather than just Vertica and ParAccel.

Assuming you have affordable, scalable query performance, the competitive differentiator can switch to additional analytic functionality. Aster, Netezza, ParAccel, Vertica, and Greenplum either offer full analytic platforms, or seem to be on the path to doing so. Teradata, which now owns Aster Data, offers substantial built-in analytic capability in its traditional products as well, and the same goes for Sybase IQ.

Continued in Part 2, where we cover some of the more difficult use cases.

Sybase said that Sybase IQ had > 2000 direct customers and >500 indirect customers (i.e., end customers of OEMs). That’s counting by customers; I know from prior discussions that Sybase IQ is running at close to two installations per customer. I also believe that Sybase counts different divisions of the same large enterprise as separate customers.

Vertica cited a figure of 500 customers as of April (end Q1?), which is close to 600 now, about 40% or a little more direct. The difference between this and a 2010 year-end figure of 328 is not only new sales, but also slow reporting by OEMs.  One cool figure — a single OEM reported 82 end sales in a single (quarterly?) report. And a number of those direct customers are substantial; Vertica’s customer logo page features lots of telcos, lots of internet companies, and the national operation of Blue Cross/Blue Shield.

Pay no attention to small inconsistencies in the number of Vertica direct customers (250 at year-end, no more than that now); Colin Mahony just estimates these numbers for me from memory, and minor inaccuracies are quite excusable.

Even cooler — Vertica reports 7 customers with a petabyte or more of user data each. About 5 of the 7 are obvious-suspect big-name firms; but unsurprisingly, those big names are NDA. I did secure permission to say that there are 2 telecom companies, a mobile gaming vendor, another internet company, and 3 financial services outfits of various kinds.

SAND Technology reported >600 total customers, including >100 direct. Since SAND has been around since the 1990s, those aren’t great average annual figures, but they’re probably more than many people (including me) thought.

Infobright reported around 200 total paying customers, 130 direct. There are surely a lot more users of open source Infobright, but precise numbers are of course hard to come by.

If I asked ParAccel in the April go-round, I’ve misplaced their answer, but back in October the figure was >30 customers, 2 of them over 100 terabytes. I’ve seen published figures of 40+ for ParAccel since.

• Web companies, many of which are already MySQL users.
• Telecommunication and similar log data, especially in OEM relationships.
• Trading/financial services, especially at mid-tier companies.

Key aspects of Infobright 4.0 include: 

• “Rough Query,” which lets you get approximate query results >10X faster than you could get precise ones, which is a good thing for iterative investigative analytics.
• The start of a plan — “DomainExpert” — to compress and otherwise optimize data in specific, commonly machine-generated patterns, such as URLs or CDRs (call detail records).
• “Distributed Load Manager” — i.e., load nodes that are separate from (and more parallelized than) query nodes.
• A Hadoop connector.
• Lots of cleanup and Bottleneck Whack-A-Mole, although I haven’t paid close attention as to which parts of that are truly new, and which were already handled in recent Infobright point releases.

Items on that list focused on the machine-generated data market include:

• DomainExpert — obviously.
• The Hadoop connector — also obviously.
• The Distributed Load Manager — why would you need such load speeds unless the data is flowing in from machines?

To understand Infobright Rough Query, recall the essence of Infobright’s architecture:

Infobright’s core technical idea is to chop columns of data into 64K chunks, called data packs, and then store concise information about what’s in the packs. The more basic information is stored in data pack nodes,* one per data pack. If you’re familiar with Netezza zone maps, data pack nodes sound like zone maps on steroids. They store maximum values, minimum values, and (where meaningful) aggregates, and also encode information as to which intervals between the min and max values do or don’t contain actual data values.

I.e., a concise, imprecise representation of the database is always kept in RAM, in something Infobright calls the “Knowledge Grid.” Rough Query estimates query results based solely on the information in the Knowledge Grid — i.e., Rough Query always executes against information that’s already in RAM.

To me, Rough Query is the most impressive part of the Infobright 4.0 announcement. DomainExpert sounds like it will be somewhat better than straightforward prefix/suffix compression, but Infobright hasn’t yet convinced me that the difference is substantial. Distributed Load Manager is indeed important, but only because Infobright doesn’t have a shared-nothing MPP (Massively Parallel Processing) option at this time. And the rest is mainly catch-up toward Infobright’s larger and more expensive peers.

Per Infobright chief scientist Dominik Slezak, the answer is

Infobright perform joins using its own optimization/execution layers (that actually include hash join algorithms and advanced knowledge-grid-based nested loop optimizations in particular).

• This is a list of Monash Advantage members.
• All our vendor clients are Monash Advantage members, unless …
• … we work with them primarily in their capacity as technology users. (A large fraction of our user clients happen to be SaaS vendors.)
• We do not usually disclose our user clients.
• We do not usually disclose our venture capital clients, nor those who invest in publicly-traded securities.
• Included in the list below are two expired Monash Advantage members who haven’t said they will renew, as mentioned in my recent post on analyst bias. (You can probably imagine a couple of reasons for that obfuscation.)

With that said, our vendor client disclosures at this time are:

• Aster Data
• Cloudera
• CodeFutures/dbShards
• Couchbase
• EMC/Greenplum
• Endeca
• IBM/Netezza
• Infobright
• Intel
• MarkLogic
• ParAccel
• QlikTech
• salesforce.com/database.com
• SAND Technology
• SAP/Sybase
• Schooner Information Technology
• Skytide
• Splunk
• Teradata
• Vertica

That list includes the two I’m obfuscating, plus one more who just emailed to say a signed renewal contract is arriving this week. It does not include others who, less concretely, have said they will sign up soon.

Also, I guess there’s a bit of a gray area for Tableau. As far as I’m concerned, I’m doing an upcoming co-sponsored webinar just for Monash Advantage member Aster Data. Indeed, I declined to contract with or bill Tableau directly for its share,  because I had no good way to do that paperwork. But even so, Tableau is a cosponsor, was involved in the planning discussions and, behind the scenes, is surely footing part of the bill.

The Gartner 2010 Data Warehouse Database Management Systems Magic Quadrant is out. I shall now comment, just as I did to varying degrees on the 2009, 2008, 2007, and 2006 Gartner Data Warehouse Database Management System Magic Quadrants.

Note: Links to Gartner Magic Quadrants tend to be unstable. Please alert me if any problems arise; I’ll edit accordingly.

In my comments on the 2008 Gartner Data Warehouse Database Management Systems Magic Quadrant, I observed that Gartner’s “completeness of vision” scores were generally pretty reasonable, but their “ability to execute” rankings were somewhat bizarre; the same remains true this year. For example, Gartner ranks Ingres higher by that metric than Vertica, Aster Data, ParAccel, or Infobright. Yet each of those companies is growing nicely and delivering products that meet serious cutting-edge analytic DBMS needs, neither of which has been true of Ingres since about 1987. 

The general list of “market forces, end-user expectations and vendors’ resulting solution approaches” at the top of the 2010 Gartner Data Warehouse Database Management System Magic Quadrant article is a mixed bag. Following Gartner’s order, I’ll address those first, and particular companies cited afterwards. Specific items and comments include:

• “Increased demand for optimization techniques and performance enhancement.“ Gartner seems to be saying that data warehouse DBMS buyers want lists of specific, esoteric performance features. Well, buyers always want their DBMS to run fast, and they’d like the products to be mature enough to have been through a few rounds of Bottleneck Whack-A-Mole, but otherwise I’m not sure I’d put that at the top of my list.
• “The argument made by purchasing departments that buying power increases when dealing with a single, incumbent vendor.“ I agree that vendor consolidation and account control are a huge part of the Oracle, Microsoft, IBM and even Teradata stories. (Vertica can prove it’s 10X more price-performant than Oracle and still not get the business.) But it’s not just about price negotiations; once annual maintenance is included, one has to squint pretty hard to see Oracle as a low-cost alternative. Also important is reducing the number of total product-specific skill-sets needed on the IT staff.
• “Prepackaged, prebalanced warehouse environments delivered using data warehouse appliances.“ Yep. To varying extents, Oracle, Microsoft, Teradata, and IBM are all committed to designed-hardware strategies.
• “Expectations for the delivery of on-site POCs.“ Honestly, not as many buyers insist on on-site Proofs of Concept as should. Still, Oracle is shameful in its reluctance to do them. (Teradata tries to avoid them too, for obvious reasons of expense, but is much more gracious about capitulating when the buyer insists.)
• “Cost controls and data warehouse performance management.“ See next comment.
• “Demands for delivering a fully mixed workload.“ I’d have phrased the workload management and administrative tools points rather differently than this, but so be it.
• “Demands for departmental analytics delivered quickly via data marts.“ Agreed. Data-mart-only installations are a huge part of the market of the analytic DBMS market. Data mart spin-out is also important.
• “Wider indexing and fast performance within clusters of data, delivered via column-based solutions.“ This bizarrely seems to conflate column stores and parallel processing (both of which are of course highly important).
• “A wave of new data warehouse implementers seeking fast-track, low-risk delivery.“ Well, yes. Netezza noticed that quite some years ago. And by now the long-gestation EDW (Enterprise Data Warehouse) is widely disliked.
• “Global organizations seeking distributed solutions as potential architecture.“ If this is the MPP point, it’s oddly phrased. If this is a suggestion that data warehouses should be partitioned across wide-area networks, it’s just plain odd. If it’s a reiteration that departments like to control their own data marts, I agree. And if it’s a comment on keep-data-in-the-country privacy laws, it could be the most prescient thing Donald Feinberg has said in many years.

Long though it is, that list of general items and issues for the 2010 Gartner Data Warehouse Database Management System Magic Quadrant has some gaps. Most glaringly, I don’t see any references to advanced analytics in general, or even to the specific case of integrated predictive analytics. There’s also nothing about solid-state memory or other storage-technology considerations, although in fairness it’s still early days for much of what vendors conceive of as competitive differentiation in those respects.

Here are some vendor-specific comments on the 2010 Gartner Data Warehouse Database Management System Magic Quadrant:

• It’s pretty bizarre to compare 1010data to database.com or Microsoft Azure. Kognitio would be a better choice. So would cloud-hosted instances of Vertica, Aster Data nCluster, or others.
• Gartner’s comments on Aster Data and nCluster are actually pretty reasonable.
• Gartner’s comments on EMC/Greenplum are a bit Kool-Aid-drinky, and don’t account for the inevitable flailing that occurs right after an acquisition. But otherwise they’re pretty reasonable.
• I don’t take IBM’s super-comprehensive-all-inclusive architectural stories as seriously as Gartner does.
• I don’t take Netezza’s small stable of OEM partners as seriously as Gartner does. I also don’t share Gartner’s optimism for the continuation of Netezza’s NEC partnership in the face of IBM’s Netezza ownership.
• I’m even more skeptical about illuminate than Gartner is.
• I’m delighted that Gartner has adopted my phrase machine-generated data (Infobright is one of several firms pushing that one).
• “Only open-source column-store DBMS” is a bit exaggerated, but Infobright is indeed the only one with serious traction, or offered by a serious analytic DBMS vendor.
• What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.
• While Gartner’s write-up of Kognitio is a bit confused, that’s excusable. Kognitio’s strategy changes often.
• I’m not persuaded by the claim of low Microsoft TCO. The days when Microsoft’s tools were vastly better than the competition’s are long gone. And using an OLTP DBMS for data warehousing generally takes more people effort than using something more purpose-built.
• Gartner is right to ding Oracle for high prices, high people costs, and unwillingness to do onsite POCs.
• Gartner is right that Exadata is a huge improvement over non-Exadata Oracle data warehousing.
• Gartner is right to suggest that Exadata can easily handle data warehouses over 20 terabytes in size, but wrong to suggest that software-only Oracle also can. Just because the pain is less than it was with earlier releases of Oracle doesn’t mean it isn’t still bad.
• Gartner’s comments on ParAccel are pretty reasonable.
• Gartner’s comments on compression in connection with SAND make no technical sense (tokenization is a key form of columnar compression, not an alternative to it). Also, SAP’s acquisition of Sybase is a business challenge for SAND, not a technical one.
• Unless I’m forgetting something, Sybase IQ has no more in-database data mining than any other Fuzzy Logix partner does.
• Gartner failed to note that, like other DBMS dating back to the 1990s and before, Sybase IQ is more complex to administer than some newer products are.
• Gartner’s take on Teradata is pretty reasonable.
• Gartner’s take on Vertica, while sloppy, is basically sensible. However, Gartner failed to note that Vertica is a laggard in non-query analytics. (I am sure those deficiencies are being addressed, but Vertica’s competitors are moving ahead as well.)