Back in April, Steve Miller suggested that approximate BI could be a growing trend, gaining speed at the expense of (often false anyway) precision. That idea of course goes well with Infobright’s recent released Rough Query feature, and also with Datameer’s year-earlier pitch.

Aster Data (now a Teradata company) is positioning itself as analyzing multi-structured data — which is my second-choice term, behind the more precise but odder-sounding “poly-structured.” I hope “poly-structured” wins, and plan to keep using it myself; but I recognize that “multi-structured” may actually be the one that prevails.

Barbara Darrow wrote a great piece on Oracle Exadata pricing. Highlights include:

• Routine Oracle software discounts are high.
• Exadata discounts are higher.
• Big/referenceable customers get the best Exadata discounts. The term “extremely deep” was used. (I’ve also heard that from Oracle competitors, with the term “free” even coming up, hyperbolically or otherwise.)
• Oracle’s hardware maintenance pricing is forcing users to trash Sun gear, even when it’s working. One guy told the story of literally crying as the Sun boxes were taken away.
• Oracle’s 22% of license maintenance fee goes up to 27% after two years. I didn’t know that.

Oracle has been making considerable messaging fuss around a win in Japan, where Softbank replaced years-old Teradata systems with vastly less new Exadata gear. I blogged that this is hardly an apples-to-apples comparison. During my visit last April, Oracle pushed back, in particular pointing out that the Softbank division that awarded the deal was very separate from the one that was an Oracle reseller. But Monday Teradata shared with me a counter-pushback, asserting that during the recent worldwide recession, Softbank assigned its underemployed systems integration division to do internal projects — including the data warehouse upgrade. I.e., Teradata stands by its claim that this replacement was strongly influenced by the Softbank/Oracle partnership.

If you’re analytically inclined, Kx Systems has some interesting ideas, manifested in kdb+ and so on. A 2009 ACM article seems as good a starting point as any, the company’s website probably aside. Confusingly, Kx is small company that evidently does most of its selling through a couple of much larger partners. Also, 1010data happens to be built on an older version of Kx’s technology.

The Gartner 2010 Data Warehouse Database Management Systems Magic Quadrant is out. I shall now comment, just as I did to varying degrees on the 2009, 2008, 2007, and 2006 Gartner Data Warehouse Database Management System Magic Quadrants.

Note: Links to Gartner Magic Quadrants tend to be unstable. Please alert me if any problems arise; I’ll edit accordingly.

In my comments on the 2008 Gartner Data Warehouse Database Management Systems Magic Quadrant, I observed that Gartner’s “completeness of vision” scores were generally pretty reasonable, but their “ability to execute” rankings were somewhat bizarre; the same remains true this year. For example, Gartner ranks Ingres higher by that metric than Vertica, Aster Data, ParAccel, or Infobright. Yet each of those companies is growing nicely and delivering products that meet serious cutting-edge analytic DBMS needs, neither of which has been true of Ingres since about 1987. 

The general list of “market forces, end-user expectations and vendors’ resulting solution approaches” at the top of the 2010 Gartner Data Warehouse Database Management System Magic Quadrant article is a mixed bag. Following Gartner’s order, I’ll address those first, and particular companies cited afterwards. Specific items and comments include:

• “Increased demand for optimization techniques and performance enhancement.“ Gartner seems to be saying that data warehouse DBMS buyers want lists of specific, esoteric performance features. Well, buyers always want their DBMS to run fast, and they’d like the products to be mature enough to have been through a few rounds of Bottleneck Whack-A-Mole, but otherwise I’m not sure I’d put that at the top of my list.
• “The argument made by purchasing departments that buying power increases when dealing with a single, incumbent vendor.“ I agree that vendor consolidation and account control are a huge part of the Oracle, Microsoft, IBM and even Teradata stories. (Vertica can prove it’s 10X more price-performant than Oracle and still not get the business.) But it’s not just about price negotiations; once annual maintenance is included, one has to squint pretty hard to see Oracle as a low-cost alternative. Also important is reducing the number of total product-specific skill-sets needed on the IT staff.
• “Prepackaged, prebalanced warehouse environments delivered using data warehouse appliances.“ Yep. To varying extents, Oracle, Microsoft, Teradata, and IBM are all committed to designed-hardware strategies.
• “Expectations for the delivery of on-site POCs.“ Honestly, not as many buyers insist on on-site Proofs of Concept as should. Still, Oracle is shameful in its reluctance to do them. (Teradata tries to avoid them too, for obvious reasons of expense, but is much more gracious about capitulating when the buyer insists.)
• “Cost controls and data warehouse performance management.“ See next comment.
• “Demands for delivering a fully mixed workload.“ I’d have phrased the workload management and administrative tools points rather differently than this, but so be it.
• “Demands for departmental analytics delivered quickly via data marts.“ Agreed. Data-mart-only installations are a huge part of the market of the analytic DBMS market. Data mart spin-out is also important.
• “Wider indexing and fast performance within clusters of data, delivered via column-based solutions.“ This bizarrely seems to conflate column stores and parallel processing (both of which are of course highly important).
• “A wave of new data warehouse implementers seeking fast-track, low-risk delivery.“ Well, yes. Netezza noticed that quite some years ago. And by now the long-gestation EDW (Enterprise Data Warehouse) is widely disliked.
• “Global organizations seeking distributed solutions as potential architecture.“ If this is the MPP point, it’s oddly phrased. If this is a suggestion that data warehouses should be partitioned across wide-area networks, it’s just plain odd. If it’s a reiteration that departments like to control their own data marts, I agree. And if it’s a comment on keep-data-in-the-country privacy laws, it could be the most prescient thing Donald Feinberg has said in many years.

Long though it is, that list of general items and issues for the 2010 Gartner Data Warehouse Database Management System Magic Quadrant has some gaps. Most glaringly, I don’t see any references to advanced analytics in general, or even to the specific case of integrated predictive analytics. There’s also nothing about solid-state memory or other storage-technology considerations, although in fairness it’s still early days for much of what vendors conceive of as competitive differentiation in those respects.

Here are some vendor-specific comments on the 2010 Gartner Data Warehouse Database Management System Magic Quadrant:

• It’s pretty bizarre to compare 1010data to database.com or Microsoft Azure. Kognitio would be a better choice. So would cloud-hosted instances of Vertica, Aster Data nCluster, or others.
• Gartner’s comments on Aster Data and nCluster are actually pretty reasonable.
• Gartner’s comments on EMC/Greenplum are a bit Kool-Aid-drinky, and don’t account for the inevitable flailing that occurs right after an acquisition. But otherwise they’re pretty reasonable.
• I don’t take IBM’s super-comprehensive-all-inclusive architectural stories as seriously as Gartner does.
• I don’t take Netezza’s small stable of OEM partners as seriously as Gartner does. I also don’t share Gartner’s optimism for the continuation of Netezza’s NEC partnership in the face of IBM’s Netezza ownership.
• I’m even more skeptical about illuminate than Gartner is.
• I’m delighted that Gartner has adopted my phrase machine-generated data (Infobright is one of several firms pushing that one).
• “Only open-source column-store DBMS” is a bit exaggerated, but Infobright is indeed the only one with serious traction, or offered by a serious analytic DBMS vendor.
• What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.
• While Gartner’s write-up of Kognitio is a bit confused, that’s excusable. Kognitio’s strategy changes often.
• I’m not persuaded by the claim of low Microsoft TCO. The days when Microsoft’s tools were vastly better than the competition’s are long gone. And using an OLTP DBMS for data warehousing generally takes more people effort than using something more purpose-built.
• Gartner is right to ding Oracle for high prices, high people costs, and unwillingness to do onsite POCs.
• Gartner is right that Exadata is a huge improvement over non-Exadata Oracle data warehousing.
• Gartner is right to suggest that Exadata can easily handle data warehouses over 20 terabytes in size, but wrong to suggest that software-only Oracle also can. Just because the pain is less than it was with earlier releases of Oracle doesn’t mean it isn’t still bad.
• Gartner’s comments on ParAccel are pretty reasonable.
• Gartner’s comments on compression in connection with SAND make no technical sense (tokenization is a key form of columnar compression, not an alternative to it). Also, SAP’s acquisition of Sybase is a business challenge for SAND, not a technical one.
• Unless I’m forgetting something, Sybase IQ has no more in-database data mining than any other Fuzzy Logix partner does.
• Gartner failed to note that, like other DBMS dating back to the 1990s and before, Sybase IQ is more complex to administer than some newer products are.
• Gartner’s take on Teradata is pretty reasonable.
• Gartner’s take on Vertica, while sloppy, is basically sensible. However, Gartner failed to note that Vertica is a laggard in non-query analytics. (I am sure those deficiencies are being addressed, but Vertica’s competitors are moving ahead as well.)

Less successful to date are efforts in the area of pure database software-as-a-service.  It seems that if somebody is going for SaaS anyway, they usually want a more complete, integrated offering. The most noteworthy exceptions I can think of to this general rule are Kognitio and Vertica, and they only have a handful of database SaaS customers each. To wit:

1.  Kognitio has built a lot of its marketing around database SaaS, which it calls DaaS for data-as-a-service, and runs primarily from its own facility.  On a small sample size, it reports a very roughly 50-50 split in new business activity (that’s customers/prospects, not revenue) between DaaS and conventionally licensed software.

2.  Vertica has expressed high hopes for its Amazon cloud offering. Actual production usage has so far only matched part of that, but it isn’t exactly zero either. Specifically, marketing chief Dave Menninger writes by email:

In addition to approximately a dozen POCs running on the cloud at any point in time we have five customers using the cloud on a regular basis. Three of these customers do short lived projects so they start up instances, run them for the duration of a project, and shut them down. They are three different types of orgs: govt agency, pharma consulting org and SaaS provider.

Two financial services companies use the cloud as spare resource/capacity.  When they need additional computing resource or capacity they will temporarily move some projects onto the cloud with the anticipation of moving them back off once the capacity constraint is relieved (new hardware arrives, other projects or systems come to an end, etc.

3.  1010data offers its data warehousing product by remote service only.  However, unlike Gartner I’m not totally convinced 1010data should be regarded as comparable to DBMS vendors; perhaps it’s more like a SaaS business intelligence provider.

Edits:

• A comment below says Gerry Cohen wrote Nomad too.
  
• Kognitio commented on Twitter that they actually use DaaS to mean Data warehouse As A Service.

Gartner’s annual Magic Quadrant for data warehouse DBMS is out.  Thankfully, vendors don’t seem to be taking it as seriously as usual, so I didn’t immediately hear about it.  (I finally noticed it in a Greenplum pay-per-click ad.)  Links to Gartner MQs tend to come and go, but as of now here are two working links to the 2008 Gartner Data Warehouse Database Management System MQ.  My posts on the 2007 and 2006 MQs have also been updated with working links.

Highlights of this year’s data warehouse DBMS Magic Quadrant include:

• Teradata is #1, Oracle is #2, and IBM is #3, with the first two if anything slightly extending their leads.  (in 2006, IBM was #2.)
• Netezza has been given a nice upwards (actually, more rightwards) bump and is now a clear #4.
• Microsoft is treading water at a clear #5.
• Greenplum and Sybase have slid back some, but depending on which dimension you weight more heavily are somewhere in the #6-8 range.
• HP joins newly, as the other #6-8 competitor, a little behind Sybase.
• Vertica joins as a first-timer, as a clear #9.
• Kognitio and SAND are next, with hefty gains in “ability to execute”, both leapfrogging Sun/MySQL.
• Ingres, iLLuminate, and 1010data straggle in at the bottom, all of them new (at least versus 2006-7).

I don’t really have a lot of quarrel with the “completeness of vision” rankings.  As I see it, important attributes of a data warehouse DBMS “vision” would include:

• A performance story across at least a reasonable range of workloads.
• Either a clear hardware architecture story, or else a clear story as to why hardware architecture is relatively unimportant.
• SQL 2003 and further features in integrated analytics.
• Reasonable OLTP-like features, from the basics — ACID compliance! — to manageability, high availability and fast-enough update/load.
• Good compatibility with third-party products.

Gartner’s rankings are not ridiculous by those standards.  Aster would surely have ranked high, but obviously they did not meet the confirmed-sale requirements for inclusion.

So what about Gartner’s “ability to execute” rankings?  These are approximately:

• Teradata at #1
• Oracle and IBM tied at #2-3
• HP, Sybase, Microsoft, and Netezza tied at #4-7
• Greenplum at #8, Vertica at #9, and everybody else trailing after

That looks like it’s basically a measure of revenue, blending overall corporate and data-warehouse-DBMS-specific figures in some way, adjusted for who can deploy the most credible-sounding executive who appears to simultaneously have his — I use the male pronoun deliberately — finger on development and revenue-generation alike.

Frankly, I think it’s that dimension that makes Gartner Magic Quadrants well-nigh meaningless.  If you asked me in which vendor’s execution-on-vision I had the most confidence, I’d stammer around unless I felt free to reframe the question and shoot back “Which PART of the vision?”  If you want to deploy a 1 terabyte data warehouse with a highly diverse workload — well, Oracle, IBM, Teradata, and to a lesser extent Microsoft have been doing that for years, and they deserve to be atop the ability-to-execute charts, with Netezza perhaps not far behind.  If you want to run fast queries on cheap hardware on 200 GB of data, Sybase IQ is a proven market leader.  If you want a cheap 100 TB data warehouse that will soon scale to over a petabyte, Oracle’s great achievements in other areas of DBMS and its clever Exadata ideas suffice merely to put it on a par with those smaller vendors that have actually deployed a few such systems each, albeit behind Teradata.

When selecting a database management system for analytic processing, confine yourself to those vendors whose products can, today, do everything you’re likely to need for the next few years. Further require that they be on track to soon deliver most of what you seriously want over that time period.  And throw the Gartner MQ into the nearest bit bucket, before it confuses your evaluation cycle irredeemably.

• I was just briefed by Vertica, and got the impression that data mart outsourcers may be Vertica’s #3 vertical market, after financial services and telecom. Certainly it seems like they are Vertica’s #3 market if you bundle together data mart outsourcers and more conventional OEMs.
• When Netezza started out, a bunch of its early customers were credit data-based analytics outsourcers like Acxiom.
• After nagging DATAllegro for a production reference, I finally got a good one — TEOCO. TEOCO specializes in figuring out whether inter-carrier telcom bills are correct. While there’s certainly a transactional invoice-processing aspect to this, the business seems to hinge mainly around doing calculations to figure out correct charges.
• I was talking with Pervasive about Pervasive Datarush, a beta product that lets you do super-fast analytics on data even if you never load it into a DBMS in the first place. I challenged them for use cases. One user turns out to be an insurance claims rule-checking outsourcer.
• One of Infobright’s references is a French CRM analytics outsourcer, 1024 Degres.
• 1010data has built up a client base of 50-60, including a number of financial and retail blue-chippers, with a soup-to-nuts BI/analysis/columnar database stack.
• I haven’t heard much about Verix in a while, but their niche was combining internal sales figures with external point-of-sale/prescription data to assess retail (especially pharma) microtrends.

To a first approximation, here’s what I think is going on.

Privacy laws force some outsourcing. It’s often OK to use credit data to decide what you’ll market at whom, even when it’s not OK to actually see the credit data itself. What’s more, in some cases data can’t leave a country, so if you don’t have critical business mass in that particular country, it’s natural to use an outsourcer who does.

Privacy even aside, owners of proprietary data are natural analytics outsourcers. Either you ship your data to your customers to do with as they please — and impose on them the expense of managing it — or you manage it for them.

Analytic “secret sauce” software providers also are natural outsourcers. Most proprietary analytic rules are pretty simple-minded. Outsourcing preserves mystique and pricing power.

The usual benefits of SaaS apply. Fast set-up, no fixed costs, etc. are all goodness, just as they are in the transactional world.

With that as background, the big change in the analytics outsourcing market is the same as the one sweeping the rest of the analytics world — interactive access to detail data is finally becoming affordable. If you just run weekly or monthly reports, and there may be no reason to distinguish between analytic and transactional processing. But if you want to allow ad-hoc query, unlimited drilldown, or live dashboards, then you’re talking a serious data mart technology stack.

And I do mean “data mart”. Outsourcing an enterprise data warehouse, with all of your proprietary transactional data, doesn’t make much sense unless you’re a complete SaaS shop already outsourcing that data in the first place.