• In general, I regard Gartner Magic Quadrants as a bad use of good research.
• Illustrating the uselessness of — or at least poor execution on — the overall quadrant metaphor, a large majority of the vendors covered are lined up near the line x = y, each outpacing the one below in both of the quadrant’s dimensions.
• I find fewer specifics to disagree with in this Gartner Magic Quadrant than in previous year’s versions. Two factors jump to mind as possible reasons:
  • This year’s Gartner Magic Quadrant for Data Warehouse Database Management Systems is somewhat less ambitious than others; while it gives as much company detail as its predecessors, it doesn’t add as much discussion of overall trends. So there’s less to (potentially) disagree with.
  • Merv Adrian is now at Gartner.
• Whatever the problems may be with Gartner’s approach, the whole thing comes out better than do Forrester’s failed imitations.

*At the time of this posting, I don’t yet have a link. However, I expect that to change quickly, and I plan to edit this paragraph accordingly. If nothing else, I hope people will drop links into the comment thread.

Specific company comments, roughly in line with Gartner’s rough single-dimensional rank ordering, include:

• The Gartner Magic Quadrant’s comments on Teradata seem pretty fair. I don’t think I’m much in disagreement when I say:
  • Teradata has the richest, most mature analytic DBMS offering.
  • Teradata has an outstanding track record both for managing large data volumes and for high-concurrency mixed workloads.
  • Aster Data was a cool Teradata acquisition, even if Teradata/Aster synergies or integration have been nominal to date.
  • Teradata still needs to get out of its own way in marketing, positioning, packaging, and/or defining its premium-priced system vs. its more moderately-priced alternatives. Indeed, as necessary as this approach may have been to fending off encroachments by Netezza and others, what Teradata really needs to do is evolve to a more pick-your-own-node-combination mix-match kind of offering.
• Gartner has talked with a lot of Oracle Exadata users who say that the product works; Gartner also has stopped beating Oracle up for its previous policy of almost never doing onsite POCs (Proofs of Concept); both parts of that ring true with me. But Gartner also rightly dings Oracle for various issues in cost and cumbersomeness. Overall, while I agree there are organizations for which Oracle should indeed be a top-ranked choice, there are many others who shouldn’t put Oracle on their short list.
• Third in the Gartner MQ rankings is IBM.
  • Gartner gets so caught up in reciting the names of various IBM product offerings that it neglects to say much good about DB2 itself. (I tend to have a similar problem.)
  • But Gartner does mention concurrency as a strength. I agree, especially if we presume that that was a reference to DB2 rather than Netezza.
  • Gartner cites Netezza’s post-acquisition annual growth rate as 30%. Gartner seems to think this is a good number. I disagree, but in Netezza’s defense, it has had to endure IBM’s post-acquisition on-boarding process.
• Arguably fourth in the Gartner Data Warehouse Magic Quadrant rankings is EMC/Greenplum.
  • In general, Gartner likes the taste of Greenplum Kool-Aid.
  • Gartner neglects to ding Greenplum for concurrency challenges, which I view as an oversight given Gartner’s general stress on that area.
  • Gartner does ding Greenplum for support challenges.
  • Gartner neglects to praise Greenplum for true hybrid row/columnar data management, a feature shared by Teradata and Vertica, among others, but not by Oracle, DB2, or Netezza.
  • Gartner located a half-petabyte Greenplum database. This doesn’t surprise me, even though Greenplum has frequently made exaggerated claims about large-size database successes in the past.
  • Gartner reports a >400 figure for Greenplum customers, which is plausible.
• In its first deviation from strict one-dimensional rank ordering, the Gartner Magic Quadrant ranks Sybase ahead of Greenplum in completeness of vision but behind in “ability to execute”.
  • If that were the other way around, it might make more sense. Greenplum promises anything and everything you might ever want for analytic data management or the associated analysis; but Sybase has vastly more analytic DBMS users than Greenplum does, running a variety of demanding workloads.
  • Gartner appears to think that Sybase IQ requires less database administration than I do.
  • Gartner seems concerned that SAP will position HANA and Sybase ASE as, between them, the only DBMS you’ll ever need, casting doubt on Sybase IQ’s future. I wouldn’t worry about that if you have a problem you want to solve today.
• The Gartner Magic Quadrant for Data Warehouse Database Management Systems ranks Microsoft sixth overall, despite noting that there isn’t a single production reference for Microsoft’s Parallel Data Warehouse. In support of this ranking, it for example cites the compression feature, which distinguishes Microsoft SQL Server from no other product on the list except Kognitio. If you have such an undemanding data warehousing problem that many different analytic DBMS could meet your needs, there’s a good chance Microsoft SQL Server can also do the job; and if you’ve bought into the Microsoft technology stack, you might as well keep going down that path. Otherwise, I don’t know why somebody should adopt Microsoft’s offering at this time.
• Seventh along the main diagonal path in the Gartner Magic Quadrant is HP Vertica. I’d rank Vertica higher than that, but in fairness I note two execution concerns. First, HP has a lousy track record, both in acquisitions and in data warehousing/analytics. Second, Vertica is bad about answering my email. Anyhow, Gartner doesn’t seem to have given Vertica credit either for its full customer count or for the multiple petabyte-scale databases Vertica runs.
• 1010data is an outlier, with Gartner noting that it only partly fits in with other “Data Warehousing Database Management” companies, and hence kind of confessing that 1010data on the Magic Quadrant is somewhat arbitrary. Stuff like that is bound to happen, given the inherent difficulties of defining market categories. Anyhow, my thoughts on 1010data include:
  • I’m nervous about the fact that 1010data doesn’t actually control its own DBMS technology, but rather relies on old code from the small private company KX Systems.
  • There are three main reasons to consider 1010data:
    • You want to enter the data mart outsourcing business in a casual way, and you like its SaaS offering.
    • You want to engage in stakeholder-facing analytics in a casual way, and you like its SaaS offering.
    • You love 1010data’s particular set of interactive analytic features and performance.
• Back to the main path winding along the Gartner Magic Quadrant main diagonal — next up is ParAccel. While I question some of the peripheral comments, I agree with Gartner’s core messages that:
  • ParAccel, the product, is blazingly fast in certain use cases.
  • ParAccel, the company, is dangerously small.
• Eighth on the Gartner MQ’s main path is Kognitio. This is too high. Kognitio positions itself as offering in-memory DBMS, yet stubbornly refuses to do any kind of data compression. That’s an awful combination of choices. As for using Kognitio’s data warehousing SaaS offering — why would you do that, when more modern products are available on a SaaS/cloud basis as well?
• Ninth in the Gartner Magic Quadrant main rankings is SAND.
  • The SAND section is not a triumph of Gartner accuracy. For example:
    • Gartner completely missed the errors in SAND’s reported customer counts.
    • Gartner refers to SAND as being “in existence for approximately nine years”, which is too low by at least a factor of 2.
    • Gartner says “SAND is a privately held company”, even though Merv knows better than that.
  • Otherwise, Gartner’s opinion on SAND seems to boil down to “Interesting technology and ideas, but dangerously small company.” I agree.
• Tenth and too low in the Gartner MQ main rankings is Infobright.
  • At least by some metrics (e.g. customer count), Infobright isn’t as dangerously small as ParAccel, SAND, Kognitio, et al.
  • That said, Infobright is small and focused on machine-generated data. So I wouldn’t be confident in Infobright’s future technology path for human-generated data use cases.
  • Infobright’s performance is uneven — blazing in cases where the Knowledge Grid helps, but not necessarily stellar by analytic DBMS standards when full table scans are called for.
  • I agree with Gartner that the possibility of Oracle/MySQL future shenanigans is a concern. But while the energy behind MySQL forking efforts doesn’t seem too great right now, I’d expect them to revive and offer a successful escape path if it seemed Oracle was going to indeed play hardball.
  • Also, given that it’s already an open source vendor, there are various kinds of assurances Infobright could give that would also help alleviate customer concerns.
• Actian, formerly Ingres, took a big tumble in Gartner’s rankings versus last year, when I simply wrote “What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.” I’m even a little harsher about Ingres/Actian’s DBMS products and prospects than Gartner is, but at least now we’re in the same ballpark.
• Along with Infobright, ParAccel, and SAND, Exasol appears to be another of the “good columnar technology/small company” crowd. As with other such products, one should be careful about fit-and-finish features that are missing today, as there is no assurance they’ll be added in a timely manner going forward.
• illuminate Solutions, which was on last year’s Gartner list, now appears to be an ex-company.

I put the question out to Twitter, and was messaged by an extremely reliable source, who had heard that illuminate has shut down and is in receivership.

illuminate’s website and CTO blog that I previously linked both appear to be rather dead sites. Archive.org emphatically confirms that perception.

I can’t find anybody on LinkedIn who says they’ve worked at illuminate more recently than May, 2011.

It would seem that illuminate Solutions is no more, has ceased to be, has kicked the bucket, has joined the choir invisible, and is an ex-company.

The Gartner 2010 Data Warehouse Database Management Systems Magic Quadrant is out. I shall now comment, just as I did to varying degrees on the 2009, 2008, 2007, and 2006 Gartner Data Warehouse Database Management System Magic Quadrants.

Note: Links to Gartner Magic Quadrants tend to be unstable. Please alert me if any problems arise; I’ll edit accordingly.

In my comments on the 2008 Gartner Data Warehouse Database Management Systems Magic Quadrant, I observed that Gartner’s “completeness of vision” scores were generally pretty reasonable, but their “ability to execute” rankings were somewhat bizarre; the same remains true this year. For example, Gartner ranks Ingres higher by that metric than Vertica, Aster Data, ParAccel, or Infobright. Yet each of those companies is growing nicely and delivering products that meet serious cutting-edge analytic DBMS needs, neither of which has been true of Ingres since about 1987. 

The general list of “market forces, end-user expectations and vendors’ resulting solution approaches” at the top of the 2010 Gartner Data Warehouse Database Management System Magic Quadrant article is a mixed bag. Following Gartner’s order, I’ll address those first, and particular companies cited afterwards. Specific items and comments include:

• “Increased demand for optimization techniques and performance enhancement.“ Gartner seems to be saying that data warehouse DBMS buyers want lists of specific, esoteric performance features. Well, buyers always want their DBMS to run fast, and they’d like the products to be mature enough to have been through a few rounds of Bottleneck Whack-A-Mole, but otherwise I’m not sure I’d put that at the top of my list.
• “The argument made by purchasing departments that buying power increases when dealing with a single, incumbent vendor.“ I agree that vendor consolidation and account control are a huge part of the Oracle, Microsoft, IBM and even Teradata stories. (Vertica can prove it’s 10X more price-performant than Oracle and still not get the business.) But it’s not just about price negotiations; once annual maintenance is included, one has to squint pretty hard to see Oracle as a low-cost alternative. Also important is reducing the number of total product-specific skill-sets needed on the IT staff.
• “Prepackaged, prebalanced warehouse environments delivered using data warehouse appliances.“ Yep. To varying extents, Oracle, Microsoft, Teradata, and IBM are all committed to designed-hardware strategies.
• “Expectations for the delivery of on-site POCs.“ Honestly, not as many buyers insist on on-site Proofs of Concept as should. Still, Oracle is shameful in its reluctance to do them. (Teradata tries to avoid them too, for obvious reasons of expense, but is much more gracious about capitulating when the buyer insists.)
• “Cost controls and data warehouse performance management.“ See next comment.
• “Demands for delivering a fully mixed workload.“ I’d have phrased the workload management and administrative tools points rather differently than this, but so be it.
• “Demands for departmental analytics delivered quickly via data marts.“ Agreed. Data-mart-only installations are a huge part of the market of the analytic DBMS market. Data mart spin-out is also important.
• “Wider indexing and fast performance within clusters of data, delivered via column-based solutions.“ This bizarrely seems to conflate column stores and parallel processing (both of which are of course highly important).
• “A wave of new data warehouse implementers seeking fast-track, low-risk delivery.“ Well, yes. Netezza noticed that quite some years ago. And by now the long-gestation EDW (Enterprise Data Warehouse) is widely disliked.
• “Global organizations seeking distributed solutions as potential architecture.“ If this is the MPP point, it’s oddly phrased. If this is a suggestion that data warehouses should be partitioned across wide-area networks, it’s just plain odd. If it’s a reiteration that departments like to control their own data marts, I agree. And if it’s a comment on keep-data-in-the-country privacy laws, it could be the most prescient thing Donald Feinberg has said in many years.

Long though it is, that list of general items and issues for the 2010 Gartner Data Warehouse Database Management System Magic Quadrant has some gaps. Most glaringly, I don’t see any references to advanced analytics in general, or even to the specific case of integrated predictive analytics. There’s also nothing about solid-state memory or other storage-technology considerations, although in fairness it’s still early days for much of what vendors conceive of as competitive differentiation in those respects.

Here are some vendor-specific comments on the 2010 Gartner Data Warehouse Database Management System Magic Quadrant:

• It’s pretty bizarre to compare 1010data to database.com or Microsoft Azure. Kognitio would be a better choice. So would cloud-hosted instances of Vertica, Aster Data nCluster, or others.
• Gartner’s comments on Aster Data and nCluster are actually pretty reasonable.
• Gartner’s comments on EMC/Greenplum are a bit Kool-Aid-drinky, and don’t account for the inevitable flailing that occurs right after an acquisition. But otherwise they’re pretty reasonable.
• I don’t take IBM’s super-comprehensive-all-inclusive architectural stories as seriously as Gartner does.
• I don’t take Netezza’s small stable of OEM partners as seriously as Gartner does. I also don’t share Gartner’s optimism for the continuation of Netezza’s NEC partnership in the face of IBM’s Netezza ownership.
• I’m even more skeptical about illuminate than Gartner is.
• I’m delighted that Gartner has adopted my phrase machine-generated data (Infobright is one of several firms pushing that one).
• “Only open-source column-store DBMS” is a bit exaggerated, but Infobright is indeed the only one with serious traction, or offered by a serious analytic DBMS vendor.
• What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.
• While Gartner’s write-up of Kognitio is a bit confused, that’s excusable. Kognitio’s strategy changes often.
• I’m not persuaded by the claim of low Microsoft TCO. The days when Microsoft’s tools were vastly better than the competition’s are long gone. And using an OLTP DBMS for data warehousing generally takes more people effort than using something more purpose-built.
• Gartner is right to ding Oracle for high prices, high people costs, and unwillingness to do onsite POCs.
• Gartner is right that Exadata is a huge improvement over non-Exadata Oracle data warehousing.
• Gartner is right to suggest that Exadata can easily handle data warehouses over 20 terabytes in size, but wrong to suggest that software-only Oracle also can. Just because the pain is less than it was with earlier releases of Oracle doesn’t mean it isn’t still bad.
• Gartner’s comments on ParAccel are pretty reasonable.
• Gartner’s comments on compression in connection with SAND make no technical sense (tokenization is a key form of columnar compression, not an alternative to it). Also, SAP’s acquisition of Sybase is a business challenge for SAND, not a technical one.
• Unless I’m forgetting something, Sybase IQ has no more in-database data mining than any other Fuzzy Logix partner does.
• Gartner failed to note that, like other DBMS dating back to the 1990s and before, Sybase IQ is more complex to administer than some newer products are.
• Gartner’s take on Teradata is pretty reasonable.
• Gartner’s take on Vertica, while sloppy, is basically sensible. However, Gartner failed to note that Vertica is a laggard in non-query analytics. (I am sure those deficiencies are being addressed, but Vertica’s competitors are moving ahead as well.)

At intervals of a little over a year, Gartner Group publishes a Data Warehouse Database Management System Magic Quadrant. Gartner’s 2009 data warehouse DBMS Magic Quadrant — actually, January 2010 — is now out.* For many reasons, including those I noted in my comments on Gartner’s 2008 Data Warehouse DBMS Magic Quadrant, the Gartner quadrant pictures are a bad use of good research. Rather than rehash that this year, I’ll merely call out some points in the surrounding commentary that I find interesting or just plain strange.

*Links to Gartner Magic Quadrants commonly break, but that one worked at the time of this posting.

• Gartner thinks that data warehouse appliances are on the rise, due to their simplicity.
• Gartner correctly says that Teradata has been a data warehouse appliance vendor from the getgo.
• Gartner characterizes IBM as being an appliance vendor as well.
• Gartner suggests that HP is having trouble living up to its technical promises for Neoview.
• Gartner further suggests — no surprise here — that HP Neoview has had very few new customers past its initial wave.
• Gartner notes IBM’s difficulties in selling data warehouse installations of DB2, despite what on paper is great-sounding technology.
• Gartner says — also no surprise — that illuminate “has seen little success in North America since opening its first office in the U.S. over two years ago.”
• Ingres has evidently gotten a few BI-centric “appliance” deals, e.g. with Jaspersoft. But basically Ingres isn’t doing well in data warehousing.
• Gartner does say Ingres has “the strongest open-source DBMS offering for data warehousing.” Being very literal about “open source,” that’s a defensible claim — but it’s pretty irrelevant in a world where Greenplum Single-Node Edition can be had for free. It also waves away all the data mart use cases in which Infobright Community Edition shines.
• Gartner says that Netezza is working out as a “complex workload” enterprise data warehouse provider, according to reference checks, in addition to its established success in data mart scenarios.
• Gartner says Oracle’s offering has finally become “accepted” in the market for databases >50 TB. I guess I can live with that fairly weak claim, but I wouldn’t go much further than that.
• Gartner asserts that, unlike software-only Oracle, Oracle Exadata isn’t significantly harder to administer than “other mixed OLTP/OLAP DBMS vendors,” because Exadata is fast enough you don’t need to jump through all those hoops any more to get tolerable performance. The money quote is “one reference reported reducing the number of indexes by a factor of 100 to fewer than five.” Note, however, that Gartner does not seem to assert that Exadata’s ease of use rivals that of the newer analytic DBMS specialists.
• Gartner confirms Oracle’s reluctance to do onsite Exadata POCs, but says it is not absolute. This is roughly compatible with what I’m hearing elsewhere, and indeed with Oracle own claims to be ramping up availability of Exadata POC hardware.
• Gartner’s criteria for inclusion include at least 10 different organizations having a product “in production.” Thus, the big surprise was ParAccel being included. The money quote there is “With approximately 20 customers in the pharmaceutical, retail, financial and media/advertising analytics sectors, ParAccel has a good reference base.” That assessment is difficult to reconcile with other information, but I’ve been told Gartner is sticking to its guns. That assessment would be even harder to believe if those 20 references were all alleged to be true production customers.
• Gartner notes that you basically can’t run a 1 TB+ MySQL data warehouse without sharding. (Of course, Infobright has an alternative, and up to a small number of terabytes so does Kickfire.)
• Gartner reports that at least some customers are pleased with Sybase IQ’s mixed workload/enterprise data warehouse capabilities.
• Gartner correctly notes that Oracle Exadata is a price-competition challenge for Teradata.
• Gartner notes that 20% of Vertica’s customers are outside the US. While not shocking, that’s more than I realized.
• Gartner notes something I don’t think I’ve posted yet, which is that Vertica has a customer with 300 TB of data. (The identity is a deep dark secret, but if I told you you probably wouldn’t recognize the name anyway.)

As does any such piece, the Gartner Data Warehouse DBMS Magic Quadrant also has outright errors.  For example:

• Aster Data isn’t really “the newest entrant to the DBMS data warehouse world.”
• Aster’s SQL/MapReduce was not new in Release 4.0.
• Greenplum isn’t yet pushing down code to the storage tier.
• I’m not sure what kind of database-tier parallelism Gartner is claiming is new in Oracle in 11g Release 2 — but I doubt it’s really new. Rather, what Oracle has done recently is make parallelism less administratively cumbersome.
• Vertica wasn’t really the first DBMS in the cloud. At most it was the first pure-play analytic DBMS to get there.

Gartner’s annual Magic Quadrant for data warehouse DBMS is out.  Thankfully, vendors don’t seem to be taking it as seriously as usual, so I didn’t immediately hear about it.  (I finally noticed it in a Greenplum pay-per-click ad.)  Links to Gartner MQs tend to come and go, but as of now here are two working links to the 2008 Gartner Data Warehouse Database Management System MQ.  My posts on the 2007 and 2006 MQs have also been updated with working links.

Highlights of this year’s data warehouse DBMS Magic Quadrant include:

• Teradata is #1, Oracle is #2, and IBM is #3, with the first two if anything slightly extending their leads.  (in 2006, IBM was #2.)
• Netezza has been given a nice upwards (actually, more rightwards) bump and is now a clear #4.
• Microsoft is treading water at a clear #5.
• Greenplum and Sybase have slid back some, but depending on which dimension you weight more heavily are somewhere in the #6-8 range.
• HP joins newly, as the other #6-8 competitor, a little behind Sybase.
• Vertica joins as a first-timer, as a clear #9.
• Kognitio and SAND are next, with hefty gains in “ability to execute”, both leapfrogging Sun/MySQL.
• Ingres, iLLuminate, and 1010data straggle in at the bottom, all of them new (at least versus 2006-7).

I don’t really have a lot of quarrel with the “completeness of vision” rankings.  As I see it, important attributes of a data warehouse DBMS “vision” would include:

• A performance story across at least a reasonable range of workloads.
• Either a clear hardware architecture story, or else a clear story as to why hardware architecture is relatively unimportant.
• SQL 2003 and further features in integrated analytics.
• Reasonable OLTP-like features, from the basics — ACID compliance! — to manageability, high availability and fast-enough update/load.
• Good compatibility with third-party products.

Gartner’s rankings are not ridiculous by those standards.  Aster would surely have ranked high, but obviously they did not meet the confirmed-sale requirements for inclusion.

So what about Gartner’s “ability to execute” rankings?  These are approximately:

• Teradata at #1
• Oracle and IBM tied at #2-3
• HP, Sybase, Microsoft, and Netezza tied at #4-7
• Greenplum at #8, Vertica at #9, and everybody else trailing after

That looks like it’s basically a measure of revenue, blending overall corporate and data-warehouse-DBMS-specific figures in some way, adjusted for who can deploy the most credible-sounding executive who appears to simultaneously have his — I use the male pronoun deliberately — finger on development and revenue-generation alike.

Frankly, I think it’s that dimension that makes Gartner Magic Quadrants well-nigh meaningless.  If you asked me in which vendor’s execution-on-vision I had the most confidence, I’d stammer around unless I felt free to reframe the question and shoot back “Which PART of the vision?”  If you want to deploy a 1 terabyte data warehouse with a highly diverse workload — well, Oracle, IBM, Teradata, and to a lesser extent Microsoft have been doing that for years, and they deserve to be atop the ability-to-execute charts, with Netezza perhaps not far behind.  If you want to run fast queries on cheap hardware on 200 GB of data, Sybase IQ is a proven market leader.  If you want a cheap 100 TB data warehouse that will soon scale to over a petabyte, Oracle’s great achievements in other areas of DBMS and its clever Exadata ideas suffice merely to put it on a par with those smaller vendors that have actually deployed a few such systems each, albeit behind Teradata.

When selecting a database management system for analytic processing, confine yourself to those vendors whose products can, today, do everything you’re likely to need for the next few years. Further require that they be on track to soon deliver most of what you seriously want over that time period.  And throw the Gartner MQ into the nearest bit bucket, before it confuses your evaluation cycle irredeemably.

The VBS also provides some data access features that can not be duplicated in any other structure. A search can be executed starting with a data value in the pool. By going from the value pool back to the index, it is possible to quickly locate every use of the value wherever is may be used in the logical record structures.

which makes sense, and

This structure also enables our incremental query capability. As the result of a query, the database returns a set of instance identifiers rather than a set of records. This is because there are no records, only pointers and values. With the response being a set of pointers, it is a simple matter to perform the next query step and then get the union or difference between the two sets of pointers for the result of the second query step. This process can be continued indefinitely with the result set shrinking or growing as the new results are merged with the old.

which still sounds like gobbledygook to me.

Perhaps a demo of that query capability would make the lightbulb go on, but right now it sounds like illuminate is pitching a whole new non-SQL database architecture on the strength of a particular BI capability that probably could be replicated in more conventional systems.

In other news, I’ve rarely seen a software company with a bigger SEO problem than illuminate. The search that worked best for me was illuminate data warehousing.

illuminate’s basic idea is one I’ve heard before, but mainly from companies with more of a search orientation*, such as Attivio: Take a collection of tables, create a big inverted index on all the values in all columns at once, and do queries on that. This, illuminate claims, obviates all sorts of database design problems and similar hassles you otherwise might have. illuminate’s buzzword for all this is “CDBMS”, where the “C” stands for correlation. The actual CDBMS product is called iLuminate; related business intelligence tools are called iCorrelate and iAnalyze. What iLuminate actually indexes is a token that holds four pieces of information: Instance identifier, table identifier, column identifier, and value.

*iLuminate has string-matching and soundex on character fields, but that’s about as far as it goes.

iLuminate also lets you use standard business intelligence tools via ODBC interfaces. A particularly common front-end for iLuminate appears to be QlikView, which Joe believes doesn’t show good performance on its own if it’s asked to load more than 100 megs or so of data into RAM. I got the impression that illuminate’s own not-SQL-based tools were used mainly for information exploration and discovery, while more repetitive reporting and dashboarding are more usually done via third-party tools. (But that may be somewhat off; we didn’t talk about the point in much detail.)

Where things get really unclear is the database size range that iLuminate is suitable for. Current customers top out at 100 gigs or so, and standard POCs are limited to 25 gigs. On the other hand, illuminate also wants to do “super-size” POCs, and would be happy for those to be in the 5-20 terabyte range. iLuminate isn’t MPP, so a big database would run on a large SMP box. Joe says they’ve tested the product internally up to 1 terabyte or so. A 100 gig database runs happily on a $15K commodity server, but I must confess upon reflection to not being particularly impressed by that fact.

All figures are user data, by the way. Expansion ratio is about 1:1. Values aren’t compressed, but tend to get repeated more in bigger databases. Hence as databases get larger, an increasing fraction of total data volume is in the indexes, and those are smaller than the raw data itself. Joe says query response times are typically 1/10 second or less, unless there are extremely large result sets.

Applications to date seem to span the BI gamut – lots of customer analysis, some product analysis, 2 insurance companies looking at agent payouts, and some random stuff. (E.g., the ability to look anywhere for an odd number of Euros was helpful in one auditing use.) But then, 30 installations in three years in a country the size of Spain isn’t trivial, so one would expect a range of uses.