• My most important post of the week was a general guide to IT vendor strategy. That one has already spawned discussion at many companies, from the tiny to the multi-billion-dollar.
• The best comment thread of the week was probably on my post about scale-out relational OLTP choices, in which people discussed the merits of various particular alternatives.
• I recommended that people strongly consider attending XLDB 5 in Menlo Park on October 18-19.

Most of the posts, however, were reactions to news events. In particular:

• Teradata announced that Teradata 14 will be hybrid-columnar, more in Vertica’s way than in Greenplum’s or Aster Data’s. (Pay no attention to the Wall Street Journal’s apparent belief that no other analytic DBMS is hybrid-columnar at all.)
• Aster announced the unsurprising news that there will be a Teradata Aster appliance. Also, Aster talked about greater analytic flexibility in the forthcoming Aster 5.0.
• With Oracle OpenWorld coming up, Oracle decided to get some of its announcing out of the way early. In particular, it announced the Oracle Database Appliance, which is small-business-friendly hardware for running the Oracle DBMS. However, the Oracle Database Appliance doesn’t seem to do much about the complexity of running the Oracle DBMS software.
• In a catch-all Hadoop post, I noted that:
  • Oracle has now clearly said it has a Hadoop appliance coming, no doubt next week at OpenWorld.
  • I still can’t see why Hadoop appliances would succeed, but a lot of smart folks seem to disagree with me.
  • Greenplum announced what looks like a nice but unimportant little product upgrade.
  • It’s a really good thing that previously reported plans to revamp Hadoop are underway.
• DataStax announced that it really is a Cassandra company after all. Pay no attention to previous marketing that seemed to put DataStax in the same Hadoop-alternative category as, say, MapR.
• Ingres has changed its name to Actian. The announcement seems like a confession that Ingres and VectorWise are going nowhere.

• Changing its name to Actian.
• Deemphasizing Ingres, the product.
• Emphasizing a set of products that don’t exist yet (or at least aren’t shipping), namely lightweight mobile apps that are business-intelligence-plus-an-action, and technology for building them. These are called “Action Apps”, and are discussed on the Actian company blog.
• Positioning all this as something to do with “big data” (what a shock).

It turns out that Actian was the name of an ancient athletic competition commemorating Augustus’ defeat of Anthony at Actium, a battle that was more recently memorialized in the movie Cleopatra. Frankly, I think Cleopatra Software might have been a more interesting company name, although that could mean execs would have to arrive at sales calls rolled up in a carpet.

One article said:

Greg Wood, chief financial officer for Actian, told V3 that while the firm would continue to develop and maintain the Ingres database platform, its would be placing the spotlight on its Cloud Action Platform and its line of Action Apps.

“The Ingres database is well-recognised and we will continue to support it, but at the same time that brand was more associated with an older-generation technology,” Wood said.

“We think Actian better reflects where we are going as a company, particularly the application strategy.”

Wood explained that the platform would look to expand on the emerging field of big data applications by adding functionality for end users. The small, specialised applications would link up with data analytics tools, providing alerts and actions when various conditions are spotted within a database.

So what about VectorWise? Notwithstanding Actian’s stated focus on “big data”, I think VectorWise’s chances for market success are slim.* Reasons include:

• The market for shared-disk columnar analytic DBMS is crowded (Sybase IQ, Infobright, SAND). Those vendors also have to compete with MPP columnar analytic DBMS offerings from Vertica and ParAccel.
• I’ve never heard anything to make me believe VectorWise is getting significant market traction.
• Indeed, Daniel Abadi’s well-known flirtation with the idea of using VectorWise in HadoopDB/Hadapt excepted, I don’t recall any marketplace mention of VectorWise at all.

*The possibility of some kind of Action App synergy leads me to elevate them to “slim” from “none”.

The Action App idea actually sounds cool, but it’s quite a change from Ingres’ previous positioning and technology, and I have no basis for judging it as likely to succeed. On the other hand, companies have occasionally made successful transitions into business intelligence from relatively unrelated businesses before, most notably Cognos in the mid-1990s.

Note: Links to Gartner Magic Quadrants tend to be unstable. Please alert me if any problems arise; I’ll edit accordingly.

In my comments on the 2008 Gartner Data Warehouse Database Management Systems Magic Quadrant, I observed that Gartner’s “completeness of vision” scores were generally pretty reasonable, but their “ability to execute” rankings were somewhat bizarre; the same remains true this year. For example, Gartner ranks Ingres higher by that metric than Vertica, Aster Data, ParAccel, or Infobright. Yet each of those companies is growing nicely and delivering products that meet serious cutting-edge analytic DBMS needs, neither of which has been true of Ingres since about 1987. 

The general list of “market forces, end-user expectations and vendors’ resulting solution approaches” at the top of the 2010 Gartner Data Warehouse Database Management System Magic Quadrant article is a mixed bag. Following Gartner’s order, I’ll address those first, and particular companies cited afterwards. Specific items and comments include:

• “Increased demand for optimization techniques and performance enhancement.“ Gartner seems to be saying that data warehouse DBMS buyers want lists of specific, esoteric performance features. Well, buyers always want their DBMS to run fast, and they’d like the products to be mature enough to have been through a few rounds of Bottleneck Whack-A-Mole, but otherwise I’m not sure I’d put that at the top of my list.
• “The argument made by purchasing departments that buying power increases when dealing with a single, incumbent vendor.“ I agree that vendor consolidation and account control are a huge part of the Oracle, Microsoft, IBM and even Teradata stories. (Vertica can prove it’s 10X more price-performant than Oracle and still not get the business.) But it’s not just about price negotiations; once annual maintenance is included, one has to squint pretty hard to see Oracle as a low-cost alternative. Also important is reducing the number of total product-specific skill-sets needed on the IT staff.
• “Prepackaged, prebalanced warehouse environments delivered using data warehouse appliances.“ Yep. To varying extents, Oracle, Microsoft, Teradata, and IBM are all committed to designed-hardware strategies.
• “Expectations for the delivery of on-site POCs.“ Honestly, not as many buyers insist on on-site Proofs of Concept as should. Still, Oracle is shameful in its reluctance to do them. (Teradata tries to avoid them too, for obvious reasons of expense, but is much more gracious about capitulating when the buyer insists.)
• “Cost controls and data warehouse performance management.“ See next comment.
• “Demands for delivering a fully mixed workload.“ I’d have phrased the workload management and administrative tools points rather differently than this, but so be it.
• “Demands for departmental analytics delivered quickly via data marts.“ Agreed. Data-mart-only installations are a huge part of the market of the analytic DBMS market. Data mart spin-out is also important.
• “Wider indexing and fast performance within clusters of data, delivered via column-based solutions.“ This bizarrely seems to conflate column stores and parallel processing (both of which are of course highly important).
• “A wave of new data warehouse implementers seeking fast-track, low-risk delivery.“ Well, yes. Netezza noticed that quite some years ago. And by now the long-gestation EDW (Enterprise Data Warehouse) is widely disliked.
• “Global organizations seeking distributed solutions as potential architecture.“ If this is the MPP point, it’s oddly phrased. If this is a suggestion that data warehouses should be partitioned across wide-area networks, it’s just plain odd. If it’s a reiteration that departments like to control their own data marts, I agree. And if it’s a comment on keep-data-in-the-country privacy laws, it could be the most prescient thing Donald Feinberg has said in many years.

Long though it is, that list of general items and issues for the 2010 Gartner Data Warehouse Database Management System Magic Quadrant has some gaps. Most glaringly, I don’t see any references to advanced analytics in general, or even to the specific case of integrated predictive analytics. There’s also nothing about solid-state memory or other storage-technology considerations, although in fairness it’s still early days for much of what vendors conceive of as competitive differentiation in those respects.

Here are some vendor-specific comments on the 2010 Gartner Data Warehouse Database Management System Magic Quadrant:

• It’s pretty bizarre to compare 1010data to database.com or Microsoft Azure. Kognitio would be a better choice. So would cloud-hosted instances of Vertica, Aster Data nCluster, or others.
• Gartner’s comments on Aster Data and nCluster are actually pretty reasonable.
• Gartner’s comments on EMC/Greenplum are a bit Kool-Aid-drinky, and don’t account for the inevitable flailing that occurs right after an acquisition. But otherwise they’re pretty reasonable.
• I don’t take IBM’s super-comprehensive-all-inclusive architectural stories as seriously as Gartner does.
• I don’t take Netezza’s small stable of OEM partners as seriously as Gartner does. I also don’t share Gartner’s optimism for the continuation of Netezza’s NEC partnership in the face of IBM’s Netezza ownership.
• I’m even more skeptical about illuminate than Gartner is.
• I’m delighted that Gartner has adopted my phrase machine-generated data (Infobright is one of several firms pushing that one).
• “Only open-source column-store DBMS” is a bit exaggerated, but Infobright is indeed the only one with serious traction, or offered by a serious analytic DBMS vendor.
• What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.
• While Gartner’s write-up of Kognitio is a bit confused, that’s excusable. Kognitio’s strategy changes often.
• I’m not persuaded by the claim of low Microsoft TCO. The days when Microsoft’s tools were vastly better than the competition’s are long gone. And using an OLTP DBMS for data warehousing generally takes more people effort than using something more purpose-built.
• Gartner is right to ding Oracle for high prices, high people costs, and unwillingness to do onsite POCs.
• Gartner is right that Exadata is a huge improvement over non-Exadata Oracle data warehousing.
• Gartner is right to suggest that Exadata can easily handle data warehouses over 20 terabytes in size, but wrong to suggest that software-only Oracle also can. Just because the pain is less than it was with earlier releases of Oracle doesn’t mean it isn’t still bad.
• Gartner’s comments on ParAccel are pretty reasonable.
• Gartner’s comments on compression in connection with SAND make no technical sense (tokenization is a key form of columnar compression, not an alternative to it). Also, SAP’s acquisition of Sybase is a business challenge for SAND, not a technical one.
• Unless I’m forgetting something, Sybase IQ has no more in-database data mining than any other Fuzzy Logix partner does.
• Gartner failed to note that, like other DBMS dating back to the 1990s and before, Sybase IQ is more complex to administer than some newer products are.
• Gartner’s take on Teradata is pretty reasonable.
• Gartner’s take on Vertica, while sloppy, is basically sensible. However, Gartner failed to note that Vertica is a laggard in non-query analytics. (I am sure those deficiencies are being addressed, but Vertica’s competitors are moving ahead as well.)

Most of what I get paid for is in some form or other consulting. (The same would be true for many other analysts.) And so I can be a bit stingy with my advice toward non-clients. But my non-clients are a distinguished and powerful group, including in their number Oracle, IBM, Microsoft, and most of the BI vendors. So here’s a bit of advice for them too.

Oracle. On the plus side, you guys have been making progress against your reputation for untruthfulness. Oh, I’ve dinged you for some past slip-ups, but on the whole they’ve been no worse than other vendors.’ But recently you pulled a doozy. The analyst reports section of your website fails to distinguish between unsponsored and sponsored work.* That is a horrible ethical stumble. Fix it fast. Then put processes in place to ensure nothing that dishonest happens again for a good long time.

*Merv Adrian’s “report” listed high on that page is actually a sponsored white paper. That Merv himself screwed up by not labeling it clearly as such in no way exonerates Oracle. Besides, I’m sure Merv won’t soon repeat the error — but for Oracle, this represents a whole pattern of behavior.

Oracle. And while I’m at it, outright dishonesty isn’t your only unnecessary credibility problem. You’re also playing too many games in analyst relations.

HP. Neoview will never succeed. Admit it to yourselves. Go buy something that can. 

Smaller BI vendors. Analytic DBMS evaluations commonly include BI strategy and tool selection as well. If an analytic DBMS expert tells you he needs to learn more about your product line, don’t blow him off. In fact, you should be particularly embracing anybody who’s shown a fondness for small DBMS vendors; maybe he or his clients will like small BI vendors as well. That means (among others) Jaspersoft, Endeca, and Tableau.

Information Builders. Is there anything about your BI products that is in any way technologically differentiated? If so, you might want to mention some examples to somebody some time.

Kalido. I’ve said this to you before, but it bears repeating — your positioning translates to “I-CASE for analytics,” and that’s not a good thing. If your product is not as cumbersome and entrapping as that sounds, you need to do a much better job of explaining why not.

SenSage. You are what you are. Sell out while the selling is good. You don’t have the corporate personality to make it into the analytic DBMS mainstream on your own.

Ingres. You need to be more engaged with analysts than you are. Ingres navel-gazed too much 25 years ago, and evidently you haven’t outgrown it yet.

TIBCO. You probably have a lot of cool analytic technology, but I don’t know of an influencer who has much relationship with or trust in you. Rethink how you’re approaching influencer relations top to bottom.

Tableau. You had a lot of mindshare, but it’s fading. Do something.

MarkLogic, graph DBMS vendors, etc. You’re clinging too hard to the NoSQL label. Nobody is out there deciding among Cassandra, neo4j, and MarkLogic. They might be deciding between MongoDB and MarkLogic, I guess, but if you admit to yourself that’s all it is you’ll probably change your messaging somewhat.

Objectivity. Get real about marketing. Infinite Graph is a cool opportunity. But I didn’t even ping you for a meeting when I’m in your area next week, because I wouldn’t have known who to reach out to.

Everybody (especially Objectivity). “First X deployed in the cloud” is almost surely an inaccurate claim. Don’t make it. And by the way, even if it were true, it probably wouldn’t be interesting.

• VectorWise is indeed a shared-everything analytic DBMS.
• The VectorWise front-end is Ingres. Ingres VectorWise supports almost all SQL that Ingres does (there are a few edge-case exceptions).
• Conversely, Ingres VectorWise doesn’t support any SQL Ingres doesn’t, most notably SQL-99 Analytics. Naturally, SQL-99 Analytics is a roadmap item for Ingres/VectorWise.
• Ingres VectorWise 1.0 is pretty purely columnar. There’s a bit of PAX, but it’s mainly automagic/under the covers. The one user-controlled exception I understood was that one can ensure that composite keys are stored together.
• The main Ingres VectorWise performance secret sauce ingredients we touched on were:
  • Vectorization of operations (hence VectorWise’s name).
  • Compression that is tuned for speed rather than to minimize storage utilization.
• We unfortunately didn’t have time to revisit the other big part of the Ingres VectorWise performance story, namely clever design for modern microprocessor architectures. High-level generalities about that do pervade the Ingres VectorWise press release, but – well, they’re very high level.
• Unlike Vertica but like most other columnar DBMS vendors, Ingres VectorWise wants you to store your data once. You can index-organize the data. You can also organize multiple tables in the same order, to make joins among them fast.
• Support for actual join indexes is an Ingres VectorWise roadmap item.
• As do ever more analytic DBMS, Ingres VectorWise has something akin to Netezza zone maps.
• When I asked Peter what had changed most from the initial VectorWise development plan, other than the above, he basically said that their performance priorities had shifted a bit. Specifically, he said.
  • They had originally been “blinded” (his word) by the TPC-H benchmark, but figured out that they were overly focused on it. (Well, duh.)
  • They learned about the importance of other things such as data loading speeds.

]]> http://www.dbms2.com/2010/06/11/ingres-vectorwise-technical-highlights/feed/ 1 http://www.dbms2.com/2010/06/10/vectorwise-press-release/ http://www.dbms2.com/2010/06/10/vectorwise-press-release/#comments Thu, 10 Jun 2010 11:38:39 +0000 Curt Monash http://www.dbms2.com/?p=2242 Ingres forgot to prebrief me on the VectorWise announcement, and despite valiant efforts hasn’t succeeded in connecting with me since they realized the lapse. Meanwhile, I took a look at the VectorWise press release, and found the quotes to be somewhat amusing.

Daniel Abadi was quoted as saying:

“VectorWise … is now proven to be a terrific fit as database technology underpinning large-scale HadoopDB applications.”

notwithstanding that HadoopDB has never been used for a production application, large-scale or otherwise. Unsurprisingly, Daniel is quite emphatic in noting he didn’t actually say that.*

*Daniel actually said “has proven,” which to him means “[not] officially proven, but proven in a more colloquial sense.”

Meanwhile, a quote from the Rohatyn Group starts:

“For the past 20 years, we’ve been searching for the killer database …”

notwithstanding that the Rohatyn Group hasn’t been around for even 10 years, let alone 20. (I infer this from the LinkedIn profile of a “founder” who’s been working there only since 2002.) However, Peter Boncz assures me via e-mail that this is a genuine quote even so, and speculates quite reasonably that this database quest started at some other entity.

Otherwise, the quotes are the usual stuff, with highlights including:

• At least some queries run much faster on VectorWise than they do on a non-analytic DBMS.
• A couple of system integration partners are enthused about how happy their VectorWise customers will be if and when they ever have any.
• Ingres and VectorWise executives think their product is really cool.

Related link

• A press release quote that REALLY went awry

• Personally selected annual lists of 12 “Most influential” companies and 36 “Companies to watch” in analytics- and database-related sectors.
• Made it clear that these are his personal selections.
• Nonetheless has called it an Editors’ Choice list, rather than Editor’s Choice.

(Actually, he’s really called it an “award.”)

People advising Doug — who come to think of it actually are Contributing Editors to Intelligent Enterprise or something like that — included Cindi Howson, Seth Grimes, three others, and me.

And if past is prologue, I will now get a flood of PR emails calling my attention to this award that I already have both participated in and blogged about.

As usual, the sense:nonsense ratio on these lists was pleasingly high. Analytic DBMS vendors cited included IBM, Microsoft, Netezza, Oracle, Sybase, and Teradata in the “Most influential” group, with Aster, Greenplum, HP, Infobright, and Vertica among the “To watch” crowd. It’s tough to argue with those selections, whose most questionable element is probably the not-ridiculous supposition that HP could do something interesting over the coming year. Cloudera and Intersystems also made the list, deservedly.

All three of QlikTech, Tableau, and TIBCO made the list, which is appropriate given the potential for and interest in interactive data exploration technology.  The BI majors, independent or otherwise, were all on as well. In text mining, Doug included Attensity and Clarabridge, which I think is exactly right. (Plus OpenCalais.)  Upon reflection, I probably should have nominated Mark Logic, even though most of its business is non-enterprise; but hey, nobody’s perfect, and the same goes for lists. Open source was well represented, with Apache, Actuate, Jaspersoft, Eclipse, Infobright, Nuxeo and R all being cited (but not Ingres or Pentaho). Kalido made the list, with my endorsement, their silly I-CASE like marketing messaging notwithstanding.

Speaking of imperfections — there only are a few category names, and so category assignments can be pretty bizarre. (In an ideal world, middleware wouldn’t be included under “enterprise applications”.) Greenplum hasn’t really “extended” its DBMS with a “cloud” option. As much as I’d like Netezza to be more influential than SAP, that’s probably not the best way to rank them. And there are a number of “This company is on a roll!” kinds of comments that I wouldn’t necessarily endorse.

But those are all nitpicks. On the whole, it’s another nice job.

At intervals of a little over a year, Gartner Group publishes a Data Warehouse Database Management System Magic Quadrant. Gartner’s 2009 data warehouse DBMS Magic Quadrant — actually, January 2010 — is now out.* For many reasons, including those I noted in my comments on Gartner’s 2008 Data Warehouse DBMS Magic Quadrant, the Gartner quadrant pictures are a bad use of good research. Rather than rehash that this year, I’ll merely call out some points in the surrounding commentary that I find interesting or just plain strange.

*Links to Gartner Magic Quadrants commonly break, but that one worked at the time of this posting.

• Gartner thinks that data warehouse appliances are on the rise, due to their simplicity.
• Gartner correctly says that Teradata has been a data warehouse appliance vendor from the getgo.
• Gartner characterizes IBM as being an appliance vendor as well.
• Gartner suggests that HP is having trouble living up to its technical promises for Neoview.
• Gartner further suggests — no surprise here — that HP Neoview has had very few new customers past its initial wave.
• Gartner notes IBM’s difficulties in selling data warehouse installations of DB2, despite what on paper is great-sounding technology.
• Gartner says — also no surprise — that illuminate “has seen little success in North America since opening its first office in the U.S. over two years ago.”
• Ingres has evidently gotten a few BI-centric “appliance” deals, e.g. with Jaspersoft. But basically Ingres isn’t doing well in data warehousing.
• Gartner does say Ingres has “the strongest open-source DBMS offering for data warehousing.” Being very literal about “open source,” that’s a defensible claim — but it’s pretty irrelevant in a world where Greenplum Single-Node Edition can be had for free. It also waves away all the data mart use cases in which Infobright Community Edition shines.
• Gartner says that Netezza is working out as a “complex workload” enterprise data warehouse provider, according to reference checks, in addition to its established success in data mart scenarios.
• Gartner says Oracle’s offering has finally become “accepted” in the market for databases >50 TB. I guess I can live with that fairly weak claim, but I wouldn’t go much further than that.
• Gartner asserts that, unlike software-only Oracle, Oracle Exadata isn’t significantly harder to administer than “other mixed OLTP/OLAP DBMS vendors,” because Exadata is fast enough you don’t need to jump through all those hoops any more to get tolerable performance. The money quote is “one reference reported reducing the number of indexes by a factor of 100 to fewer than five.” Note, however, that Gartner does not seem to assert that Exadata’s ease of use rivals that of the newer analytic DBMS specialists.
• Gartner confirms Oracle’s reluctance to do onsite Exadata POCs, but says it is not absolute. This is roughly compatible with what I’m hearing elsewhere, and indeed with Oracle own claims to be ramping up availability of Exadata POC hardware.
• Gartner’s criteria for inclusion include at least 10 different organizations having a product “in production.” Thus, the big surprise was ParAccel being included. The money quote there is “With approximately 20 customers in the pharmaceutical, retail, financial and media/advertising analytics sectors, ParAccel has a good reference base.” That assessment is difficult to reconcile with other information, but I’ve been told Gartner is sticking to its guns. That assessment would be even harder to believe if those 20 references were all alleged to be true production customers.
• Gartner notes that you basically can’t run a 1 TB+ MySQL data warehouse without sharding. (Of course, Infobright has an alternative, and up to a small number of terabytes so does Kickfire.)
• Gartner reports that at least some customers are pleased with Sybase IQ’s mixed workload/enterprise data warehouse capabilities.
• Gartner correctly notes that Oracle Exadata is a price-competition challenge for Teradata.
• Gartner notes that 20% of Vertica’s customers are outside the US. While not shocking, that’s more than I realized.
• Gartner notes something I don’t think I’ve posted yet, which is that Vertica has a customer with 300 TB of data. (The identity is a deep dark secret, but if I told you you probably wouldn’t recognize the name anyway.)

As does any such piece, the Gartner Data Warehouse DBMS Magic Quadrant also has outright errors.  For example:

• Aster Data isn’t really “the newest entrant to the DBMS data warehouse world.”
• Aster’s SQL/MapReduce was not new in Release 4.0.
• Greenplum isn’t yet pushing down code to the storage tier.
• I’m not sure what kind of database-tier parallelism Gartner is claiming is new in Oracle in 11g Release 2 — but I doubt it’s really new. Rather, what Oracle has done recently is make parallelism less administratively cumbersome.
• Vertica wasn’t really the first DBMS in the cloud. At most it was the first pure-play analytic DBMS to get there.

• VectorWise, the product, will be an open-source columnar analytic DBMS. (But that’s not quite true. Pending productization, it’s more accurate to call the VectorWise technology a row/column hybrid.)
• VectorWise is due to be introduced in 2010. (Peter Boncz said that to me more clearly than I’ve seen in other coverage.)
• VectorWise and Ingres have a deal in which Ingres will at least be the exclusive seller of the VectorWise technology, and hopefully will buy the whole company.
• Notwithstanding that it was once named something like “MonetDB,” VectorWise actually is not the same thing as MonetDB, another open source columnar analytic DBMS from the same research group.
• The MonetDB and VectorWise research groups consist in large part of academics in Holland, specifically at CWI (Centrum voor Wiskunde en Informatica). But Ingres has a research group working on the project too. (Right now there are about seven “highly experienced” people each on the VectorWise and Ingres sides, although at least the VectorWise folks aren’t all full-time. More are being added.)
• Ingres and VectorWise haven’t agreed exactly how VectorWise and Ingres Classic will play together in the Ingres product line. (All of the obvious possibilities are still on the table.)
• VectorWise is shared-everything, just as Ingres is. But plans — still tentative — are afoot to integrate VectorWise with MapReduce in Daniel Abadi’s HadoopDB project.

The MonetDB project is led by Martin Kersten, with whom I chatted at SIGMOD in June (standing up and not taking notes, so I may have some details wrong). I get the impression, based on that conversation, my VectorWise call, and other data:

• Martin has been researching analytic DBMS (mainly but not only relational) since the late 1970s, and has been based at CWI since 1985.
• Peter Boncz has been either second in command of that crew or close to it.
• Martin Kersten, Peter Boncz, and the CWI/MonetDB team in general have gotten all sorts of computer science glory for their work.
• Martin has enjoyed generously stable government research funding for his group, but has found commercialization of the technology more difficult than he might at, stay, Stanford. The figure of 15 MonetDB researchers comes to mind, although I see from Martin’s bio that he oversees a team of ~55 in total.
• One early attempt at commercializing MonetDB turned into a company called Data Distilleries that was sold to SPSS. Peter Boncz was chief architect of Data Distilleries.
• Besides VectorWise, there are at least two other recent spin-off companies from the MonetDB project. One is a zero-headcount shell, set up to facilitate MonetDB project members (and others) consulting to users of the open source MonetDB technology. The other is in stealth mode, focusing on some vertical market.

I further get the impression that VectorWise was actually Marcin Zukowksi’s Master’s Ph.D project, with Peter Boncz being his advisor. VectorWise also boasts another Peter Boncz student, who wrote about updating column stores.

As one might expect from the name, VectorWise does vector processing. I.e., the hard part of Marcin’s work was developing vectorized algorithms for one SQL operation after another. Vectorization, pipelining, and FPGAs might all seem to go together — XtremeData certainly seems to think so — but the VectorWise folks preferred to develop for Intel CPUs anyway, for pretty much the usual reasons. Another major theme is trying to get the right things into CPU cache, because in their opinion RAM cache is just sooooo painfully slow.

Our discussion of VectorWise’s compression was interesting. Highlights included:

• The design requirement is that decompression work at a rate of 3 gigabytes/second or so. That way the system is faster overall than if it operated at 1 gigabyte/second on uncompressed data, which I gather is the alternative.
• VectorWise takes 4-5 steps CPU cycles to decompress a tuple.
• VectorWise says it sacrificed compression ratio to achieve speed. That said, VectorWise claims 3-4X compression on TPC-H data, which is no worse than what ParAccel reported, and enjoys higher compression rates on other kinds of data.
• VectorWise decompresses data before manipulating it, and claims that the advantages of operating on compressed data are only significant if — like Vertica but apparently unlike VectorWise — the database stores columns in multiple sort orders each.
• VectorWise’s compression is mainly on numerical and numerical-like (e.g. date) datatypes. An exception is that VectorWise uses dictionary compression on string data when it makes sense to do so.

Other notes include:

• VectorWise has technology akin to Microsoft SQL Server’s Shared Scans, in which multiple queries that require similar table scans don’t have to repeat all the redundant scanning work. I need to get better at figuring out which other analytic DBMS do similar things.
• While VectorWise hasn’t yet been open-sourced, its code is in the hands of some other academic institutions, used mainly for computer science research (as opposed to, say, as a data store for some kind of scientific experiment).
• VectorWise’s scalability has only been tested up to eight cores.

• Ingres is much of the way toward SQL 92 compatibility.
• Ingres also has added some industry-common proprietary SQL extensions. E.g., it appears multiple vendors have adopted Oracle’s way of doing substrings along with the standard way.
• Development chief Bill Maimone points out that when Ingres was still owned by CA, a lot of work was done to increase availability. (Arguably, Ingres’ main use at CA was as the underpinning for CA Unicenter.)
• I’m sure Ingres would disagree with this phrasing, but it sounds as if Ingres’ appliance strategy is more bait-and-switch than anything else. E.g., Ingres appliances don’t have the Red Hat version of Linux many users want.
• That said, Ingres’ BI appliance has enjoyed a “fair” number of small sales. Ingres claims – not unreasonably — that with a mature optimizer and parallel query, its product is suitable for at least lightweight reporting despite a lack of more advanced data warehousing features.
• Ingres does say, using the appliances as an example, that it built an installer much easier than what it had before.
• Although Ingres said something last August to Seth Grimes that sounded like it was going to integrate MonetDB, that’s not actually happening. Ingres does seem to have some kind of data warehousing speed-up project in the works, but was coy as to details.
• Ingres believes it will soon leapfrog closed-source vendors in its geospatial support, helped by significant community contributions. (Geospatial was the only part of Ingres it didn’t have the licensing rights to open-source, which is a bit part of the reason it’s being replaced.)
• However, it didn’t sound as if Ingres was doing much else in datatype extension.
• Speaking of data-warehousing-oriented enhancements, another example of nontrivial community contribution is partition-compatible hash aggregates.
• Another example is column encryption.
• Generally, Ingres echoes what other open source vendors say: The community will work on what it wants to work on, and many of the vendor’s priorities have to be implemented by the vendor itself.
• I didn’t ask directly, but I get the sense Ingres has concentrated some of it sales efforts in the financial industry, for three reasons. First, Ingres says it is doing a lot of Sybase replacements. Second, when Ingres said it hasn’t had as much success replacing Oracle, but has some hopes that this will change, it used financial companies as an example. Third, that’s the market Ingres used to say it was focusing on.
• Ingres says it also is doing OK in Microsoft replacements. But MySQL replacements are harder because MySQL is so non-standard that porting applications is more challenging.
• In another slap at MySQL, Ingres suggested that it’s hard to reference a BLOB in MySQL without pulling the whole thing down to the client.
• While conceding Postgres is good technology, Ingres asserted having a support organization is a competitive advantage. I asked a couple of times “What about EnterpriseDB?”, but didn’t get a response.

]]> http://www.dbms2.com/2009/04/02/ingres-update/feed/ 6