My current customer list among the analytic DBMS specialists

(This is an updated version of an August, 2008 post.)

One of my favorite pages on the Monash Research website is the list of many current and a few notable past customers. (Another favorite page is the one for testimonials.) For a variety of reasons, I won’t undertake to be more precise about my current customer list than that. But I don’t think it would hurt anything to list the analytic/data warehouse DBMS/appliance specialists in the group. They are:

• Aster Data
• Greenplum
• Infobright
• Kickfire
• Kognitio
• Microsoft
• Netezza (my biggest client this year, probably, because of all the Enzee Universe appearances)
• Sybase
• Teradata
• Vertica
• Attivio, which may or may not be construed as being in the analytic DBMS business
• Clearpace, ditto

All of those are Monash Advantage members.

If you care about all this, you may also be interested in the rest of my standards and disclosures.

Ingres update

I talked with Ingres today. Much of the call was fluff — open-source rah-rah, plus some numbers showing purported success, but so finely parsed as to be pretty meaningless. (To Ingres’ credit, they did offer to let me talk w/ their CFO, even if they offered no promises as to whether he’d offer any more substantive information.) Highlights included:

Read more

Draft slides on how to select an analytic DBMS

I need to finalize an already-too-long slide deck on how to select an analytic DBMS by late Thursday night.  Anybody see something I’m overlooking, or just plain got wrong?

Edit: The slides have now been finalized.

Gartner’s 2008 data warehouse database management system Magic Quadrant is out

Gartner’s annual Magic Quadrant for data warehouse DBMS is out.  Thankfully, vendors don’t seem to be taking it as seriously as usual, so I didn’t immediately hear about.  (I finally noticed it in a Greenplum pay-per-click ad.)  Links to Gartner MQs tend to come and go, but as of now here are two working links to the 2008 Gartner Data Warehouse Database Management System MQ.  My posts on the 2007 and 2006 MQs have also been updated with working links. Read more

Patent nonsense in the data warehouse DBMS market

There are two recent patent lawsuits in the data warehouse DBMS market. In one, Sybase is suing Vertica. In another, an individual named Cary Jardin (techie founder of XPrime, a sort of predecessor company to ParAccel) is suing DATAllegro. Naturally, there’s press coverage of the DATAllegro case, due in part to its surely non-coincidental timing right after the Microsoft acquisition was announced and in part to a vigorous PR campaign around it. And the Sybase case so excited a troll who calls himself Bill Walters that he posted identical references to it on about 12 different threads in this blog, as well as to a variety of Vertica-related articles in the online trade press. But I think it’s very unlikely that any of these cases turn out to much matter. Read more

Netezza update

In my usual dual role, I called Phil Francisco of Netezza to lay some post-Microsoft/DATAllegro consulting on him late on a Friday night — and then took the opportunity of being on the phone with him to get a general Netezza update. Netezza’s July quarter just ended, so they’re still in quiet period, so I didn’t press him for a lot of numerical detail. More generally, I didn’t find a lot out that wasn’t already covered in my May Netezza update. But notwithstanding all those disclaimers, it was still a pretty interesting chat. Read more

The IRS data warehouse

According to a recent Eric Lai Computerworld story and a 2006 Sybase.com success story,

• The IRS has a data warehouse running on Sybase IQ, with 500 named users, called the CDW (Compliance Data Warehouse). (Computerworld)
• By some metric, it’s a 150 TB warehouse. (Computerworld)
• By some metric, they add 15-20 TB/year, with a 4 hour load time. (Computerworld)
• As of 2006, there were 20-25 TB of “input data”, with a “70% compression rate”. (Sybase)

I can’t entirely reconcile those numbers, but in any case the database sounds plenty big.

Computerworld also said:

the research division also uses Microsoft Corp.’s SQL Server to store all of the metadata for the data warehouse and the rest of the agency. Managing and cleaning all of that metadata — 10,000 labels for 150 databases — is a huge task in itself,

ANTs bails out of the DBMS market

ANTs Data Server — i.e., the ANTs DBMS — has been sold off to a company called 4Js. It is now to be called Genero DB. Actually, 4Js has been selling or working on a version of the product called Genero DB since 2006, specifically an Informix-compatible one.

I’m not totally clear on why an Informix-compatible DBMS is needed in a world that already has Informix SE, but maybe IBM is overcharging for maintenance even on the low-end version of the product.

Meanwhile, ANTs, which had originally tried to get enterprises to migrate away from Oracle, is now focused on middleware called the ANTs Compatibility Server to help them migrate to Oracle, specifically/initially from Sybase.

Positioning the data warehouse appliances and specialty DBMS

There now are four hardware vendors that each offer or seem about to announce two different tiers of data warehouse appliances: Sun, HP, EMC, and Teradata. Specifically:

• Sun partners with both Greenplum and ParAccel.

• HP sells Neoview, and also is partnered with Vertica.

• EMC (together with Dell in North America and Bull in Europe) sells DATAllegro. Now EMC is also entering a partnership with ParAccel.

• Teradata is pretty far down the road toward releasing a low-end product.

Read more

Odd article on Sybase IQ and columnar systems

Intelligent Enterprise has an article on Sybase IQ and columnar systems that leaves me shaking my head. E.g., it ends by saying Netezza has a columnar architecture (uh, no). It also quotes an IBM exec as saying only 10-20% of what matters in a data warehouse DBMS is performance (already an odd claim), and then has him saying columnar only provides a 10% performance gain (let’s be generous and hope that’s a misquote).

Also from the article — and this part seems more credible — is:

“Sybase IQ revenues were up 70% last year,” said Richard Pledereder, VP of engineering. … Sybase now claims 1,200 Sybase IQ customers. It runs large data warehouses powered by big, multiprocessor servers. Priced at $45,000 per CPU, those IQ customers now account for a significant share of Sybase’s revenues, although the company won’t break down revenues by market segment.

Read more

Load speeds and related issues in columnar DBMS

Please do not rely on the parts of the post below that are about ParAccel. See our February 18 post about ParAccel instead.

I’ve already posted about a chat I had with Mike Stonebraker regarding Vertica yesterday. I naturally raised the subject of load speed, unaware that Mike’s colleague Stan Zlodnik had posted at length about load speed the day before. Given that post, it seems timely to go into a bit more detail, and in particular to address three questions:

1. Can columnar DBMS do operational BI?
2. Can columnar DBMS do ELT (Extract-Load-Transform, as opposed to ETL)?
3. Are columnar DBMS’ load speeds a problem other than in issues #1 and #2?

Read more

Vertica update

I chatted with Andy Ellicott and Mike Stonebraker of Vertica today. Some of the content is embargoed until February 19 (for TDWI), but here are some highlights of the rest.

• Vertica now is “approaching” 50 paid customers, up from 15 or so in early November. (Compared to most of Vertica’s fellow data warehouse specialists, that’s a lot.) Many — perhaps most — of these customers are hedge funds or telcos.
• Vertica’s typical lag from sale to deployment is about one quarter.
• Vertica’s typical initial selling price is $250K. Or maybe it’s $100-150K. The Vertica guys are generally pretty forthcoming, but pricing is an exception. Whatever they charge, it’s strictly per terabyte of user data. They think they are competitive with other software vendors, and cheaper, all-in, than appliance vendors.
• One subject on which they’re totally non-forthcoming (lawyers’ orders) is the recent patent lawsuit filed by Sybase. They wouldn’t even say whether they thought it was bogus because they didn’t infringe, or whether they thought it was bogus because the patent shouldn’t have been granted.
• Average Vertica database size is a little under 10 terabytes of user data, with many examples in the 15-20 Tb range. Lots of customers plan to expand to 50-100 Tb.
• Vertica claims sustainable load speeds of 3-5 megabytes/sec/node, irrespective of database size. Data is sucked into RAM uncompressed, then written out a gig/node at a time, compressed. Gigabyte chunks are then merged on disk, which is superfast as it doesn’t involve sorting. (30 megabytes/second.) Mike insists this doesn’t compromise compression.

We also addressed the subject of Vertica’s schema assumptions, but I’ll leave that to another post.

IBM acquires SolidDB to compete with Oracle TimesTen

IBM is acquiring Solid Information Technology, makers of solidDB. Some quick comments:

• solidDB is actually a very interesting hybrid disk/in-memory memory-centric database management system. However, the press release announcing the deal makes it sound as if solidDB is in-memory only.
• That strongly suggests that IBM is buying Solid mainly to compete with Oracle TimesTen. As of last June, solidDB was already IBM’s TimesTen answer via a partnership; this deal just solidifies that arrangement.
• This probably isn’t good news for Solid’s MySQL engine. That’s a pity, since solidDB technically has the potential to be the best MySQL engine around.
• Notwithstanding IBM’s presumed intentions, Solid’s main market success historically is as an embedded system in telecommunications equipment, network software, and similar systems.
• Last year I wrote a white paper on memory-centric data management, showcasing four products. IBM now has bought two of them, namely Solid’s and Applix’s (via Cognos).
• Comparisons to IBM’s embedded Java DBMS Cloudscape are pointless. That’s just a failed product vs. solidDB or Sybase SQL Anywhere, and IBM long ago cut its losses.

Read more

A quick survey of data warehouse management technology

There are at least 16 different vendors offering appliances and/or software that do database management primarily for analytic purposes.* That’s a lot to keep up with,. So I’ve thrown together a little overview of the analytic data management landscape, liberally salted with links to information about specific vendors, products, or technical issues. In some ways, this is a companion piece to my prior post about data warehouse appliance myths and realities.

*And that’s just the tabular/alphanumeric guys. Add in text search and you run the total a lot higher.

Numerous data warehouse specialists offer traditional row-based relational DBMS architectures, but optimize them for analytic workloads. These include Teradata, Netezza, DATAllegro, Greenplum, Dataupia, and SAS. All of those except SAS are wholly or primarily vendors of MPP/shared-nothing data warehouse appliances. EDIT: See the comment thread for a correction re Kognitio.

Numerous data warehouse specialists offer column-based relational DBMS architectures. These include Sybase (with the Sybase IQ product, originally from Expressway), Vertica, ParAccel, Infobright, Kognitio (formerly White Cross), and Sand. Read more

Gartner 2007 Magic Quadrant for Data Warehouse Database Management Systems

It’s early autumn, the leaves are turning in New England, and Gartner has issued another Magic Quadrant for data warehouse DBMS.  (Edit: As of January, 2009, that link is dead but this one works.) The big winners vs. last year are Greenplum and, secondarily, Sybase. Teradata continues to lead. Oracle has also leapfrogged IBM, and there are various other minor adjustments as well, among repeat mentionees Netezza, DATAllegro, Sand, Kognitio, and MySQL. HP isn’t on the radar yet; ditto Vertica. Read more

Marketing versus reality on the one-petabyte barrier

Usually, I don’t engage in the kind of high-speed quick-response blogging I have over the past couple of days from the Teradata Partners conference (and more generally have for the past week or so). And I’m not sure it’s working out so well.

For example, the claim that Teradata has surpassd the one-petabyte mark comes as quite a surprise to variety of Teradata folks, not to mention at least one reliable outside anonymous correspondent. That claim may indeed be true about raw disk space on systems sold. But the real current upper limit, according to CTO Todd Walter,* is 5-700 terabytes of user data. He thinks half a dozen or so customers are in that range. I’d guess quite strongly that three of those are Wal-Mart, eBay, and an unspecified US intelligence agency.

*Teradata seems to have quite a few CTOs. But I’ve seen things much sillier than that in the titles department, and accordingly shan’t scoff further — at least on that particular subject.

On the other hand, if anybody did want to buy a 10 petabyte system, Teradata could ship them one. And by the way, the Teradata people insist Sybase’s claims in the petabyte area are quite bogus. Teradata claims to have had bigger internal systems tested earlier than the one Sybase writes about.

The petabyte machine

EMC has announced a machine — a virtual tape library — that supposedly stores 1.8 petabytes of data. Even though that’s only 584 terabytes uncompressed, it shows that the 1 petabyte barrier will be broken soon no matter how unhyped the measurement.

I just recently encountered some old notes in which Sybase proudly announced a “1 gigabyte challenge.” The idea was that 1 gig was a breakthrough size for business databases.

Time flies.

Naming the DBMS disruptors

Edit: This post has largely been superseded by this more recent one defining mid-range relational DBMS.

I find myself defining a new product category – midrange OLTP/multipurpose DBMS. (Or just midrange DBMS for brevity.) Nothing earthshaking here; I’m simply referring to those products that: Read more

Deal prospects for data warehouse DBMS vendors

The fourth Monash Letter is now posted for Monash Advantage members (just 3 pages this time). It’s about forthcoming M&A in data warehouse DBMS, something that seems likely just because of the large number of current players. Some of the observations are:

• Oracle needs to buy somebody, because of its rather dire product problems at the data warehouse high end. And it’s very much in keeping with their recent behavior to do so.
• Teradata could be acquired sooner than people think. While there are tax considerations preventing an outright sale, these should be obviated if all of the current NCR is taken private. What’s more NCR minus Teradata is exactly the kind of healthy, slow-growth, niche company that private equity loves.
• DATAllegro is a natural merger partner for somebody. Their technical differentiation is almost DBMS-independent, so it could be easy to roll them into a larger overall product strategy. And they have enough market traction to have proved some non-trivial value.
• Kognitio seems desperate these days, with several odd or even underhanded marketing tactics. But they do have MPP bitmap software, something Sybase sorely lacks. So there’s an obvious potential combination between those two.

Mike Stonebraker on database compression — comments

In my opinion, the key part of Mike Stonebraker’s fascinating note on data compression was (emphasis mine):

The standard wisdom in most row stores is to use block compression. Hence, a storage block is compressed using a single technique (say Lempel-Ziv or dictionary). The technique chosen then compresses all the attributes in all the columns which occur on the block. In contrast, Vertica compresses a storage block that only contains one attribute. Hence, it can use a different compression scheme for each attribute. Obviously a compression scheme that is type-specific will beat an implementation that is “one size fits all”.

It is possible for a row store to use a type-specific compression scheme. However, if there are 50 attributes in a record, then it must remember the state for 50 type-specific implementations, and complexity increases significantly.

In addition, all row stores we are familiar with decompress each storage block on access, so that the query executor processes uncompressed tuples. In contrast, the Vertica executor processes compressed tuples. This results in better L2 cache locality, less main memory copying and generally much better performance.

Of course, any row store implementation can rewrite their executor to run on compressed data. However, this is a rewrite – and a lot of work.

Read more

The end of the single-server DBMS vendor

For all practical purposes, there are no DBMS vendors left advocating single-server strategies. Oracle was the last one, but it just acquired in-memory data management vendor TimesTen, which will be used as a cache in front of high-performance Oracle databases. (It will also continue to be sold for stand-alone uses, especially in the financial trading and defense/intelligence markets.)

IBM’s Viper is a server-and-a-half story, with lots of integration over a dual-server (one relational, one native XML) base. IBM also is moving aggressively in data integration/federation, with Ascential and many other acquisitions. It also sells a broad range of database products itself, including two DB2s, several Informix products, and so on.

Microsoft also has a multi-server strategy. In its case, relational, text, and MOLAP storage are more separate than in Oracle’s or even IBM’s products; again, there’s a thick layer of technology on top integrating them. An eventual move to native XML storage will, one must imagine, be handled in the same way.

Smaller vendors Sybase and Progress also offer multiple DBMS each.

Teradata is a pretty big player with only one DBMS — but it’s specialized for data warehousing. Teradata is the first to tell you you should use something else for your classical transaction processing.

The Grand Unified Integrated Database theory is, so far as I can tell, quite dead. Some people just refuse to admit that fact.

Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

• DBMS2 covers database management, analytics, and related technologies.
• Text Technologies covers text mining, search, and social software.
• Strategic Messaging analyzes marketing and messaging strategy.
• The Monash Report examines technology and public policy issues.
• Software Memories recounts the history of the software industry.

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.

• Recent posts

  • User data vs. raw disk space as a marketing metric
  • The TPC-H schema
  • Notes on columnar/TPC-H compression
  • Storage humor
  • NoSQL?

• Categories

  • About this blog
  • Analytic technologies
    • Business intelligence
    • Data mart outsourcing
    • Data warehousing
    • MOLAP
  • Application areas
    • Games and virtual worlds
    • Investment research and trading
    • Scientific research
    • Telecommunications
    • Web analytics
  • Buying processes
    • Benchmarks and POCs
  • Companies and products
    • 1010data
    • Ab Initio Software
    • Aleri and Coral8
    • Alpha Five
    • Amazon and its cloud
    • ANTs Software
    • Aster Data
    • Business Objects
    • Calpont
    • Cast Iron Systems
    • Clearpace
    • Cloudera
    • Cogito and 7 Degrees
    • Cognos
    • CouchDB
    • DATAllegro
    • Dataupia
    • Elastra
    • EMC
    • EnterpriseDB and Postgres Plus
    • Exasol
    • Expressor
    • FileMaker
    • Google
    • Greenplum
    • Hadoop
    • HP and Neoview
    • IBM and DB2
      • pureXML
    • illuminate Solutions
    • Infobright
    • Informatica
    • Information Builders
    • Inforsense
    • Ingres
    • Intel
    • Intersystems and Cache'
    • Jaspersoft
    • Kalido
    • Kickfire
    • Kognitio
    • Mark Logic
    • McObject
    • Microsoft and SQL*Server
    • MySQL
    • Netezza
    • Oracle
      • Exadata
    • Oracle TimesTen
    • ParAccel
    • Pentaho
    • Pervasive Software
    • PostgreSQL
    • Progress, Apama, and DataDirect
    • QlikTech and QlikView
    • SAND Technology
    • SAP AG
    • SAS Institute
    • ScaleDB
    • Software AG
    • solidDB
    • StreamBase
    • Sybase
    • Tableau Software
    • Talend
    • Teradata
    • Tokutek
    • Truviso
    • Vertica Systems
    • VoltDB and H-Store
  • Data integration and middleware
    • Application servers
    • EAI, EII, ETL, ELT, ETLT
  • Data types
    • GIS and geospatial
    • Native XML
    • Object
    • RDF and graphs
    • Text
  • DBMS product categories
    • Archiving and information preservation
    • Data warehouse appliances
    • Mid-range
    • OLTP
    • Open source
  • Emulation, transparency, portability
  • Fun stuff
    • Humor
  • Market share
  • Memory-centric data management
    • Cache
    • Complex event processing (CEP)
    • In-memory DBMS
  • Michael Stonebraker
  • Parallelization
    • MapReduce
  • Presentations
  • Pricing
  • Software as a Service (SaaS)
    • Cloud computing
  • Specific users
    • eBay
    • Facebook and Cassandra
    • Fox and MySpace
    • TEOCO
    • Yahoo
  • Storage
    • Solid-state memory
  • Theory and architecture
    • Columnar database management
    • Data models and architecture
    • Database compression
    • Database diversity
    • Petabyte-scale data management
  • TransRelational
  • Uncategorized

• Date archives

  Select Month July 2009  (6) June 2009  (25) May 2009  (19) April 2009  (30) March 2009  (23) February 2009  (29) January 2009  (18) December 2008  (10) November 2008  (13) October 2008  (29) September 2008  (60) August 2008  (28) July 2008  (23) June 2008  (7) May 2008  (16) April 2008  (18) March 2008  (22) February 2008  (25) January 2008  (35) December 2007  (18) November 2007  (12) October 2007  (31) September 2007  (16) August 2007  (11) July 2007  (9) June 2007  (16) May 2007  (6) April 2007  (18) March 2007  (17) February 2007  (8) January 2007  (13) December 2006  (1) November 2006  (1) October 2006  (7) September 2006  (17) August 2006  (6) July 2006  (12) June 2006  (1) May 2006  (7) April 2006  (7) March 2006  (2) February 2006  (4) January 2006  (13) December 2005  (9) November 2005  (9) October 2005  (10) September 2005  (1) August 2005  (6)

• Links

  • Monash Research
  • Webcasts
  • White Papers

• Admin

  • Log in

• Home
• About
• Contact
• Feeds