As I asked above — are there any unnatural acts of partitioning reflected in the SQL to get that kind of scalability?

Any serious DBMS can scale almost arbitrarily large if you just put a lot of database instances side by side …

CAM

You might want to look through http://www.dbms2.com/category/database-theory-practice/columnar-database-management/ for some ideas and answers. ParAccel and SAP would say that columnar architectures make memory-centric processing easier. Vertica and Infobright would say they make compression easier. DATAllegro and other row-based vendors, however, would offer the same skeptical questions you did.

Best,

CAM

where is the novelty of column-oriented DBMS? Is this storage architecture another name for vertical partitioning in traditional RDBMS?

Ileana

I’m trying to remember when Bob Epstein of Sybase first enthused to me about the Expressway acquisition, and I think it was a little earlier than the timeframe you’re suggesting.

Anyhow, after looking at your website I have a few suggestions:

1. If your main claim is speed, don’t have the benchmark link be dead.

2. Developer pricing is a bad business model in most markets.

3. Your web site doesn’t really say very much .

4. You need a copy editor who is a native English speaker.

Best regards,

CAM

well, BEFORE you have invent Vertica, and BEFORE Sybase have ship its column-oriented product, yet in 1998 year was introduced Valentina Database (www.paradigmasoft.com), with major development started at 1994-1995.

Intresting to compare

Nice paper! Did you guys sponsor it? I didn’t see any disclosure statements about that, but I noticed that “evaluation” was in quotes in the title.

Either way, I’m a great admirer of Philip Howard’s unrelentingly optimistic view of technology, as per http://www.dbms2.com/2006/05/15/philip-howard-likes-viper/. And I wonder whether it’s really true that the appliance vendors don’t do tokenization/dictionary compression. If they don’t, they surely should, and probably will soon.

Seriously, I’d be interested to learn what unnatural acts you did or didn’t have to perform to scale that high. And I’d really like to learn about the complexity you do or don’t offer in text analysis, since I’ve long thought that columnar relational indexing and text indexing were apt to fit very well together.

CAM

Bloor Research recently published an excellent evaluation white paper on Sybase IQ, authored by Philip Howard, which addresses (among other subjects) the Sybase IQ approach to parallelization.

http://www.sybase.com/content/1035804/SybaseIQ-12.7-010407-wp.pdf

As for Sybase IQ’s ability to scale — it has been dramatically demonstrated in a number of benchmark exercises (up to 155 TB) and customer implementations (40+ TB in production). A few examples:

http://www.sybase.com/detail_list?id=49108
http://www.sybase.com/detail?id=1027323
http://www-03.ibm.com/systems/p/solutions/sybase/iq/index.html

The entry of Vertica and other players into the column-based database market helps to demonstrate the growth potential of this space. We can expect to see more such entrants as database sizes continue to increase and organizations continue to look for technology that can reliably handle their analytics requirements.

Phil Bowermaster
Sybase