August 25, 2009

Sybase IQ technical highlights

General highlights of the Sybase IQ technical story include:

Highlights of the Sybase IQ compression story include:

Highlights of the Sybase IQ update and load story include:

Highlights of the Sybase IQ concurrency, scalability, and workload management story include:

Related links

Comments

11 Responses to “Sybase IQ technical highlights”

  1. Sybase IQ business notes | DBMS2 -- DataBase Management System Services on August 25th, 2009 5:17 am

    [...] Sybase IQ technical highlights [...]

  2. Daniel Lemire on August 25th, 2009 8:39 am

    Thoughts: I’m guessing that by “projection” they mean bit packing. In any case, it would be really useful if people in this industry agreed to some common definitions, or, minimally, to share their definitions. Reinventing the whole database language for each and every company is painful.

  3. Mark Callaghan on August 25th, 2009 12:49 pm

    Bitmap indexes in Oracle support high-cardinality columns and their effectiveness in that case is determined by the number of rows per distinct index value, not by the index cardinality. The cost to update them is not as high as you mention for ‘High Group’ indexes in Sybase IQ. Is Sybase IQ storing one bitmap per index value and replacing the entire bitmap on update?

  4. Justin Swanhart on August 25th, 2009 5:59 pm

    I’m under the impression that Sybase IQ supports compressed ‘binned’ bitmap indexes. These would require a lot of work to update, though I don’t think they would have to recalculate the entire bitmap.

  5. Jon Smirl on August 25th, 2009 11:25 pm

    A couple of years ago there was a TechWave conference where they talked about their ability to do comparison operations on compressed data without decompressing it. I thought that was a cool idea.

  6. Justin Swanhart on August 26th, 2009 12:40 pm

    Jon,

    That is possible. An example of such a bitmap index is implemented as Fastbit, an LGPL project.

    It implements a type of bitmap compression which supports logical operations between bitmaps w/out decompressing.

    You can find it here:
    https://sdm.lbl.gov/fastbit/

  7. Julian Hyde on August 27th, 2009 12:58 pm

    Jon, Justin,

    Those capabilities are quite straightforward if the database uses dictionary compression and compresses to a fixed number of bits per value. LucidDB does the same in its compressed column stores.

    http://www.luciddb.org

  8. Curt Monash on August 27th, 2009 1:06 pm

    I haven’t drilled down to the level of detail I’d like on the various Sybase index kinds, so I’ll duck questions like Mark’s.

    CAM

  9. Justin Swanhart on August 27th, 2009 5:42 pm

    Fastbit doesn’t use dictionaries. It uses Word-Aligned-Hybrid compression (for which they have established an “open” patent) and is available LGPL.

    I used it briefly as part of a POC at AdBrite and was VERY impressed. I’ve though about integrating it as a storage engine for MySQL, but MySQL is row engine oriented which sucks for a column store like FastBit.

    Eigenbase is pretty cool, and would make a nice frontend to add support for joins and other such SQL constructs which Fastbit currently doesn’t support.

  10. Enterprise headlines and summaries, 2009-08-25 « Next Gen Enterprise on August 28th, 2009 1:44 am

    [...] Sybase IQ technical highlights General highlights of the Sybase IQ technical story include: * Sybase IQ is an analytic DBMS with a columnar/column-store architecture * Unlike most analytic DBMS, Sybase IQ has a shared-disk architecture. * The Sybase IQ indexing story is a bit complicated, with a bunch of different index kinds. Most are focused on columns with low cardinality, and it least in some cases are a lot like bitmaps. (Sybase IQ when first introduced was a pure bitmap index product, with a single index type “Fast Project”.) But one index kind, “High Group” — designed for columns with high cardinality – is an exception to most generalities about other Sybase IQ index kinds, and instead is more akin to a b-tree. [...]

  11. Mike Stonebraker on “real column stores” | DBMS 2 : DataBase Management System Services on January 12th, 2011 9:46 am

    [...] A lesser oopsie is Mike’s criterion “IO-1″, which is written so confusingly that it technically seems not to be met by any of the vendors cited — including Vertica, which introduced Vertica FlexStore in mid-2009.  And while I’m at it — Aster Data nCluster definitely meets criterion IO-3; I confirmed that by asking Tasso Agyros. Mike’s “No” for Sybase IQ on his criterion CPU-5 is also pretty questionable, given that Sybase IQ operates on compressed data until “the last possible moment.” [...]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.