Sybase IQ technical highlights
General highlights of the Sybase IQ technical story include:
- Sybase IQ is an analytic DBMS with a columnar/column-store architecture
- Unlike most analytic DBMS, Sybase IQ has a shared-disk architecture.
- The Sybase IQ indexing story is a bit complicated, with a bunch of different index kinds. Most are focused on columns with low cardinality, and it least in some cases are a lot like bitmaps. (Sybase IQ when first introduced was a pure bitmap index product, with a single index type “Fast Project”.) But one index kind, “High Group” — designed for columns with high cardinality – is an exception to most generalities about other Sybase IQ index kinds, and instead is more akin to a b-tree.
- Unlike Vertica, Sybase stores each column of data only once. I don’t see how it would make sense to have multiple indexes on the same column, but I didn’t actually ask whether doing so is possible or common.
- Sybase estimates that Sybase IQ requires ¼ the DBA effort of, say, Oracle. (Frankly, that’s not a particularly good figure.) Obviously, this is just a broad-brush average.
- Sybase recently repurposed an acquired ETL tool to be focused on Sybase IQ. IQ of course also works with various third-party tools, certified or otherwise.
- Sybase’s Power Designer CASE (Computer-Aided Software Engineering)/database design tool works with Sybase IQ.
- Sybase is proud of Sybase IQ’s new in-database analytics capabilities, but I haven’t yet grasped what, if anything, is differentiated about them.
- Sybase has an ILM (Information Lifecycle Management) story built around the point that different columns can be stored on different kinds of media.
Highlights of the Sybase IQ compression story include: Read more
| Categories: Analytic technologies, Columnar database management, Data warehousing, Database compression, EAI, EII, ETL, ELT, ETLT, Sybase, Theory and architecture | 11 Comments |
Sybase IQ business notes
As specialized analytic DBMS go, Sybase is near the top of the charts both in age (Sybase IQ was first introduced in the mid 1990s) and adoption. That’s even more true, of course, if we restrict the discussion strictly to columnar DBMS, aka column stores. Basic Sybase IQ adoption claims include:
- >1500 users
- >3000 installations (Sybase has variously cited 2.1 and 2.5+ as the installation/user ratio)
- At least ~50-60 installations with >5 terabytes of user data
Note that 98% of Sybase IQ installations are under 5 terabytes; the heart of Sybase IQ’s business is the sub-terabyte data warehouse market.* Read more
| Categories: Analytic technologies, Data mart outsourcing, Data warehousing, Investment research and trading, Sybase | 3 Comments |
Teradata highlights some analytic use cases
A couple of slides in my recent briefing on Teradata’s Active Enterprise Data Warehouse Story contained long lists of analytic use cases, at a finer level of granualarity than I’m focusing on for a September speaking tour. I think they’re interesting to pass along. Read more
| Categories: Analytic technologies, Data warehousing, Teradata | 2 Comments |
Teradata’s Active Enterprise Data Warehouse story
Teradata used to tell a one-size-fits-all Enterprise Data Warehouse (EDW) story. That’s no longer the case. Last year, Teradata introduced a range of products. I think Teradata is serious about selling its full product range, and by now has achieved buy-in from its sales force for that strategy. I base these beliefs on data points such as:
- Teradata says so, repeatedly and persuasively.
- At least in passing, Teradata cites non-trivial sales figures for the appliance product lines.
- Competitors are less unanimous in asserting that Teradata’s lower-end products are presented on just a bait-and-switch basis.
But that raises the question: How does Teradata pitch the advantages of its top-end product line these days? At least at the corporate level, the answer seems to focus less on the “EDW” concept than it used to, and more on “Active.” Teradata – which actually has been talking about “Active Data Warehousing” for about a decade — indeed calls its top-end 55xx series the “Teradata Active Enterprise Data Warehouse.”
Teradata proudly told me that it has >100 customers who have truly adopted an “Active” EDW. When we discussed what that meant, supported by a whole lot of named examples, it became clear that “Active” data warehousing: Read more
| Categories: Analytic technologies, Data warehousing, Teradata | 6 Comments |
Social network analysis, aka relationship analytics
A number of applications lend themselves to graph-oriented analytics, including:
- Finding bad guys (national intelligence)
- Finding bad guys (anti-fraud)
- Data mining the social graph (e.g., for advertising optimization on social networks, or to identify influencers)
There are plenty more graph-oriented applications, of course, such as the identification of biochemical pathways. But I want to focus for now on ones like those on my list. My key points are:
- There are Big Data problems that lend themselves to graphical data models.
- So far as I can tell, the database management community isn’t doing enough to address them. (If I’m wrong about that, please tell me. I plan to arrive in Lyon for VLDB/XLDB Wednesday of next week, and of course I can always be reached by email.)
Here’s what I mean. Read more
| Categories: Analytic technologies, Cogito and 7 Degrees, Data models and architecture, Data types, RDF and graphs, Theory and architecture | 22 Comments |
Bottleneck Whack-A-Mole
Developing a good software product is often a process of incremental improvement. Obviously, that can happen in the case of feature addition or bug-fixing. Less obviously, there’s also great scope for incremental improvement in how the product works at its core.
And it goes even further. For example, I was told by a guy who is now a senior researcher at Attivio: “How do you make a good speech recognition product? You start with a bad one and keep incrementally improving it.”
In particular, I’ve taken to calling the process of enhancing a product’s performance across multiple releases “Bottleneck Whack-A-Mole” (rhymes with guacamole). This is a reference to the Whack-A-Mole arcade game,* the core idea of which is:
- An annoying mole pops its head up.
- You whack it with a mallet.
- Another pops its head up.
- You whack that one.
- Repeat, as mole_count increments to a fairly large integer.
| Categories: Data warehousing, Exadata, Fun stuff, Netezza, Oracle, Theory and architecture | 24 Comments |
Kickfire’s FPGA-based technical strategy
Kickfire’s basic value proposition is that, if you have a data warehouse in the 100s of gigabytes, they’ll sell you – for $32,000 – a tiny box that solves all your query performance problems, as per the Kickfire spec sheet. And Kickfire backs that up with a pretty cool product design. However, thanks in no small part to what was heretofore Kickfire’s penchant for self-defeating secrecy, the Kickfire story is not widely appreciated.
Fortunately, Kickfire is getting over its secrecy kick. And so, here are some Kickfire technical basics.
- Kickfire is MySQL-based, with all the SQL functionality and lack of functionality that entails.
- The Kickfire/MySQL DBMS is columnar, with the usual benefits in compression and I/O reduction.
- Kickfire is based on FPGAs (Field-Programmable Gate Arrays).
- The Kickfire DBMS is ACID-compliant.
- Kickfire runs only as a single-box appliance.
- While Kickfire earlier estimated that, at least for data sets that compressed well, a Kickfire box could hold 3-10 terabytes of user data, more recent figures I’ve heard from Kickfire have been in the 1-1 /2 terabyte range. (Edit: Karl Van Der Bergh subsequently wrote in to say that the 1 1/2 TB is raw disk figure, not user data.)
The new information there is that Kickfire relies on an FPGA; Read more
| Categories: Analytic technologies, Columnar database management, Data warehouse appliances, Data warehousing, Database compression, Kickfire, MySQL, Theory and architecture | 16 Comments |
Sorting out Netezza and Oracle Exadata data warehouse appliance pricing
Netezza recently announced a new generation of data warehouse appliance called TwinFin. TwinFin’s clearest stated list price is “a little under $20,000 per terabyte of user data,” which in my opinion immediately became the new industry reference point for discussing prices in the data warehouse appliance category. Vigorous discussion ensued, especially in the comment thread to the first of the two posts linked above. Here’s some followup.
Netezza should not have claimed a “10-15X price/performance improvement,” based on a 3-5X performance improvement and a 3X decrease in price/terabyte, and I should have grilled Netezza harder when it first made the claim. In fact, there is no unit of performance that you can, in a reasonable blended average, get 10-15X more of per dollar in TwinFin than you can in the predecessor NPS series.
| Categories: Data warehousing, Exadata, Netezza, Oracle, Pricing | 19 Comments |
What does Netezza do in the FPGAs anyway, and other questions
The news of Netezza’s new TwinFin product family has generated a lot of comments and questions, some pretty reasonable, some quite silly. E.g., I’ve seen it suggested privately or publicly that
- Netezza’s older products only handle one query at a time (nonsense, and I’m going to loyally protect the identity of the person who emailed that odd suggestion to me)
- A Netezza node can be a single point of failure (also nonsense, although performance degradation from a node failure might be considerable)
- Netezza has a cache consistency problem (also hardly true, except insofar as it’s an issue to overcome in future development as Netezza moves toward parallelizing bulk loads, transactional updates, and/or trickle feeds).
Netezza’s Phil Francisco addressed some points of this nature in a recent blog post.
More reasonable is the question:
Now that Netezza has changed its architecture, what are all those FPGAs (Field-Programmable Gate Arrays) being used for anyway?
The short answer is: Read more
| Categories: Data warehouse appliances, Data warehousing, Netezza, Theory and architecture | 6 Comments |
Dataupia is officially for sale
Dataupia marketing VP Samantha Stone — who by the way has been one heck of a trooper through Dataupia’s troubles — is joining the exodus from the company. General graciousness aside, the heart of Samantha’s farewell email reads:
Unfortunately, we have had to reduce our burn rate as we seek an acquirer for our technology.
We have a group of loyal employees remaining on staff focused on current production customers and the acquisition efforts.
As part of the most recent staff reductions I will be leaving Dataupia.
Two years ago I wrote:
[Dataupia would] make a great acquisition for a BI company or DBMS vendor who could then say “Oh, no, this isn’t a DBMS appliance – it’s merely a data warehouse accelerator.” When you look at it that way, their chances of prospering look distinctly higher.
But at this point I think there probably would be more appealing ways for those vendors to meet the same needs.