September 20, 2006

Netezza’s chip story

In addition to its software story, Netezza of course has a rather unique chip story. Where other vendors might have standard disk controllers and high-powered microprocessors, Netezza respectively has a FPGA (Field-Programmable Gate Array) and lesser microprocessor (PowerPC). Netezza claims that two major advantages of these choices are:

The main function of the FPGA, other than generically getting data on and off disk, is to restrict and project tables (i.e., execute single-table WHERE clauses). Netezza claims that their FPGAs can perform these operations on the streaming data at least as quickly as an expensive, hot, power-hungry top-end microprocessor would, and indeed faster. The key word is “streaming”, which they contrast to the microprocessor’s need to get the data in and then back out of RAM (cache or otherwise).

I’ll be interested to see whether somebody can muster a ringing refutation to Netezza’s claims.

Comments

12 Responses to “Netezza’s chip story”

  1. DBMS2 — DataBase Management System Services»Blog Archive » No locks, no logs — no problem? on September 20th, 2006 4:15 am

    […] There’s another cool-sounding part to the Netezza story, which straddles their chips and their software: The FPGA takes over the work of assuring database consistency. If the system attempts to read and write a record at the same time, the FPGA keeps thing straight. This eliminates the need for locks — at least if you don’t care about transactional integrity — and some of the reason for logs. (I guess that in lieu of any kind of rollback/rollforward they just rely on failover to mirrored disks.) […]

  2. Stuart Frost on September 20th, 2006 11:40 pm

    Curt,

    According to your post, Netezza claims two major advantages from their use of FPGAs: 5X performance and lower heat and power consumption. Both are very easy to refute.

    On performance, our latest version pretty much maxes out the disks at around 70MBps each. This is when running queries, not just simple table scans. Based on recent proof of concept projects, we’ve seen that Netezza does roughly the same. However, our performance is achieved through the use of smart software on commodity CPUs, rather than their combo of a CPU and an FPGA.

    IMHO, our performance is even more impressive when you consider that we have just two dual-core CPUs for every twelve disks. In contrast, Netezza uses one CPU and one FPGA for every disk. Hence, our CPU:disk ratio is 1:3 compared with Netezza’s 2:1.

    This goes a long way in explaining our price advantage – our CPU cost per disk is much lower and this feeds directly into a significant advantage on both price/performance and price/capacity.

    It also leads to a refutation of their supposed heat/power advantage. A single FPGA certainly does use less power than a dual core Xeon CPU. However, a typical Netezza installation uses far more chips for a given performance and capacity level.

    To be specific, a Netezza 8650z has 33TB capacity and uses 672 SPUs with one Power CPU, an FPGA and one disk each. That’s a total of 1,344 ‘CPUs’ and 672 disks. In contrast, a DATAllegro P33 has 33TB capacity and uses 66 nodes with 2 dual core CPUs and 12 disks each for a total of 132 dual core CPUs and 792 disks.

    With a similar number of disks being scanned at a similar rate, the two systems will have similar performance (until we take into account our sophisticated multi-level partitioning, which gives us a huge performance advantage on most queries). However, our system has less than one tenth the number of CPUs. I don’t know how much power Netezza’s FPGAs use, but I doubt it’s less than a tenth of that used by one Xeon.

    Our P33 is also less than half the price.

    Stuart

  3. Stuart Frost on October 3rd, 2006 10:34 am

    Curt,

    Just read another article that put Netezza’s power consumption at around 30W per SPU. Since each SPU has one disk, that equates to 30W per disk.

    Our nodes use around 420W each and have 12 disks, so that’s 35W per disk. Our P Series gets slightly more usable space per disk than Netezza, so the consumption per TB is about the same.

    If we compare to our C series, we get around four times the capacity per disk, so power consumption per TB is much lower.

    Stuart
    DATAllegro

  4. DBMS2 — DataBase Management System Services»Blog Archive » Philip Howard on Netezza on October 4th, 2006 2:37 am

    […] Philip Howard has published a write-up based on Netezza’s user conference, entertainly mixing fantasy and reality in his usual manner. Notably, he confuses Netezza’s zone maps, which are basically a very limited form of range partitioning, with something that can substitute for real indices. And the mind boggles at his implication that Netezza has neglected the FPGA in its overall market messaging. More understandable is his regurgitation of Netezza’s claims about heat and power, but although I must confess to not having checked either side’s arithmetic, I find Stuart Frost’s rebuttal in the comments to this thread pretty interesting. […]

  5. DBMS2 — DataBase Management System Services»Blog Archive » Data integration appliance vendor Cast Iron Systems on January 4th, 2007 8:51 pm

    […] As for the hardware/platform side, it’s similar to what I’m hearing from other appliance vendors (Netezza and their emphasis on an FPGA are a real outlier). Namely: […]

  6. DBMS2 — DataBase Management System Services»Blog Archive » Data warehouse appliance hardware strategies on January 27th, 2007 3:38 am

    […] Netezza is Type 0 all the way. The product, if not the company, is built around its FPGA. […]

  7. Intrigued on July 14th, 2007 12:21 pm

    I’ve seen many articles like this where Stuart tries to position Datallegro over Netezza, the problem is Datallegro has hardly any customers on board, at least hardly any willing to talk about their success publicly, compare that to the momentum Netezza has in the marketplace, that speaks volumes to me !

  8. Curt Monash on July 14th, 2007 12:26 pm

    Up until the second half of last year, I used to complain that Netezza would talk to me about its customers but not its technology, and DATAllegro would talk about its technology but not its customers.

    They still both have those orientations, but neither is nearly as extreme any more as they were.

    CAM

  9. diskpharma on September 28th, 2007 11:41 pm

    At one of Netezza’s User Conference technical breakout sessions this week they basically opened the kimona.
    They discussed in great detail with animated slides the technology utilized starting from the user query to
    the front end software optimization and scheduling, how the query is distributed & executed on the SPUs, how
    data is manipulated, and ultimately sent back to the host and to the user for the answer to their query.

    Also several of their customers gave presentations at their business track sessions disclosing their DW
    deployment and reported performance gains of 2 and 3 orders of magnitude. So something is missing in the
    technical analysis just comparing disk data rates which on average cannot go faster than 70 MB/s. Any clues??

  10. Curt Monash on September 29th, 2007 8:11 am

    Hiya.

    Lots of clues elsewhere in this blog. Index-light MPP architectures such as Netezza’s or DATAllegro’s are waaaay better for large data warehouses than traditional index-heavy SMP systems like Oracle or Microsoft SQL*Server.

    More debatable is the choice between FPGAs (Netezza) and more conventional chips (pretty much everybody else). Both choices have a lot of merit.

    CAM

  11. Infology.Ru » Blog Archive » Стратегии аппаратного обеспечения комплексов для хранилищ данных on August 21st, 2008 2:14 pm

    […] Netezza всю дорогу является Типом 0. Продукт, если даже не компания, построен вокруг их FPGA. […]

  12. Xtreme Data readies a different kind of FPGA-based data warehouse appliance | DBMS2 -- DataBase Management System Services on June 29th, 2009 12:09 pm

    […] for core SQL processing, after the data is ingested via conventional I/O. This is different from Netezza’s approach to FPGA-based data warehouse appliances, in which the FPGA sits in the place of a disk controller […]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.