• From comment threads:
  • The eBay guys didn’t like my phrasing around their decision to replace Greenplum with Teradata.
  • Dwight Merriman and some other folks thought I underrated the importance of schema flexibility as a reason to go NoSQL.
  • In Section 5 of a paper linked in the comment thread to my recent post on advanced analytics, Joe Hellerstein, Greenplum, et al. argued that one can do some parallel-statistics things in SQL that perhaps seemed only possible to do in other ways.
  • An interesting older comment thread is one in April, 2010 on examples of machine-generated data.
• Ray Wang offered a survey of enterprise software innovation. He argues that consumer technology is the source of substantially all ideas for enterprise software innovation. If that sounds a bit extreme, it is — Ray oddly identified “analytics” as a consumer technology, and ignored any MPP that isn’t in the cloud. Still, his overall point is excellent, and most of his subpoints are instructive.
• Research is underway to let your individual organs broadcast data helpful to your health care. Or to put it starkly, they’re working on good reasons for you to get chipped. Yet another reason to find ways to protect us against wrongful data use.
• Now that I’ve met Quentin Gallivan — and had dinner with Mayank Bawa — my previously favorable views of Aster Data’s management transition are unchanged.
  • And by the way: It’s been months since Aster Data’s head of sales Mark Cranney left. Business seems to have been great in the interim, and the salespeople and managers he hired did not leave with him.
• Xtreme Data offered a couple of win reports.

• Aster Data showed me a lot of customer names and deal sizes, across a bunch of industries (mainly enterprise rather than web). Yes, Aster’s market success is for real. (But almost all those details are NDA.)
• Sybase’s product plans for IQ are pretty impressive. (But the most interesting parts are, you guessed it, NDA.)
• I’ve kissed and made up* with ParAccel, now that they’ve replaced their CEO, replaced their marketing chief, and stopped the worst of the marketing nonsense I used to complain about. ParAccel has some interesting plans for ParAccel 3.0 which are, naturally, NDA.
• The Peoplesoft guys are doing it over again at Workday. Only this time, their platform isn’t a relational DBMS. Rather, it’s an in-memory, completely object-oriented data model, with disk used only on a “Just in case the power ever goes out” basis. (Thankfully, nothing at all about our conversation was NDA.)
• I’m finally feeling good about Northscale’s memcached-compatible persistent store Membase. The main reason is that they showed me a near-term path to interfaces that are richer than key-value. Also, Todd Hoff reassured me that even pure persistent memcached has a place.
• Rumor says that even the one app for which Facebook was using Cassandra — in-box search — has been decommissioned. On the other hand, numerous other scale-0ut DBMS (SQL or otherwise) seem to have Facebook footholds. But details are — all together now! — NDA.

*If you know ParAccel’s new marketing exec Michael Weir, you surely guessed I mean that only in a figurative sense.

From elsewhere:

• Daniel Abadi offered his analysis of Kickfire’s demise. In general I agree, but Daniel neglected to mention one hugely important factor — the chicken-egg negative effect of Kickfire’s lack of market or marketing traction. Customers were extremely reluctant to buy from Kickfire because they perceived, correctly, that Kickfire’s survivability was far from assured.
• While the InfiniDB forums suggest that there are at least a couple of production users of Calpont’s free InfiniDB, Calpont seemingly has a long way to go to be even as successful as Kickfire. But Calpont does have a bit of money to spend on lead generation; maybe some day they’ll even have actual customers.
• In a response to a question I messaged over, XtremeData tells me they have actual customers now. Press releases to follow.
• The admiration for the job Mark Hurd did at HP is in my opinion overstated. Sure, the financial/operational management appeared to work, but HP did little on Hurd’s watch to strengthen its reputation or customers’ loyalty. In particular:
  • HP’s analytics efforts have accomplished little.
  • HP’s data warehouse appliance efforts have failed pathetically.
  • From what I hear, HP’s execution in its Exadata partnership was not good.
  • HP’s server business in general is distinguished mainly by HP being a big company.
  • HP’s EDS acquisition has been rocky, not that EDS was sailing so smoothly on its own beforehand.
  • HP’s success in PCs amounts to “arguably, HP sucks a little less than the other guys”.
  • HP’s elite reputation is long gone (admittedly, for the most part that predates Hurd).
• Doug Henschen evidently favors really strong intellectual property protection for software, even forbidding plug-compatible reverse engineering. I agree with Doug up to the point that it should be forbidden to copy proprietary software, but I don’t see why he (or a court) would view such behavior as copying.

• XtremeData still hasn’t sold any dbX stuff (they’ve had a side business in generic FPGA-based boards paying the bills for years). Well, there may have been some paid POCs (proofs of concept) or something, but real sales haven’t come through yet.
• XtremeData does have three prospects who have said “Yes”, and expects one order to come through this month.
• XtremeData continues to believe it shines when:
  • Data models are complex
  • In particular, there are complex joins
  • In particular, two large tables have to be joined with each other, under circumstances where no product can avoid doing vast data redistribution
• XtremeData insists that all the nice things Bill Inmon – including in webinars — has said about it has not been for pay or other similar business compensation. That’s quite unusual.
• XtremeData is coming out with a new product, codenamed the Personal Data Warehouse (PDW), which:
  • Is ready to go into beta test
  • Should be launched in a month and a half or so
  • Will have a different name when it is launched

Naming aside,

• The XtremeData PDW consists of XtremeData software running on a Cray CX1 box.
• Thus, the XtremeData PDW will plug into a 20 amp wall power socket. It consumes 1600 watts.
• The XtremeData PDW also inherits the Cray CX1′s noise cancellation feature.
• Bottom line on the form factor: The XtremeData PDW is meant to be stuck in the corner of a business analyst’s office, not a computer room.
• The XtremeData PDW will have 16 1 TB disks (going up in size later), for 5 TB of uncompressed user data.
• Pricing isn’t finalized for the XtremeData PDW, but it will be around XtremeData’s usual figure — $20K/TB of uncompressed user data.
• XtremeData hasn’t “released” compression yet, but it’s “ready to go.”
• The XtremeData PDW will not include FPGAs, unlike other XtremeData dbX appliances. It will just run the XtremeData dbX software on 8 Nehalem chips.
• XtremeData calls this a “3-node” machine. I didn’t bother asking why it wasn’t 4-node. (Perhaps there’s a head node of some kind that properly isn’t counted.)

Some comparative notes:

• A Netezza Skimmer has similar size and price to the XtremeData PDW, seems to draw less power, has less uncompressed user data capacity (but already has compression), is also in essence a three-node system (I think), and of course has a lot of software connectivity. If XtremeData can match Netezza’s compression, the XtremeData PDW will have a 2X or so price/TB advantage over Netezza Skimmer – but Netezza’s compression is of course a moving target. I don’t know how happy Skimmer is outside a computer room.
• Kickfire manages similar amounts of data on a smaller box (5 rack units vs. 7), drawing less power (600 watts vs.1600), also with a lot of BI and ETL tool connectivity.

One reason everybody talks about the “Netezza (list) price” is that it hasn’t been changing much, seemingly staying stable at $50-60K/terabyte for a long time. And thus Teradata’s 2550 and Oracle’s larger-disk Exadata configuration — both priced more or less in the same range — have clearly been price-competitive with Netezza since their respective introductions.

That just changed. Netezza is cutting its pricing to the $20K/terabyte range imminently, with further cuts to come. So where does that leave competitors?

• The Teradata 1550 is in the Netezza price range (still a little below, actually).
• Oracle basically has nothing price-competitive with Netezza.
• Microsoft has stated it plans to introduce Madison below the old DATAllegro price points; conceivably, that could be competitive with Netezza’s new pricing, although I haven’t checked as to how much it now costs simply to buy a lot of SQL Server licenses (which presumably would be a Madison lower bound, and might except for hardware be the whole thing, since Microsoft likes to create large product bundles).
• XtremeData just launched in the new Netezza price range.
• Troubled Dataupia is hard to judge. While on the surface Dataupia’s prices sound very low, you can’t use a Dataupia box unless you also have a brand-name DBMS (license and hardware) alongside it. That obviously affects total cost significantly.
• Kickfire seems unaffected, as it doesn’t and most likely won’t compete with Netezza (different database size ranges).
• For the most part, software-only vendors are free to adapt or not as they choose. Hardware prices generally don’t need to be over $10K/terabyte, and in some cases could be a lot less. So the question is how far they’re willing to discount their software.

• XtremeData is announcing a single pricing metric — $20,000 per terabyte of user data.
• DBx currently has no compression – so when XtremeData adds compression to DBx, price/TB will naturally go down further.
• XtremeData’s DBx node hardware is based on a board that combines an Intel-compatible CPU with some FPGAs, called the XtremeData In-Socket Accelerator (ISA). In addition there is a head node and a data loading node. Failover/high availability for the head node seem to mainly be futures.
• DBx software is based on PostgreSQL. XtremeData says it kept PostgreSQL’s front end and replaced the execution engine. I haven’t checked exactly which PostgreSQL features are in or out of DBx.
• (This subject edited, after Dave DeWitt pointed out how unclear the first version was.) XtremeData’s DBx of course does complete parallel redistribution of data after every intermediate result set. The basic idea of DBx’s data redistribution is that after each intermediate result set, DBx recalculates histograms and redistributes data — and hence work — approximately evenly accordingly. This recalculation is done in the FPGAs. XtremeData claims that this constant re-setting of the execution plan is more extensive, and results in more even data distribution, than rival vendors’ strategies. (Based outside Chicago, XtremeData was founded by high-performance computing (HPC) guys, and its design priorities reflect that. XtremeData’s database management guys, by the way, are in Bangalore.)
• XtremeData’s DBx uses Infiniband to support the resulting large amount of data movement.
• At least in theory, XtremeData’s DBx scales up to 1024 nodes, the limit at which its Infiniband switches can support full bandwidth.
• XtremeData’s smallest DBx product is a half-rack system with 8 nodes, rated at 30 TB of user data.
• Because of the its data redistribution strategy, XtremeData says DBx doesn’t much care about the physical distribution of data. Hash distribution is the default, but it has less benefit than in other MPP analytic DBMS systems.
• XtremeData also claims that DBx is particularly good at being schema-agnostic, in that competing MPP analytic DBMS products shine most for schemas that don’t lead to a lot of data redistribution (e.g., one big fact table, N small dimension tables that can be replicated at each node). However, I’m skeptical about that point. E.g., Oracle Exadata also uses Infiniband, and features no more data redistribution per query than DBx does, so where exactly is the bottleneck Exadata faces but DBx doesn’t?
• XtremeData says it has tested DBx up to 10-15 concurrent queries. There seem to be no workload management features in the first DBx release, but naturally there’s a technical roadmap in that direction.
• XtremeData claims a variety of DBx beta tests, successful proofs-of-concept (POCs), and even customers intending to buy, but no actual sales to date.

XtremeData has kindly permitted me to post its DBx launch slide deck. Three specific POC/prospect price/performance comments may be found on Slide 9.

XtremeData says that the clock speed on the FPGAs it uses is 200 megahertz, clearly much less than an Intel-compatible CPU’s. However, XtremeData also says 100s or 1000s of steps can be done at once on an FPGA. The reason for this seems to be “pipelining” much more than on-chip parallel streams. XtremeData’s explanation seemed to focus on the point that many rows of data could be processed independently of each other, and hence at once. I’m not wholly convinced that this is a standard use of the word “pipelining”. The point may be moot anyway, in that XtremeData’s reported performance advantages are nowhere what one would get by naively assuming DBx can do ~1000 times as many steps per clock cycle at 1/12th – 1/16th a normal clock speed.

Related link

• XtremeData now has the obvious URL, but at the time of this posting it’s a work in progress.

• Xtreme Data’s basic idea is to take a custom board and build a data warehouse appliance around it.
• An Xtreme Data board looks a lot like a conventional two-socket board, but has only one four-core CPU. In addition, it sports some FPGAs (Field-Programmable Gate Arrays).
• In the Xtreme Data appliance, the FPGAs will be used for core SQL processing, after the data is ingested via conventional I/O. This is different from Netezza’s approach to FPGA-based data warehouse appliances, in which the FPGA sits in the place of a disk controller and touches the data first, before passing it off to a more or less conventional CPU.
• While preparing entry into the data warehouse appliance business, Xtreme Data has sold its board to 150 other outfits, many quite impressive. Buyers seem to be FPGA users who previously had to craft their own custom boards. According to Xtreme Data, major uses by these customers include:
  • Military/intelligence/digital signal processing.
  • Military/intelligence/cybersecurity (a newish area for Xtreme Data)
  • Bioinformatics/high-throughput gene sequencing (a “handful” of customers)
  • Medical imaging
  • More or less pure university research of various sorts (around 50 customers)
  • … but not database management.
• Xtreme Data’s website has a non-obvious URL.

So far as I can tell, Xtreme Data’s 1.0 product will — like most other 1.0 analytic database management products — be focused on price/performance, without little or no positive differentiation in the way of features.