• XtremeData still hasn’t sold any dbX stuff (they’ve had a side business in generic FPGA-based boards paying the bills for years). Well, there may have been some paid POCs (proofs of concept) or something, but real sales haven’t come through yet.
• XtremeData does have three prospects who have said “Yes”, and expects one order to come through this month.
• XtremeData continues to believe it shines when:
  • Data models are complex
  • In particular, there are complex joins
  • In particular, two large tables have to be joined with each other, under circumstances where no product can avoid doing vast data redistribution
• XtremeData insists that all the nice things Bill Inmon – including in webinars — has said about it has not been for pay or other similar business compensation. That’s quite unusual.
• XtremeData is coming out with a new product, codenamed the Personal Data Warehouse (PDW), which:
  • Is ready to go into beta test
  • Should be launched in a month and a half or so
  • Will have a different name when it is launched

Naming aside,

• The XtremeData PDW consists of XtremeData software running on a Cray CX1 box.
• Thus, the XtremeData PDW will plug into a 20 amp wall power socket. It consumes 1600 watts.
• The XtremeData PDW also inherits the Cray CX1’s noise cancellation feature.
• Bottom line on the form factor: The XtremeData PDW is meant to be stuck in the corner of a business analyst’s office, not a computer room.
• The XtremeData PDW will have 16 1 TB disks (going up in size later), for 5 TB of uncompressed user data.
• Pricing isn’t finalized for the XtremeData PDW, but it will be around XtremeData’s usual figure — $20K/TB of uncompressed user data.
• XtremeData hasn’t “released” compression yet, but it’s “ready to go.”
• The XtremeData PDW will not include FPGAs, unlike other XtremeData dbX appliances. It will just run the XtremeData dbX software on 8 Nehalem chips.
• XtremeData calls this a “3-node” machine. I didn’t bother asking why it wasn’t 4-node. (Perhaps there’s a head node of some kind that properly isn’t counted.)

Some comparative notes:

• A Netezza Skimmer has similar size and price to the XtremeData PDW, seems to draw less power, has less uncompressed user data capacity (but already has compression), is also in essence a three-node system (I think), and of course has a lot of software connectivity. If XtremeData can match Netezza’s compression, the XtremeData PDW will have a 2X or so price/TB advantage over Netezza Skimmer – but Netezza’s compression is of course a moving target. I don’t know how happy Skimmer is outside a computer room.
• Kickfire manages similar amounts of data on a smaller box (5 rack units vs. 7), drawing less power (600 watts vs.1600), also with a lot of BI and ETL tool connectivity.

• Netezza’s software runs on well-designed proprietary hardware. Aster runs on hardware that’s more off-the-shelf.
• Aster was first to ship, and will also be first to ship an IDE (Integrated Development Environment).
• MapReduce is central to Aster’s approach. Netezza TwinFin(i) supports MapReduce too, specifically a Hadoop implementation, but I don’t get the sense that everything Netezza does is built on MapReduce underpinnings.
• Both Aster and Netezza try to provide rich functionality for creating in-memory data structures parallel analytic programs can use. Both seem to let you escape from the pure relational-table paradigm more easily than, say, Teradata’s new persistent memory capabilities do.
• Aster and Netezza have made different choices about what kinds of prebuilt analytic packages to offer. Netezza could actually leapfrog Aster in this regard, but let’s see where each vendor is by, say, mid-year. If you care about the details of built-in analytic functions, you really should consider executing non-disclosure agreements with both those companies.
• Both Aster and Netezza stress that you can run analytic functions out-of-process, greatly reducing the chance that they crash the database. Netezza and I’m pretty sure also Aster also retain the option of running in-process, which provides maximum performance. (In Netezza’s case C++ is the only in-process language supported, and I think Aster has a similar limitation.)
• Like Aster, Netezza is integrating SQL queries and other analytic processing under the same workload management rubric.
• Much like Aster, Netezza is tap-dancing by implying much richer forthcoming SAS support than anything currently announced. (The crunch-per-paragraph ratio in either vendor’s SAS-related press releases to date is distressingly low.)

More specifically, here are some highlights of what I know, am guessing, and/or am allowed to say about Netezza TwinFin(i) at this time.

• The foundation for the analytic add-ons in Netezza TwinFin(i) is some sort of low-level “analytic executables.” Not understanding exactly what these are is my biggest area of confusion in the whole TwinFin(i) stack. Are they all C++, with everything translated into same? Is there Java all the way down as an alternative? (E.g., Hadoop is written in Java.) Anyhow, whatever it is, it’s surely a big improvement on Netezza’s prior Verilog-based generation of analytic extensibility technology.
• The announced list of languages supported in Netezza TwinFin(i) is Java, Python, Fortran, R, and C/C++. More are coming.
• Netezza has named a lot of analytic functions it is adding, and hinting about more to come. It has named CRAN/R and GNU libraries, saying those have 1900 or more functions each. Netezza has also built its own linear algebra library for TwinFin(i), called nzMatrix. And as previously noted, TwinFin(i) also boasts a Hadoop implementation.
• I haven’t heard about much in the way of TwinFin(i)-specific IDE support.
• I don’t really have details as to what kinds of in-memory data structures Netezza TwinFin(i) does or doesn’t support.

*Links to Gartner Magic Quadrants commonly break, but that one worked at the time of this posting.

• Gartner thinks that data warehouse appliances are on the rise, due to their simplicity.
• Gartner correctly says that Teradata has been a data warehouse appliance vendor from the getgo.
• Gartner characterizes IBM as being an appliance vendor as well.
• Gartner suggests that HP is having trouble living up to its technical promises for Neoview.
• Gartner further suggests — no surprise here — that HP Neoview has had very few new customers past its initial wave.
• Gartner notes IBM’s difficulties in selling data warehouse installations of DB2, despite what on paper is great-sounding technology.
• Gartner says — also no surprise — that illuminate “has seen little success in North America since opening its first office in the U.S. over two years ago.”
• Ingres has evidently gotten a few BI-centric “appliance” deals, e.g. with Jaspersoft. But basically Ingres isn’t doing well in data warehousing.
• Gartner does say Ingres has “the strongest open-source DBMS offering for data warehousing.” Being very literal about “open source,” that’s a defensible claim — but it’s pretty irrelevant in a world where Greenplum Single-Node Edition can be had for free. It also waves away all the data mart use cases in which Infobright Community Edition shines.
• Gartner says that Netezza is working out as a “complex workload” enterprise data warehouse provider, according to reference checks, in addition to its established success in data mart scenarios.
• Gartner says Oracle’s offering has finally become “accepted” in the market for databases >50 TB. I guess I can live with that fairly weak claim, but I wouldn’t go much further than that.
• Gartner asserts that, unlike software-only Oracle, Oracle Exadata isn’t significantly harder to administer than “other mixed OLTP/OLAP DBMS vendors,” because Exadata is fast enough you don’t need to jump through all those hoops any more to get tolerable performance. The money quote is “one reference reported reducing the number of indexes by a factor of 100 to fewer than five.” Note, however, that Gartner does not seem to assert that Exadata’s ease of use rivals that of the newer analytic DBMS specialists.
• Gartner confirms Oracle’s reluctance to do onsite Exadata POCs, but says it is not absolute. This is roughly compatible with what I’m hearing elsewhere, and indeed with Oracle own claims to be ramping up availability of Exadata POC hardware.
• Gartner’s criteria for inclusion include at least 10 different organizations having a product “in production.” Thus, the big surprise was ParAccel being included. The money quote there is “With approximately 20 customers in the pharmaceutical, retail, financial and media/advertising analytics sectors, ParAccel has a good reference base.” That assessment is difficult to reconcile with other information, but I’ve been told Gartner is sticking to its guns. That assessment would be even harder to believe if those 20 references were all alleged to be true production customers.
• Gartner notes that you basically can’t run a 1 TB+ MySQL data warehouse without sharding. (Of course, Infobright has an alternative, and up to a small number of terabytes so does Kickfire.)
• Gartner reports that at least some customers are pleased with Sybase IQ’s mixed workload/enterprise data warehouse capabilities.
• Gartner correctly notes that Oracle Exadata is a price-competition challenge for Teradata.
• Gartner notes that 20% of Vertica’s customers are outside the US. While not shocking, that’s more than I realized.
• Gartner notes something I don’t think I’ve posted yet, which is that Vertica has a customer with 300 TB of data. (The identity is a deep dark secret, but if I told you you probably wouldn’t recognize the name anyway.)

As does any such piece, the Gartner Data Warehouse DBMS Magic Quadrant also has outright errors.  For example:

• Aster Data isn’t really “the newest entrant to the DBMS data warehouse world.”
• Aster’s SQL/MapReduce was not new in Release 4.0.
• Greenplum isn’t yet pushing down code to the storage tier.
• I’m not sure what kind of database-tier parallelism Gartner is claiming is new in Oracle in 11g Release 2 — but I doubt it’s really new. Rather, what Oracle has done recently is make parallelism less administratively cumbersome.
• Vertica wasn’t really the first DBMS in the cloud. At most it was the first pure-play analytic DBMS to get there.

That said, highlights of my Netezza Skimmer briefing included:

• In essence, Netezza Skimmer is 1/3 of Netezza’s previously smallest appliance, for 1/3 the price.
• I.e., Netezza Skimmer has 1 S-blade and 9 disks, vs. 3 S-blades and 24 disks on the Netezza TwinFin 3.
• With 1 disk reserved as a hot spare, that boils down to a 1:1:1 ratio among CPU cores, FPGA cores, and 1-terabyte disks on Netezza skimmer. The same could pretty much be said of Netezza TwinFin, the occasional hot-spare disk notwithstanding.
• Netezza Skimmer costs $125K.
• With 2.8 or so TB of space for user data before compression, that’s right in line with the Netezza price point of slightly <$20K/terabyte of user data.
• That assumes Netezza’s usual 2.25X compression. I forgot to ask when 4X compression was actually being shipped.
• I forgot to ask, but it seems obvious that Netezza Skimmer uses identical or substantially similar components to Netezza TwinFin’s.
• Netezza Skimmer is 7 rack units high.
• In place of the SMP hosts on TwinFin Systems, Netezza Skimmer has a host blade.
• Netezza (specifically Phil Francisco) mentioned that when Kalido uses Netezza Skimmer for its appliance, there will be an additional host computer, but when it uses TwinFin for the same software, the built-in host will suffice. (Even so, I suspect it might be too strong to say that Skimmer’s built-in host computer is underpowered.)
• Netezza also suggested that more appliance OEMs are coming down the pike specifically focused on the affordable Skimmer.

Obviously, Netezza Skimmer isn’t breaking any new technical ground. If Netezza had just called Skimmer “TwinFin 1,” nobody should have objected. So the main news here is that you can buy a Netezza box for $125K, plug it in, load a few terabytes of data, and be good to go with a pretty solid data warehouse.  For enterprises and data mart outsourcers with databases of the appropriate size, that could be a pretty attractive deal.

Is Netezza Skimmer as cheap as buying your own hardware and putting (free) Greenplum Single-Node Edition software on it? Not even close, especially since Greenplum’s free option limits you to lower overall compute power. Does Netezza Skimmer have as high availability as more expensive alternatives? In some cases, surely not. Skimmer is neither the cheapest thing around nor an utterly high-end product.

But Netezza Skimmer belongs on a lot of short lists even so.

• Oracle thinks flash memory is the most important hardware technology of the decade, one that could lead to Oracle being “bumped off” if they don’t get it right.
• Juan believes the “bulk” of Oracle’s business will move over to Exadata-like technology over the next 5-10 years. Numbers-wise, this seems to be based more on Exadata being a platform for consolidating an enterprise’s many Oracle databases than it is on Exadata running a few Especially Big Honking Database management tasks.

And by the way, Oracle doesn’t make its storage-tier software available to run on anything than Oracle-designed boxes.  At the moment, that means Exadata Versions 1 and 2. Since Exadata is by far Oracle’s best DBMS offering (at least in theory), that means Oracle’s best database offering only runs on specific Oracle-sold hardware platforms.

*E.g., I was sitting upstairs in my parents’ apartment in Columbus, OH having the call while their doctor, who I’ve never met, was visiting downstairs. He offered to make a special trip back Saturday afternoon because he missed me Wednesday, but he’s notorious for not coming when he says he will. Update: He didn’t come Saturday. On Saturday he said he’d come Sunday. He didn’t do that either.

Other high- and lowlights of our conversation included:

• Flash is the main new hardware element in Exadata Version 2. Otherwise, Exadata 2 is just an annual refresh of Exadata Version 1 to include updated components (Nehalem chips, bigger disk drives, etc.)
• Juan thinks it’s suboptimal to use flash memory through the bottleneck of disk controllers, favoring PCIe cards instead. (I emphatically agree.)
• Juan resolutely ducked questions about actual Exadata production deployment. Literally the only fact he shared in that regard is that there are at least 2 Exadata production systems running that each have 2 or more racks cabled together.
• Juan stressed that Exadata runs apps written over Oracle DBMS unchanged.
• When making mixed-workload claims for Exadata 2, Juan stressed consolidation of multiple databases, some OLTP and some analytic. He didn’t really argue with my skepticism about integrating OLTP and analytics in the same database, with one exception:
• Juan pointed out that in major OLTP apps such as ERP systems, there often is actually more processing going on in reporting and other batch stuff than there is in true OLTP.
• Exadata 2’s flash memory is designed as a disk cache, smarter than LRU (Least Recently Used). The two examples Juan gave of “smarter than LRU” are that backups and table scans don’t flush the cache.
• I forget whether this is new in Exadata 2 (I think it is), but anyhow – Exadata has a “Storage Index” that’s a lot like a Netezza zone map. I.e., for each megabyte or so of data it stores the min and max value of every column; if a query predicate rules out those ranges, that megabyte is never retrieved.
• Oracle has long offered what sounds like flexible workload management capability, and this has now been extended to specifically include I/O resources on the storage tier.
• This isn’t Exadata-specific, but Oracle has built a file system on top of its DBMS, optimized for speed, which helps with, e.g., ELT (Extract/Load/Transform). Evidently, it’s not at all the same thing as Mark Benioff’s 1990s Microsoft-annoying IFS (Internet File System) project, which seems to have morphed into a content management SDK.

Highlights specifically in the area of parallelization included:

• Juan stressed that all databases consolidated onto an Exadata machine are/should be striped across all storage units.
• On the other hand, Juan said that different databases should be confined to specific cores or CPUs on the database tier.
• But on the third hand, Juan also stressed – in what could be called a “private cloud” pitch – that there’s great elasticity as to which databases are matched to which server CPUs.
• Contrary to what I thought he and/or his colleagues told me a year ago, Juan said RAC (Real Application Clusters) is a big part of Oracle’s data warehouse processing.
• However, Juan says that what I regard(ed) as a major objection to Oracle’s database-tier parallelization — the need to manually specify “degrees of parallelism” — has now been obviated by automation. Juan thinks that few data warehouse DBAs will now need to manually tune parallelism, with minor exceptions. One exception he cites is that if a nightly report really is non-urgent, it can just be forced to run on a single core with no chance to grab more resources. (However, Juan thinks manual tuning of parallelism will continue to play a greater role in OLTP.)

OK. That’s all I can get done tonight (see above re: inconvenience of timing). Follow-on subjects I’d like to and indeed plan to post about include:

• What Juan said about hybrid columnar compression
• Oracle’s delightfully non-confidential slide deck, and a few comments about same

As we went through the contest entries in detail, it was readily apparent that today’s data warehousing solutions are either massively expensive or non-existent,” said Curt Monash, Founder of Monash Research. “Clearly, there is major dual-market opportunity for a product such as the Kickfire appliance that can not only provide an affordable data warehousing solution to small companies; but can also target larger companies that have made an initial investment in high-end solutions, yet still need to add some affordable query processing power in other areas of the organization.”

In reality:

• I spent a few minutes reviewing summaries of eight stories selected by Kickfire from the entrants, and emailed comments back to Kickfire about them. I have no further role to play in the contest.
• The part of the “quote” that slams Kickfire’s competitors is not reflective of my views.
• The “market opportunity” is in line with the positioning I’ve encouraged Kickfire to adopt. A good shorthand for it is the “Sybase IQ market.” In essence I see Kickfire as an interesting Sybase IQ alternative. But Sybase IQ is a formidable competitor, and there are many other competitors as well. This is hardly an untapped market ripe for Kickfire’s plucking.

I’m satisfied that this is all a case of lousy marketing execution – something Kickfire has a history of — rather than deliberate deception. Kickfire has recently turned over its VP of Marketing (twice) and PR resource (at least once). Scott Humphrey, Kickfire’s new outside PR guy, says he was incorrectly told by his predecessor that the press release and quote in question had been approved, and put it out without fact-checking. I believe him. I hope Kickfire CEO Bruce Armstrong will be able to add stronger marketing leadership soon. Bruce seems aware of the need, and is making reasonable marketing strategy decisions himself in the mean time, so there’s some basis for optimism.

And by the way – I don’t let vendors write press release quotes for me anyway. I let them edit in precise product names and so on, but otherwise the words are mine. The last occasion on which I recall bending this policy was inadvertent and over a year ago, when Greenplum emailed something to me — which was genuinely similar to my opinion — while I was on the phone with Aster at a particularly frenzied time, and I didn’t immediately realize the words weren’t my own.

The top two points from Q&A probably were:

• Big Data and the cloud actually have relatively little to do with each other, a few exceptions notwithstanding, especially if the data is in a shared-nothing DBMS (as opposed to, say, a MapReduce-oriented file cluster). Two principal reasons are:
  • Redistributing data from node to node is a little slow, undermining some of the elasticity benefits of the cloud.
  • Getting data into the cloud in the first place is a lot slow.
• The NoSQL movement is a lot like the Ron Paul campaign — it consists of people who are dissatisfied with the status quo, whose dissatisfaction has a lot to do with insufficient liberty and/or excessive expenditure, and who otherwise don’t have a whole lot in common with each other.

Anyhow, here are my notes for the talk, edited in just a couple of places for readability or linkage.

Quick introduction

• Big Data vs. cloud
• How big is Big Data?
• At the low end of that range, there’s little you can’t do with conventional technology if you have:
  • An unlimited budget for hardware
  • An unlimited budget for software
  • An unlimited budget for people, especially Oracle DBAs

Big Data in OLTP

• Hard-core OLTP
  • Focus of DBMS technology for a long-time
  • Big budgets because each transaction has significant value
  • Tough to get users to change technologies
• Lighter-weight OLTP
  • Classic example = web companies
    • Big ones — retail-oriented ones (eBay, Amazon) partially excepted — rolled their own technology stacks
    • Reluctant to give money to anybody
      • Open source, etc.
  • Difficulty finding market
    • Product vs. feature
      • Clustering/HA/DR/whatever
      • Ditto cloud enablement
    • True products haven’t found much traction yet

Analytic Big Data use cases

• Kinds of data for analytics
  • More of same != big
  • More detail and/or new kinds
    • Complete data sets
    • Transactions
    • Call details
    • Tick/trade history
    • Web clickstreams
    • Network event logs
    • Other machine-generated data
    • CAM bottom line
      • Anything human-generated should and will be retained in its entirety
      • Quantities of machine-generated data retained should and will grow roughly in line w/ computing cost reductions (Moore’s Law, etc.)
• Analytic uses of Big Data
  • Analytics is mainly about three things
    • Problem detection
    • Customer relationship improvement
      • (Those overlap when the customer relationship is bad)
    • Financial statements on steroids
  • Main kinds of analytics
    • What BI vendors traditionally sell
      • General reporting and dashboards
      • Ad-hoc query (now driven from those reports and dashboards)
      • Planning (allegedly integrated with BI)
    • Research
      • Ad hoc relational query (worth mentioning twice because it drives so much of the market)
      • Data mining
      • Most web search and web mining
    • Operational/near-real-time
    • Archiving/compliance
  • What gets Big?
    • Mainly research and archiving
    • But when reporting or operational get Big, you have really interesting computing problems

Technology issues and trends

• Moore’s Law
  • CPUs — All about cores, hence parallelism is key
  • RAM
  • SSDs – hence replace disks
  • Sensors – hence generate lots more data
• Kryder’s Law
  • But rotational speeds up only 12.5X since Eisenhower Administration
  • Hence solid-state memory (or RAM) will soon take over
• In the mean time, I/O bottlenecks have had to be beaten
  • Hence sequential scans
  • Hence index-light architectures
  • Hence columnar
• DBMS “overhead”
  • Raw license and maintenance fees – software increasing fraction of total
  • OLTP vestiges – locking and all that
  • DBAs
    • People costs = huge fraction of total
    • Index-lightness addresses
    • So does appliance
  • Many people don’t really know how to write SQL
• Configuration
  • Appliance/tightly-balanced
    • Netezza
    • Teradata earlier
    • Greenplum/Sun
    • Oracle
    • IBM
    • Microsoft/Madison
  • Commodity/do what you want
    • Vertica
    • Greenplum now
    • Infobright, Aster and others
    • MapReduce-oriented file systems
  • Extreme rigidity is silly
    • Teradata, Oracle have both signaled moving to more modularity
    • Big driver of that = heterogeneous storage
      • Cheap disk
      • Expensive disk
      • Solid-state
      • RAM
    • CPU/storage ratio is even more of a driver

Theoretically defensible ways to segment the market

• Latency requirements
  • High availability and low latency go together
• Query types
  • Simultaneous users for same
• Database size
• Budget

Actual segments right now

• Utter ADW/EDW
• Data mart
  • Size
  • Naturally columnar vs. naturally row-based
• Operational/frontline
• Less dramatic/smaller EDW

• Teradata’s future lies in solid-state memory. That’s in line with what Carson Schmidt told me six months ago.
• To Teradata’s surprise, the solid-state future is imminent. Teradata is 6-9 months further along with solid-state drives (SSD) than it thought a year ago it would be at this point.
• Short-term, Teradata is going to increase the number of appliance kinds it sells. I didn’t actually get details on anything but the new SSD-based Blurr, but it seems there will be others as well.
• Teradata’s eventual future is to mix and match parts (especially different kinds of storage) in a more modular product line. Teradata Virtual Storage is of pretty limited value otherwise. I probably believe Teradata will go modular more emphatically than Teradata itself does, because I think doing so will meet users needs more effectively than if Teradata relies strictly on fixed appliance configurations.
  

In addition, some non-SSD componentry tidbits from Carson Schmidt include:

• Teradata really likes Intel’s Nehalem CPUs, with special reference to multi-threading, QuickPath interconnect, and integrated memory controller. Obviously, Nehalem-based Teradata boxes should be expected in the not too distant future.
• Teradata really likes Nehalem’s successor Westmere too, and expects to be pretty fast to market with it (faster than with Nehalem) because Nehalem and Westmere are plug-compatible in motherboards.
• Teradata will go to 10-gigabit Ethernet for external connectivity on all its equipment, which should improve load performance.
• Teradata will also go to 10-gigabit Ethernet to play the Bynet role on appliances. Tests are indicating this improves query performance.
• What’s more, Teradata believes there will be no practical scale-out limitations with 10-gigabit Ethernet.
• Teradata hasn’t decided yet what to do about 2.5” SFF (Small Form Factor) disk drives, but is leaning favorably. Benefits would include lower power consumption and smaller cabinets.
• Also on Carson’s list of “exciting” future technologies is SAS 2.0, which at 6 gigabits/second doubles the I/O bandwidth of SAS 1.0.
• Carson is even excited about removing universal power supplies from the cabinets, increasing space for other components.
• Teradata picked Intel’s Host Bus Adapters for 10-gigabit Ethernet. The switch supplier hasn’t been determined yet.

Let’s get back now to SSDs, because over the next few years they’re the potential game-changer. The big news on SSDs is that after last year’s Teradata Partners conference, a stealth supplier* introduced itself and convinced Teradata it offers really great SSD technology. For example, not a single SSD it has provided Teradata has ever failed. (In hardware, that is. There have of course been firmware bugs, suitably squashed.) I think SSD performance is also exceeding Teradata’s expectations. This supplier is where the 6-9 month time-to-market gain comes from.

*Based on how often the concept of “stealth” and “name is NDAed” came up, I do not believe this is the SSD company another vendor told me about that is going around claiming it has a Teradata relationship.

Teradata SSD highlights include:

• I/O speeds on “random medium blocks” are 520 megabytes/second, vs. 15 MB/second on their fastest disks. And that’s limited by SAS 1.0, load-balanced across two devices, not the hardware itself. (2 x 300+ MB/sec turns out to be 520 MB/sec in this case.) No wonder Carson is excited about SAS 2.0.
• Teradata is using SAS interfaces for its SSDs, and believes that’s unusual, in that other companies are using SATA or Fibre Channel.
• Never having had a part fail, Teradata has no real basis to make MTTF (Mean Time To Failure) estimates for its SSDs.
• Teradata’s SSD appliance design includes no array controllers. The biggest reason is that right now array controllers can’t keep up with the SSDs’ speed.
• In its SSD appliance, Teradata has abandoned RAID, doing mirroring instead via a DBMS feature called Fallback that’s been around since Teradata’s earliest days. (However, unlike Oracle in Exadata, Teradata continues to use RAID for disks.)
• Useful life for Teradata’s SSDs is estimated at 5-7 years.
• Teradata’s SSDs are SLC (Single-Level Cell), as opposed to MLC (Multi-Level Cell).

• Teradata ascribes considerable importance to a Virtual Storage technology whose main purpose is to allow mixing of heterogeneous storage devices in a single system. And the discussion rarely suggests that these parts will be in a rigid fixed relationship.
• Netezza — as Teradata keeps reminding me — often sells boxes with the expectation that they won’t be filled with data, so as to increase spindle count and hence performance.
• Oracle/Sun have dropped some comments about Exadata being more flexibly configured going forward.
• Kickfire’s new “high-end” appliance lets you attach fairly arbitrary amounts of external storage.
• And of course, software-only analytic DBMS vendors run their software in all sorts of hardware and storage environments.

What’s more, the claim never made a lot of sense anyway. With the rarest of exceptions, even a single data warehouse’s workload will contain different queries that strain different parts of the system in different ratios. Calculating the “ideal” hardware configuration for that single workload would be forbiddingly difficult. And even if one could calculate it, it almost surely would be different than another user’s “ideal” configuration. How a single hardware configuration can be “ideally balanced” for a broad class of use cases boggles the imagination.

]]> http://www.dbms2.com/2009/10/25/data-warehouse-balanced-hardware-configuration/feed/ 6 http://www.dbms2.com/2009/10/19/teradata-partners-2009/ http://www.dbms2.com/2009/10/19/teradata-partners-2009/#comments Mon, 19 Oct 2009 13:07:31 +0000 Curt Monash http://www.dbms2.com/?p=1150 Teradata tells me that its press embargoes are ending at 9:00 this morning. Here are some highlights of what’s going on, although names, dates, and details will have to await conversations and press releases this week.

• Teradata is productizing “private cloud,” under names including “Teradata Enterprise Analytics Cloud,” “Teradata Agile Analytics Cloud,” and “Teradata Elastic Mart Builder.” I.e., Teradata hopes to leapfrog Greenplum in its “Enterprise Data Cloud” strategy. This is only fair, in that Greenplum lifted the idea from Teradata and eBay in the first place. It also provides major support for what I think is an extremely sensible trend. Give or take issues of who announces and ships what a couple months before or after a competitor, my early thinking is that the main differences between Greenplum and Teradata in this regard will be:
  • Virtual as opposed to just physical data marts, based on robust workload management software. (Advantage: Teradata)
  • Pricing, deployment options. (Advantage: Greenplum)
  • Features that don’t directly relate to enterprise/private cloud. (Advantage: Either, often Teradata.)
• Teradata is generally strengthening its data movement technology, e.g. for making various appliances work in sync. I’m not too clear yet on the details of that. I think this is what Teradata’s phrase “ecosystem management” refers to.
• Teradata is (pre-)announcing – at least as a statement of direction — an appliance based on solid-state drives (SSDs). I’ve thought for a while that Teradata was a leader in thinking through the issues around solid-state memory in data warehousing, so it makes sense that they’re among the leaders in actually coming to market as well. I plan to say more after meeting with, e.g., Carson Schmidt.
• Teradata has achieved a 300%ish speed-up in geospatial processing. I gather this is largely a byproduct of the parallel analytics work Teradata did around strengthening its SAS integration. However, there don’t seem to be a lot of Teradata geospatial users yet.
• Teradata Express, Teradata’s free Windows-based crippleware, is being ported to Amazon EC2 and VMware as well. Presumably to avoid cannibalizing Teradata product sales, there are quite a few limitations on Teradata Express, including system capacity, database size, and “no production use.”
• Teradata continues to extend its optimizations to handle queries issued by business intelligence tools. Previously, the focus of what Teradata discussed in this regard was query rewrite. But soon automatic recommendation and creation of Aggregate Join Indexes – i.e.., materialized views – will be included as well.