December 14, 2008

The “baseball bat” test for analytic DBMS and data warehouse appliances

More and more, I’m hearing about reliability, resilience, and uptime as criteria for choosing among data warehouse appliances and analytic DBMS. Possible reasons include:

The truth probably lies in a combination of all these factors.

Making the most fuss on the subject is probably Aster Data, who like to talk at length both about mission-critical data warehouse applications and Aster’s approach to making them robust. But I’m also hearing from multiple vendors that proofs-of-concept now regularly include stress tests against failure, in what can be – and indeed has been – called the “baseball bat” test. Prospects are encouraged to go on a rampage, pulling out boards, disk drives, switches, power cables, and almost anything else their devious minds can come up with to cause computer carnage. The goal is to see whether systems keep running, how well they perform when impaired, or how fast they recover if they indeed do go down.

Teradata benchmark honcho Gene Erickson definitely encourages this kind of behavior. Kognitio – which sees “resilience” as a competitive advantage and reports that prospects increasingly care about same — is friendly to such behavior as well. And that’s surely not a complete list.

Of course, some data warehouse users have cared about robustness for years, even up to the point of replication and hot standbys. Even so, I think there’s recently been a change in the market. For example, when Vertica and before that its research predecessor C-Store were rolled out, Mike Stonebraker repeatedly called attention to the potential for implementing them with high redundancy, but I and most other observers basically yawned.

Selecting analytic DBMS products is even more fun than it used to be. 🙂


6 Responses to “The “baseball bat” test for analytic DBMS and data warehouse appliances”

  1. Steve Wooledge on December 15th, 2008 11:07 am

    I love the baseball bat analogy, Curt. Makes me wish it was summer.

    Another important aspect to consider is the elimination of “planned” downtime (e.g., loading data, upgrading the system, etc.). Aster Data is also focused on eliminating/reducing this aspect as well. More here:

    There is also a funny video about using a baseball bat on servers here (we used too small of a sledgehammer, but the effect was similar):

  2. Andy E on December 15th, 2008 3:36 pm

    We see baseball bat testing by Vertica evaluators a lot. Comcast talked about the “swings” they took in a webcast they did last year –

    Curt, is there a reason high availability (baseball bat) testing wasn’t included in your list of analytic database assessment criteria in ?

  3. Curt Monash on December 15th, 2008 8:37 pm


    It was kind of implicit in the references to POCs and SLAs.

    Mainly, however, I was talking about universals, and not so much stressing things that are important to some users but not others.

  4. Joe Harris on December 16th, 2008 5:41 am

    Are companies putting their DW’s in Iraqi data centers or what?

    In the large enterprises that I’ve worked with the S.O.P. is to have a completely duplicate “disaster recovery” version of the DW. Expensive but reassuring (a bit like Teradata…).

    I really don’t see them moving away from this approach for a number of years, even though it is clearly wasteful and unnecessary.

    I think the reality is that many CIOs are not technically proficient enough to be convinced that a “highly resilient” and/or “distributed” data warehouse can remove the need for a DR standby.

  5. Oracle says they do onsite Exadata POCs after all | DBMS2 -- DataBase Management System Services on February 1st, 2009 4:16 am

    […] And vendor-site POCs can be done quite respectably.  For example, the guy who first focused me on baseball-bat testing is Teradata’s chief of in-house […]

  6. The Netezza guys propose a POC checklist | DBMS2 -- DataBase Management System Services on February 18th, 2009 3:48 pm

    […] The Netezza guys at “Data Liberators” are being a bit too cute in talking about FULL DISCLOSURE yet not actually saying they’re from Netezza — but only a bit, in that their identity is pretty clear even so.  That said, they’ve proposed a not-terrible checklist of how to conduct POCs.  Of course, vendor-provided as it is, it’s incomplete; e.g., there’s no real mention of a baseball-bat test. […]

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.