Winter Corporation on Exadata
The most ridiculous analyst study I can recall — at least since Aberdeen pulled back from the “You pay; we say” business — is Winter Corporation’s list of large data warehouses. (Failings include that it only lists warehouses run by software from certain vendors; it doesn’t even list most of the largest warehouses from those vendors; and its size metrics are in my opinion fried.) So it was with some trepidation that I approached what appears to be an Oracle-sponsored Winter Corporation white paper about Exadata.*
*Edit: As Ken Jacobs points out in the comments below, Oracle’s sponsorship is indeed disclosed straightforwardly. Disclosure isn’t crystal-clear. But it’s identified as part of a Winter Corporation “Sponsored research program,” and I can’t imagine who ELSE would have sponsored the work, except maybe HP. Besides, I found out about the paper via an “announcement” in Kevin Closson’s blog.
There are indeed some eyebrow-raisers in the paper. E.g. the paper’s introduction says that Winter Corporation was “able to observe” Exadata’s “modular expandability” and “resilience to failure,” but its body provides not a smidgeon of evidence Winter tested or observed any such thing. Still, I see no reason to doubt the paper’s essential conclusions, which are that:
- Oracle Exadata ran four concurrent complex queries quickly in an independent test.
- It did so more (cost)-effectively than other Oracle implementation alternatives.
- Running a few queries at once can actually improve average performance, because in the single-query case disk and CPU are more apt to sit idle waiting for each other. Edit: I’m drawing a blank on who exactly it was, but — if memory serves — one of Kickfire/Exasol/ParAccel told me that both they and a second member of that group had more than the minimum number of “users” in their TPC-H submissions. I didn’t check the submissions myself, but basically that sounds like the same idea.
Comments
5 Responses to “Winter Corporation on Exadata”
Leave a Reply
Hi Curt. Just one comment … there is nothing sneaky or underhanded herer and there is no lack of clarity regarding disclosure in the Winter Report. I thought I would mention this, since the bulk of your commentary is not about the findings of the report or the technical merits of Exadata and thus leaves the wrong impression.
The second paragraph of the executive summary reads …
“This paper presents the results of the first independently validated tests of the performance of Exadata for data warehousing. Winter Corp, an industry expert in large scale data management,
was retained by Oracle to review the design of the tests; audit the test process; analyze the results; and, provide an independent technical assessment of the product.”
In case you don’t see it on second reading, it says, “Winter Corp … was retained by Oracle”.
Nuff said.
Ken
Thanks, Ken. I missed that.
When I do a sponsored paper, there’s a big vendor logo on the front page, and one or more paragraphs disclosing the the sponsorship. I guess that dulled my reading skills with respect to Winter Corp’s more subtle disclosure. Sorry.
CAM
We all make mistakes sometimes … me included. Thanks for the edits.
The WinterCorp report on Exadata is labelled as sponsored research on the front cover, in addition to the descriptions in the text. The opinions on modular expansion and resilience were made over several months of involvement with Oracle in the benchmark tests and with Oracle customers who had participated in the beta program.
The WinterCorp studies on database size, which have been conducted every few years since 1995, are open to all users. The data is vigorously validated by WinterCorp approved scripts run by each user and reviewed by WinterCorp tech staff. The process is the same regardless of vendor. We separately measure user data, indexes, summaries, free space and work space. The 100 TB database you mention in your article was measured to contain 89 TB of user data.
The data, as noted on our website,was accurate as of September, 2005, when the survey closed. We look forward to providing updated data, and when it is available, it will be published.
If you heard of larger databases at that time, chances are that you heard a figure based on total disk, not data.
Richard
Richard,
Thanks for posting. But c’mon. The 10th largest database in that survey was under 18 TB of user data. Do you seriously believe that all of eBay, Wal-Mart, Sears, Bank of America, or Chase were under 18 TB at the time of the survey? Do you seriously believe that any of them were under 18 TB? Do you seriously believe that any of them were under 50 TB?
CAM
Edit: Wait. I just looked at the survey, and you’re talking about more than user data — indices and aggregates as well. And you still got numbers that low. Wow.