Therefore Pentaho literally not comparable to Exasol, ParAccel, Vertica. Pentaho is a different genre of product.

I’m pretty sure that, say, Exasol, ParAccel, QlikView, and SAP BI Accelerator all do a lot better jobs than row-based DBMS’ caches do. Compression lets you put more in RAM. Convincing the cache to preload exactly what you want isn’t always as straightforward as running the right query at the right time. Etc.

CAM

But yeah, I’d say there was something quite confusing about how the statement was framed. With the numbers that far out of whack, the task has to have been something very different from what we commonly think of as “load”.

CAM

I’m quite certain they don’t have it, at least not yet. This is why they only used memory based solutions at the very bottom of the scale factors (100GB, 300GB and 1000GB).

It would seem that the memory based solutions (ParAccel, Exasol) are only effective if all of the required data is in memory. For example take this customer benchmark from Exasol. The first run of the queries took 20 minutes compared to 24 minutes on the customer’s existing system. The explanation from Exasol is that during the first run EXASolution “completely reorganized the internal data and performed internal optimizations, e.g. it generated an index”. Of course, the subsequent runs took significantly less time, but let’s be realistic, that is not a new trick. Both DB2 and Oracle have features (Query Patroller and Results Cache) that can just return the result if a given query is run more than once. IMO so called customer benchmarks where the same queries are executed more than once are quite unimpressive.

Now if my data warehouse or data mart follows most, nightly bulk load, then automated KPI reports (or similar) this is there any advantage with a product like this (ParAccel, Exasol)? I don’t know the answer but I am interested in knowing if you know.

WRT to the validity of TPC-H or conclusions drawn from them:

Whether or not you think TPC-H is valid or not, there are audited and validated metrics that are in the full disclosure reports that would probably allow you cross-check some of the metrics that you report on. For instance, in your post on TEOCO you wrote:

“Oracle couldn’t get the load time for 100 million call detail records (CDRs) below 24 hours“

This full disclosure report shows that an Oracle database was able to load the data for entire 30TB scale factor (almost 260 billion rows) in just over 16 hours. Loading data is not rocket science, but it appears that with TEOCO there would appear that there was a bit of EBKAC going on. This would seem to be also confirmed from Paul’s comment. Would you agree Curt?

CAM

Perhaps I should have said that most conclusions drawn from TPCs are jokes. I wouldn’t say that TPCs provide no evidence for any claim at any time.

But if you think about it, in the post I mainly was illustrating what TPCs did NOT show — namely, great disk-centric performance for ParAccel. They may have it, but the TPCs don’t show that, because the TPCs weren’t done on a disk-centric configuration.

CAM