I don’t know what you’re talking about, because I don’t take benchmark descriptions seriously enough to read and remember them. See the “Benchmarking” section here or search on “TPC” to see what I mean.

CAM

We have worked on a new database/platforms for last 6 years we have been running enterprises on in memory model with many points being listed in this article. From our research we concur, the relational DB model is no good. It is not only about performance but also inflexibility and the resultant introduction of unnecessary complexity.

We have gone much further than all this. Combination of these DB thoughts with semantics leads to all kinds of other weird and amazing properties for db. Radical reductions in size of DB are possible. Atomisation of databases increases speed etc. etc. To discover what is right all the angles have to be investigated. Ultimately DB is just a component to keep/act on knowledge/information.

Pawel Lubczonok

The H-Store claim is that they get an order of magnitude or whatever single-processor performance advantage over disk-based systems, even for database sizes that disk-based systems can put entirely in cache. (Indeed, the starting number is more like 15X, but that’s the really rose-colored view.) That’s supposed to make up for any parallelization awkwardness.

The same team did C-Store/Vertica, and they followed a similar approach. I’m not aware of Vertica having the same level of interprocessor data movement (or, better yet, data movement prevention) sophistication as some of its competitors. But based on their sales figures, it seems they’re winning a lot of POCs even so.

Best,

CAM

I’d like to expand my point about idle processors. Basically, you can always make a database application faster by throwing hardware at it. For many applications, the big modern database products don’t scale very well with multiple machines, so typically you have to buy faster computers, which is expensive because price goes up much faster than computer speed. That is, to get a box that’s twice as fast, you pay lots more than twice as much. What H-Store is trying to take advantage of is that buying twice as many computers of the same speed costs exactly twice as much (less if you get bulk discounts ). So it’s less expensive to have a database product that scales and buy several cheap computers to run it on than to buy one that doesn’t scale and buy big iron.

However, if you have idle processors, that changes the equation. If your processors are idle half the time then you have to buy four computers, not just two. Suddenly the equation doesn’t look quite so good for multiple computers. At that rate, you are better off going with a computer that is twice as fast, so long as it is less than four times as expensive.

Thanks for your comments in the first two paragraphs of 3)!

I’m glad they’re working on better synchronization; despite quite a bit of time talking with them I never got to exactly that point.

Where you confused me is where you seemed to be saying that a stored procedure has to have all of the following properties:

A. No application logic.
B. Single ACID transaction.
C. Coarse-grained and doing a lot of work.

While that’s what I read, it surely can’t be exactly what you meant.

Best,

CAM

1. Idle processors: Yes, I think that’s true. I’m not sure it’s a problem. The real measure of a system is whether it gets the latency and throughput that you want. Saying that idle processor are a drawback is like saying that not using the whole disk is a drawback. So what if it leaves processors idle? Processors are cheap. Scaling is what matters. (You may or may not buy this point…)

2. Moving data: Yes, that’s right. The paper calls this “general transactions”: the ones where the different processors need to interact during the execution of commands. It admits that these won’t work well. The hope is that the great majority of the commands whose speed is critical can be done as single-node or one-shot transactions (see paper). The degree to which this will work depends a lot on your schema and your queries. In some cases, it’s very easy to make this work. For example, in the database of one prominent company I know, there are only two kinds of data: data that is specific to one user account, and a small amount of slow-changing configuration data. The former can be partitioned trivially, and the latter just gets replicated everywhere. Now all transactions can run on a single node. In other applications, it’s much harder to make this work. We won’t know until and unless H-store gets productized and applied to a wide variety of applications. This is certainly a potential drawback of the H-store approach.

3. Synchronization: Yes, you’re right. H-store has a synchronization mechanism. The one in the paper depends on guaranteed hard maximum realtime limits on network transmission, which is unfortunately not a condition obtainable in real-world circumstances. However, the authors of the paper are working on improving this (private communication). Assuming they do, the claim is that the overhead will be amortized over a relatively large amount of actual work per command, so that its percentage of overhead will be acceptably low. Also, a single-node read transaction can all happen on one processor so it doesn’t run into this problem, and probably the hope is that those will be relatively common.

What appears to be the big issue is that you need, as you say, to do program re-writing. You have to arrange things so that your commands to H-store are of high enough granularity (lots of actual work per command). Furthermore, a single command always performs one ACID transaction. So you can’t interleave application logic with transaction execution. Some applications will be easy to rewrite this way and some will be hard. Again, they’ll need more experience to see how this works out.

About “not having any persistent store”, please read my earlier post, which I feel takes care of this issue.

About doing 5-way joins: well, it’ll also depend a lot on the application. If the five-way joins are on five tables that are generally used together, they can be co-located on single hosts, and ther’s nothing to stop H-store from acquiring a more sophisticated query optimizer down the line if it turns out that it’s needed more often than the paper says to expect. So I don’t think this is an inherent problem with the H-store concept.

On the whole, I agree that only time will tell, and it all depends on the specifics of the particular application and database structure.