The H-Store claim is that they get an order of magnitude or whatever single-processor performance advantage over disk-based systems, even for database sizes that disk-based systems can put entirely in cache. (Indeed, the starting number is more like 15X, but that’s the really rose-colored view.) That’s supposed to make up for any parallelization awkwardness.

The same team did C-Store/Vertica, and they followed a similar approach. I’m not aware of Vertica having the same level of interprocessor data movement (or, better yet, data movement prevention) sophistication as some of its competitors. But based on their sales figures, it seems they’re winning a lot of POCs even so.

Best,

CAM

I’d like to expand my point about idle processors. Basically, you can always make a database application faster by throwing hardware at it. For many applications, the big modern database products don’t scale very well with multiple machines, so typically you have to buy faster computers, which is expensive because price goes up much faster than computer speed. That is, to get a box that’s twice as fast, you pay lots more than twice as much. What H-Store is trying to take advantage of is that buying twice as many computers of the same speed costs exactly twice as much (less if you get bulk discounts ). So it’s less expensive to have a database product that scales and buy several cheap computers to run it on than to buy one that doesn’t scale and buy big iron.

However, if you have idle processors, that changes the equation. If your processors are idle half the time then you have to buy four computers, not just two. Suddenly the equation doesn’t look quite so good for multiple computers. At that rate, you are better off going with a computer that is twice as fast, so long as it is less than four times as expensive.

Thanks for your comments in the first two paragraphs of 3)!

I’m glad they’re working on better synchronization; despite quite a bit of time talking with them I never got to exactly that point.

Where you confused me is where you seemed to be saying that a stored procedure has to have all of the following properties:

A. No application logic.
B. Single ACID transaction.
C. Coarse-grained and doing a lot of work.

While that’s what I read, it surely can’t be exactly what you meant.

Best,

CAM

1. Idle processors: Yes, I think that’s true. I’m not sure it’s a problem. The real measure of a system is whether it gets the latency and throughput that you want. Saying that idle processor are a drawback is like saying that not using the whole disk is a drawback. So what if it leaves processors idle? Processors are cheap. Scaling is what matters. (You may or may not buy this point…)

2. Moving data: Yes, that’s right. The paper calls this “general transactions”: the ones where the different processors need to interact during the execution of commands. It admits that these won’t work well. The hope is that the great majority of the commands whose speed is critical can be done as single-node or one-shot transactions (see paper). The degree to which this will work depends a lot on your schema and your queries. In some cases, it’s very easy to make this work. For example, in the database of one prominent company I know, there are only two kinds of data: data that is specific to one user account, and a small amount of slow-changing configuration data. The former can be partitioned trivially, and the latter just gets replicated everywhere. Now all transactions can run on a single node. In other applications, it’s much harder to make this work. We won’t know until and unless H-store gets productized and applied to a wide variety of applications. This is certainly a potential drawback of the H-store approach.

3. Synchronization: Yes, you’re right. H-store has a synchronization mechanism. The one in the paper depends on guaranteed hard maximum realtime limits on network transmission, which is unfortunately not a condition obtainable in real-world circumstances. However, the authors of the paper are working on improving this (private communication). Assuming they do, the claim is that the overhead will be amortized over a relatively large amount of actual work per command, so that its percentage of overhead will be acceptably low. Also, a single-node read transaction can all happen on one processor so it doesn’t run into this problem, and probably the hope is that those will be relatively common.

What appears to be the big issue is that you need, as you say, to do program re-writing. You have to arrange things so that your commands to H-store are of high enough granularity (lots of actual work per command). Furthermore, a single command always performs one ACID transaction. So you can’t interleave application logic with transaction execution. Some applications will be easy to rewrite this way and some will be hard. Again, they’ll need more experience to see how this works out.

About “not having any persistent store”, please read my earlier post, which I feel takes care of this issue.

About doing 5-way joins: well, it’ll also depend a lot on the application. If the five-way joins are on five tables that are generally used together, they can be co-located on single hosts, and ther’s nothing to stop H-store from acquiring a more sophisticated query optimizer down the line if it turns out that it’s needed more often than the paper says to expect. So I don’t think this is an inherent problem with the H-store concept.

On the whole, I agree that only time will tell, and it all depends on the specifics of the particular application and database structure.

As I said before, you always have to deal with b-tree consistency and other problems around simultaneous access to shared data. H-Store doesn’t avoid the problem; H-store simply replaces locking with partitioning (or I should say that H-store uses spatial partitioning as opposed to ADS which uses what might be called temporal partitioning). But spatial partitioning is not a magical solution; it has its own overheads:

1. Idle processors: each processor can only work on a particular subset of data, so if there is no work to do on that subset of data then the processor is idle.
2. Moving data: any work that requires sharing data among multiple processors requires moving data around.
3. Synchronization: any work that requires distributed commits has the overhead of synchronizing the distribute commit.

The claim of this paper is essentially that for “OLTP applications” 2 and 3 are either not significant portions of the work or can be largely eliminated by program re-writing. This claim remains to be proven and I rather doubt from my own experience that the claim is correct. The paper doesn’t address 1 at all.

Then there is the issue of having no persistent store. Other commenters have pointed out the problems with this.

My experience is that applications are never what the model says they should be. There was a comment in the paper to the effect that OLTP applications don’t do 5-way joins. I remember when we used to think that. We also thought that performance critical applications would use prepared statements and wouldn’t update indexed columns and would use tree-like schemas and other things. I fondly remember those days of optimistic naivete :-).

So, while I grant that Stonebreaker knows much more about databases that I do, I’m skeptical as to whether this will work. ANTs also is able to get 80X performance over commercial databases for certain kinds of work loads. But when we do real applications, the actual benefit is more like 3X to 10X.

1. ACID isn’t being taken away.
2. See my later post on ObjectGrid vs. H-Store. You’re not all wrong.
3. In essence, this is an attempt to roll a lot of other technologies into one, which can lead to savings in all areas of TCO (programming, admin, hardware/power, etc.)

CAM

So here’s a question… how will this “H-Store” view of the world be different from the raft of transaction coordinating middleware that’s out there already? Most MQ’s/TC’s are tuned for throughput rather than persistence, support 3/4GL’s instead of SQL and have little or no persistent storage. The only thing Mike &c are taking away is the ACID properties of the transactions?

I guess what I’m saying is - hasn’t this already been invented and stabilized as a discrete set of products for 10 years or more already?

Regards

Jeremy Huppatz
EDS Australia
Data Engineering Capability

I agree, as per http://blogs.zdnet.com/BTL/?p=8055

However, the H-Store guys do have one interesting rejoinder. Namely, assume that all the data is being streamed to a data warehouse (different software, probably WITH disk of its own). Then that argument is weakened a lot.

Even so — I think fairly frequent checkpointing is the way to go. Leaving it out doesn’t save enough to make up for the doubt caused by discussions like this.

CAM

If you are running all the same code on all the same OS, then your distributed system is still vulnerable to security failures (dos/intrusion/virus/worms), and time and data-induced bugs. To address these failure scenarios properly you would need an ecosystem of machines running different OS’s and independent implementations of the same software. Who can afford that, just to avoid writing to disk?

Geographically-distributed RAM replicas are swell - we use them all the time. However, IMHO, they can only for promise unrelenting availability of the data. My point is that these ensure data resiliency, not its persistence. Recovery from disk in both cases is taken for granted, the only differentiator being is the data’s staleness. A backup is only updated until the point in time in which it was taken, while persistence usually provides fresher data.

IH