Looking at VectorNova’s site, however, it sounds like VectorStar is based on arrays rather than columns, perhaps like http://www.dbms2.com/2006/10/04/sas-intelligence-storage/

Thanks for the clarification – that makes much more sense now.

I believe that in the scenario you describe – loading, the graph wouldn’t have much value, whereas when you update (hence you must enforce consistency and coherency), it would be interesting.

DK

I don’t think I follow the analogy. There’s a big distinction between loads and updates. In inserts/loads, there’s no “coherency” needed beyond knowing which data was part of what transaction (timestamping, as you say). From the perspective of a node processing a load stream, each row loaded “belongs” on one of the nodes in the cluster, so it can be sent to that node directly and nobody else needs to know about it. (If a query requests that row, the node processing the query knows where to look.)

So from the perspective of a node doing a load in a cluster of N nodes, it processes a stream of data and sends it to N-1 places (keeping 1/Nth for itself). The node will also receive data from N-1 other nodes if they are processing loads. So if all nodes are loading data at the same time, each node would expect to receive as much data as it sends. So load speed scales linearly with the number of nodes as long as:
1. A node can handle N incoming load streams. (This translates to a memory requirement, mostly.)
2. The interconnect can distribute all the data, full duplex, with all nodes talking to all others at once on the backplane. (Currently this is true of relatively cheap GigE switches with 32 or 64 ports.)

I’m sorry, but I don’t understand the concept behind the graph. It seems to me like for a given load size and frequency, either the system will keep up or it won’t. Wouldn’t it be more intersting to know what the maximum load rate is (GB/min) for combinations of parameters like # of nodes in the cluster and # of GB in each load?

Thanks for the elaboration. Here’s why I ask…

When I think about the problem from an architectural standpoint (computer architecture that is), it’s exactly analogous to an update cache coherency policy.

The E/T steps are local, and those probably scale reasonably well. What isn’t going to scale is the 1:N communication of distributing the data out to every node, and the resulting acknowledgements. You have to update all N nodes, and you can’t really get around that.

IIRC, you guys use timestamping, so you don’t have to redo any in-flight transactions because of an update.

As I said, what would be interesting would be a 3D graph of:

Data load size (x GB), data load frequency (Y loads/hour) and performance (Z seconds)

DK

We did see around 4x load speed compared to 1 node in this test, which is far more than we can say for a competing row store that uses a shared disk architecture and saw 2.5x. Likewise, a competing shared-nothing row store without a parallel load feature didn’t get anywhere near 4x on 4 nodes, as load saw 1x while index and MV build saw 4x.

Of course other products out there do it the same way we do and don’t suffer these limitations, but I repeat my claim that there’s no theoretical reason why a column store would be beaten on load performance. Nor do we ever get beaten (on apples-to-apples hardware of course).

Stay tuned for more complete presentations on our numbers in an upcoming paper, as well as bigger cluster and data sizes.