1. Shipping rows around between execution plan fragments sounds OK for small amounts of data, but with large volumes, it’s far more efficient to move data around in large blocks to avoid the overhead of many small movements (especially on GigE). We’ve been able to handle all queries put before us, so I don’t see any inherent advantages in terms of functionality.

2. The last time I looked, Postgres was multi-process and not multi-threaded and I don’t think that’s changed. My guess is that there’s a lot of time spent waiting for rows with this kind of approach.

3. I don’t understand why the approach described above leads to the conclusion that ‘native indexing and highly optimized aggregations’ can’t be easily done with different architectures. We certainly manage it.

4. In any real-world DW, what use are bit-mapped indexes with cardinality of up to 10,000? We generally deal with tables of billions of rows and cardinality in the tens or hundreds of millions.

5. The network throughput numbers quoted by Sun don’t make any sense. How do I get 1GBps through four GigE links? Each link will max out at 80MBps, so that gives only 320MBps. Also, even with a TOE, you’d see a lot of CPU load with that kind of data movement. With such light CPU power relative to the number of disks, how does the system scale under concurrency?

6. There’s also a claim of one TB per minute table scans from ten servers floating around. I presume that’s calculated by just multiplying Sun’s claim of 2GBps disk throughput x 10 x 60. Seems unlikely that Postgres could get anywhere near that in practice with just two dual core CPUs. Even if a simple table scan could run at that speed (which I doubt), our experience with Postgres is that it’s MUCH slower than Ingres when running actual queries.

Stuart
DATAllegro

WRT “query shipping”, I was actually referring to a simpler approach used by others, not ours. My admittedly subtle but I think important point was that the whole problem of supporting arbitrary DBMS work is that you have to get inside the database engine and implement optimization at the “execution plan” level and not the “query plan” level. Rather than “repartitioning on the fly”, we pipeline rows through the interconnect among execution plan fragments in real time. We do this because of the performance, generality and ease of adding future capabilities associated with a DBMS internal architecture. I think this is critical and you should expect us to continue to have advantages like supporting a rich assortment of native indexing and highly optimized aggregations that can’t be done easily without our architecture.

Let’s see if this sparks some comments!