Comments on: Relational data warehouse Expansion (or Explosion) Ratios http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/ Choices in data management and analysis Fri, 16 May 2008 18:54:17 +0000 http://wordpress.org/?v=2.5 By: DBMS2 — DataBase Management System Services»Blog Archive » Progress Software progress report http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-27913 DBMS2 — DataBase Management System Services»Blog Archive » Progress Software progress report Sun, 29 Apr 2007 02:20:27 +0000 http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-27913 [...] size vs. Oracle for the same data, due to, for example, variable record lengths. Such factors are not unheard of in data warehousing, of course; but for OLTP his comment was quite the jaw-dropper. On the other hand, OLTP databases [...] […] size vs. Oracle for the same data, due to, for example, variable record lengths. Such factors are not unheard of in data warehousing, of course; but for OLTP his comment was quite the jaw-dropper. On the other hand, OLTP databases […]

]]> By: Curt Monash http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-9317 Curt Monash Tue, 03 Oct 2006 07:49:07 +0000 http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-9317 William, Yes, but the question is how much you need in the way of indexing and summaries to get the same performance from different systems. Also, the grand hash partition that starts off a typical MPP installation saves some index space. And if table scans are fast enough a lot of the time, you need fewer indicies than if they almost never are. Anyhow, if you look up to my IBM/Teradata post, IBM claims that indeed their installations wind up very comparable to Teradata's. But then, IBM and Teradata also have pretty similar overall technical approaches now. William,

Yes, but the question is how much you need in the way of indexing and summaries to get the same performance from different systems.

Also, the grand hash partition that starts off a typical MPP installation saves some index space. And if table scans are fast enough a lot of the time, you need fewer indicies than if they almost never are.

Anyhow, if you look up to my IBM/Teradata post, IBM claims that indeed their installations wind up very comparable to Teradata’s. But then, IBM and Teradata also have pretty similar overall technical approaches now.

]]> By: DBMS2 — DataBase Management System Services»Blog Archive » IBM and Teradata too http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-9298 DBMS2 — DataBase Management System Services»Blog Archive » IBM and Teradata too Tue, 03 Oct 2006 05:52:07 +0000 http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-9298 [...] When I posted a few days ago about Expansion Ratios, I passed along Teradata’s estimate that there was a significant gap between Teradata and IBM. IBM’s own estimate, however, is that a typical Expansion Ratio for DB2 is a Teradata-like 2-3X in DB2 Version 8, and that it’s falling further in Version 9 due to compression. (DB2’s compression sounds like it’s the most aggressive in the business, specialty columnar or MOLAP products perhaps excepted.) Incidentally, this suggests that the indexing features DB2 has that Teradata doesn’t – e.g., alternate datatypes like geospatial – aren’t heavily used by a large fraction of the customer base. [...] […] When I posted a few days ago about Expansion Ratios, I passed along Teradata’s estimate that there was a significant gap between Teradata and IBM. IBM’s own estimate, however, is that a typical Expansion Ratio for DB2 is a Teradata-like 2-3X in DB2 Version 8, and that it’s falling further in Version 9 due to compression. (DB2’s compression sounds like it’s the most aggressive in the business, specialty columnar or MOLAP products perhaps excepted.) Incidentally, this suggests that the indexing features DB2 has that Teradata doesn’t – e.g., alternate datatypes like geospatial – aren’t heavily used by a large fraction of the customer base. […]

]]> By: William McKnight http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-8469 William McKnight Fri, 29 Sep 2006 20:26:14 +0000 http://www.dbms2.com/2006/09/28/expansion-ratio-data-warehouse-explosion/#comment-8469 Curt, I believe that end users of data warehouse technologies have good control over the expansion, or uptake, ratio. It really depends on the level of performance that is needed and the amount of the data that they actually access in online queries. Some sites index everything and others only do it when absolutely necessary. Same goes for creating summary tables and cubes. TPC numbers represent the high end of anything that would be implemented at a real site because the vendors use TPC to brag most about the performance figures, not the price/performance. The Teradata numbers seem about right in practice. By the way, Teradata has an interesting new feature called "multi temperature" which allows sites to pay less for the data they don't access as much. Many have been implementing concepts like this for some time to find a good price/performance balance. Curt, I believe that end users of data warehouse technologies have good control over the expansion, or uptake, ratio. It really depends on the level of performance that is needed and the amount of the data that they actually access in online queries. Some sites index everything and others only do it when absolutely necessary. Same goes for creating summary tables and cubes. TPC numbers represent the high end of anything that would be implemented at a real site because the vendors use TPC to brag most about the performance figures, not the price/performance. The Teradata numbers seem about right in practice.

By the way, Teradata has an interesting new feature called “multi temperature” which allows sites to pay less for the data they don’t access as much. Many have been implementing concepts like this for some time to find a good price/performance balance.

]]>