Comments on: Data warehouse storage options — cheap, expensive, or solid-state disk drives http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/ Choices in data management and analysis Thu, 09 Feb 2012 21:52:17 +0000 hourly 1 http://wordpress.org/?v=3.0.3 By: Curt Monash http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-219661 Curt Monash Sun, 24 Apr 2011 16:11:52 +0000 http://www.dbms2.com/?p=768#comment-219661 I think you'd do best to check with Sybase on that. Prices change too often for me to have that memorized. On the plus side, they often have fairly clear web pages with their list pricing. I think you’d do best to check with Sybase on that. Prices change too often for me to have that memorized.

On the plus side, they often have fairly clear web pages with their list pricing.

]]> By: sai http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-219658 sai Sun, 24 Apr 2011 16:04:16 +0000 http://www.dbms2.com/?p=768#comment-219658 Dear all, Does anybody know what is the cost of additional storage of 1 TB added to an existing Warehouse. My client company is having SYbase IQ Datawarehouse, and I'm just curious to know what would be the incremental cost of 1TB, coz they might add upto 3. Regards, Sai. Dear all,

Does anybody know what is the cost of additional storage of 1 TB added to an existing Warehouse. My client company is having SYbase IQ Datawarehouse, and I’m just curious to know what would be the incremental cost of 1TB, coz they might add upto 3.

Regards,

Sai.

]]> By: eBay followup — Greenplum out, Teradata > 10 petabytes, Hadoop has some value, and more | DBMS 2 : DataBase Management System Services http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-186898 eBay followup — Greenplum out, Teradata > 10 petabytes, Hadoop has some value, and more | DBMS 2 : DataBase Management System Services Sat, 09 Oct 2010 14:39:36 +0000 http://www.dbms2.com/?p=768#comment-186898 [...] Teradata and Greenplum, Oliver previously indicated he was inclined to attribute this more to specific Sun Thumper hardware/storage choices than to [...] [...] Teradata and Greenplum, Oliver previously indicated he was inclined to attribute this more to specific Sun Thumper hardware/storage choices than to [...]

]]> By: Curt Sampson http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-167713 Curt Sampson Sat, 08 May 2010 06:33:45 +0000 http://www.dbms2.com/?p=768#comment-167713 Or someone can just go write a sensible DBMS that doesn't force you to link the logical format with the physical. There's no reason that several normalized relations can't be stored as a single denormalized table on disk, if that happens to be best for the query load. Column-oriented systems are an example of a different storage method under a relational front-end, though they suffer just as badly from not being able to store things in a row-oriented manner when that makes more sense. Or someone can just go write a sensible DBMS that doesn’t force you to link the logical format with the physical. There’s no reason that several normalized relations can’t be stored as a single denormalized table on disk, if that happens to be best for the query load. Column-oriented systems are an example of a different storage method under a relational front-end, though they suffer just as badly from not being able to store things in a row-oriented manner when that makes more sense.

]]> By: Revisiting disk vibration as a data warehouse performance problem | DBMS2 -- DataBase Management System Services http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-167698 Revisiting disk vibration as a data warehouse performance problem | DBMS2 -- DataBase Management System Services Sat, 08 May 2010 04:06:07 +0000 http://www.dbms2.com/?p=768#comment-167698 [...] April, I wrote about the problems disk vibration can cause for data warehouse performance. Possible performance hits exceeded 10X, wild as that [...] [...] April, I wrote about the problems disk vibration can cause for data warehouse performance. Possible performance hits exceeded 10X, wild as that [...]

]]> By: Curt Monash http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-121147 Curt Monash Tue, 12 May 2009 14:22:02 +0000 http://www.dbms2.com/?p=768#comment-121147 Robert, I understand the appeal of saying something like "The reason we need to be aware of physical design is largely complex query performance. Complex query performance is an issue mainly because of I/O. If we have better storage technology, that problem goes away, and we can start ignoring physical design the way the theorists have always wanted us to." But I think we're a long way from reaching that ideal, at best. Data warehouses are BIG, and getting bigger. They'll push the limits of hardware technology for a long time to come. Robert,

I understand the appeal of saying something like “The reason we need to be aware of physical design is largely complex query performance. Complex query performance is an issue mainly because of I/O. If we have better storage technology, that problem goes away, and we can start ignoring physical design the way the theorists have always wanted us to.”

But I think we’re a long way from reaching that ideal, at best. Data warehouses are BIG, and getting bigger. They’ll push the limits of hardware technology for a long time to come.

]]> By: Robert Young http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-121137 Robert Young Tue, 12 May 2009 12:48:24 +0000 http://www.dbms2.com/?p=768#comment-121137 Check Andandtech for reviews of SSD. The latest is from 20 March 2009. Deals explicitly with some of the issues here. An earlier review dealt with the "block" write versus read. The value of SSD is not going to be in highly redundant, flat-file (called whatever you want) style datastores; price will be too high. The value will be in high NF relational databases. Now, in my opinion (which you can read, and I am not alone), SSD will be the motivator that merges back OLTP with its various replicants. SSD, and the flash versions (both MLC and SLC) are only the latest low-end implementations (check Texas Memory Systems for one example of industrial strength SSD), removes the join penalty from 3/4/5NF databases. The bottleneck will be in finding folks with enough smarts to embrace (again) Dr. Codd's vision. The xml folk are not those kind of folk. My candidate is Larry Ellison. The reason is that the Oracle architecture, MVCC, is superior for OLTP (IBM finally just capitulated with entrpriseDB). With SSD, he can use the Oracle database, appropriately normalized, to support both without stars and snowflakes. A true one stop solution. Check Andandtech for reviews of SSD. The latest is from 20 March 2009. Deals explicitly with some of the issues here. An earlier review dealt with the “block” write versus read.

The value of SSD is not going to be in highly redundant, flat-file (called whatever you want) style datastores; price will be too high. The value will be in high NF relational databases. Now, in my opinion (which you can read, and I am not alone), SSD will be the motivator that merges back OLTP with its various replicants. SSD, and the flash versions (both MLC and SLC) are only the latest low-end implementations (check Texas Memory Systems for one example of industrial strength SSD), removes the join penalty from 3/4/5NF databases.

The bottleneck will be in finding folks with enough smarts to embrace (again) Dr. Codd’s vision. The xml folk are not those kind of folk. My candidate is Larry Ellison. The reason is that the Oracle architecture, MVCC, is superior for OLTP (IBM finally just capitulated with entrpriseDB). With SSD, he can use the Oracle database, appropriately normalized, to support both without stars and snowflakes. A true one stop solution.

]]> By: Mark Callaghan http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-120317 Mark Callaghan Thu, 07 May 2009 05:09:37 +0000 http://www.dbms2.com/?p=768#comment-120317 Curt, I agree with you that workload has something to do with performance. Ignore the poor wording. I mean that you won't get 10X more MB/s or IOPs from 15k SAS versus 7200 RPM SATA. Teradata has done clever things with track aligned reads to optimize disk performance. I would much rather read about that. Curt,

I agree with you that workload has something to do with performance. Ignore the poor wording. I mean that you won’t get 10X more MB/s or IOPs from 15k SAS versus 7200 RPM SATA. Teradata has done clever things with track aligned reads to optimize disk performance. I would much rather read about that.

]]> By: Curt Monash http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-119632 Curt Monash Sat, 02 May 2009 22:42:34 +0000 http://www.dbms2.com/?p=768#comment-119632 Mark, Maybe the eBay guys diagnosed their situation correctly and maybe they didn't, but I can't begin to fathom your basis for saying that workload has nothing to do with it. CAM Mark,

Maybe the eBay guys diagnosed their situation correctly and maybe they didn’t, but I can’t begin to fathom your basis for saying that workload has nothing to do with it.

CAM

]]> By: Mark Callaghan http://www.dbms2.com/2009/04/28/data-warehouse-storage-options-cheap-expensive-or-solid-state-disk-drives/#comment-119585 Mark Callaghan Sat, 02 May 2009 13:46:40 +0000 http://www.dbms2.com/?p=768#comment-119585 @Michael - you are the first person to ever claim that a 15k enterprise-grade SAS disk can do 10x more IOPs than a 7200 RPM consumer-grade SATA disk. Congratulations. @Curt - workload has nothing to do with it. Oliver has made a controversial claim with no substantiation. That is marketing and nothing else. @Michael – you are the first person to ever claim that a 15k enterprise-grade SAS disk can do 10x more IOPs than a 7200 RPM consumer-grade SATA disk. Congratulations.

@Curt – workload has nothing to do with it. Oliver has made a controversial claim with no substantiation. That is marketing and nothing else.

]]>