An interesting part of my conversation with Dataupia’s CTO John O’Brien came when we talked about data warehousing in general. On the one hand, he endorsed the view that using Oracle probably isn’t a good idea for data warehouses larger than 10 terabytes, with SQL Server’s limit being well below that. On the other hand, he said he’d helped build 50-60 terabyte warehouses in Oracle years ago.
The point is that to build warehouses that big in Oracle or other traditional DBMS, you have to pull out a large bag of tricks. (John gave a figure of 1-2 dozen.) Areas include schemas, partitioning, specialized SQL, “very specialized file layouts”, and so on. Summing up, John said that using Oracle above 10 terabytes was very much like working with columnar database structures.
Things are a lot simpler in the specialist world. Buy a data warehouse appliance, load it up in any halfway sensible way, and you’ll probably have decent performance from the get go. Tuning is still a really good idea, of course. But your struggles are to make the system run better, or to make a few annoying performance problems go away. You’re not performing heroic acts just to get it running in the first place.
Please subscribe to our feed!