Flash is coming, well …
I really, really wanted to title this post “Flash is coming in a flash.” That seems a little exaggerated — but only a little.
- Netezza now intends to come out with a flash-based appliance earlier than it originally expected.
- Indeed, Netezza has suspended — by which I mean “scrapped” — prior plans for a RAM-heavy disk-based appliance. It will use a RAM/flash combo instead.*
- Tim Vincent of IBM told me that customers seem ready to adopt solid-state memory. One interesting comment he made is that Flash isn’t really all that much more expensive than high-end storage area networks.
Uptake of solid-state memory (i.e. flash) for analytic database processing will probably stay pretty low in 2010, but in 2011 it should be a notable (b)leading-edge technology, and it should get mainstreamed pretty quickly after that.
*So far as I can tell, that’s one of the two significant roadmap changes between the 2009 and 2010 editions of Enzee Universe. The other one is that the robust form of appliance-to-appliance replication technology is coming out later than Netezza had originally planned and hoped.
There also is increasing reason to think that the issues with flash memory wearing out are overwrought. And by the way, the entire history of enterprise solid-state memory use is basically shorter than the time in which these products supposedly will wear out, so it’s not as if there have been a lot of real-life failures out there.
- First, clever things are being done in the area of error correction codes, although for the most part I defer that part of the discussion to Petascan’s Camuel Gilyadov. 🙂 E.g., this seems to be the idea behind Anobit.
- Second, analytic DBMS are pretty much an ideal use case for flash reliability. Suppose, as is the case for many products and implementations, you only write things in big blocks. Then you are, ipso facto, resetting the flash bits only in big blocks. Thus, at least in theory, you automatically have pretty perfect wear leveling.
Comments
4 Responses to “Flash is coming, well …”
Leave a Reply
analytic DBMS are pretty much an ideal use case for Flash reliability
Absolutely. Except that flash allows fast random read access which most analytic DBMS don’t leverage.
Well, the way I phrased it is correct. 🙂
But yes — a decade of performance innovation revolving around sequential-vs.-random I/O will soon be a lot less important.
“Soon” is the real question…I think it may be a few years yet.
per RAID1 LUN:
10k drives: ~115MB/s
15k drives: ~140MB/s
You don’t need to stack many spindles to saturate one of the many bottlenecks in play for a scan based stack: PCI,CPU,Storage Processor, Database engine etc.
At the same time, you have to stack a lot more spindles to achieve peak IOPs rate. SSD’s deliver far more *realizable” IOP’s than they do scan rate as other components in the stack bottleneck scan throughput at much lower values (in the form factors SSD throughput can currently be delivered in).
It’s not that hard to build a model that shows where the price point has to move to make SSD price competitive for scan based systems. Looks to be a ways out yet to me.
With that said, great potential for targeted operations (like temp) that inherently induce random IO. Tiered storage looks ideal in scan based workloads.
[…] been getting some DB2 briefings, which is why I’ve blogged about some specialized technical points from same. But I can’t yet say why the theoretically great-sounding data warehousing […]