Analysis of hybrid memory-centric DBMS solidDB and its developer Solid Technology. Related subjects include:
Billy Newport of IBM sees a lot of similarities between his app-server-based product ObjectGrid and H-Store. In both cases, constrained tree schemas are assumed, and OLTP performance goodness ensues. A couple of points I noted on a quick skim through his blog:
- He calls out RAM consumption as a challenge for this kind of architecture.
- He points out that it’s a big advantage to have data called and used in the same address space.
Being based in RAM is obviously a huge part of the H-Store scheme. But so is having transaction execution be close to the database.
IBM now has both ObjectGrid and a memory-centric DBMS (solidDB) that they’ve been using as a front end for DBMS. Integration of the two could be pretty interesting.
|Categories: Cache, IBM and DB2, Memory-centric data management, OLTP, solidDB, Theory and architecture, VoltDB and H-Store||Leave a Comment|
IBM is acquiring Solid Information Technology, makers of solidDB. Some quick comments:
- solidDB is actually a very interesting hybrid disk/in-memory memory-centric database management system. However, the press release announcing the deal makes it sound as if solidDB is in-memory only.
- That strongly suggests that IBM is buying Solid mainly to compete with Oracle TimesTen. As of last June, solidDB was already IBM’s TimesTen answer via a partnership; this deal just solidifies that arrangement.
- This probably isn’t good news for Solid’s MySQL engine. That’s a pity, since solidDB technically has the potential to be the best MySQL engine around.
- Notwithstanding IBM’s presumed intentions, Solid’s main market success historically is as an embedded system in telecommunications equipment, network software, and similar systems.
- Last year I wrote a white paper on memory-centric data management, showcasing four products. IBM now has bought two of them, namely Solid’s and Applix’s (via Cognos).
- Comparisons to IBM’s embedded Java DBMS Cloudscape are pointless. That’s just a failed product vs. solidDB or Sybase SQL Anywhere, and IBM long ago cut its losses.
|Categories: Cache, Cognos, IBM and DB2, In-memory DBMS, Memory-centric data management, MySQL, OLTP, Oracle TimesTen, solidDB, Sybase||5 Comments|
I’m sorry for the short notice, but — well, never mind what the distractions have been. This Wednesday, at 2:00 pm Eastern time, I’m doing a webinar on behalf of Solid. The core subject is memory-centric OLTP data management. I will of course also cover some DBMS and memory-centric generalities.
More info and sign-up can be found here.
I had the chance to talk at length with Solid Information Technology tech guru Antoni Wolski about their memory-centric DBMS technology architecture. The most urgent topic was what made in-memory database managers inherently faster than disk-based ones that happened to have all the data in cache. But we didn’t really separate that subject from the general topic of how they made their memory-centric technology run fast, from its introduction in 2002 through substantial upgrades in the most recent release.
There were 4 main subtopics to the call:
1. Indexing structures that are very different from those of disk-based DBMS.
2. Optimizations to those indexing structures.
3. Optimizations to logging and checkpointing.
4. Miscellaneous architectural issues.
It’s just at the proof-of-concept stage, but Solid has a nice write-up about SolidDB being used as a front-end cache for DB2. Well, it’s a marketing document, so of course there’s a lot of pabulum too, but interspersed there’s some real meat as well. Highlights include 40X throughput improvement and 1 millisecond average response time (something that clearly can’t be achieved with disk-centric technology alone).
Analogies to Oracle/TimesTen are probably not coincidental; this is exactly the upside scenario for the TimesTen acquisition, as well as being TimesTen’s biggest growth area towards the end of its stint as an independent company.
|Categories: Cache, IBM and DB2, Memory-centric data management, OLTP, Oracle, Oracle TimesTen, solidDB||1 Comment|
MySQL 4.0 is an OLTP joke. MySQL 5.0, however, shows a lot of progress in terms of real transactions, foreign keys, referential integrity, triggers, stored procedures and so on. In anticipation of the MySQL user conference next week, I got a quick briefing from Paola Lubet and Murat Demiroglu at Solid Information Technology, whose SolidDB is one of the two transactional storage engines for MySQL (the other is InnoDB, now owned by Oracle).
The layer provided by MySQL actually does most of what I think of as “language processing” – parsing, optimization, drivers, triggers, stored procedures, referential integrity, etc. SolidDB is a storage engine providing actual execution. Its features and virtues include:
• Online backup. (Note: Apparently, the extra-cost InnoDB online backup product isn’t showing up on price lists these days.)
• Optimistic (as well as pessimistic) concurrency control. This can be a good performance feature for applications that have a whole lot of Adds and very few Changes.
• General reliability. Unless they really botched the port, Solid benefits from a long history of very reliable operation.
• High availability. Scheduled for alpha in early summer and beta in the fall is a high-availability option. This initial-release will be master-slave synchronous replication. More sophisticated replication could come later on, as could memory-centric performance, if market conditions seem to warrant it (I’m betting they will).
Edit: This post has largely been superseded by this more recent one defining mid-range relational DBMS.
I find myself defining a new product category – midrange OLTP/multipurpose DBMS. (Or just midrange DBMS for brevity.) Nothing earthshaking here; I’m simply referring to those products that: Read more
|Categories: Actian and Ingres, EnterpriseDB and Postgres Plus, IBM and DB2, Intersystems and Cache', Microsoft and SQL*Server, Mid-range, MySQL, OLTP, Open source, Oracle, Progress, Apama, and DataDirect, solidDB, Sybase||8 Comments|
Oracle made a slick move in picking up Tangosol, a leader in object/data caching for all sorts of major OLTP apps. They do financial trading, telecom operations, big web sites (Fedex, Geico), and other good stuff. This is a reminder that the list of important memory-centric data handling technologies is getting fairly long, including:
- Object caching (e.g., Tangosol, Progress ObjectStore)
- In-memory RDBMS (e.g., Oracle TimesTen, Solid BoostEngine, McObject eXtremeDB)
- Stream processing (e.g., Progress Apama, Streambase)
And that’s just for OLTP; there’s a whole other set of memory-centric technologies for analytics as well.
When one connects the dots, I think three major points jump out:
- There’s a lot more to high-end OLTP than relational database management.
- Oracle is determined to be the leader in as many of those areas as possible.
- This all fits the market disruption narrative.
I write about Point #1 all the time. So this time around let me expand a little more on #2 and #3.
|Categories: Cache, Complex event processing (CEP), Database diversity, Memory-centric data management, MySQL, Netezza, OLTP, Oracle, Oracle TimesTen, Progress, Apama, and DataDirect, solidDB, StreamBase, Theory and architecture||3 Comments|
Most of what I’ve written lately about database management seems to have been focused on analytic technologies. But I have a lot to say on the OLTP (OnLine Transaction Processing) side too. So let’s start by clearing the decks. Here’s a list of some consensus views that I in essence agree with:
- Oracle is the top of the line, and has nothing wrong with it other than cost of ownership and the non-joys of doing business with Oracle Corporation.
- DB2/mainframe is a fine product, but only if you like IBM mainframes.
- DB2/open systems is another fine product, but it’s hard to think of reasons to use it over Oracle.
- Microsoft SQL Server has great cost of ownership if you’re a Windows (server) shop anyway, especially on the administrative side. It does most but not all of what Oracle does.
- Sybase Adaptive Server Enterprise is a lot like SQL Server, but without the Windows dependence or the great Microsoft tools. If you have it installed or are Chinese, you should strongly consider using it, but otherwise there are better alternatives.
- Progress’ DBMS is great if you don’t need any of the features it’s missing. Administration, for example, is a super-low-cost breeze. But why use it unless you’re also using the Progress development tools?
- Intersystems’ Cache’ is another fine mid-range product that involves buying into the vendors’ whole tool set – all the more so because it isn’t relational.
- Small-footprint embedded DBMS, from vendors such as Sybase’s iAnywhere division or Solid Information Technologies, are off in their own little world. Mainly, that world is telecom, with a satellite in medical devices, although other kinds of networked equipment also sometimes use these products.
- IBM’s non-DB2 database management products – IMS, Informix, etc. – are fine things to stick with until you have to change. Ditto products from Software AG, Computer Associates, Cincom, etc.
- MySQL Version 4 is an OLTP joke, but it’s a joke many people share. (Hey — a lot of blogs, including mine, run on WordPress and MySQL 4.)
- Until Ingres is meaningfully marketed and sold outside its installed base, it’s not worth worrying about.
- PostgreSQL is more significant as the underpinning of other products — mainly EnterpriseDB in the OLTP space — than it is in its own right.
I have finally finished and uploaded the long-awaited white paper on memory-centric data management.
This is the project for which I origially coined the term “memory-centric data management,” after realizing that the prevalent “in-memory DBMS” creates all sorts of confusion about how and whether data persists on disk. The white paper clarifies and updates points I have been making about memory-centric data management since last summer. Sponsors included:
- Applix, vendors of in-memory/memory-centric MOLAP tool TM1
- Progress Software, vendors of ObjectStore, an OODBMS that has more impressive references in-memory or otherwise memory-centric than it does in classical disk-based configurations, and also of the Apama stream processing products
- SAP, vendors of the BI Accelerator functionality of SAP NetWeaver, or whatever tortured name they want to give it this month — basically, that’s a very cool in-memory columnar data mart technology
- Solid Information Technology, vendor of hybrid in-memory/disk-based OLTP RDBMS. Historically focused on the embedded systems market, especially telecom and networking, they’ve recently been in the news because of a deal with MySQL that is designed to extend their reach.
- Intel, makers of the processors used to run a lot of the other sponsors’ products (including all BI Accelerator installations to date).
If there’s one area in my research I’m not 100% satisfied with, it may be the question of where the true hardware bottlenecks to memory-centric data management lie (it’s obvious that the bottleneck to disk-centric data management is random disk access). Is it processor interconnect (around 1 GB/sec)? Is it processor-to-cache connections (around 5 GB/sec)? My prior pronouncements, the main body of the white paper, and the Intel Q&A appendix to the white paper may actually have slightly different spins on these points.
And by the way — the current hard limit on RAM/board isn’t 2^64 bytes, but a “mere” 2^40. But don’t worry; it will be up to 2^48 long before anybody actually puts 256 gigabytes under the control of a single processor.
|Categories: Cognos, Companies and products, In-memory DBMS, Intel, Memory-centric data management, MOLAP, Open source, Progress, Apama, and DataDirect, SAP AG, solidDB||2 Comments|