I’m sorry for the short notice, but — well, never mind what the distractions have been. This Wednesday, at 2:00 pm Eastern time, I’m doing a webinar on behalf of Solid. The core subject is memory-centric OLTP data management. I will of course also cover some DBMS and memory-centric generalities.
More info and sign-up can be found here.
Technorati Tags: SolidDB, in-memory database
I had the chance to talk at length with Solid Information Technology tech guru Antoni Wolski about their memory-centric DBMS technology architecture. The most urgent topic was what made in-memory database managers inherently faster than disk-based ones that happened to have all the data in cache. But we didn’t really separate that subject from the general topic of how they made their memory-centric technology run fast, from its introduction in 2002 through substantial upgrades in the most recent release.
There were 4 main subtopics to the call:
1. Indexing structures that are very different from those of disk-based DBMS.
2. Optimizations to those indexing structures.
3. Optimizations to logging and checkpointing.
4. Miscellaneous architectural issues.
Read the rest of this entry »
It’s just at the proof-of-concept stage, but Solid has a nice write-up about SolidDB being used as a front-end cache for DB2. Well, it’s a marketing document, so of course there’s a lot of pabulum too, but interspersed there’s some real meat as well. Highlights include 40X throughput improvement and 1 millisecond average response time (something that clearly can’t be achieved with disk-centric technology alone).
Analogies to Oracle/TimesTen are probably not coincidental; this is exactly the upside scenario for the TimesTen acquisition, as well as being TimesTen’s biggest growth area towards the end of its stint as an independent company.
Technorati Tags: Solid Information Technology, solidDB, DB2, TimesTen
Mike Stonebraker wrote in with one “nit pick” about yesterday’s blog. I had credited Truviso for strong DBMS/stream processor integration. He shot back that StreamBase has Sleepycat integrated in-process. He further pointed out that a Sleepycat record lookup takes only 5 microseconds if the data is in cache. Assuming what he means is that it’s in Sleepycat’s cache, that would be tight integration indeed.
I wonder whether StreamBase will indefinitely rely on Sleepycat, which is of course now an Oracle product …
Want to continue getting great research about DBMS, analytics, data integration, and other technologies related to data management? Get a FREE subscription by RSS/Atom or e-mail! We recommend taking the integrated feed for all our blogs, but blog-specific ones are also easily available. Technorati Tags: StreamBase, Sleepycat
After my call with Truviso and blog post referencing same, I had the chance to discuss stream processing with Mike Stonebraker, who among his many distinctions is also StreamBase’s Founder/CTO. We focused almost exclusively on the financial trading market. Here are some of the highlights. Read the rest of this entry »
When Mike Stonebraker and I discussed RDF yesterday, he quickly turned to suggesting fast ways of implementing it over an RDBMS. Then, quite characteristically, he sent over a paper that allegedly covered them, but actually was about closely related schemes instead. 
Edit: The paper has a new, stable URL. Hat tip to Daniel Abadi.
All minor confusion aside, here’s the story. At its core, an RDF database is one huge three-column table storing subject-property-object triples. In the naive implementation, you then have to join this table to itself repeatedly. Materialized views are a good start, but they only take you so far. Read the rest of this entry »
Thus spake Mike Stonebraker to me, on a call we’d scheduled to talk about several other things altogether. This was one day after I was told at the Text Analytics Summit that the US government is going nuts for RDF. And I continue to get confirmation of something I first noted last year — Oracle is pushing RDF heavily, especially in the life sciences market.
Evidently, the RDF data model is for real … unless, of course, you’re the kind of purist who cares to dispute whether RDF is a true “data model” at all.
Technorati Tags: RDF, Semantic Web, database
I’ve been implying that the short list for native XML database engine vendors should be Mark Logic, IBM, and maybe Microsoft, on the theory that Progress and Intersystems tried the market and pulled back. Well, add Intersystems to the list, and not necessarily in last place. They’ve long had a very fast nonrelational engine in Cache’. Perhaps building Ensemble on it has induced them to sharpen up the XML capabilities again.
Anyhow, while I’m not at liberty to explain more of my reasoning (i.e., to disclose my evidence) — Cache’ should be taken seriously as an XML DBMS alternative … even if I never can seem to get a proper DBMS briefing from them (which is far from entirely being their fault).
Want to continue getting great research about DBMS, analytics, data integration, and other technologies related to data management? Then get a FREE subscription, by RSS/Atom or e-mail! We recommend taking the integrated feed for all our blogs, but blog-specific ones are also easily available.
Technorati Tags: XML database, Intersystems, Cache’
The word from Vertica is that the product will go GA in the fall, and that they’ll have blow-out benchmarks to exhibit.
I find this very credible. Indeed, the above may even be something of an understatement.
Vertica’s product surely has some drawbacks, which will become more apparent when the product is more available for examination. So I don’t expect row-based appliance innovators Netezza and DATAllegro to just dry up and blow away. On the other hand, not every data warehousing product is going to live long and prosper, and I’d rate Vertica’s chances higher than those of several competitors that are actually already in GA.
Want to continue getting great research about DBMS, analytics, data integration, and other technologies related to data management? Then get a FREE subscription, by RSS/Atom or e-mail! We recommend taking the integrated feed for all our blogs, but blog-specific ones are also easily available.
I’ve just started a research project on the IT-like technology of games and virtual worlds, especially MMORPGs. My three recent posts on Guild Wars attracted considerable attention in GW’s community, and elicited some interesting commentary, especially for the revelation of Guild Wars’ very simple database architecture. Specifically, pretty much all character information is banged into a BLOB or two, and stored as a string of tokens, with little of the record-level detail one might expect. By way of contrast, Everquest is run on Oracle (and being transitioned to EnterpriseDB), at least one console-based game maker uses StreamBase, and so on.
Much of the attention has focused on the implications for the in-game economy – how can players buy and sell to their hearts’ content if there’s no transactional back-end. Frankly, I think that’s the least of the issues. For one thing, without a nice forms-based UI you probably won’t create enough transactions to matter, and integrating that into the game client isn’t trivial. For another, virtual items can be literally created and destroyed by the computer, with no negative effect on game play, a factor which drastically reduces the integrity burdens the game otherwise would face.
Rather, where I think the Guild Wars developers at ArenaNet may be greatly missing out is in the areas of business intelligence, data mining, and associated game control. Here are some examples of analyses they surely would find it helpful to do. Read the rest of this entry »
ANTs Software’s primary focus isn’t really even on DBMS any more. Even so, it just announced a deal to replace Informix in a large retail chain’s in-store systems. (In its 1990s heyday, Informix wound up running in-store systems at an impressive list of major retailers. Of course, Informix was long ago acquired by IBM.)
EnterpriseDB has probably passed ANTs in the DBMS plug-compability business. And taken together they’re still pretty small. Even so, plug-compatible DBMS replacement has to be taken seriously as a (possibly) emerging trend. Economically, it makes all the sense in the world.
Want to continue getting great research about DBMS, analytics, data integration, and other technologies related to data management? Then get a FREE subscription, by RSS/Atom or e-mail! We recommend taking the integrated feed for all our blogs, but blog-specific ones are also easily available.
I have the enviable task of researching online game and virtual world technology for an upcoming Network World column. My first interview, quite naturally, was with the lead developers of a game I actually play – Guild Wars. The overview is in another post; that may provide context for this one, which focuses on the database technology. (I also did a short post just on the implications for Guild Wars players.) It also has a brief description of what Guild Wars is – namely, a MMORPG (Massively MultiPlayer Role-Playing Game) with the unusual feature that most of the game world is instanced rather than utterly shared.
First, some scope. ArenaNet (Guild Wars’ developer, now a subsidiary of NCsoft) runs Microsoft SQL Server, mainly Enterprise Edition, having just switched to 2005 4 months ago. They run 1500-2500 transactions/second all day, spiking up to 5000 in their busiest periods. They have no full-time DBA, and when the developers started this project they didn’t know SQL. They’ve only had one major SQL Server failure in the 2+ years the game has been running, and that was (like most of their bugs) a network driver problem more than an issue with the core system.
As for what’s going on — there are a few different kinds of database things that happen in an instanced MMORPG. Read the rest of this entry »
I was consulting recently to a client that needs to build really big relational data warehouses, and also is attracted to native XML. Naturally, I suggested they consider DB2. They immediately shot back that they were Linux-based, and didn’t think DB2 ran (or ran well) on Linux. Since IBM often leads with AIX-based offerings in its marketing and customer success stories, that wasn’t a ridiculous opinion. On the other hand, it also was very far from what I believed.
So I fired some questions at IBM,
I’ve been a DBMS industry analyst, in one guise or another, since 1981. So by now I’ve witnessed a whole lot of claims and debates about scalability. And there’s one observation I’d like to call out.
What matters most isn’t what kind of capacity or throughput you can get with heroic efforts. Rather, what matters most is the capacity and throughput you get without any kind of special programming or database administraton.
Of course, when taken to extremes, that point could become silly. DBMS are used by professionals, and requiring a bit of care and tuning is par for the course. But if you have a choice between two systems that can get the job done for you, one of which requires you to perform unnatural acts and one doesn’t – go for the one that works straightforwardly. Your overall costs will wind up being much lower, and you’ll probably get a lot more useful work done. A system that has to strain even to meet known requirements will probably fail altogether at meeting the as-yet-unknown ones that are sure to arise down the road.
Want to continue getting great research about DBMS, analytics, data integration, and other technologies related to data management? Get a FREE subscription by RSS/Atom or e-mail! We recommend taking the integrated feed for all our blogs, but blog-specific ones are also easily available.
StreamBase is a decently-established startup, possibly the largest company in its area. Truviso, in the process of changing its name from Amalgamated Insight, has a dozen employees, one referenceable customer, and a product not yet in general availability. Both have ambitious plans for conquering the world, based on similar stories. And the stories make a considerable amount of sense.
Both companies’ core product is a memory-centric SQL engine designed to execute queries without ever writing data to disk. Of course, they both have persistence stories too — Truviso by being tightly integrated into open-source PostgreSQL, StreamBase more via “yeah, we can hand the data off to a conventional DBMS.” But the basic idea is to route data through a whole lot of different in-memory filters, to see what queries it satisfies, rather than executing many queries in sequence against disk-based data. Read the rest of this entry »
Unfortunately, the first draft of this post got eaten. I’m now trying again.
In response to its small but vocal constituency, I got myself briefed on the FileMaker story. My conclusion, in a nutshell, is that FileMaker sometimes is a good alternative to low-end use of a standard relational DBMS. If you do feel able to use more standard-style products, you often should, for all sorts of obvious flexibility and future-proofing reasons. But if you can’t, or if you’re really confident the project won’t grow past a certain level, the FileMaker class of products can be a very appealing alternative.
Make no mistake; FileMaker is very different from conventional DBMS/app dev tool combos (and that’s the right comparison, as it combines aspects of both product categories into one). Read the rest of this entry »
You are currently browsing the DBMS2 — DataBase Management System Services weblog archives for June, 2007.