January 22nd, 2008 Curt Monash
For very high-end applications, the list of viable database management systems is short. Scalability can be a problem. (The rankings of most scalable alternatives differ in the OLTP and data warehouse realms.) Extreme levels of security can be had from only a few DBMS. (Oracle would have you believe there’s only one choice.) And if you truly need 99.99% uptime, there only are a few DBMS you even should consider.
But for most applications at any enterprise – and for all applications at most enterprises – super high-end DBMS aren’t required. There are relatively few applications that wouldn’t run perfectly well on PostgreSQL or EnterpriseDB today. Ingres and Progress OpenEdge aren’t far behind (they’re a little lacking in datatype support). Ditto Intersystems Cache’, although the nonrelational architecture will be off-putting to many. And to varying degrees, you can also do fine with MySQL, Pervasive PSQL, MaxDB, or a variety of other products – or for that matter with the cheap or free crippled versions of Oracle, SQL Server, DB2, and Informix.
What’s more, these mid-range database management systems can have significant advantages over their high-end brethren. Read the rest of this entry »
Posted in EnterpriseDB and Postgres Plus, IBM and DB2, Ingres, Intersystems and Cache', Microsoft and SQL*Server, Mid-range DBMS, MySQL, Open source RDBMS, Oracle, Pervasive Software, PostgreSQL, Progress, Apama, and DataDirect, Relational database management systems, SAP, BI Accelerator, and MaxDB | 14 Comments »
January 16th, 2008 Curt Monash
As previously noted, I’ve been writing about an Oracle/BEA merger since 2002. So like many observers, I find I have little more to say on the subject. Let’s go straight to the bullet points: Read the rest of this entry »
Posted in HP and Neoview, IBM and DB2, Oracle, Oracle TimesTen, SAP, BI Accelerator, and MaxDB | 1 Comment »
January 14th, 2008 Curt Monash
I’m getting a flood of press releases today, because many of the companies I write about were selected to Intelligent Enterprise’s list of 12 most influential vendors plus 36 more to watch in the areas Intelligent Enterprise covers (which seems to be pretty much the analytics-related parts of what I write about here and on Text Technologies). It looks like a pretty reasonable list, although I think they forced the issue in some of the small analytics vendors they selected, and of course anybody can quibble with some of the omissions.
Among the companies they cited, you can find topical categories here for IBM (and Cognos), Informatica, Microsoft, Netezza, Oracle, SAP/Business Objects (both), SAS, and Teradata; QlikTech; Cast Iron, Coral8, DATAllegro, HP, ParAccel, and StreamBase; and Software AG. On Text Technologies you’ll find categories for some of the same vendors, plus Attensity, Clarabridge, and Google. There also are categories for some of these vendors on the Monash Report.
Posted in Business Objects, Cast Iron Systems, Coral8, DATAllegro, HP and Neoview, IBM and DB2, Informatica, Microsoft and SQL*Server, Netezza, Oracle, ParAccel, QlikTech and QlikView, SAP, BI Accelerator, and MaxDB, SAS Institute, Software AG and ADABAS, StreamBase, Teradata | No Comments »
November 12th, 2007 Curt Monash
Analyst conference calls about merger announcements are generally pretty boring. Indeed, the companies involved tend to feel they are legally barred from saying anything interesting, by mandate of both the antitrust regulators and the SEC.
Still, such calls are joyful events, full of strategic happy talk. If one is really lucky, there may a virtuouso tap dancing exhibition as well. On today’s IBM/Cognos call, Cognos CEO Rob Ashe was asked whether he thought Cognos’ independence or lack thereof was as important today as he said it was after SAP announced its BOBJ takeover. Without missing a beat, he responded that there were two kinds of openness:
- Database openness (not important)
- ERP/business process openness (indeed important)
Hmm. I’m not so sure I agree. To begin with, there aren’t just two major points of potential integration. There’s also a whole lot of middleware: obviously data integration, but also app servers, portals, and query execution acceleration as well. Read the rest of this entry »
Posted in Analytics and analytic technologies, Business Objects, Business intelligence, Cognos and Applix TM1, IBM and DB2, Memory-centric data management, ParAccel, SAP, BI Accelerator, and MaxDB | No Comments »
October 12th, 2007 Curt Monash
In the past month or so, both Dennis Moore and Nimish Mehta have left SAP. Their reasons are well-known among Oracle alumni to be — at least in large part — discomfort with SAP’s direction. (My unnamed sources on that are highly reliable.) And of course Shai Agassi left earlier this year. It now looks as if my contrarian viewpoint pooh-poohing the importance of Shai’s departure was probably wrong.
Based on all that, I don’t think there’s much reason for optimism about SAP’s system software futures, except perhaps for those that are placed wholly under the control of the Business Objects division. NetWeaver? Already a creaking omnibus. MaxDB? They didn’t get it right the first time around; what will be different now? BI Accelerator? That one actually could do well under Business Objects. The dream of other kinds of appliances? Not likely to achieve take-off. TREX? They weren’t really enhancing that much anyway. The rest of the search-related vision Dennis outlined for me? That’s another one that actually could thrive under Business Objects, but I expect a considerable number of false starts at best before they work out a coherent new strategy.
The high-end app business, the new SaaS business, the new Business Objects subsidiary — any and all of those could do well. But the attempts to become a broad-based system software player rivaling Oracle, Microsoft, and/or IBM are looking a lot less healthy than they used to.
Keep getting great research about enterprise applications, analytics and related technologies. Get a FREE subscription by RSS or email!
Technorati Tags: SAP, NetWeaver, Business Objects, TREX, BI Accelerator
Posted in Business Objects, Business intelligence, SAP, BI Accelerator, and MaxDB | No Comments »
October 12th, 2007 Curt Monash
Jeff Nolan has a great post on the Oracle/BEA deal. Yeah, he still has some of his old SAP good/Oracle evil reflexes, but he can be forgiven those and the tinfoilhattishness associated with them. His analysis of sellers’ and buyers’ deal habits is revealing and sound. Ditto the start of his remarks on Oracle product delays and internal politics, and SAP/Oracle competition. Even better, nothing in his analysis seems to disagree with mine. 
What Oracle now needs to do is make Oracle Application Server be a seamless “upgrade” from Weblogic. Then they can integrate in whatever kitchen-sink stuff they want from Oracle data caching (already there), app and/or dev tool run times, TimesTen, Tangosol, and so on, creating an app server stack that’s a worthy counterpart to the Oracle database in how it meets high-end OLTP needs. Meanwhile, Weblogic should remain as a not-bloated app-server-for-the-rest-of-us. Read the rest of this entry »
Posted in Oracle, SAP, BI Accelerator, and MaxDB | No Comments »
October 8th, 2007 Curt Monash
SAP is acquiring Business Objects. There’s nothing inherent in BI Accelerator’s design that ties it to NetWeaver, SAP star schema InfoCubes, or any other particular current implementation detail. So BI Accelerator could become a lot more than an afterthought.
Combine that with Cognos’s acquisition of Applix and the continued success of upstart QlikView, and we could finally see a general memory-centric BI boom.
Maybe. There have been a lot of false alarms before.
Technorati Tags: Business intelligence, BI, QlikView, Applix
Posted in Analytics and analytic technologies, Business Objects, Business intelligence, Cognos and Applix TM1, Memory-centric data management, QlikTech and QlikView, SAP, BI Accelerator, and MaxDB | 2 Comments »
October 4th, 2007 Curt Monash
Way back in January, 2006, I wrote that MaxDB was not getting merged into MySQL. Given that, it makes sense for SAP to take back control of the product. As The Reg reports, that’s exactly what’s happening.
The bigger question is — how’s MySQL’s SAP certification coming along? Whether or not MySQL gets SAP-certified and included in the SAP product catalog will be a huge indicator of whether it’s ready for OLTP prime time.
Anybody want to place bets on which midrange OLTP DBMS gets certified for SAP first, MySQL or EnterpriseDB? MySQL has a large head start, but if my friends and clients and EnterpriseDB have their priorities straight, they might wind up lapping MySQL even so.
Keep getting great research about database management and related technologies. Get a FREE subscription by RSS/Atom or e-mail!
Technorati Tags: MySQL, MaxDB, SAP
Posted in EnterpriseDB and Postgres Plus, Mid-range DBMS, MySQL, OLTP database management, SAP, BI Accelerator, and MaxDB | 4 Comments »
September 25th, 2007 Curt Monash
I’ve written extensively in the past about the differences between Oracle and SAP’s technical paradigms. (In a nutshell, Oracle is first and foremost about data, while SAP is about business process.) Last week, the respective companies’ CEOs outlined very different business strategies as well. Specifically, SAP’s Henning Kagermann called SAP’s new ByDemand SaaS offering “most important announcement I’ve made in my career,” while Oracle’s Larry Ellison outlined a continued high-end strategy as follows (excerpted from Oracle’s September 20 conference call transcript):
Our strategy for growth is to find a way to add more value to the same customers we already serve, which are the large end of the mid-market and large companies. What we’re doing here is moving beyond ERP to industry specific software. So in the telecommunications industry that would be billing systems and network provisioning systems and network inventory systems; core applications to run their business, to run telco. Core applications to run a bank. Core applications to run a retail chain of stores. Core applications to run a utility. That’s our focus, and that allows us to leverage the existing relationships that we have because we already sell databases to these companies, we sell middleware to these companies. We sell ERP and CRM to these companies, and now we want to sell this industry-specific software.
Now, when a CEO says that something is a company’s “most important announcement ever,” it’s time to check your hyperbole meter. (E.g., I recall Larry saying that about, of all things, a release of Oracle’s application development tools.) Still, there are at least three strong reasons to take last week’s statements more or less seriously: Read the rest of this entry »
Posted in Oracle, SAP, BI Accelerator, and MaxDB | 1 Comment »
March 24th, 2007 Curt Monash
I’ve recently made a lot of posts about database compression. 3X or more compression is rapidly becoming standard; 5X+ is coming soon as processor power increases; 10X or more is not unrealistic. True, this applies mainly to data warehouses, but that’s where the big database growth is happening. And new kinds of data — geospatial, telemetry, document, video, whatever — are highly compressible as well.
This trend suggests a few interesting possibilities for hardware, semiconductors, and storage.
- The growth in demand for storage might actually slow. That said, I frankly think it’s more likely that Parkinson’s Law of Data will continue to hold: Data expands to fill the space available. E.g., video and other media have near-infinite potential to consume storage; it’s just a question of resolution and fidelity.
- Solid-state (aka semiconductor or flash) persistent storage might become practical sooner than we think. If you really can fit a terabyte of data onto 100 gigs of flash, that’s a pretty affordable alternative. And by the way — if that happens, a lot of what I’ve been saying about random vs. sequential reads might be irrelevant.
- Similarly, memory-centric data management is more affordable when compression is aggressive. That’s a key point of schemes such as SAP’s or QlikTech’s. Who needs flash? Just put it in RAM, persisting it to disk just for backup.
- There’s a use for faster processors. Compression isn’t free. What you save on disk space and I/O you pay for at the CPU level. Those 5X+ compression levels do depend on faster processors, at least for the row store vendors.
Keep getting great research about database management, analytics, and related technologies. No hassle, no spam!
Technorati Tags: relational databases, storage, processors, memory, flash memory, compression
Posted in Data warehousing, Database compression, Memory-centric data management, QlikTech and QlikView, SAP, BI Accelerator, and MaxDB | 6 Comments »
March 16th, 2007 Curt Monash
IBM sent over a bunch of success stories recently, with DB2’s new aggressive compression prominently mentioned. Mike Stonebraker made a big point of Vertica’s compression when last we talked; other column-oriented data warehouse/mart software vendors (e.g. Kognitio, SAP, Sybase) get strong compression benefits as well. Other data warehouse/mart specialists are doing a lot with compression too, although some of that is governed by please-don’t-say-anything-good-about-us NDA agreements.
Compression is important for at least three reasons:
- It saves disk space, which is a major cost issue in data warehousing.
- It saves I/O, which is the major performance issue in data warehousing.
- In well-designed systems, it can actually make on-chip execution faster, because the gains in memory speed and movement can exceed the cost of actually packing/unpacking the data. (Or so I’m told; I haven’t aggressively investigated that claim.)
When evaluating data warehouse/mart software, take a look at the vendor’s compression story. It’s important stuff.
EDIT: DATAllegro claims in a note to me that they get 3-4x storage savings via compression. They also make the observation that fewer disks ==> fewer disk failures, and spin that — as it were — into a claim of greater reliability.
Posted in DATAllegro, Data warehouse appliances, Data warehousing, Database compression, IBM and DB2, Relational database management systems, SAP, BI Accelerator, and MaxDB, Vertica Systems | 2 Comments »
February 13th, 2007 Curt Monash
QlikTech has a pretty interesting story, and a number of customers seem to agree. Their flagship product QlikView is a BI suite that runs off an in-memory copy of the data. Specifically, that copy is logically relational and physically columnar. In an important feature, QlikView is happy to import data from multiple sources at once, such as a warehouse plus an operational data store.
So the QlikTech pitch is essentially “Buy our stuff, and you can start doing BI immediately, running any queries and reports you want to. No reason to limit your queries to any kind of dimensional model. No need to prepare the data.” More precisely, QlikTech claims to do away with some kinds of data preparation; obviously, cleaning and so on might still be necessary. Indeed, they describe their classic use case as being the combination of data partly from an operational store and partly from a pre-existing warehouse.
Read the rest of this entry »
Posted in Business intelligence, Memory-centric data management, QlikTech and QlikView, SAP, BI Accelerator, and MaxDB | 1 Comment »
January 22nd, 2007 Curt Monash
The best known columnar RDBMS is surely Sybase’s IQ Accelerator, evolved from a product acquired in the mid-1990s. Problem – it doesn’t have a shared-nothing architecture of the sort needed to exploit grid/blade technology. Whoops. The other recognized player is SAND, but I don’t know a lot about them. Based on their website, it would seem that grids and compression play a big part in their story. Less established but pretty interesting is Kognitio, who are just beginning to make marketing noise outside the UK. SAP’s BI Accelerator is also a compressed columnar system, but operates entirely in-memory and hence is limited in possible database size. Mike Stonebraker’s startup Vertica is of course the new kid on the block, and there are other columnar startups as well whose names currently escape me.
Read the rest of this entry »
Posted in Data warehousing, Kognitio and WX2, Relational database management systems, SAP, BI Accelerator, and MaxDB, TransRelational | 2 Comments »
September 22nd, 2006 Curt Monash
The last person I spoke with at the Netezza conference on Tuesday was a customer/presenter that the company had picked out for me. One thing he said baffled me — he claimed that Netezza was a real appliance vendor, but DATallegro wasn’t, presumably due to administrability issues. Now, it wasn’t clear to me that he’d ever evaluated DATallegro, so I didn’t take this too seriously, but still the exchange brought into focus the great differences between data warehouse products in the area of administration. For example:
- Netezza has no indices at all. And no caches. And the hardware is preconfigured. This all makes administration pretty simple.
- DATallegro has almost no indices, and also has preconfigured hardware. But it has some partitioning, optionally.
- Teradata also has preconfigured hardware. It does have indices, but rather simple ones. Plus it has join indices. And it has a few more configuration options in other areas (e.g., block size) than the other appliance vendors. (Yes, I count Teradata among the appliances.)
- If you go through all the fuss of installing SAP’s applications and BI technology anyway, the incremental administration of just SAP BI Accelerator is pretty light.
- Oracle and IBM have mammothly complex indexing options, but have put large amounts of work into tools to lessen the resulting administrative burden.
- IBM offers preconfigured hardware units to simplify some installation issues.
- Come to think of it, I don’t really know how hard it is to administer columnar systems (e.g., Sybase IQ).
Posted in DATAllegro, Data warehouse appliances, Data warehousing, Greenplum, IBM and DB2, Netezza, Oracle, Relational database management systems, SAP, BI Accelerator, and MaxDB, Teradata | 2 Comments »
September 20th, 2006 Curt Monash
I wrote about SAP’s BI Accelerator quite a bit in my white paper on memory-centric data management, but otherwise I seem not to have posted much about it here. In essence, it’s a product that’s all RAM-based, and generally geared for multi-hundred-gigabyte data marts. The basic design is a compression-heavy column-based architecture, evolved from SAP’s text-indexing technology TREX. Like data warehouse appliances, it eschews indexing, relying instead on blazingly fast table scans.
I asked Lothar Schubert of SAP how BIA was doing in the market in its early going. This was his response:
Read the rest of this entry »
Posted in Analytics and analytic technologies, Business intelligence, Data warehouse appliances, Data warehousing, Database compression, Memory-centric data management, Relational database management systems, SAP, BI Accelerator, and MaxDB | 5 Comments »
September 19th, 2006 Curt Monash
A lot of evidence is pointing to a major paradigm shift in data warehouse RDBMS, along the lines of:
Old way: Assume I/O is random; lower total execution time by improving selectivity and thus lowering the amount of I/O.
New way: Drive the amount of random I/O to near zero, and do as much sequential I/O as necessary to achieve this goal.
Examples include:
Read the rest of this entry »
Posted in DATAllegro, Data warehouse appliances, Database theory and practice, Memory-centric data management, Relational database management systems, SAP, BI Accelerator, and MaxDB, TransRelational | 3 Comments »
August 10th, 2006 Curt Monash
QlikTech — the vendor of QlikView — contacted me to tell their memory-centric BI story. A Swedish company with >$23 million in estimated license revenue last year and a 100%ish growth rate, they claim to be the leader in that space, pulling ahead of Applix. But for now, I’ll call them “a” leader, and say that their story sounds like a hybrid between those of Applix (TM1 product) and SAP (BI Accelerator).
Read the rest of this entry »
Posted in Business intelligence, Cognos and Applix TM1, Memory-centric data management, QlikTech and QlikView, SAP, BI Accelerator, and MaxDB | 1 Comment »
August 8th, 2006 Curt Monash
An eWeek article suggests that ANTs is repositioning with a strong emphasis on memory-centricity. ANTs’ website, frankly, doesn’t support this theory, giving a more balanced tech overview in line with how they pitched me in a briefing last November. Still, it’s an interesting possibility to watch.
The main focus of the article actually wasn’t ANTs, but rather SAP’s wildest dreams in expanding the scope of its BI Accelerator technology. But the new-to-me part was the positioning of ANTs.
Posted in ANTs Software, Memory-centric data management, Relational database management systems, SAP, BI Accelerator, and MaxDB | 2 Comments »
May 22nd, 2006 Curt Monash
If we define a “data warehouse appliance” as “a special-purpose computer system, with appliance administratibility, that manages a data warehouse,” then there are two major contenders: Netezza and DATAllegro, both startups, both with a small number of disclosed customers. Past contenders would include Teradata and White Cross (which seems to have just merged into Kognitio), but neither would admit to being in that market today. (I suspect this is a mistake on Teradata’s part, but so be it.) IBM with DB2 on the z-Series wouldn’t be properly regarded as an appliance player either, although IBM is certainly conscious of appliance competition. And SAP’s BI Accelerator does not persist data at this time.
In principle, the Netezza and DATAllegro stories are similar — take an established open source RDBMS*, build optimized hardware to run it, and optimize the software configuration as well. Much of the optimization is focused on getting data on and off disk sequentially, minimizing any random accesses. This is why I often refer to data warehouse appliances as being the best alternative to memory-centric data management. Beyond that, the optimizations by the two vendors differ considerably.
*Netezza uses PostgreSQL; DATAllegro uses Ingres.
Hmm. I don’t feel like writing more on this subject at this very moment, yet I want to post something urgently because there’s an IOU in my Computerworld column today for it. OK. More later.
Posted in DATAllegro, Data warehouse appliances, IBM and DB2, Ingres, Memory-centric data management, Open source RDBMS, Products and vendors, Relational database management systems, SAP, BI Accelerator, and MaxDB | No Comments »
May 8th, 2006 Curt Monash
I have finally finished and uploaded the long-awaited white paper on memory-centric data management.
This is the project for which I origially coined the term “memory-centric data management,” after realizing that the prevalent “in-memory DBMS” creates all sorts of confusion about how and whether data persists on disk. The white paper clarifies and updates points I have been making about memory-centric data management since last summer. Sponsors included:
- Applix, vendors of in-memory/memory-centric MOLAP tool TM1
- Progress Software, vendors of ObjectStore, an OODBMS that has more impressive references in-memory or otherwise memory-centric than it does in classical disk-based configurations, and also of the Apama stream processing products
- SAP, vendors of the BI Accelerator functionality of SAP NetWeaver, or whatever tortured name they want to give it this month — basically, that’s a very cool in-memory columnar data mart technology
- Solid Information Technology, vendor of hybrid in-memory/disk-based OLTP RDBMS. Historically focused on the embedded systems market, especially telecom and networking, they’ve recently been in the news because of a deal with MySQL that is designed to extend their reach.
- Intel, makers of the processors used to run a lot of the other sponsors’ products (including all BI Accelerator installations to date).
If there’s one area in my research I’m not 100% satisfied with, it may be the question of where the true hardware bottlenecks to memory-centric data management lie (it’s obvious that the bottleneck to disk-centric data management is random disk access). Is it processor interconnect (around 1 GB/sec)? Is it processor-to-cache connections (around 5 GB/sec)? My prior pronouncements, the main body of the white paper, and the Intel Q&A appendix to the white paper may actually have slightly different spins on these points.
And by the way — the current hard limit on RAM/board isn’t 2^64 bytes, but a “mere” 2^40. But don’t worry; it will be up to 2^48 long before anybody actually puts 256 gigabytes under the control of a single processor.
Posted in Cognos and Applix TM1, Intel, MOLAP, Memory-centric data management, Open source RDBMS, Products and vendors, Progress, Apama, and DataDirect, Relational database management systems, SAP, BI Accelerator, and MaxDB, solidDB | 1 Comment »
April 8th, 2006 Curt Monash
Dan Farber’s blog from SAP’s developer conference isn’t, frankly, his best piece of work, since the quotes are sometimes so garbled as to be a bit unreadable. Still, it helps flesh out what we already knew about SAP’s strategy.
Basically, they claim to be reengineering their whole product line for the new services-based architcture I keep writing about. And they insist this truly is a new platform architecture. In that regard, I buy into and agree with their pitch.
They further insist that the mid-market will be a big part of their business going forward, but SaasS will not. I don’t buy into that as fully.
I’ll spell out why in another post, but not until Monday at the earliest. Watch the comments section on this one for trackbacks.
Posted in Database theory and practice, SAP, BI Accelerator, and MaxDB | No Comments »
January 26th, 2006 Curt Monash
I’ve had a chance to clarify and correct my understanding of the relationship between SAP, MaxDB, and MySQL. The story is this:
- MySQL has the right to sell MaxDB, but apparently isn’t focusing much on that.
- The MySQL and MaxDB code lines are NOT merging, for technical reasons. For example, the older MaxDB does a lot of its own thread management, while MySQL relies on the operating system for that.
- When SAP thinks a DBMS is capable of running SAP’s apps, it adds the DBMS to its product catalog and resells it. Yes, even Oracle. That’s why all my discussions with SAP of MySQL’s enterprise-readiness quickly come back to an exhaustive multi-year certification process.
- My personal best guess as to when MySQL will be in SAP’s product catalog is 1 1/2 - 3 years from now.
And by the way, MaxDB’s share in SAP’s user base is about the same as DB2’s (at least DB2 for open systems). MaxDB is being aggressively supported, and nobody should get any ideas to the contrary!
Posted in IBM and DB2, MySQL, Open source RDBMS, Oracle, SAP, BI Accelerator, and MaxDB | 4 Comments »
December 9th, 2005 Curt Monash
I just spent a couple of days at SAP’s analyst meeting, and realized something I’d somewhat forgotten – much of the DBMS2 concept was inspired by SAP’s technical strategy. That’s not to say that SAP’s techies necessarily agree with me on every last point. But I do think it is interesting to review SAP’s version of DBMS2, to the extent I understand it.
1. SAP’s Enterprise Services Architecture (ESA) is meant to be, among other things, an abstraction layer over relational DBMS. The mantra is that they’re moving to a “message-based architecture” as opposed to a “database architecture.” These messages are in the context of a standards-based SOA, with a strong commitment to remaining open and standards-based, at least on the data and messaging levels. (The main limitation on openness that I’ve detected is that they don’t think much of standards such as BPEL in the business process definition area, which aren’t powerful enough for them.)
2. One big benefit they see to this strategy is that it reduces the need to have grand integrated databases. If one application manages data for an entity that is also important to another application, the two applications can exchange messages about the entity. Anyhow, many of their comments make it clear that, between partner company databases (a bit of a future) and legacy app databases (a very big factor in the present day), SAP is constantly aware of situations in which a single integrated database in infeasible.
3. SAP is still deeply suspicious of redundant transactional data. They feel that with redundant data you can’t have a really clean model – unless, of course, you code up really rigorous synchronization. However, if for some reason synchronization is preferred – e.g., for performance reasons — it can be hidden from users and most developers.
4. One area where SAP definitely favors redundancy and synchronization is data warehousing. Indeed, they have an ever more elaborate staging system to move data from operational to analytic systems.
5. In general, they are far from being relational purists. For example, Shai Agassi referred to doing things that you can’t do in a pure relational approach. And Peter Zencke reminded me that this attitude is nothing new. SAP has long had complex business objects, and even done some of its own memory management to make them performant, when they were structured in a manner that RDBMS weren’t well suited for. (I presume he was referring largely to BAPI.)
6. That said, they’re of course using relational data stores today for most things. One exception is text/content, which they prefer to store in their own text indexing/management system TREX. Another example is their historical support for MOLAP, although they seem to be edging as far away from that as they can without offending the MOLAP-loving part of their customer base.
Incidentally, the whole TREX strategy is subject to considerable doubt too. It’s not a state-of-the-art product, and they currently don’t plan to make it into one. In particular, they have a prejudice against semi-automated ontology creation, and that has clearly become a requirement for top-tier text technologies.
7. One thing that Peter said which confused me a bit is when we were talking about nonrelational data retrieval. The example he used was retrieving information on all of a specific sales reps’ customers, or perhaps on several sales reps’ customers. I got the feeling he was talking about the ability to text search on multiple columns and/or multiple tables/objects/whatever at once, but I can’t honestly claim that I connected all the dots.
And of course, the memory-centric ROLAP tool BI Accelerator — technology that’s based on TREX — is just another example of how SAP is willing to go beyond passively connecting to a single RDBMS. And while their sponsorship of MaxDB isn’t really an example of that, it is another example of how SAP’s strategy is not one to gladden the hearts of the top-tier DBMS vendors.
Posted in Database theory and practice, EII, ETL, and/or EAI, Hierarchies, networks, graphs, and trees, MOLAP, OLTP database management, Relational database management systems, SAP, BI Accelerator, and MaxDB | 7 Comments »
November 14th, 2005 Curt Monash
I’m writing more and more about memory-centric data management technology these days, including in my latest Computerworld column. You may be wondering what that term refers to. Well, I’ve basically renamed what are commonly called “in-memory DBMS,” for what I think is a very good reason: Most of the products in the category aren’t true DBMS, aren’t wholly in-memory, or both! Indeed, if you catch me in a grouchy mood I might argue that “in-memory DBMS” is actually a contradiction in terms.
I’ll give a quick summary of the vendors and products I am focusing on in this newly-named category, and it should be clearer what I mean:
- TimesTen (now owned by Oracle): TimesTen is the quintessentional “in-memory DBMS.” It’s a fairly full relational DBMS, but if you want to persist memory to disk it has to be handed off to a conventional DBMS. Historically, that has usually been MySQL or Oracle. TimesTen’s biggest market penetration has been in financial trading.
- Solid Information Technology’s BoostEngine: Solid is a Finnish company (or was — it’s pretty American now) specializing in embedded DBMS sold mainly for telecommunication uses. Big OEM customers include several well-known telecom equipment manufacturers and HP (for OpenView). “Embedded” often means no DBA, no monitor, no keyboard — they box manufacturer installs it and there it stays for the life of the product. Solid has to offer strong replication capabilities, since its products are often used in highly distributed (e.g., multiblade, multibox) environments. So it’s taken the next step and exploited the replication by allowing customers to use some instances of the product disklessly.
- Event-stream products from Streambase and Progress: The canonical application for event-stream products is automating financial trading decisions based on the flow of market information. Mike Stonebraker, the brains behind Streambase, has recently popularized the idea; Progress bought Apama, who actually have been in the business longer. These applications require even more speed than the financial trading apps that TimesTen handles, and they discard most of the information they look at. In-memory is the only way to go.
- Progress’s ObjectStore: ObjectStore comes from the company Object Design, which merged into Excelon, which was acquired by Progress. It’s really a toolkit for building DBMS and similar systems, which is why it’s at various times been marketed as an OODBMS and an XML DBMS, without a lot of success either way. But there have been a few sterling apps built in ObjectStore even so, including a key part of the Amazon bookstore Despite this limited market success, a significant fraction of Progress’s best engineering talent has moved over to the Real-Time Division to focus on ObjectStore and other memory-centric products. The memory-centric aspect of ObjectStore is this: ObjectStore’s big virtue is that it gets objects from disk to memory and vice-versa very efficiently, then distributes and caches them around a network as needed. This was originally invented for client/server processing, but works fine in a multi-server thin client setup as well. And object processing, of course, relies on a whole lot of pointers. And pointer-chasing is pretty much the worst way to deal with the disk speed barrier, unless you do it in main memory.
- Applix’s TM1: Like many companies in the analytics area, Applix has had trouble deciding whether it sells applications, BI system software, or both. But in any case its core technology is TM1, a memory-centric MOLAP offering. Traditional MOLAP products reside on the horns of a nasty dilemma: They rely on precalculation to give good performance, but that causes ghastly database explosion. Applix gets out of this problem by doing no precalculation whatsoever, loading the data into main memory, and executing all queries on the fly.
- SAP’s BI Accelerator: SAP is building out an elaborate technology stack with NetWeaver, especially in the BI area. One important aspect is that the full data warehouse is logically broken (or copied) into a series of data marts called “InfoCubes.” BI Accelerator takes the logical next step, loading an entire InfoCube into main memory. Almost every query is executed via a full table scan, which would be insane on disk but makes perfect sense when the data is already in RAM.
So there you have it. There are a whole lot of technologies out there that manage data in RAM, in ways that would make little or no sense if disks were more intimately involved. Conventional DBMS also try to exploit RAM and limit disk access, via caching; but generally the data access methods they use in RAM are pretty similar to those they use when going out to disk. So memory-centric systems can have a major advantage.
Posted in Cognos and Applix TM1, Complex event/stream processing (CEP), Data types, MOLAP, Memory-centric data management, OLTP database management, Objects, Oracle TimesTen, Progress, Apama, and DataDirect, SAP, BI Accelerator, and MaxDB, solidDB | 2 Comments »
August 8th, 2005 Curt Monash
MySQL is like a star high school athlete — impressive skills and potential, but it still only excels at a limited range of mainly simple things. Will it grow into a robust, adult star? I think so, and here’s a big part of the reason why: MaxDB and SAP certification.
MaxDB is a database product that bounced among all the major German computer hardware and software companies: Nixdorf, Siemens, Software AG, and SAP. (What little fame it ever had was primarily under the name Adabas-D.) SAP eventually shipped MaxDB as the underlying DBMS at many R3 installations. This is a huge sign of OLTP industrial-strengthness; if a DBMS can run SAP’s apps, it can run pretty much anything. OK, not necessarily retail banking, airline reservations, and so on — but pretty much anything else.
Well, two years ago MySQL (the company) and SAP agreed to what amounts to a slow-motion merge between MySQL (the product) and MaxDB. The resulting joint product (currently still quite separate from MySQL 5.0) is undergoing a multi-year process of achieving SAP certification. Everybody involved clearly expects this certification to eventually succeed — in 2-3 years, probably, or perhaps less if they were being really coy with me.
And when that happens, there will be a version of MySQL that is unquestionably fit for rigorous OLTP.
Technorati Tags: Database, DBMS, DBMS2, MySQL, SAP, Software
Posted in MySQL, Relational database management systems, SAP, BI Accelerator, and MaxDB | 4 Comments »