• In May I surveyed relational vs. non-relational pros and cons at some length.
• Last November I mused about when it might be OK to do without joins.
• The question is implicit in a variety of posts about, say, document-oriented or object-oriented DBMS.

Before going further in that vein, I’d like to do a quick review of what E. F. “Ted” Codd was getting at with the relational model in the first place. 

The first sentence of Codd’s famous 1970 paper introducing the relational database concept reads:

Future users of large data banks must be protected from having to know how the data is organized in the machine (the internal representation).

In modern terms, that means “all you have to know to use the database is its logical schema; you don’t need to know anything about its physical representation.”

Over the next 15 years, Codd’s thinking — and his employer IBM’s technology — evolved to the point that Codd proposed 12 rules for a relational DBMS, the three most fundamental of which are:

Foundation Rule
A relational database management system must manage its stored data using only its relational capabilities.

Information Rule
All information in the database should be represented in one and only one way — as values in a table.

Guaranteed Access Rule
Each and every datum (atomic value) is guaranteed to be logically accessible by resorting to a combination of table name, primary key value and column name.

I.e., Codd was positively asserting that a database should have a fixed logical schema, in a tabular form. The clear implication was that programmers could or should be able to write anything they wanted to against that schema, without database performance being unduly compromised.

Of course, things never quite worked out that way. For most of the history of tabular DBMS, the best-performing short-request and analytic DBMS have been designed quite differently from each other.* Non-relational systems — from IBM’s own IMS to various object-oriented DBMS — outperformed relational DBMS on particular applications. Designers of high-performance applications were sensitive to the database’s physical design, sometimes even going to the extreme of non-transparent sharding. But on the whole, it was generally agreed that programming against a fixed logical schema is a good thing.

*Codd acknowledged this himself by promoting multidimensional OLAP over traditional RDBMS. (I regard the multidimensional/relational divide to be a distinction without significant difference; it’s all just fixed-logical-schema tabular processing with different data manipulation languages.)

In my next post, I’ll return to the subject of why fixed schemas might not always be such a good idea after all.

Bit bucket

• Kinds of data likely to be included: Logs, other technical/external
• Likely use styles: Staging/ETL, investigative
• Canonical example: Log files in a Hadoop cluster
• Stresses: TCO, scale-out, transform/big-query performance, ETL functionality

With the explosion of machine-generated data has come the need for a place to put it all, sometimes called the big bit bucket. This is like the investigative data mart for big databases, but more poly-structured. In some cases it is focused on data staging and transformation; but it can also be used for analysis in place.

The list of candidate technologies to run your bit bucket starts with Hadoop and Splunk.

Archival data store

• Kinds of data likely to be included: Operational, CDR (call detail record), security log
• Likely use styles: Archival, reporting (for compliance), possibly also investigative
• Examples: Any long-term detailed historical store
• Stresses: TCO, compression, scale-out, performance (if multi-use)

Analytic DBMS vendors have been insulting each other with the claim “that’s just an archival data store,” dating back at least to the first time Greenplum was deployed on an underpowered Sun Thumper system. Perhaps only Rainstor truly embraces the archival positioning, and I’ve become pretty dubious about their technical claims and their company alike.

Still, there’s a legitimate need for data stores — especially relational analytic DBMS that:

• Store data cheaply, with high rates of compression.
• Have decent performance if you do want to query the data.
• May have archiving/compliance-specific features as well.

Along with Rainstor, SAND and SenSage have at least partially targeted that use case. In addition, appliance vendors such as Teradata and Netezza try to have an archive-oriented product version in their lineups.

Outsourced data mart

• Kinds of data likely to be included: All
• Likely use styles: Traditional BI, investigative analytics, staging/ETL
• Examples: Advertising tracking, SaaS CRM
• Stresses: Performance, TCO, reliability, concurrency

Much of what happens in analytic database management can also be outsourced. Some applications that run via SaaS (Software as a Service) are analytic. I’ve had three different clients whose main business is picking marketing targets in various vertical segments; others who wanted to add analytics to what were historically OLTP applications; and others yet who just offered online business intelligence. Also, if your fundamental business is gathering data and reselling it to a variety of user organizations, that’s an analytic data management challenge. The possibilities expand from there.

Data outsourcers are in the IT business, and so their IT development is — hopefully! — more serious and less politically encumbered than at many conventional enterprises. Thus, legacy systems and master data management issues are commonly less prevalent, or at least more aggressively disposed of. The same, up to a point, goes for vendor politics.*  Multitenancy is commonly an issue, as is running in the cloud.

*Even so, there’s often That Guy who doesn’t want to migrate away from Oracle, no matter what.

Vertica gets the nod in a number of these cases; it’s cloud-friendly, and often the problem is naturally columnar. Other columnar products can be good choices too, with added brownie points for Infobright if the shop is MySQL-oriented anyway. Running Netezza or other appliances makes sense mainly if you’re pretty sure you want to keep operating your own data centers, but some data outsourcers are just fine with that assumption.

Operational analytic(s) server

• Kinds of data likely to be included: Customer-centric, log, financial trade
• Likely use styles: Advanced operational analytics
• Examples:
  • Lower latency: Web or call-center personalization, anti-fraud
  • Higher latency: Customer profiling, Basel 3 risk analysis
• Stresses: Performance, reliability, analytic functionality, perhaps concurrency

Even with eight different choices, I need a “catch-all” category; this is it.

Suppose you want to do reasonably sophisticated analytics, then use the results in operations. This is the classical challenge in integrating short-request and analytic processing. There are multiple ways to tackle it, embodying different trade-offs in cost, convenience, or analytic accuracy. If the platform on which you want to run your investigative analytics also has the reliability and concurrency appropriate for mission-critical operations, you’re set. Otherwise, you may want to pipe derived data into a more “industrial-strength” DBMS, ideally the one that runs your operational apps anyway

Another option is to integrate a limited amount of analytics immediately into your short-request processing system. For example, as bad as they are at the kinds of queries that require joins, NoSQL systems are often fast at simple aggregations. As MapReduce/NoSQL integrations mature, that option may not require pumping the data anywhere else for deeper analytics; even if it does, at least you’re starting out with the data in a convenient bit bucket.

Streaming/CEP-centric architectures could come into play as well. And it goes on from there. The possibilities in this last category are just too varied to generalize about.

So did I get them all? Or are there yet other analytic data management use cases that I don’t fit into my eight categories?

Let’s try eight categories instead. While no categorization is ever perfect, these each have at least some degree of technical homogeneity. Figuring out which types of analytic database you have or need — and in most cases you’ll need several — is a great early step in your analytic technology planning. 

Enterprise data warehouse (Full or partial)

• Kinds of data likely to be included: All, but especially operational
• Likely use styles: All
• Canonical example: Central EDW for a big enterprise
• Stresses: Concurrency, reliability, workload management

The enterprise data warehouse (EDW) ideal says that you copy all your data into one place, and drive all decision-making from there. Full EDWs are pipedreams. Still, a partial EDW makes sense for most large enterprises, and many indeed already have one. The first product lines to consider for classical EDWs are Teradata, DB2, Exadata, and maybe Microsoft SQL Server, especially if you’re going to stress concurrency and/or operational use cases.

Traditional data mart

• Kinds of data likely to be included: All
• Likely use styles: Business intelligence, budgeting/consolidation, investigative
• Examples: Reporting servers, planning/consolidation servers, anything MOLAP, etc.
• Stresses: Performance, concurrency, TCO

Whether or not you have something like an enterprise data warehouse, it’s common to have lighter-weight data marts as well. A traditional data mart might drive reports and dashboards. Or it might be specialized for budgeting, planning, and/or consolidation.  Some investigative analytics may be in the mix as well.

Any DBMS that can support an EDW can also support a data mart, but it may not be the most cost-effective way to do so. Columnar DBMS might have more attractive performance and TCO (Total Cost of Ownership); the same goes for Netezza. Some of them — e.g. Sybase IQ and Vertica — have excellent track records in concurrent usage as well. Ted Codd pushed what amounts to MOLAP (Multidimensional OnLine Analytic Processing) systems for these use cases. But relational DBMS commonly do a better job, which is one reason most major MOLAP products have wound up at RDBMS companies.

Investigative data mart — agile

• Kinds of data likely to be included: All, especially customer-centric
• Likely use styles: Investigative
• Canonical example: A few analysts getting a few TB to examine
• Stresses: Ease of setup/load, ease of admin, price/performance

Besides the traditional data mart, there are at least two other kinds. Both are focused on investigative analytics, but they’re differentiated by database size.

If you have just a few analysts,* looking at no more than a few terabytes of data (perhaps even just some gigabytes) — and if that data is “single-subject” and fairly homogenous — your watchwords should be “cheap”, “easy”, and “fast”. You don’t need to invest in much hardware, in expensive software, in much administrative effort (the analysts can be their own DBAs),  nor should you endure much set-up time. Just grab a product, grab some data, and start running queries (or extracts into the statistical tool of your choice).

*If you have dozens or even hundreds of analysts hitting the same database, you’re probably back to the more concurrency-oriented scenarios outlined above.

Infobright is often cost-effective among columnar analytic DBMS. Other vendors might cut you a price break as well. If you have multiple terabytes of data, don’t rule out Netezza’s lowest-end products (even if they’d really rather sell you something bigger). Or, if you’re in the sub-terabyte range, maybe you can get by with an in-memory BI tool such as QlikView, and not do anything special on the DBMS side at all.

Investigative data mart — big

• Kinds of data likely to be included: All, especially customer-centric, logs, financial trade, scientific
• Likely use styles: Investigative
• Canonical example: Single-subject 20 TB – 20 PB relational database
• Stresses: Performance, scale-out, analytic functionality

But if you’re looking at tens of terabytes of relational data, or even more, you really do have a “big data” problem. Performance and scalability are major challenges, usually best addressed by MPP (Massively Parallel Processing) systems, such as Netezza, Vertica, Aster Data, ParAccel, Teradata, or Greenplum. Performance POCs (Proofs Of Concept) are a big part of the buying process. Vendor price negotiations are crucial too.

Actually, in the low tens of terabytes you might be able to get away with a shared-disk system that has excellent compression — e.g., columnar products like Sybase IQ, Infobright, or SAND, rather than just Vertica and ParAccel.

Assuming you have affordable, scalable query performance, the competitive differentiator can switch to additional analytic functionality. Aster, Netezza, ParAccel, Vertica, and Greenplum either offer full analytic platforms, or seem to be on the path to doing so. Teradata, which now owns Aster Data, offers substantial built-in analytic capability in its traditional products as well, and the same goes for Sybase IQ.

Continued in Part 2, where we cover some of the more difficult use cases.

*Actually, he didn’t admit that what he was advocating was a different kind of DBMS, namely a MOLAP one — but he was. And he was wrong anyway about the necessity for MOLAP. But let’s overlook those details.

Nonetheless, relational DBMS dominate the market. As I see it, the reasons for relational dominance cluster into four areas (which of course overlap):

• Data re-use. Ted Codd’s famed original paper referred to shared data banks for a reason.
• The benefits of normalization, which include:
  • You only have to do programming work of writing something once …
  • … and you don’t have to do the programming work of keeping multiple versions of the information consistent.
  • You only have to do processing work of writing something once.
  • You only have to buy storage to hold each fact once.
• Separation of concerns.
  • Different people can worry about programming and “database stuff.”
  • Indeed, even performance optimization can sometimes be separated from programming (i.e., when all you have to do to get speed is implement the correct indexes).
• Maturity and momentum, as reflected in the availability of:
  • People.
  • A broad variety of mature relational DBMS.
  • Vast amounts of packaged software that “talks” SQL.

Generally speaking, I find the reasons for sticking with relational technology compelling in cases such as: 

• You’re building a low-volume, medium-complexity suite of applications that will evolve over time. This is the use case for which relational DBMS were invented, and they’re still great for it.
• Your (duplicated) data volumes would be ridiculous if you didn’t do a reasonable amount of normalization. Once you need to normalize, you need to do joins — and if you’re doing joins, you’re in relational territory.
• You simply don’t see a cost/benefit advantage to moving away from proven legacy technology. If you’re looking for an off-the-shelf answer to your needs — or if you’re inventorying your own technological shelves — relational-oriented technology has overwhelming share.

For many enterprises, that third point alone should be decisive in a large fraction of cases.

But the advantages of relational technology are less clear when you’re doing serious engineering of path-breaking new applications, where by “serious engineering” I mean:

• The problem is big enough that you simply want the best solution, with only loose coupling needed to the rest of your technical environment.
• Long-lasting “strategic” or legacy technology is not a great concern; you’re willing to keep “rebuilding the 747 while it’s flying” if that’s what’s necessary to get the best possible result.
• You have access to sufficient quantities of sufficiently smart people.

For example:

• I recently suggested that innovative SaaS vendors could adopt object-oriented database technology.
• Major web applications are rarely very relational. Until recently, the default approach to scaling out web databases was memcached/sharded MySQL, hardly a whole-hearted adoption of relational technology. Now NoSQL DBMS are vigorous competitors.
• Analytic challenges that amount to teasing out signals from streams of data are sometimes handled non-relationally as well, although it’s often nice to be able to do a few joins to mix in information from more relationally-structured data.

Not coincidentally, in a lot of those cases, throwing performance concerns “over the wall” to the database administrator isn’t going to work.

*I do expect the pendulum to swing back a bit as high-performance/highly-scalable MySQL implementations mature, but there are relatively few supporting examples to date.

To look at it another way, it’s right to be skeptical about relational DBMS when you can defeat all of the reasons to favor them. For example:

• Data re-use may not arise when applications are self-contained and rapidly-changing.
• Sometimes you don’t need to normalize your data.
• It’s not obvious that the relational approach to separation of concerns is the best one. Perhaps you’d be better off with the people who understand a specific application best being responsible for all the decisions connected with it.
• As for that maturity and momentum:
  • People don’t actually learn much SQL in school.
  • Are any of the mature relational DBMS what you really want?
  • Is any of that packaged software out there really helpful for your specific problem?

I should probably stop there. But in an appeal to authority, I’ll close instead with a quote from Codd’s own OLAP paper:

IT should never forget that technology is a means to an end, and not an end in itself. Technologies must be evaluated individually in terms of their ability to satisfy the needs of their respective users. IT should never be reluctant to use the most appropriate interface to satisfy users’ requirements. Attempting to force one technology or tool to satisfy a particular need for which another tool is more effective and efficient is like attempting to drive a screw into a wall with a hammer when a screwdriver is at hand: the screw may eventually enter the wall but at what cost?

Related link

• My exchange with Mike Stonebraker highlighting our shared advocacy for database diversity

1.  Not many terms I coin gets marketing traction, but machine-generated data has grown some legs. Clients (Infobright, Cloudera) and non-clients alike have adopted it. I need to follow up with a more official description/definition of the concept. The Wikipedia article on same doesn’t get the job done yet. (Edit: Here’s my take on defining machine-generated data. Be sure to read through to Daniel Abadi’s response.)

2.  Merv Adrian is going to Gartner Group. Expect great improvement in Gartner’s DBMS coverage, in areas beyond the straightforward “This is what users say they are doing” Gartner already excels at. That said, Merv is probably not starting at Gartner soon enough to help make the 2010 analytic DBMS Magic Quadrant any better than the Gartner 2009 data warehouse database management system magic quadrant, the Gartner 2008 data warehouse database management system magic quadrant, and so on.

In particular, Merv has a good understanding of trends and technology on analytic DBMS and related markets. Judging by his Twitter stream, James Kobielus at Forrester if anything overrates the shift to general “analytic platforms.” And I of course am expected to help define the “analytic platform”/”advanced analytics”/whatever category. Taking all those analyst efforts together, it’s reasonable to expect a lot more market awareness — and also market confusion — around these areas.

3.  All that plugs into a larger project I was working on before my family issues came crashing in. The enterprise data warehouse is a myth, and that’s just the first reason that the old EDW vs. data mart bifurcation is grossly inadequate for understanding analytic data management choices. So I’m working on some ideas to categorize types of data warehouse/mart/whatever according to what kind of data you have and how you use that data. Multiple industry players (OK, vendors) have offered interesting and useful feedback in this process, although I’m still waiting for Teradata and IBM. (Edit: My bad. Teradata actually had sent a helpful response some time ago.)

In connection with that effort, the last outline I did back in October of analytic data use styles read:

• Traditional BI
  • Reporting, dashboards, & light-weight ad-hoc query
  • (Even if you make this more into data exploration, you’re probably not stressing the underlying DBMS much more than traditional BI does)
  • (If integrated into operational apps, your DBMS choice for this may be constrained by your choice of operational apps)
• Near-real-time BI
  • E.g., dashboards w/ constant or 1-minute refresh
  • (Actually, this isn’t a great fit for most analytic DBMS yet)
  • (Also, it’s not a big market yet, except in specialized niches such as trading or network control)
• Budgeting & consolidation
  • (MOLAP is still strong here)
  • (I took out the word “planning” because it has several meanings)
• Investigative analytics*
  • Can be but doesn’t have to be long-running
  • Example technologies include:
    • Heavy ad-hoc query
    • Data mining/machine learning/predictive analytics modeling
    • Simulation
    • Other advanced analytics
• (Advanced) operational analytics
  • Inputs to operational apps
  • Technologically similar to investigative analytics
    • Data mining/machine learning/predictive analytics scoring
    • Simulation
    • Other advanced analytics
  • Example applications include:
    • Customer classification or scoring
    • Wholesale telecom pricing
    • Basel 3 risk analysis
• Pre-processing, staging, and ETL
• Archive & compliance
• (Test/dev)

The data warehouse/mart categories weren’t in exact one-to-one correlation to those use styles, but the connection was of course pretty close.

*I’ve really struggled with terminology in the area of data exploration (over-used already)/discovery analytics (sounds weird)/research analytics (caused confusion when I tried it). Investigative analytics is my latest try.

4.  And finally — like most people, I find the terms unstructured or semi-structured data to be misleading, for at least two reasons:

• When the data is human-generated, what’s really happening is usually that the structure is just in a different place — structured databases generally tend to hold unstructured data, and vice-versa.
• In the case of machine-generated data, you really can start out with unstructured sets of individually unstructured logs. So what do you do then? You derive data, which has some kind of structure, and do most of your operations on that.

So I’ve been playing for a couple of years with the thought of introducing the term polystructured data. This is not a finished concept, because there are at least three different things I could mean by it:

• “Polystructured data is data that has considerable structure, but whose structure is in some important way unpredictable.” That’s a direct quote from a draft of a never-published paper. The paper, conceived before the days of NoSQL, was meant to be very XML-centric.
• “Polystructured data is data whose structure is apt to be interpreted in different ways at different times” — e.g., data that will variously get referenced by free text and structured searches. The example I gave illustrates part of the problem with that version, as increasingly many software vendors think it’s a dandy idea to do free-text searches across various columns of relational tables.
• “Polystructured data is data that gets restructured over time.” That’s the derived data point.

It may take a while to find, but I think there’s a pony in there somewhere.

Edit: Here’s the definition of poly-structured database I eventually came up with.

The Bottom Line  – SAP’s Turning The Corner

Credit must be given to SAP for charting a new course.  A shift in the management philosophy and product direction will take years to realize, however, its not too late for change.  SAP must remember its roots and become more German and less American.  The renewed focus must put customer requests and priorities ahead of SAP’s bureaucracy.  The emphasis must focus on the relationship.  When that reemerges in how SAP works with customers, partners, influencers, and its own employees, SAP will be back in good graces. In the meantime, its  time to get to work and deliver.  Oracle’s Fusions Apps are coming soon and competitors such as IBM, Microsoft, Epicor, IFS, and SalesForce.com will not relent.

I recall the 1980s, when SAP’s main differentiator, at least in the English-speaking US, was a total commitment to customer success, and when it could be taken for granted that SAP would do business ethically. Things change, and not always for the better.

Anyhow, the reason I’m highlighting Ray’s post is that he makes reference to a number of interesting SAP-cetric technology trends or initiatives. In no particular order, Ray suggests:

• SAP’s and Oracle’s (Fusion) efforts to meld memory-centric analytics with operational apps will be crucial for large enterprises — but perhaps only around the middle of the next decade. (I basically agree, although I’d note that:
  • Wisely, Ray suggested a very long time frame.
  • BI/operational app integration has been, on the whole, glacial.
  • The idea that you have to put pre-built aggregates into RAM to get performance is an indictment of market-leading RDBMS — but it’s a fair indictment.
  • I’m not sure whether memory-centric OLAP will wind up in RAM or Flash. If the data stores are updated at near-transactional speeds, RAM may make more sense. Otherwise, Flash should have major advantages.)
• SAP’s long-standing attempts to support third-party development of SAP add-ons are a technological mess, in line with my fears of a couple of years ago. However, the business-relationship part of the effort is vastly stronger.
• As SAP focused more on the mid-market, it is partnering closely with Microsoft. (If you think about it, that makes all kinds of sense.)
• Energy/environmental/safety tracking — i.e., sustainability — tools are a big deal. (See also The Economist on that point.)

What are reasonable MDX performance expectations?

MDX doesn’t come up in my life very much, and I don’t have much intuition about it. E.g., I don’t know whether one can slap an MDX-to-SQL converter on top of a fast analytic RDBMS and go to town. What’s more, I’m heading off on vacation and don’t feel like researching the matter myself in the immediate future.

So here’s the long form of the question. Any thoughts?

I have a general question on assessing the performance of an OLAP technology using a set of MDX queries. I would be interested to know if there are any benchmark MDX performance tests/results comparing different OLAP technologies (which may be based on different underlying DBMS’s if appropriate) on similar hardware setup, or even comparisons of complete appliance solutions. More generally, I want to determine what performance limits I could reasonably expect on what I think are fairly standard servers.

In my own work, I have set up a star schema model centered on a Fact table of 100 million rows (approx 60 columns), with dimensions ranging in cardinality from 5 to 10,000. In ad hoc analytics, is it expected that any query against such a dataset should return a result within a minute or two (i.e. before a user gets impatient), regardless of whether that query returns 100 cells or 50,000 cells (without relying on any aggregate table or caching mechanism)? Or is that level of performance only expected with a high end massively parallel software/hardware solution? The server specs I’m testing with are: 32-bit 4 core, 4GB RAM, 7.2k RPM SATA drive, running Windows Server 2003; 64-bit 8 core, 32GB RAM, 3 Gb/s SAS drive, running Windows Server 2003 (x64).

I realise that caching of query results and pre-aggregation mechanisms can significantly improve performance, but I’m coming from the viewpoint that in purely exploratory analytics, it is not possible to have all combinations of dimensions calculated in advance, in addition to being maintained.

So there are a whole lot of other worthy subjects I doubt I’ll be getting to any time soon.  In some cases, of course, other people are doing great jobs of writing about same. Here are pointers to a few links that I am glad to recommend:

• I wrote recently that I’ve discovered a number of different in-memory OLAP engines. Cindi Howson far outdid that, writing at length for Intelligent Enterprise on in-memory analytics, in an article that seems to itself be a teaser for a longer, free white paper on the subject.
• CouchDB posted an eye-catching, risque slide presentation promoting CouchDB and, more generally, key-value stores, at least for internet applications.  And yes, they’ve integrated MapReduce.
• Merv Adrian posted favorably about Birst, with special reference to its OEM efforts.  As previously noted, I was highly unimpressed with Birst’s end-user BI story at the time of its September roll-out, and Jerome Pineau’s recent examination did nothing to reassure me.  But perhaps OEM is a different matter.
• Merv also offers an interesting post about data integration upstart Expressor, and a highly favorable one about “visualization” vendor Tableau.
• Ann All interviewed Nigel Pendse, who grumped that BI features are overrated, and what end users really want is great query performance. I’m not so sure about the features side of that, but I’m hugely in agreement about the performance. That’s a big part of why the analytic DBMS industry is so vibrant. It’s also why in-memory OLAP is suddenly so hot.

• The combined Aleri has around 100 employees, 60-40 from Aleri vs. Coral8.
• The combined Aleri has around 80 customers. All of Aleri’s, with one sort-of exception at Banks.com, were in financial services. A large minority of Coral8′s were in financial services too.
• However, half of Aleri’s marketing spend going forward is budgeted outside the financial services markets. Not unreasonably, John presents this as a proof point Aleri is serious about selling to other markets.
• Aleri had 12-14 people in the UK pre-merger. Coral8 had none in Europe.
• Coral8 had 15 OEMs pre-merger, some actually generating revenue. Aleri had substantially none.
• Coral8 had been closing a “couple” of customers/quarter in online commerce. But recently, that rate ramped up to a “few.”
• Aleri’s engine is used to handle “many” hundreds of thousands of messages per second. Coral8′s highest-throughput user processes 100-150,000 messages/second.

John is sticking by the company line that there will be an integrated Aleri/Coral8 engine in around 12 months, with all the performance optimization of Aleri and flexibility of Coral8, that compiles and runs code from any of the development tools either Aleri or Coral8 now has. While this is a lot faster than, say, the Informix/Illustra or Oracle/IRI Express integrations, John insists that integrating CEP engines is a lot easier. We’ll see.

I focused most of the conversation on Aleri’s forthcoming efforts outside the financial services market. John sees these as being focused around Coral8′s old “Continuous (Business) Intelligence” message, enhanced by Aleri’s Live OLAP. Aleri Live OLAP is an in-memory OLAP engine, real-time/event-driven, fed by CEP. Queries can be submitted via ODBO/MDX today. XMLA is coming. John reports that quite a few Coral8 customers are interested in Live OLAP, and positions the capability as one Coral8 would have had to develop had the company remained independent.

I’m a bit confused about how new or mature Aleri Live OLAP is. Although a “5.0″ version was announced last May, John seemed to describe Live OLAP as a new technology just coming to market.

Generally, the applications that kept coming up were anti-fraud and ad/interaction-targeting, across multiple markets (including online gaming). An energy management deal to be announced soon seems to be an exception. I’m a little unclear as to how much is dashboards and how much is integrated operational BI. John views a natural progression as being dashboard-to-hard-coded-operational-BI. But I’m not year clear as to whether the initial dashboards provide much business value, or whether they are more just tools to get executive buy-in for the real opportunities.

]]> http://www.dbms2.com/2009/03/25/aleri-update/feed/ 4 http://www.dbms2.com/2009/02/07/analytics-role-in-a-frightening-economy/ http://www.dbms2.com/2009/02/07/analytics-role-in-a-frightening-economy/#comments Sat, 07 Feb 2009 11:55:15 +0000 Curt Monash http://www.dbms2.com/?p=683 I chatted yesterday with the general business side (as opposed to the trading operation) of a household-name brokerage firm, one that’s in no immediate financial peril. It seems their #1 analytic-technology priority right now is changing planning from an annual to a monthly cycle.* That’s a smart idea. While it’s especially important in their business, larger enterprises of all kinds should consider following suit.

*By the way, they seem to want use Applix technology, now owned by IBM/Cognos, to do it, more for the planning tools than for the cool in-memory OLAP engine itself. Your mileage may vary.

If you don’t go for fancy slice/dice tools, then do something else to make sure your drilldown or exploration are up to snuff. Just about every enterprise is going to be seeing some distressing numbers right now. But that doesn’t mean every part of every line of business is in equal trouble. Teasing that apart is important at times like this, or employment and investment could get strangled do to overreaction.

I have a small enough business that I can keep things like this in my head, without bothering to run precise three-significant-figures calculations. But I’ll tell you this — my revenue in analytics is sure healthier than my business in OLTP or custom publishing.

And don’t necessarily stop there. We’ve heard the horror stories of investment models failing in this unusual economy. Well, what about your own predictive analytic models? If you have formal models of buyer behavior, what makes you think the future will be like the past? Times have changed. No matter what your usual schedule is, you should be revalidating and perhaps strengthening your more important models now. If that overtaxes your infrastructure, and you can’t afford the capital investment to do something about that — well, appliances are pretty cheap, and some SaaS data warehousing offerings are (at least in the short term) even cheaper.