That said,  

• Raj said that Sybase’s Aleri acquisition was, if anything, tracking ahead of expectations.
• Raj didn’t seem the slightest bit focused on the Coral8/Aleri CEP-based BI strategy that John Morell had long championed.
• Raj reminded me that Sybase SQL Anywhere has numerous OEMs, not just on the true desktop/laptop or smaller, but also in a return to its server/workgroup roots. Sybase SQL Anywhere even added geospatial indexing recently.

Raj also spoke glowingly of SAP’s in-memory database technology and the potential for Sybase of same — until I asked a follow-up question. At that point, he confessed that he didn’t really know much about about SAP’s in-memory database technology yet. As I said before, I believe SAP is fairly sincere about its belief that its in-memory database technology will conquer the world — but this is a naive and poorly-founded opinion even so.

One tidbit I did get is that SAP’s in-memory database technology is not just son-of-T-REX. A Korean (Raj thinks) company SAP had acquired is also in the mix. Raj also had the impression SAP’s in-memory technology can do rows, columns, or hybrid structures. On the one hand, that makes sense. On the other, it’s not a perfect fit with what Hasso Plattner said last year.

To understand Sybase’s plans for Aleri and CEP, it helps to understand Sybase’s current CEP-oriented offering, Sybase RAP. So far as I can tell, Sybase RAP has to date only been sold in the form of Sybase RAP: The Trading Edition. In that guise, Sybase RAP has been sold to >40 outfits since its May, 2008 launch, mainly big names in the investment banking and stock exchange sectors. If I understood correctly, the next target market for Sybase RAP is telcos, for real-time network tuning and management.

In addition to any domain-specific applications, Sybase RAP has three layers:

• CEP (Complex Event Processing). Sybase RAP CEP is based on a version of the Coral8 engine Sybase licensed and has been subsequently developing.
• In-memory DBMS. Sybase’s IMDB is part of (but I guess separable from) and has the same API as Sybase’s OLTP DBMS Adaptive Server Enterprise (ASE, aka Sybase Classic).
• Sybase IQ. Actually, Sybase used the phrase “based on Sybase IQ,” but I’m guessing it’s just Sybase IQ.

In theory, there could be a DBMS other than Sybase IQ, such as Sybase ASE or even Oracle, because Sybase IMDB can talk to a variety of DBMS. I didn’t get the impression, however, that in practice there were any Sybase RAP installations whose persistent DBMS was anything other than Sybase IQ.

Aleri had all along had something called Project Ohio, to merge Coral8 with Aleri Classic. Now Sybase’s own CEP engineering team is being added to the mix, schedules are being reconsidered and haven’t been disclosed yet. (If one woman can produce one baby in nine months, how long does it take nine women to produce a baby?) Apparently Sybase has a dozen programmers in the CEP area, plus ~20 more on Sybase RAP, not counting QA, documentation, etc.; that represents a significant bump to the overall Aleri development team.

Sybase doesn’t seem to have decided what to do yet with the various business intelligence/real-time OLAP engine products and technologies it is inheriting from Aleri.

And finally, some metrics:

• The Sybase/Aleri guys estimate that 1/3 of of Aleri’s customers and even less of its revenue came from outside the financial services sector. They did say the non-financial-services business was “starting to pick up,” but not very convincingly.
• Sybase IQ is now up to >1800 customers, with >200 new ones in 2009.
• Sybase IQ indeed has users taking in market feeds up to 3 terabytes a day, so it probably matches Vertica in having at least several-hundred-terabyte databases in the financial sector.

• The folks at Sybase still haven’t figured out when to prebrief me. (Edit: I’ve been briefed subsequently.)
• Sybase/Aleri is a potentially powerful combination, if they can effectively address the point I just made about integrating disparate latencies. That said, I’m not expecting a lot, because the CEP industry always disappoints me.
• Microsoft, IBM, and (somewhat less clearly) Oracle are all trying to do CEP inhouse. Sybase is making a good choice in having serious CEP inhouse itself
• Surely the main focus and financial justification for the Sybase/Aleri acquisition is the financial services market.
• Specifically, I expect the focus of technical integration between Aleri and Sybase’s DBMS products to start with Sybase IQ.
• Coral8 had some interesting ideas about how to integrate CEP with OLTP/operational BI, but I’m not aware that they got much traction.
• I bet there are use cases where Sybase tries and fails to sell Adaptive Server SQL Anywhere that CEP would be a better technical fit, but I don’t immediately see much practical business significance to that observation.
• While this deal could easily strengthen the Vertica/StreamBase partnership, I don’t see any reason why it would lead those two companies to actually merge.

Related link

• Thinking about analytic latency

• Most independent CEP vendors have some kind of application story in the capital markets vertical, such as packaged applications, ISV partners with packaged applications, application frameworks, and so on.
• CEP vendors offer lots of connectors to specific financial industry price/quote/trade feeds, as well as the usual other kinds of database connectivity (SQL, XML, etc.)
• Aleri/Coral8 (separately and now together) like to call attention to their business intelligence/analytics offerings. Analytics is front-and-center on Truviso’s web site too, not that Truviso does much to call attention to itself, period. (Roman Bukary once said he’d outline Truviso’s new strategy to me in 6-8 weeks or so … it’s now 14 months and counting.)

So far as I can tell, the areas of applications and analytics are fairly uncontroversial. Different CEP vendors have implemented different kinds of things, no doubt focusing on those they thought they would find easiest to build and then sell. But these seem to be choices in business execution, not in core technical philosophy.

In CEP application development, however, real philosophical differences do seem to arise. There are at least three different CEP application development paradigms:

• DML (Data Manipulation Language) extensions to handle time windows, etc., which are then embodied into a conventional application development stack. Microsoft exemplifies an extreme form of this strategy, but most vendors offer it at least as an option.
• Visual event-oriented programming language. This is StreamBase’s claim to fame; StreamBase says that 95% of its customers do 100% of their development in its Eclipse-based visual tool, which truly creates executable programs without any kind of code generation step. (Other “this isn’t just a toy” buzzwords StreamBase offered were “modularity, ” “parametrizability,” and, best of all, “visual debugger.”)
• Rules engine. When Progress Apama first told me how its preferred tool (I think a precursor of what is now Apama Event Modeler) worked, it sounded a lot like a RETE-based expert system shell. The Apama folks assured me that this really wasn’t RETE, and I see that there’s definitely more than just a rules language in Apama MonitorScript. Still, the paradigm seems to basically be that you write a bunch of rules, which are then executed by a suitable engine.

Even more fundamental, however, is the question of event-driven programming. That’s a term which first was popular in the 1990s, signifying programs that — rather than being wholly procedural — listened for and responded to events, often in user interfaces. More generally, event-driven programming is inherent in almost any kind of loosely-coupled architecture, be it client-server, service-oriented, or whatever. But to hear some CEP proponents tell it, all that isn’t event-driven enough.

In particular, StreamBase recommended to me Gregor Hohpe’s paper Programming Without a Call Stack – Event-driven Architectures, which argues that if there’s any concept of procedure or method invocation at all, the whole thing is too “command-and-control” (Boo!!) and insufficiently event-driven to be well-suited for CEP. Rather, everything should be done on a fine-grained publish-subscribe basis. It’s an interesting argument. Usually, the problem with extremist programming paradigms is that the thing you write has to interface with the rest of the world, and by the time it accommodates itself to them, the paradigm is violated anyway. But if the essence of the paradigm is loose coupling to begin with, maybe that pitfall can for once be avoided.

]]> http://www.dbms2.com/2009/05/21/notes-on-cep-application-development/feed/ 5 http://www.dbms2.com/2009/05/21/notes-on-cep-performance/ http://www.dbms2.com/2009/05/21/notes-on-cep-performance/#comments Thu, 21 May 2009 08:20:36 +0000 Curt Monash http://www.dbms2.com/?p=787 I’ve been talking to CEP vendors on and off for a few years. So what I hear about performance is fairly patchwork. On the other hand, maybe 1-2+ year-old figures of per-core performance are still meaningful today. After all, Moore’s Law is being reflected more in core count than per-core performance, and it seems CEP vendors’ development efforts haven’t necessarily been concentrated on raw engine speed.

So anyway, what do you guys have to add to the following observations?

• Super-low-latency financial services industry tasks are often “embarrassingly parallel.” Thus, near-linear scale-out is common.
• That said, good parallelism seems fairly new in CEP engines (of course, CEP engines are fairly new themselves — for all I know, some have been parallel since inception).
• I’ve heard claims of up to 400,000 messages/second/core for simple queries or patterns.
• I’ve heard claims of 70,000 messages/core for not-so-simple queries or patterns, and probably higher than that depending on what the meaning of “simple” is.
• IBM just disclosed >15,000 messages/core on a pretty low-powered processor.
• I’ve heard that Coral8, Apama, and StreamBase rarely lost deals due to performance or throughput problems. I’ve heard that the same is not as true of Aleri.
• StreamBase proudly says it’s been fully multithreaded since academic research-project days.  For Apama multithreading is evidently a more recent feature. But does it matter much?

• The combined Aleri has around 100 employees, 60-40 from Aleri vs. Coral8.
• The combined Aleri has around 80 customers. All of Aleri’s, with one sort-of exception at Banks.com, were in financial services. A large minority of Coral8’s were in financial services too.
• However, half of Aleri’s marketing spend going forward is budgeted outside the financial services markets. Not unreasonably, John presents this as a proof point Aleri is serious about selling to other markets.
• Aleri had 12-14 people in the UK pre-merger. Coral8 had none in Europe.
• Coral8 had 15 OEMs pre-merger, some actually generating revenue. Aleri had substantially none.
• Coral8 had been closing a “couple” of customers/quarter in online commerce. But recently, that rate ramped up to a “few.”
• Aleri’s engine is used to handle “many” hundreds of thousands of messages per second. Coral8’s highest-throughput user processes 100-150,000 messages/second.

John is sticking by the company line that there will be an integrated Aleri/Coral8 engine in around 12 months, with all the performance optimization of Aleri and flexibility of Coral8, that compiles and runs code from any of the development tools either Aleri or Coral8 now has. While this is a lot faster than, say, the Informix/Illustra or Oracle/IRI Express integrations, John insists that integrating CEP engines is a lot easier. We’ll see.

I focused most of the conversation on Aleri’s forthcoming efforts outside the financial services market. John sees these as being focused around Coral8’s old “Continuous (Business) Intelligence” message, enhanced by Aleri’s Live OLAP. Aleri Live OLAP is an in-memory OLAP engine, real-time/event-driven, fed by CEP. Queries can be submitted via ODBO/MDX today. XMLA is coming. John reports that quite a few Coral8 customers are interested in Live OLAP, and positions the capability as one Coral8 would have had to develop had the company remained independent.

I’m a bit confused about how new or mature Aleri Live OLAP is. Although a “5.0″ version was announced last May, John seemed to describe Live OLAP as a new technology just coming to market.

Generally, the applications that kept coming up were anti-fraud and ad/interaction-targeting, across multiple markets (including online gaming). An energy management deal to be announced soon seems to be an exception. I’m a little unclear as to how much is dashboards and how much is integrated operational BI. John views a natural progression as being dashboard-to-hard-coded-operational-BI. But I’m not year clear as to whether the initial dashboards provide much business value, or whether they are more just tools to get executive buy-in for the real opportunities.

]]> http://www.dbms2.com/2009/03/25/aleri-update/feed/ 4 http://www.dbms2.com/2009/03/09/independent-cep-vendors-continue-to-flounder/ http://www.dbms2.com/2009/03/09/independent-cep-vendors-continue-to-flounder/#comments Mon, 09 Mar 2009 15:46:57 +0000 Curt Monash http://www.dbms2.com/?p=720 Independent CEP (Complex/Event Processing) vendors continue to flounder, at least outside the financial services and national intelligence markets.

• StreamBase once planned to conquer the world, making an impact as big as database management’s. Now it has retreated into niche markets.
• Progress Software, a decent-sized company, put a large fraction of its energy into Apama. Little has happened outside the financial service sector.
• Coral8 has some great-sounding ideas. But Coral8 now has merged into Aleri, basically a financial-markets specialist.
• Mike Franklin says some ambitious things on behalf of Truviso, but I haven’t noticed much traction there either.

CEP’s penetration outside of its classical markets isn’t quite zero. Customers include several transportation companies (various vendors), Sallie Mae (Coral8), a game vendor or two (StreamBase, if I recall correctly), Verizon (Aleri, I think), and more. But I just wrote that list from memory — based mainly on not-so-recent deals — and a quick tour of the vendors’ web sites hasn’t turned up much I overlooked. (Truviso does have a recent deal with Technorati, but that’s not exactly a blue chip customer these days.)

So far as I can tell, this is a new version of a repeated story. A clever alternative to relational DBMS was invented. It proved superior in some specific applications and vertical markets. It failed to achieve much broader adoption. Initial high hopes got dashed, companies failed to grow rapidly, and shareholders grew tired.

So where will things go from here? My best guesses include:

• The financial trading market isn’t going away for CEP. Super-low-latency is really needed there.
• As much as I love the idea of CEP-infused BI, it will be adopted only at the rate broader-based BI vendors can support.
• A few niches will generate some business for CEP in data reduction. Leading candidates are the ones where there’s been a little traction to date — national intelligence, transportation, web analytics, and so on.
• Sadly, that’s about it.

]]> http://www.dbms2.com/2009/03/09/independent-cep-vendors-continue-to-flounder/feed/ 7 http://www.dbms2.com/2008/10/20/coral8-proposes-cep-as-a-bi-data-platform/ http://www.dbms2.com/2008/10/20/coral8-proposes-cep-as-a-bi-data-platform/#comments Mon, 20 Oct 2008 17:30:27 +0000 Curt Monash http://www.dbms2.com/?p=614 It used to be that Coral8 and StreamBase were the two complex event/stream processing (CEP) vendors most committed to branching out beyond the super-low-latency algorithmic trading marketing. But StreamBase seems to have pulled in its horns after a management change, focusing much more on the financial market (and perhaps the defense/intelligence market as well). Aleri, Truviso, and Progress Apama, while each showing signs of branching out, don’t seem to have gone as far as Coral8 yet. And so, though it’s a small company with not all that many dozens of customers, my client Coral8 seems to be the one to look at when seeing whether CEP really is relevant to a broad range of mainstream – no pun intended – applications.

Coral8 today unveiled a new product release – the not-so-concisely named “Coral8 Engine and Portal Release 5.5” – and a new buzzphrase — “Continuous Intelligence.” The interesting part boils down to this:

Coral8 is proposing CEP — excuse me, “Continuous Intelligence” — as a data-store-equivalent for business intelligence.

This includes both operational BI (the current sweet spot) and dashboards (the part with cool, real-time-visualization demos).

Technically, this adds a layer of complexity – since the persistent data store isn’t going away, CEP is just additive to the environment. And if an enterprise has invested in an operational-style warehouse with sufficiently low latency, it’s not always obvious how much CEP adds. But how low is sufficient? Consider the use cases:

• Identify which network issues are actually hurting a website user’s experience, and fix them. Coral8 reports multiple customers have bought into this one.
• Capture website click behavior and get it to the call center in time for the frustrated web users call. Coral8 reports that at least Sallie Mae has adopted this app.

The latter seems to require true sub-minute latency, and the former isn’t that far behind. It’s quite conceivable that CEP could be a faster and easier way of implementing this capability than would a full operational warehouse build.

There’s one more point in CEP’s favor – it’s really good at sharing intermediate results among multiple queries. Of course, there are multiple ways of doing something like this, from Teradata’s query optimizer to various DBMS vendors’ results caches to various BI vendors’ ways of querying against report results to Oracle’s and IBM/Applix’s MOLAP accelerations. Still, at least in theory, the CEP approach to KPI (Key Performance Indicator) tracking has a lot of merit.

Coral8 has identified three groups of apps where there seems to be customer traction or recognized need for these capabilities, to wit (my names, not theirs):

• Algorithmic trading
• CRM/customer experience (including antifraud)
• Keeping operations running smoothly

Perhaps not coincidentally, those areas comprise a significant fraction of the vibrant use cases for analytic technology overall.

• I believe that both of the previously mentioned petabyte+ databases on Greenplum will feature clickstream data.
• Aster Data’s largest disclosed database, by almost two orders of magnitude, is at MySpace.
• Clickstream analytics is a big application area for Vertica Systems.
• Clickstream analytics is a big application area for Netezza.
• Infobright’s customer success stories appear to be concentrated in clickstream analytics.
• Coral8 tells me that CEP is also being used for clickstream data, although I suspect that a lot of Coral8’s evidence in that regard comes from a single flagship account. Edit: Actually, Coral8 has a bunch of clickstream customers.

But what surprised me a bit was to discover that clickstream data is joined at the hip to more general network event data. I hadn’t heard much about that until recently. But over the past month or so, Greenplum, Aster, Vertica, Coral8, and some very big users have all told me the same thing:

Where there’s clickstream data, there’s usually also network event data – and the latter is in even higher volumes.

It’s obvious what one does with clickstream data (or at least what one tries to do — current web analytics are still highly primitive, for reasons I’ll lay out in another post). But what one does with network event data is a little murkier, and how one integrates clickstream and network event data is very unclear. I hear phrases like “All that TIBCO data is just falling on the floor,” from users and vendors alike. (Come to think of it, I’ve heard that exact phrasing from one vendor and one user, and the user is that vendor’s biggest customer … coincidence?) I hear the general sentiment that network event data is useful for optimizing network operations. But I haven’t gotten a lot more detail than that. Vertica did mention some network event management VARs – e.g., OpenService and nMetrics are on Vertica’s customer list – but I got the sense those might be network event management pure plays, not outfits looking at the integration of clickstream and network data.

The one clear application message I’ve gotten so far for clickstream/network event integration came from Coral8. The idea is:

• You track where on your websites there is anomalous user data – e.g., unusually high rates of abandonment – which might be indicative of technical difficulties.
• You track network anomalies, the vast majority of which are false alarms.
• If a website anomaly and a network anomaly do match up – boom, the network jocks spring into action and fix the problem as fast as they can.

Of course, coming from Coral8, that was a real-time story. But it seems as if similar scenarios would make sense on a data warehousing basis as well.

But my biggest takeaway from this pair of calls was that Coral8 and Truviso are penetrating general BI. It’s not a huge trend yet — maybe 20-30 customers total between them. Most of those are Coral8’s, and specifically focused on clickstream analysis and the like. Truviso’s apps are still double-deep secret stealth mysteries, but Roman Bukary promises they’ll be unveiled in a couple of months or so.

I gave both Roman and John Morell of Truviso a choice between three types of use:

1. Feeding operational apps directly
2. Feeding a dashboard
3. Feeding a data warehouse for subsequent analytics (reporting, data mining, dashboard, whatever)

Both indicated that their companies were active in the first choice, feeding operational apps. Roman said Truviso is also driving dashboards, but isn’t doing a lot of what amounts to ETL for warehouses. Coral8, meanwhile, is feeding warehouses more than directly driving dashboards.

Meanwhile, I continue to think that event processing technology is the future of dashboards.

Please subscribe to our feed!