• This is a good idea, and many software vendors should do it if they aren’t already.
• It’s no big deal.

For reasons why, let me quote an email I just sent to an inquiring reporter:

• StreamBase sells mainly to the financial services and intelligence community markets. Neither group will share much in the way of core algorithms.
• But both groups are pretty interested in open source software even so. (I think for both the price and customizability benefits.)
• Open source software commonly gets community contributions for connectors, adapters, and (national) language translations.
• But useful contributions in other areas are much rarer.
• Linden Labs is one of StreamBase’s few significant customers outside its two core markets.
• All of the above are consistent with the press release (which quotes only one StreamBase customer — guess who?).

• Most independent CEP vendors have some kind of application story in the capital markets vertical, such as packaged applications, ISV partners with packaged applications, application frameworks, and so on.
• CEP vendors offer lots of connectors to specific financial industry price/quote/trade feeds, as well as the usual other kinds of database connectivity (SQL, XML, etc.)
• Aleri/Coral8 (separately and now together) like to call attention to their business intelligence/analytics offerings. Analytics is front-and-center on Truviso’s web site too, not that Truviso does much to call attention to itself, period. (Roman Bukary once said he’d outline Truviso’s new strategy to me in 6-8 weeks or so … it’s now 14 months and counting.)

So far as I can tell, the areas of applications and analytics are fairly uncontroversial. Different CEP vendors have implemented different kinds of things, no doubt focusing on those they thought they would find easiest to build and then sell. But these seem to be choices in business execution, not in core technical philosophy.

In CEP application development, however, real philosophical differences do seem to arise. There are at least three different CEP application development paradigms:

• DML (Data Manipulation Language) extensions to handle time windows, etc., which are then embodied into a conventional application development stack. Microsoft exemplifies an extreme form of this strategy, but most vendors offer it at least as an option.
• Visual event-oriented programming language. This is StreamBase’s claim to fame; StreamBase says that 95% of its customers do 100% of their development in its Eclipse-based visual tool, which truly creates executable programs without any kind of code generation step. (Other “this isn’t just a toy” buzzwords StreamBase offered were “modularity, ” “parametrizability,” and, best of all, “visual debugger.”)
• Rules engine. When Progress Apama first told me how its preferred tool (I think a precursor of what is now Apama Event Modeler) worked, it sounded a lot like a RETE-based expert system shell. The Apama folks assured me that this really wasn’t RETE, and I see that there’s definitely more than just a rules language in Apama MonitorScript. Still, the paradigm seems to basically be that you write a bunch of rules, which are then executed by a suitable engine.

Even more fundamental, however, is the question of event-driven programming. That’s a term which first was popular in the 1990s, signifying programs that — rather than being wholly procedural — listened for and responded to events, often in user interfaces. More generally, event-driven programming is inherent in almost any kind of loosely-coupled architecture, be it client-server, service-oriented, or whatever. But to hear some CEP proponents tell it, all that isn’t event-driven enough.

In particular, StreamBase recommended to me Gregor Hohpe’s paper Programming Without a Call Stack – Event-driven Architectures, which argues that if there’s any concept of procedure or method invocation at all, the whole thing is too “command-and-control” (Boo!!) and insufficiently event-driven to be well-suited for CEP. Rather, everything should be done on a fine-grained publish-subscribe basis. It’s an interesting argument. Usually, the problem with extremist programming paradigms is that the thing you write has to interface with the rest of the world, and by the time it accommodates itself to them, the paradigm is violated anyway. But if the essence of the paradigm is loose coupling to begin with, maybe that pitfall can for once be avoided.

]]> http://www.dbms2.com/2009/05/21/notes-on-cep-application-development/feed/ 5 http://www.dbms2.com/2009/05/21/notes-on-cep-performance/ http://www.dbms2.com/2009/05/21/notes-on-cep-performance/#comments Thu, 21 May 2009 08:20:36 +0000 Curt Monash http://www.dbms2.com/?p=787 I’ve been talking to CEP vendors on and off for a few years. So what I hear about performance is fairly patchwork. On the other hand, maybe 1-2+ year-old figures of per-core performance are still meaningful today. After all, Moore’s Law is being reflected more in core count than per-core performance, and it seems CEP vendors’ development efforts haven’t necessarily been concentrated on raw engine speed.

So anyway, what do you guys have to add to the following observations?

• Super-low-latency financial services industry tasks are often “embarrassingly parallel.” Thus, near-linear scale-out is common.
• That said, good parallelism seems fairly new in CEP engines (of course, CEP engines are fairly new themselves — for all I know, some have been parallel since inception).
• I’ve heard claims of up to 400,000 messages/second/core for simple queries or patterns.
• I’ve heard claims of 70,000 messages/core for not-so-simple queries or patterns, and probably higher than that depending on what the meaning of “simple” is.
• IBM just disclosed >15,000 messages/core on a pretty low-powered processor.
• I’ve heard that Coral8, Apama, and StreamBase rarely lost deals due to performance or throughput problems. I’ve heard that the same is not as true of Aleri.
• StreamBase proudly says it’s been fully multithreaded since academic research-project days.  For Apama multithreading is evidently a more recent feature. But does it matter much?

• StreamBase once planned to conquer the world, making an impact as big as database management’s. Now it has retreated into niche markets.
• Progress Software, a decent-sized company, put a large fraction of its energy into Apama. Little has happened outside the financial service sector.
• Coral8 has some great-sounding ideas. But Coral8 now has merged into Aleri, basically a financial-markets specialist.
• Mike Franklin says some ambitious things on behalf of Truviso, but I haven’t noticed much traction there either.

CEP’s penetration outside of its classical markets isn’t quite zero. Customers include several transportation companies (various vendors), Sallie Mae (Coral8), a game vendor or two (StreamBase, if I recall correctly), Verizon (Aleri, I think), and more. But I just wrote that list from memory — based mainly on not-so-recent deals — and a quick tour of the vendors’ web sites hasn’t turned up much I overlooked. (Truviso does have a recent deal with Technorati, but that’s not exactly a blue chip customer these days.)

So far as I can tell, this is a new version of a repeated story. A clever alternative to relational DBMS was invented. It proved superior in some specific applications and vertical markets. It failed to achieve much broader adoption. Initial high hopes got dashed, companies failed to grow rapidly, and shareholders grew tired.

So where will things go from here? My best guesses include:

• The financial trading market isn’t going away for CEP. Super-low-latency is really needed there.
• As much as I love the idea of CEP-infused BI, it will be adopted only at the rate broader-based BI vendors can support.
• A few niches will generate some business for CEP in data reduction. Leading candidates are the ones where there’s been a little traction to date — national intelligence, transportation, web analytics, and so on.
• Sadly, that’s about it.

]]> http://www.dbms2.com/2009/03/09/independent-cep-vendors-continue-to-flounder/feed/ 7 http://www.dbms2.com/2008/01/14/intelligent-enterprise/ http://www.dbms2.com/2008/01/14/intelligent-enterprise/#comments Mon, 14 Jan 2008 17:21:54 +0000 Curt Monash http://www.dbms2.com/2008/01/14/intelligent-enterprise/ I’m getting a flood of press releases today, because many of the companies I write about were selected to Intelligent Enterprise’s list of 12 most influential vendors plus 36 more to watch in the areas Intelligent Enterprise covers (which seems to be pretty much the analytics-related parts of what I write about here and on Text Technologies). It looks like a pretty reasonable list, although I think they forced the issue in some of the small analytics vendors they selected, and of course anybody can quibble with some of the omissions.

Among the companies they cited, you can find topical categories here for IBM (and Cognos), Informatica, Microsoft, Netezza, Oracle, SAP/Business Objects (both), SAS, and Teradata; QlikTech; Cast Iron, Coral8, DATAllegro, HP, ParAccel, and StreamBase; and Software AG. On Text Technologies you’ll find categories for some of the same vendors, plus Attensity, Clarabridge, and Google. There also are categories for some of these vendors on the Monash Report.

Commonly, these are data reduction apps – i.e., there’s a gushing stream of inputs, and the CEP engine filters and “enhances” it, so that only a small, modified subset is sent forward. In other cases, disk-based systems could do the job perfectly well from a performance standpoint, but the pattern matching and filtering requirements are just a better fit for the CEP paradigm.

For example, StreamBase, Apama, and Coral8 each have some degree of activity in text filtering.

Also, I have a little more detail beyond what I already wrote about some of Coral8’s other applications.

Highlights include:

1. One disclosed network security application is by Solutionary. Solutionary is a managed service provider that correlates different kinds of threat markers in real time to try to separate innocent unusual events from actual bad ones. Many of these are fairly slow (log-ins to unusual services, log-ins from scary countries, multiple log-in attempts to the same account); others presumably are at network speed; and the whole thing is definitely a pattern-matching exercise with time windows playing a core role.

2. One large Coral8 customer manages tens of thousands of trucks all over the United States. The main application is to notice promptly if a truck isn’t where it’s supposed to be, by matching GPS tracking against schedules. But there also are other sensors; e.g., in refrigerator trucks temperature is monitored every minute.

Mark notes that this sensor data is natively in XML format, without every having been spawned from a relational database. In general, he points out, if data comes from other systems, it’s likely to be in XML format. To which I add: If the data comes from any kind of inherently real-time system, then almost by definition it’s unlikely to have come from an RDBMS. Ditto if it comes from any kind of sensor network.

3. Speaking of sensor data, IBM has announced that Coral8 is being bundled into its WebSphere RFID Information Center v1.1.

4. Mark and I talked briefly about fraud detection and prevention. He pointed out that fraud applications are all about events having happened in a certain order. I see that as basically true, albeit slightly exaggerated in that it depends a lot on the kind of fraud (e.g., less true in insurance fraud than some other cases).

However, the question arises: For what kinds of fraud prevention is overnight analysis (for example) not sufficient? One example Mark came up with is bad cashier behavior in huge casino restaurants, which apparently is hard to document after the fact because of how much it relies on the physical cash drawer. Beyond that – well, we didn’t cover a lot more detail, but a couple of areas subsequently occurred to me. First, there are a lot of cases where it might be much more comfortable for a business to block an online or telephone transaction in the first place than it is for them to reverse it later. Also, any case where credit authorizations are made on behalf of bricks-and-mortar retailers has to be handled either real-time or not at all.

1. Mark believes Coral8 has significantly lower latency than StreamBase. E.g., the Wombat/Coral8 combo achieves sub-millisecond latency, with Coral8 itself consuming less than a tenth of that. The best comparable figures from StreamBase that I currently know of are almost an order of magnitude slower.

Top-end speed aside, Mark believes that Coral8 is fundamentally better suited for complex queries and pattern recognition, while StreamBase works well with simpler queries. For example, his other performance claims notwithstanding, he concedes that StreamBase is at least comparable to Coral8 in its throughput for huge numbers of simple queries. (The number he mentioned was ½ million queries/second.) Indeed, while we barely talked about customer/marketing issues, Mark asserts that the companies’ respective customer bases reflect this complex/simple distinction.*

*I don’t think I can judge that claim overall yet. However, it seems consistent with one particular data point. The intelligence market – which StreamBase seems to dominate – probably does feature very high volumes of relatively simple filters.

2. Mark thinks Coral8 has a much richer set of language primitives than StreamBase. In particular, he calls attention to Coral8’s primitives for synchronizing multiple data streams. That particular example seems to be core of his claim that StreamBase was subject to risks of nondeterminism. However, he did seem to concede StreamBase might be just as deterministic as Coral8 with sufficiently careful coding.* In another example, Mark thinks subqueries are important, whereas Mike Stonebraker told me a while ago he doesn’t think they arise much in real life CEP.

*To a first approximation, I continue to doubt that determinism is more than a theoretical/edge-case issue. I don’t think many indeterminate, untestable programs are actually being written using any of these engines. On the other hand, it may be the case that programming effort to assure determinism varies significantly between different systems. For a rough analogy here, think of referential integrity. Few database application systems truly suffer from lack of integrity (but feel free to insert the MySQL snark of your choice). Even so, the effort needed to assure integrity can vary widely among different DBMS.

3. Mark positions StreamBase as having been developed with a limited-power query-specification GUI, with the SQL-based language coming only after the fact. By way of contrast, Coral8 was SQL-based all along. He attributes this to the different university research projects they are respectively based on, and asserts that most StreamBase customers he knows of still use the GUI rather than the language. This is central to his argument that StreamBase may be fine for simple queries, but has trouble with more complex ones.

Other highlights included:

• Coral8 is written in C++, while most of the competition uses Java. Mark hypothesizes that this may provide a performance advantage.
• Coral8 automatically handles locks, while in StreamBase you (sometimes?) need to code them specifically.
• The same goes for asynchronous persistence (used for high availability/failover). Coral8 boasts a flexible, no-programming-required configurable clustering capability.
• As part of the “we handle more complex stuff” theme, he thinks Coral8’s optimization is more intricate than StreamBase’s.
• Mark thinks Coral8’s homegrown portal is more flexible, with more user control, than competitors’ licensed tools.

The most confusing of the three is determinism. Apparently, you can run the same query twice against substantially the same event streams, but if the time stamps are a little different or you parallelize differently, you can get different answers. StreamBase suggests this is a problem for Coral8 but not StreamBase. Coral8 assures me they have never proven non-deterministic in a practical customer test, and furthermore have a theoretical example in which StreamBase is non-deterministic. Apama has “determinism” mentioned on one of its slides, although (by my choice) we focused on other stuff during yesterday’s briefing. All told, I’m still in the dark as to whether the determinism problem just arises in theoretical edge cases, whether it really occurs in significant production situations, or indeed whether it’s a noticeable problem at all.

Parallelism comes up primarily as a subpoint to determinism or performance. However, Coral8 is proud of its cluster manager. Apparently, a major way to parallelize is akin to range partitioning – e.g., price data for different securities gets routed to different processors. One also clusters for high availability/hot standby, of course. Coral8 claims as a distinguishing feature that all this parallelization can be just configured from a GUI rather than coded. What’s more, one processor can serve as a hot standby to multiple “range-partitioned” parallel active nodes.

Naturally, one of the top areas for competitive debate is performance. StreamBase claims to beat the others, including Coral8, by a factor of 10 or better on performance. Coral8, however, cites figures of 60-80 microsecond latency for its portion of a 1-2 millisecond total latency (partnered with Wombat), and I’m pretty sure StreamBase doesn’t claim to be 10 times as fast as that. And Apama, which claims to be involved in trading most asset classes for most major investment banks, claims never to have lost a customer benchmark.

So what’s going on here? Well, some of it is surely “My next release beats the pants off of the competition’s old release,” a common feature of competitive jockeying. And some of it no doubt depends on who’s system is most expertly tuned in which situation. Beyond that, it may not all be apples-to-apples. Indeed, Apama has a slide suggesting there are at least ten different dimensions to performance. And while that’s somewhat padded, at least three dimensions are real:

• Absolute latency (should be low).
• Total throughput (should be high).
• The complexity of the queries/patterns/filters being used when the first two factors are observed.

And then there’s the jockeying about customer lists. StreamBase doesn’t actually publish customer lists yet, leading to this blog post from Apama suggesting StreamBase has only two disclosed customers, one of them a no-name and the other a game company. Well, I poked around StreamBase’s web site some, and I found the no-name described as a “Goldman Sachs company.” I also found a reference to NASA as a customer. I also saw a claim that over half the biggest investment firms were customers, although that’s vastly less substantial than what Apama can claim (and name). Also, StreamBase is known to be active in the classified/intelligence market. Perhaps that particular blog post could use some light editing …

• Like StreamBase, they do complex event/stream processing in a SQL-based language. They fondly think they did this before StreamBase. They further think their language is currently richer than StreamBase’s. They’re not as optimistic about industry language standardization as Mike Stonebraker is.
• Like every other vendor I’ve talked with in this space, they’re proud of their app dev tools.
• They don’t have a government division. Hence, no classified/intelligence business to date. They also didn’t start out by attacking the financial services business. However, algorithmic trading is their hottest area now, largely via a partnership with an outfit called Wombat.
• Their other leading business is web clickstream analysis.
• They identify three further significant market areas, all via OEMs – RFID, security/network processing, and “generic infrastructure.” RFID is of course limited by general RFID adoption. But stay tuned for partnerships – i.e., Coral8 embeddings — with humongous app vendors. Security/network processing is, presumably, self-explanatory. “Generic infrastructure” is, for example, embedding into messaging middleware.
• Like Apama, they mention fraud detection as an application area. That makes sense, since it clearly has a real-time aspect.
• One cool app (at Sallie Mae, which seems to generally be a flagship customer) is to capture information about an abandoned web site visit and immediately send it to the call center apps, so as to better serve the customer who picked up the phone and dialed in frustration. Some of the application details mentioned sounded very nice indeed.
• They seem to handle XML natively. When I asked whether there was much XML that didn’t originate in relational databases, they assured me there was. One example is PFML, a Wall Street derivatives standard. They have a lovefest with IBM/DB2 in connection with their XML capabilities.

Adapters to Data Feeds

Your blog comment that adapters doesn’t seem like a key competitive differentiator is accurate, and since adapters are so straightforward to develop with StreamBase as part of a customer engagement, we’ve never found adapters to be a key competitive differentiator. The comment by a competitor that their advantage over StreamBase comes from their having developed more adapters suggests they cannot distinguish themselves based on the other functional capabilities that are important to customers. In reality, our speed/performance and scalability are orders of magnitude superior to competitors, as is the speed with which StreamBase applications are developed, deployed, and modified when business needs change. (If it were easy to develop applications with certain competitive systems, then one might assume they would make free evaluation versions of their product available for download from their websites!)

That being said, StreamBase offers adapters to a broad array of data feeds. Most of these are offered out-of-the-box by StreamBase, including the following:
* Financial Market Data: processes data from Reuters® RMDS™ and Reuters Triarch™
* TIBCO® Rendezvous™: converts Rendezvous message into StreamBase tuples and vice versa.
* StreamBase Adapter for JDBC: connects StreamBase to enterprise databases, allowing submission of SQL queries to external resources such as IBM® DB2™, Oracle®, Microsoft® SQLServer™, and Sybase®.
* StreamBase Adapter for JMS: integrates StreamBase with any JMS-compliant message bus.
* StreamBase Adapter for Microsoft Excel™: allows applications to publish data to Excel or read data from Excel.
* StreamBase CSV Adapters: allow applications to read data from, and write data to, comma-separated value (CSV) files.
* StreamBase SMTP adapter: taps into the IP stack on a running system to process live data, converts the IP packets into a TCP data stream, or reads IP packets from captured files.
* StreamBase XML Adapter: streams XML-formatted data records into and out of StreamBase applications

We also can connect to financial exchanges either using our own adapters or through a third-party partnership. Below you’ll find a listing of those.

The “below” part turned out to be

Additional connectivity includes most of the following:

EQUITIES

* NYSE – New York Stock Exchange
* AMEX – American Stock Exchange
* US Regional Exchanges – (e.g. Boston, Midwest, Pacific, Philadelphia, Cincinnati)
* NASDAQ
* Euronext Exchanges – Paris, Amsterdam, Brussels, Lisbon
* Borsa Italiana
* London Stock Exchange
* Deutsche Borse
* Swiss Exchange
* RTS – Russian Trading System Stock Exchange
* Athens Stock Exchange
* Virt-X
* Irish Stock Exchange
* Stockholm
* Oslo
* Copenhagen
* Helsinki
* Madrid
* Vienna
* Luxembourg
* Stuttgart (EUWAX)
* German Regional Markets – Berlin, Dusseldorf, Hamburg, Hannover, Munchen

DERIVATIVES

* EUREX
* IDEM – Italian Derivatives Market
* Euronext LIFFE – London International Financial Futures and Options Exchange
* LME – London Metal Exchange
* ICE – Intercontinental Exchange
* Euronext Exchanges – including AEX Options, MONEP, MATIF
* MEFF – Spanish Derivatives Market
* NYBOT – New York Board Of Trade – CSCE, FINEX, NYCE, NYFE
* CME – Chicago Mercantile Exchange
* NYMEX – New York Mercantile Exchange
* COMEX – New York Mercantile Exchange
* CBOT – Chicago Board of Trade
* KCBT – Kansas City Board of Trade
* ADEX – Athens Derivatives Exchange
* Euronext Exchanges – including AEX Options, MONEP, MATIF
* MACE – Mid-American Commodity Exchange
* OPRA – Options Price Reporting Authority
* CBOE – Chicago Board Options Exchange
* SIMEX – Singapore Mercantile Exchange
* SFE – Sydney Futures Exchange
* NZFE – New Zealand Futures Exchange
* RTS – Russian Trading System Stock Exchange
* MICEX – Moscow Interbank Currency Exchange
* MGE – Minneapolis Grain Exchange

FOREIGN EXCHANGE

* Dresdner Frankfurt
* Dresdner Singapore
* OHV
* Tullett Prebon
* Allied Irish Bank
* Barclays
* CMC
* ECB
* Geneva International Bank
* Merita Bank
* OKO Bank
* Sampo
* Saxo Bank
* Trinkaus
* UBS
* Composite
* Realtime Forex Geneva
* Rada Forex NY
* Khanani & Kallia
* Reems UAE
* HotSpot FXi
* EFX

I must confess to trouble reconciling this with what I thought was Progress’s claim, namely that StreamBase only had two adapters or so.