• Most independent CEP vendors have some kind of application story in the capital markets vertical, such as packaged applications, ISV partners with packaged applications, application frameworks, and so on.
• CEP vendors offer lots of connectors to specific financial industry price/quote/trade feeds, as well as the usual other kinds of database connectivity (SQL, XML, etc.)
• Aleri/Coral8 (separately and now together) like to call attention to their business intelligence/analytics offerings. Analytics is front-and-center on Truviso’s web site too, not that Truviso does much to call attention to itself, period. (Roman Bukary once said he’d outline Truviso’s new strategy to me in 6-8 weeks or so … it’s now 14 months and counting.)

So far as I can tell, the areas of applications and analytics are fairly uncontroversial. Different CEP vendors have implemented different kinds of things, no doubt focusing on those they thought they would find easiest to build and then sell. But these seem to be choices in business execution, not in core technical philosophy.

In CEP application development, however, real philosophical differences do seem to arise. There are at least three different CEP application development paradigms:

• DML (Data Manipulation Language) extensions to handle time windows, etc., which are then embodied into a conventional application development stack. Microsoft exemplifies an extreme form of this strategy, but most vendors offer it at least as an option.
• Visual event-oriented programming language. This is StreamBase’s claim to fame; StreamBase says that 95% of its customers do 100% of their development in its Eclipse-based visual tool, which truly creates executable programs without any kind of code generation step. (Other “this isn’t just a toy” buzzwords StreamBase offered were “modularity, ” “parametrizability,” and, best of all, “visual debugger.”)
• Rules engine. When Progress Apama first told me how its preferred tool (I think a precursor of what is now Apama Event Modeler) worked, it sounded a lot like a RETE-based expert system shell. The Apama folks assured me that this really wasn’t RETE, and I see that there’s definitely more than just a rules language in Apama MonitorScript. Still, the paradigm seems to basically be that you write a bunch of rules, which are then executed by a suitable engine.

Even more fundamental, however, is the question of event-driven programming. That’s a term which first was popular in the 1990s, signifying programs that — rather than being wholly procedural — listened for and responded to events, often in user interfaces. More generally, event-driven programming is inherent in almost any kind of loosely-coupled architecture, be it client-server, service-oriented, or whatever. But to hear some CEP proponents tell it, all that isn’t event-driven enough.

In particular, StreamBase recommended to me Gregor Hohpe’s paper Programming Without a Call Stack – Event-driven Architectures, which argues that if there’s any concept of procedure or method invocation at all, the whole thing is too “command-and-control” (Boo!!) and insufficiently event-driven to be well-suited for CEP. Rather, everything should be done on a fine-grained publish-subscribe basis. It’s an interesting argument. Usually, the problem with extremist programming paradigms is that the thing you write has to interface with the rest of the world, and by the time it accommodates itself to them, the paradigm is violated anyway. But if the essence of the paradigm is loose coupling to begin with, maybe that pitfall can for once be avoided.

]]> http://www.dbms2.com/2009/05/21/notes-on-cep-application-development/feed/ 5 http://www.dbms2.com/2009/05/21/notes-on-cep-performance/ http://www.dbms2.com/2009/05/21/notes-on-cep-performance/#comments Thu, 21 May 2009 08:20:36 +0000 Curt Monash http://www.dbms2.com/?p=787 I’ve been talking to CEP vendors on and off for a few years. So what I hear about performance is fairly patchwork. On the other hand, maybe 1-2+ year-old figures of per-core performance are still meaningful today. After all, Moore’s Law is being reflected more in core count than per-core performance, and it seems CEP vendors’ development efforts haven’t necessarily been concentrated on raw engine speed.

So anyway, what do you guys have to add to the following observations?

• Super-low-latency financial services industry tasks are often “embarrassingly parallel.” Thus, near-linear scale-out is common.
• That said, good parallelism seems fairly new in CEP engines (of course, CEP engines are fairly new themselves — for all I know, some have been parallel since inception).
• I’ve heard claims of up to 400,000 messages/second/core for simple queries or patterns.
• I’ve heard claims of 70,000 messages/core for not-so-simple queries or patterns, and probably higher than that depending on what the meaning of “simple” is.
• IBM just disclosed >15,000 messages/core on a pretty low-powered processor.
• I’ve heard that Coral8, Apama, and StreamBase rarely lost deals due to performance or throughput problems. I’ve heard that the same is not as true of Aleri.
• StreamBase proudly says it’s been fully multithreaded since academic research-project days.  For Apama multithreading is evidently a more recent feature. But does it matter much?

• StreamBase once planned to conquer the world, making an impact as big as database management’s. Now it has retreated into niche markets.
• Progress Software, a decent-sized company, put a large fraction of its energy into Apama. Little has happened outside the financial service sector.
• Coral8 has some great-sounding ideas. But Coral8 now has merged into Aleri, basically a financial-markets specialist.
• Mike Franklin says some ambitious things on behalf of Truviso, but I haven’t noticed much traction there either.

CEP’s penetration outside of its classical markets isn’t quite zero. Customers include several transportation companies (various vendors), Sallie Mae (Coral8), a game vendor or two (StreamBase, if I recall correctly), Verizon (Aleri, I think), and more. But I just wrote that list from memory — based mainly on not-so-recent deals — and a quick tour of the vendors’ web sites hasn’t turned up much I overlooked. (Truviso does have a recent deal with Technorati, but that’s not exactly a blue chip customer these days.)

So far as I can tell, this is a new version of a repeated story. A clever alternative to relational DBMS was invented. It proved superior in some specific applications and vertical markets. It failed to achieve much broader adoption. Initial high hopes got dashed, companies failed to grow rapidly, and shareholders grew tired.

So where will things go from here? My best guesses include:

• The financial trading market isn’t going away for CEP. Super-low-latency is really needed there.
• As much as I love the idea of CEP-infused BI, it will be adopted only at the rate broader-based BI vendors can support.
• A few niches will generate some business for CEP in data reduction. Leading candidates are the ones where there’s been a little traction to date — national intelligence, transportation, web analytics, and so on.
• Sadly, that’s about it.

]]> http://www.dbms2.com/2009/03/09/independent-cep-vendors-continue-to-flounder/feed/ 6 http://www.dbms2.com/2008/02/01/dan-weinreb-on-objectstore/ http://www.dbms2.com/2008/02/01/dan-weinreb-on-objectstore/#comments Fri, 01 Feb 2008 12:09:45 +0000 Curt Monash http://www.dbms2.com/2008/02/01/dan-weinreb-on-objectstore/ Dan Weinreb was one of the key techies at Object Design, the company that made the object-oriented database management system ObjectStore. (Object Design later merger into Excelon, which was eventually sold to Progress, which has deemphasized but still supports ObjectStore.) Recently he wrote a pair of long and fascinating articles about Object Design, ObjectStore, and OODBMS, the first of which makes the case that “object-oriented database management systems succeeded.”

That’s not a ridiculous interpretation, as he explains, but it’s kind of a glass-half-full one. By Dan’s own analysis, ObjectStore did OK but not great in the CAD/CAM/CAE and CASE markets for a while, but eventually fell back due to:

1. Lack of critical mass.
2. Moore’s Law making performance advantages less important.
3. Fewer performance benefits in Java than in C++.

A later attempt to front-end RDBMS by object-caching also fell short, perhaps due to competition from object-relational mapping vendors such as Tangosol (now part of Oracle).

So far as I can tell, the other OODBMS pioneers — Versant, Servio, et al. — did even less well than ObjectStore. Versant did sell a few OLTP accounts as sort of an early Intersystems Cache’, but that’s ancient history.

I recommend the articles highly. Dan was one of the most articulate techies I knew way back in Symbolics’ heyday, as well as being a very straight shooter. He’s still going strong.

Please subscribe to our feed!

Technorati Tags: ObjectStore, Progress Software, object-oriented database, OODBMS

But for most applications at any enterprise – and for all applications at most enterprises – super high-end DBMS aren’t required. There are relatively few applications that wouldn’t run perfectly well on PostgreSQL or EnterpriseDB today. Ingres and Progress OpenEdge aren’t far behind (they’re a little lacking in datatype support). Ditto Intersystems Cache’, although the nonrelational architecture will be off-putting to many. And to varying degrees, you can also do fine with MySQL, Pervasive PSQL, MaxDB, or a variety of other products – or for that matter with the cheap or free crippled versions of Oracle, SQL Server, DB2, and Informix.

What’s more, these mid-range database management systems can have significant advantages over their high-end brethren. The biggest is often price, for licenses and maintenance alike. Beyond that, they can be much easier to administer then their more complex counterparts. For example, Progress OpenEdge and Informix SE have long been reseller favorites, in large part because they can be installed at small businesses and locations that lack technical staff, and rarely if ever require DBA attention. Programming and hardware costs can sometimes be lower as well.

And what these mid-range DBMS don’t do today, they likely will do soon. In the 1990s, Microsoft SQL Server was the mid-range entry threatening to disrupt the market. But it’s grown up quite nicely. EnterpriseDB is equal or superior in every way I can think of to Oracle7, a few security certifications perhaps excepted. (They’d probably argue the release number in that claim should be 1 or 2 higher, but I’d have to compare their multimedia support to what I recall of Oracle 8.1.5 before I agreed.)

Will these mid-range database management systems truly “disrupt” the DBMS market, as many open source advocates hope? Or will they be largely co-opted into the oligopoly, as Microsoft SQL Server was? That’s a discussion for another time. For now, please just keep your mind open to DBMS alternatives – the high-end approach is not always the best.

EDIT: For a contrary view, please see my follow-up post making the opposite case.

Please subscribe to our feed!

1. Mark believes Coral8 has significantly lower latency than StreamBase. E.g., the Wombat/Coral8 combo achieves sub-millisecond latency, with Coral8 itself consuming less than a tenth of that. The best comparable figures from StreamBase that I currently know of are almost an order of magnitude slower.

Top-end speed aside, Mark believes that Coral8 is fundamentally better suited for complex queries and pattern recognition, while StreamBase works well with simpler queries. For example, his other performance claims notwithstanding, he concedes that StreamBase is at least comparable to Coral8 in its throughput for huge numbers of simple queries. (The number he mentioned was ½ million queries/second.) Indeed, while we barely talked about customer/marketing issues, Mark asserts that the companies’ respective customer bases reflect this complex/simple distinction.*

*I don’t think I can judge that claim overall yet. However, it seems consistent with one particular data point. The intelligence market – which StreamBase seems to dominate – probably does feature very high volumes of relatively simple filters.

2. Mark thinks Coral8 has a much richer set of language primitives than StreamBase. In particular, he calls attention to Coral8’s primitives for synchronizing multiple data streams. That particular example seems to be core of his claim that StreamBase was subject to risks of nondeterminism. However, he did seem to concede StreamBase might be just as deterministic as Coral8 with sufficiently careful coding.* In another example, Mark thinks subqueries are important, whereas Mike Stonebraker told me a while ago he doesn’t think they arise much in real life CEP.

*To a first approximation, I continue to doubt that determinism is more than a theoretical/edge-case issue. I don’t think many indeterminate, untestable programs are actually being written using any of these engines. On the other hand, it may be the case that programming effort to assure determinism varies significantly between different systems. For a rough analogy here, think of referential integrity. Few database application systems truly suffer from lack of integrity (but feel free to insert the MySQL snark of your choice). Even so, the effort needed to assure integrity can vary widely among different DBMS.

3. Mark positions StreamBase as having been developed with a limited-power query-specification GUI, with the SQL-based language coming only after the fact. By way of contrast, Coral8 was SQL-based all along. He attributes this to the different university research projects they are respectively based on, and asserts that most StreamBase customers he knows of still use the GUI rather than the language. This is central to his argument that StreamBase may be fine for simple queries, but has trouble with more complex ones.

Other highlights included:

• Coral8 is written in C++, while most of the competition uses Java. Mark hypothesizes that this may provide a performance advantage.
• Coral8 automatically handles locks, while in StreamBase you (sometimes?) need to code them specifically.
• The same goes for asynchronous persistence (used for high availability/failover). Coral8 boasts a flexible, no-programming-required configurable clustering capability.
• As part of the “we handle more complex stuff” theme, he thinks Coral8’s optimization is more intricate than StreamBase’s.
• Mark thinks Coral8’s homegrown portal is more flexible, with more user control, than competitors’ licensed tools.

The most confusing of the three is determinism. Apparently, you can run the same query twice against substantially the same event streams, but if the time stamps are a little different or you parallelize differently, you can get different answers. StreamBase suggests this is a problem for Coral8 but not StreamBase. Coral8 assures me they have never proven non-deterministic in a practical customer test, and furthermore have a theoretical example in which StreamBase is non-deterministic. Apama has “determinism” mentioned on one of its slides, although (by my choice) we focused on other stuff during yesterday’s briefing. All told, I’m still in the dark as to whether the determinism problem just arises in theoretical edge cases, whether it really occurs in significant production situations, or indeed whether it’s a noticeable problem at all.

Parallelism comes up primarily as a subpoint to determinism or performance. However, Coral8 is proud of its cluster manager. Apparently, a major way to parallelize is akin to range partitioning – e.g., price data for different securities gets routed to different processors. One also clusters for high availability/hot standby, of course. Coral8 claims as a distinguishing feature that all this parallelization can be just configured from a GUI rather than coded. What’s more, one processor can serve as a hot standby to multiple “range-partitioned” parallel active nodes.

Naturally, one of the top areas for competitive debate is performance. StreamBase claims to beat the others, including Coral8, by a factor of 10 or better on performance. Coral8, however, cites figures of 60-80 microsecond latency for its portion of a 1-2 millisecond total latency (partnered with Wombat), and I’m pretty sure StreamBase doesn’t claim to be 10 times as fast as that. And Apama, which claims to be involved in trading most asset classes for most major investment banks, claims never to have lost a customer benchmark.

So what’s going on here? Well, some of it is surely “My next release beats the pants off of the competition’s old release,” a common feature of competitive jockeying. And some of it no doubt depends on who’s system is most expertly tuned in which situation. Beyond that, it may not all be apples-to-apples. Indeed, Apama has a slide suggesting there are at least ten different dimensions to performance. And while that’s somewhat padded, at least three dimensions are real:

• Absolute latency (should be low).
• Total throughput (should be high).
• The complexity of the queries/patterns/filters being used when the first two factors are observed.

And then there’s the jockeying about customer lists. StreamBase doesn’t actually publish customer lists yet, leading to this blog post from Apama suggesting StreamBase has only two disclosed customers, one of them a no-name and the other a game company. Well, I poked around StreamBase’s web site some, and I found the no-name described as a “Goldman Sachs company.” I also found a reference to NASA as a customer. I also saw a claim that over half the biggest investment firms were customers, although that’s vastly less substantial than what Apama can claim (and name). Also, StreamBase is known to be active in the classified/intelligence market. Perhaps that particular blog post could use some light editing …

Complex event/stream processing (CEP) is all about looking for many patterns at once. Reality – the stream(s) of data – is checked against these patterns for matches. In Apama, these patterns are kept in a kind of tree – they call it a hypertree — and John says the work to check them is only logarithmic in the number of patterns.

Since patterns commonly have multiple parts — and usually also take time to unfold — what really goes on is that partial matches are found, after which what’s being matched against is the REMAINDER of the pattern. Thus, there’s constant pruning and rebalancing of the tree. What’s more, a large fraction of all patterns – at least in the financial trading market — involve a short time window, which again creates a need for ongoing, rapid tree modification.

The basic development interface/paradigm to Apama is the rule/frame:

when pattern-is-matched, then do-action-1, do-action-2, …

An action can be a normal program operation step, or it can be to enter new, transformed information into the stream for other patterns to check against.

John asserts that, under the covers, Apama doesn’t look a lot like the classical models of rules engines or expert system shells, specifically RETE. That said, I suspect it’s fairer to view Apama’s approach as “improvement on RETE” rather than “unrelated to RETE.” If left-hand-sides of rules are arranged in a tree, and incoming facts modify the tree, there’s at least a whiff of RETE in the air.

Other highlights included:

• Apama is of course designed to tightly integrate the filtering/enhancement of data with the actions based on the data processing. John fondly thinks this is a significant performance benefit, by eliminating the overhead that rival systems have in their quasi-SQL calls. I wasn’t convinced.
• John also thinks that Apama can put a quasi-SQL interface on their system if the market demands it. I was convinced.
• Apama’s core programming language MonitorScript is OO-like but not really OO (i.e., it lacks inheritance). In the 1990s we called this “object-based” or “event-driven.” He calls it “event-oriented.” You can also write in Java; there’s a JVM built-in.
• A geospatial datatype (which is to say, Cartesian coordinates) has been built in from the beginning. With Apama’s strong focus on financial trading, however, it’s rarely or never been used in a deployed system. There are finally some prototypes now being built in telecom (location-based services) and logistics (trucking gets mentioned a lot in discussions with CEP vendors these days).
• Besides the developer studio/IDE, there’s a power-user facility called Scenario Modeler. It looks somewhat more formidable than what BI power users normally face, but in the same league. Certainly anybody who can be a SAS programmer should be able to cope with it as well.

With that as backdrop, where is all this compatibility/portability/transparency stuff going to lead? Potentially, pretty far. The technical barriers aren’t great. Given that, issues of cost, market dynamics, and so on suggest there should eventually be just as much database porting as there is porting between UNIX-derived hardware and operating systems. I.e., a lot.

A DBMS, first and foremost, is a big language interpreter. Therefore, if you want DBMS transparency, most of what you have to do is run the language, without major performance hits. In principle, this shouldn’t be hard, because SQL dialects are pretty similar and pretty simple. And in practice it’s not too hard either.

How do I know it’s not too hard in practice? Because DataDirect (now owned by Progress) has been doing it for years! DataDirect’s main business is translating ODBC and JDBC to native SQL, and it does a bang-up job. Performance hits are usually trivial, and sometimes there even are performance gains from the translation. True, there’s more to full stored-procedure-language compatibility than is commonly found in DataDirect drivers. But EnterpriseDB has been doing a much fuller job of Oracle compatibility without too much evident difficulty. And there’s the Ants example as well.

Of course, it’s not that simple. But much of the non-simplicity lies in this: If you port from one DBMS to another, you may need to totally retune your system to get comparable performance to what you had before. And that’s a costly hassle. Except – tuning is getting ever more automatic and easier. And as hardware gets cheaper, tuning is getting somewhat less important. And specifically on the analytic side, MPP so dominates SMP in price-performance that a port is likely to gain you a lot of throughput with very little tuning effort at all.

“Wait a moment!”, one might say. “The big DBMS companies make huge investments in their products. Unless that effort is utterly wasted, surely they do many things better than some upstart does.” But then, one might also have used the same reasoning to suggest that Linux could never outpace Unix. Or that Unix could never supplant VMS. Or that VMS could never compete with MVS …

Reasons that all of these arguments fail include that lots of the development effort at the big, established leaders:

• Is to compensate for the products’ old and now somewhat obsolete architectures.
• Is to compensate for the products’ giant-hairball natures.
• Is for high-end reliability, availability, security, and/or absolute-scalability features that most users don’t need.
• Achieves extreme generality (portability, language support, etc.) that most users don’t need.
  Is for other specialized functionality that most users don’t need.
• Suffers from “Mythical Man-Month” inefficiencies.

Industry leaders have been successfully challenged many times in the history of DBMS. As I’ve noted many times in this blog, I think it’s happening again. And transparency/portability/compatibility could well wind up being part – albeit just a part – of that general trend.

Adapters to Data Feeds

Your blog comment that adapters doesn’t seem like a key competitive differentiator is accurate, and since adapters are so straightforward to develop with StreamBase as part of a customer engagement, we’ve never found adapters to be a key competitive differentiator. The comment by a competitor that their advantage over StreamBase comes from their having developed more adapters suggests they cannot distinguish themselves based on the other functional capabilities that are important to customers. In reality, our speed/performance and scalability are orders of magnitude superior to competitors, as is the speed with which StreamBase applications are developed, deployed, and modified when business needs change. (If it were easy to develop applications with certain competitive systems, then one might assume they would make free evaluation versions of their product available for download from their websites!)

That being said, StreamBase offers adapters to a broad array of data feeds. Most of these are offered out-of-the-box by StreamBase, including the following:
* Financial Market Data: processes data from Reuters® RMDS™ and Reuters Triarch™
* TIBCO® Rendezvous™: converts Rendezvous message into StreamBase tuples and vice versa.
* StreamBase Adapter for JDBC: connects StreamBase to enterprise databases, allowing submission of SQL queries to external resources such as IBM® DB2™, Oracle®, Microsoft® SQLServer™, and Sybase®.
* StreamBase Adapter for JMS: integrates StreamBase with any JMS-compliant message bus.
* StreamBase Adapter for Microsoft Excel™: allows applications to publish data to Excel or read data from Excel.
* StreamBase CSV Adapters: allow applications to read data from, and write data to, comma-separated value (CSV) files.
* StreamBase SMTP adapter: taps into the IP stack on a running system to process live data, converts the IP packets into a TCP data stream, or reads IP packets from captured files.
* StreamBase XML Adapter: streams XML-formatted data records into and out of StreamBase applications

We also can connect to financial exchanges either using our own adapters or through a third-party partnership. Below you’ll find a listing of those.

The “below” part turned out to be

Additional connectivity includes most of the following:

EQUITIES

* NYSE – New York Stock Exchange
* AMEX – American Stock Exchange
* US Regional Exchanges – (e.g. Boston, Midwest, Pacific, Philadelphia, Cincinnati)
* NASDAQ
* Euronext Exchanges – Paris, Amsterdam, Brussels, Lisbon
* Borsa Italiana
* London Stock Exchange
* Deutsche Borse
* Swiss Exchange
* RTS – Russian Trading System Stock Exchange
* Athens Stock Exchange
* Virt-X
* Irish Stock Exchange
* Stockholm
* Oslo
* Copenhagen
* Helsinki
* Madrid
* Vienna
* Luxembourg
* Stuttgart (EUWAX)
* German Regional Markets – Berlin, Dusseldorf, Hamburg, Hannover, Munchen

DERIVATIVES

* EUREX
* IDEM – Italian Derivatives Market
* Euronext LIFFE – London International Financial Futures and Options Exchange
* LME – London Metal Exchange
* ICE – Intercontinental Exchange
* Euronext Exchanges – including AEX Options, MONEP, MATIF
* MEFF – Spanish Derivatives Market
* NYBOT – New York Board Of Trade – CSCE, FINEX, NYCE, NYFE
* CME – Chicago Mercantile Exchange
* NYMEX – New York Mercantile Exchange
* COMEX – New York Mercantile Exchange
* CBOT – Chicago Board of Trade
* KCBT – Kansas City Board of Trade
* ADEX – Athens Derivatives Exchange
* Euronext Exchanges – including AEX Options, MONEP, MATIF
* MACE – Mid-American Commodity Exchange
* OPRA – Options Price Reporting Authority
* CBOE – Chicago Board Options Exchange
* SIMEX – Singapore Mercantile Exchange
* SFE – Sydney Futures Exchange
* NZFE – New Zealand Futures Exchange
* RTS – Russian Trading System Stock Exchange
* MICEX – Moscow Interbank Currency Exchange
* MGE – Minneapolis Grain Exchange

FOREIGN EXCHANGE

* Dresdner Frankfurt
* Dresdner Singapore
* OHV
* Tullett Prebon
* Allied Irish Bank
* Barclays
* CMC
* ECB
* Geneva International Bank
* Merita Bank
* OKO Bank
* Sampo
* Saxo Bank
* Trinkaus
* UBS
* Composite
* Realtime Forex Geneva
* Rada Forex NY
* Khanani & Kallia
* Reems UAE
* HotSpot FXi
* EFX

I must confess to trouble reconciling this with what I thought was Progress’s claim, namely that StreamBase only had two adapters or so.