Notes on CEP application development
While performance may not be all that great a source of CEP competitive differentiation, event processing vendors find plenty of other bases for technological competition, including application development, analytics, packaged applications, and data integration. In particular:
- Most independent CEP vendors have some kind of application story in the capital markets vertical, such as packaged applications, ISV partners with packaged applications, application frameworks, and so on.
- CEP vendors offer lots of connectors to specific financial industry price/quote/trade feeds, as well as the usual other kinds of database connectivity (SQL, XML, etc.)
- Aleri/Coral8 (separately and now together) like to call attention to their business intelligence/analytics offerings. Analytics is front-and-center on Truviso’s web site too, not that Truviso does much to call attention to itself, period. (Roman Bukary once said he’d outline Truviso’s new strategy to me in 6-8 weeks or so … it’s now 14 months and counting.)
So far as I can tell, the areas of applications and analytics are fairly uncontroversial. Different CEP vendors have implemented different kinds of things, no doubt focusing on those they thought they would find easiest to build and then sell. But these seem to be choices in business execution, not in core technical philosophy.
In CEP application development, however, real philosophical differences do seem to arise. There are at least three different CEP application development paradigms:
- DML (Data Manipulation Language) extensions to handle time windows, etc., which are then embodied into a conventional application development stack. Microsoft exemplifies an extreme form of this strategy, but most vendors offer it at least as an option.
- Visual event-oriented programming language. This is StreamBase’s claim to fame; StreamBase says that 95% of its customers do 100% of their development in its Eclipse-based visual tool, which truly creates executable programs without any kind of code generation step. (Other “this isn’t just a toy” buzzwords StreamBase offered were “modularity, ” “parametrizability,” and, best of all, “visual debugger.”)
- Rules engine. When Progress Apama first told me how its preferred tool (I think a precursor of what is now Apama Event Modeler) worked, it sounded a lot like a RETE-based expert system shell. The Apama folks assured me that this really wasn’t RETE, and I see that there’s definitely more than just a rules language in Apama MonitorScript. Still, the paradigm seems to basically be that you write a bunch of rules, which are then executed by a suitable engine.
Even more fundamental, however, is the question of event-driven programming. That’s a term which first was popular in the 1990s, signifying programs that — rather than being wholly procedural — listened for and responded to events, often in user interfaces. More generally, event-driven programming is inherent in almost any kind of loosely-coupled architecture, be it client-server, service-oriented, or whatever. But to hear some CEP proponents tell it, all that isn’t event-driven enough.
In particular, StreamBase recommended to me Gregor Hohpe’s paper Programming Without a Call Stack – Event-driven Architectures, which argues that if there’s any concept of procedure or method invocation at all, the whole thing is too “command-and-control” (Boo!!) and insufficiently event-driven to be well-suited for CEP. Rather, everything should be done on a fine-grained publish-subscribe basis. It’s an interesting argument. Usually, the problem with extremist programming paradigms is that the thing you write has to interface with the rest of the world, and by the time it accommodates itself to them, the paradigm is violated anyway. But if the essence of the paradigm is loose coupling to begin with, maybe that pitfall can for once be avoided.
Comments
5 Responses to “Notes on CEP application development”
Leave a Reply
The use of Rete engines for complex event processing is a contraversial topic. Tibco’s engine is built on Rete. SAP Research has published a couple of papers on the subject in the last year or two, and the topic has been rattling around the academic community for several years now. My own view is that Rete engines are a useful tool in the CEP developer’s armoury, but are better suited to the upper reaches of an event processing network, close to the analytics and automated business processes, rather than close to ‘raw’ event observation.
Microsoft’s use of their monadic LINQ technology makes huge sense on the MS platform where LINQ has already found very broad acceptance and is well-supported by ‘syntactic sugar’ in C#, F# and VB.NET. There may be support for managed C++ as well. LINQ allows MS to support CEP development in a natural fashion directly within mainstream GPLs rather than having to provide specialised DSLs and without having to modify those GPLs in any way.
Thanks for good article! I think that most of products, I mean tools for speeding up programming, have no tools for visual constructing a model of application.
Actually, you can argue that CEP tools exploit all 3 of “data manipulation” (a.k.a. continuous query) languages, visual event-oriented programming, and rules (whether Rete-based, inferencing, or whatever).
As Charles said, TIBCO’s BusinessEvents CEP tool utilizes Rete for rules (Rete = a pattern matching algorithm, hence its applicability to general CEP should be obvious regardless of its usage in “conventional” rule engines). But TIBCO also does the continuous query thing, and uses drag n drop state models as a type of “visual event oriented programming”. Other vendors’ ESP and CEP tools also provide combinations of these 3 mechanisms (although possibly not yet Rete, unless you add Drools 5 to the mix).
The main philosophical aspect with “CEP” programming is the desire to process events as they occur, for whatever event patterns they impact. Truly “event driven” rather than “store and process in silos”…
Cheers
Paul,
I’ll admit to knowing much less about TIBCO’s CEP than other CEO offering. That’s because nobody at the company seems too concerned about changing this state of affairs. 😉
CAM
I see that only 10%-20% of leads that I talk to have performance requirements that we in the CEP world would think of as “high performance”. Most of these problems can also be split on multiple independent servers, leaving only a few % which actually need monster performance on a single server or a cluster of servers.