Monitoring the Apache2 log file in the current event processing applications we are working on provides only 3 precursor events of a total set of nearly 350 event and precursor events (none of which are a part of the same event stream).

Our team has not decided (yet) to go with R, BTW. We don’t need to “learn statistics” as much as we need to find the right architecture, especially since none of the current products calling themselves “CEP engines” do the trick.

Actually, I find a lot new in what we have been doing, because I am not focused solely on streaming rule-based processing and systems (as you seem to be). Processing event streams with rules is an area which I was starting working on over 20 years ago in the event processing (network and security) space.

It is, however, encouraging to read your posts recently as you “catch up” and learn about detection theory and the limitations of rule-based stream processing to detecting complex (and changing) situations and event.

I am encouraged by how much your have learned about event processing over the past two to three years. Keep up the good work!

Yours sincerely, Tim

Perhaps you can share with us exactly what you are doing at Credit Suisse.

My experience has been that folks in financial services are all excited about a stream processing rules engine.

What do you actually “detect” in real time?

Thanks!

In certain fields, though, we do see a lot of hand coded, event driven BI. I question why, if there is really demand beyond those fields, we do not see more custom solutions cropping up to fill the need now.

In the longer term, I imagine that as various products make developing event-driven software easier, demand will arise in areas where we don’t see it now. But that is long term and certainly not something on which a vendor can make money today.

Tim, if you don’t see anything new in CEP/EP, then I am surprised to find you still talking about it. One would think that a better use of your time would be to find something that you think is new, and talk about that. Perhaps in monitoring your Apache logs and learning about statistics and R, you will find this new thing that you seek.

Event processing and complex event processing are not dependent on “event driven architecture”. Events can be “fired and forgotten” or they can be requested by the CEP/EP server.

In other words, there is nothing about CEP/EP that makes it dependent on EDA. Event processing is as “old as the hills”, and there is nothing new from a technical perspective in any of the products on the market that use CEP as a marketing term.

What is new?

Please tell us !!

For years, the BI and data warehousing industry have been responding to customer interest in getting ever-faster access to data. But so far, the speed of enhancement in relational data warehousing technology has been enough to meet this need.

I’m of the opinion customers shouldn’t be satisfied, and should demand a revolution in BI. But there are a lot of needed revolutions in BI, and my preferred one in better alerting and metrics management is by no means necessarily one that will soon occur.

As for my views on the independent companies — there are few enough that matter that we can just look at them one by one. Aleri — apparently focused on financial services (but I have a briefing scheduled with them soon for more detail). Coral8 — sold to Aleri. StreamBase — retreated into a couple of niches. And so on.

For that matter, will the event-driven computing market ever grow beyond the industries that use it now? Is the entire event-driven computing market big enough to support a large company in the long term?

It’s not just CEP that faces these questions.

Were I a potential CEP investor, I would wonder how you came to believe that big players will dominate this field. Historically speaking, niche industries, especially deep niches, have been exactly where medium sized technology players can survive and produce good returns on a relatively modest investment.

I’m not saying you’re wrong. Just pointing out that there are several valid sides to the story.

I question how many application areas there are where known patterns have enough complexity that standard database languages (SQL, MDX, text search., etc.) are inappropriate to express them. (Assuming some standard temporal extensions if needed.)

You are exactly right and I am really pleased to read your posts. In capital markets, where there does not appear to have been a generic event processsing platform available, the term “CEP” has been adopted because the platforms marketing themselves as “CEP” have made progress. The adoption is not related to “CEP” per se but software marketing their “wares” as “CEP”.

Outside of processing marketdata, there have been folks willing to test these platforms, and a “few bowling pins” have wobbled (as you say); but a few wobbling bowling pins are not “main street” by any leap in imagination.

Recently we evaluated so called “CEP engines” to monitor events on a high performance web site with a real eye for using a so called “CEP product”. After spending much time and effort, we picked Zabbix for processing network-centric events. Zabbix works great and is free, open source, extensible, and does all the basic time window, rule-based processing the site needed. The “CEP offerings” took too much work because there have been optimized for processing market data, not network “event soup” data.

The piece we are current missing is future event prediction (outages, server load problems, etc.) and none of the self-described CEP engines are close to this stage of event processing (at this stage of the game). We are considering integrating the R statistical package; but have not made a decision yet.

Cheers and Best Regards, Tim

PS: I was a bit surprised to read some negative comments about your posts over at David Luckham’s forum. Hang in there!

I think CEP is one of those markets that can be usefully viewed through the old Geoffrey Moore lenses — early adoption, chasm, “bowling alley”, “tornado”, “Main Street”, etc. The bowling alley stage appeared to be getting started — but additional pins, while wobbling, aren’t so far as I can tell falling over in droves.

That’s part of what I mean by “floundering”.

The other part is that it seems, if and when CEP regains momentum outside its niche markets, the beneficiaries will be large, diversified, usually DBMS-selling software companies. I used the word “independent” in the title of the post for a reason.