August 3, 2007

A deeper dive into Apama

My recent non-technical Apama briefing has now had a much more technical sequel, with charming founder and former Cambridge professor John Bates. He still didn’t fully open the kimono – trade secrets and all that — but here’s the essence of what’s going on.

Complex event/stream processing (CEP) is all about looking for many patterns at once. Reality – the stream(s) of data – is checked against these patterns for matches. In Apama, these patterns are kept in a kind of tree – they call it a hypertree — and John says the work to check them is only logarithmic in the number of patterns.

Since patterns commonly have multiple parts — and usually also take time to unfold — what really goes on is that partial matches are found, after which what’s being matched against is the REMAINDER of the pattern. Thus, there’s constant pruning and rebalancing of the tree. What’s more, a large fraction of all patterns – at least in the financial trading market — involve a short time window, which again creates a need for ongoing, rapid tree modification.

The basic development interface/paradigm to Apama is the rule/frame:

when pattern-is-matched, then do-action-1, do-action-2, …

An action can be a normal program operation step, or it can be to enter new, transformed information into the stream for other patterns to check against.

John asserts that, under the covers, Apama doesn’t look a lot like the classical models of rules engines or expert system shells, specifically RETE. That said, I suspect it’s fairer to view Apama’s approach as “improvement on RETE” rather than “unrelated to RETE.” If left-hand-sides of rules are arranged in a tree, and incoming facts modify the tree, there’s at least a whiff of RETE in the air.

Other highlights included:

Comments

4 Responses to “A deeper dive into Apama”

  1. Text Technologies»Blog Archive » More on text processing in CEP on August 3rd, 2007 9:21 pm

    [...] isn’t the only complex event/stream processing (CEP) vendor doing text processing. Progress Apama is as well. Stemming, fuzzy matching, and so on seem to happen all the time. But there’s also [...]

  2. DBMS2 — DataBase Management System Services » Blog Archive » Competitive claims in CEP on April 25th, 2008 12:07 am

    [...] mentioned on one of its slides, although (by my choice) we focused on other stuff during yesterday’s briefing. All told, I’m still in the dark as to whether the determinism problem just arises in [...]

  3. Infology.Ru » Blog Archive » Быстрый обзор технологий хранилищ данных on March 24th, 2009 12:27 pm

    [...] событий/потоков (CEP, ESP), таких как Coral8, StreamBase, или Progress Apama (то же самое). Я думаю, впереди нас ждёт некоторая [...]

  4. Memory-centric data management when locality matters | DBMS 2 : DataBase Management System Services on July 15th, 2012 9:13 pm

    [...] moving objects, something I heard directly from the CEP (Complex Event Processing) vendors in the 2007 – 2009 period as [...]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.