May 13, 2009

IBM System S Streams, aka InfoSphere Streams, aka stream processing, aka “please don’t call it CEP”

IBM has hastily announced System S Streams, a product that was supposed to be called InfoSphere Streams and introduced only in 2010. Apparently, the rush is because senior management wanted to talk about it later this week, and perhaps also because it was implicitly baked into some of IBM’s advertising already. Scrambling ensued. Even so, Jeff Jones and team got to me fast, and briefed me — fairly non-technically, unfortunately, but otherwise how I like it, namely on a harmless embargo and without any NDAs. That’s more than can be said for my clients at Microsoft, who also introduced CEP this week, but I digress …

*Indeed, as I draft this post-Celtics-game, the embargo is already expired.

Marketing aside, IBM System S/InfoSphere Streams is indeed a CEP/stream processing engine + language (with an Eclipse-based development environment). Apparently, IBM’s thinks InfoSphere Streams (if that’s what it winds up being renamed to) is or will be differentiated from other CEP packages in:

IBM Streams seems to be a six-year-old project, with about 15 installations, about half of them true sales. The original and main customer appears to be the US government, with allied governments also in the picture. This usage involves a lot of text/voice/video/whatever analysis, but the technology for that is government-developed; IBM’s own complex datatype technology is one of the features left out to achieve the earlier-than-planned product release. Besides governments, it seems there have been actual InfoSphere Streams sales to universities.

Besides those true sales of System S Streams, there are prototypes and — in a new-to-me phrase — “First of a Kind” projects, which I gather are IP-based services engagements. One is with TD Bank Financial — that’s the one on Blue Gene — handling 5 million messages per second, which is an order of magnitude or so higher than any figure I’ve ever heard from StreamBase, Progress Apama, Aleri/Coral8, or Truviso. Latency is in the 150 microsecond range. IBM believes it is indeed close to a couple of true sales to financial services firms, presumably for super-low-latency algorithmic trading.

As is usually the case for CEP vendors, the financial services market seems to be the only one that cares about super-low latency. Everything else IBM talked about seemed to be in the area of data reduction, although IBM likes to think of that as identifying which data matches certain patterns and then transforming it accordingly. Examples mentioned include:

As I’ve previously noted, the independent CEP vendors aren’t all that active in data reduction, Aleri/Coral8 (somewhat) and StreamBase’s intelligence-community efforts excepted. I’m not in a position to discuss other software generalists’ CEP efforts at the moment, but let’s just say I think data reduction is a generally under-served market, with lots of different niches to fill. IBM System S/InfoSphere Streams was well worth bringing to market.

IBM sent a couple of PDFs with more detail on applications and architecture. I can’t find them online, and I think I have permission to post them, so here they are(Edit: Jeff Jones subsequently sent over an official IBM white paper link.)

IBM has its own SQL-based stream processing language called SPADE, with an Eclipse-based IDE. SPADE is not StreamSQL-compatible, although some prototype work suggests IBM could support StreamSQL if necessary. The overall IBM InfoSphere Streams product delivery roadmap is, and I quote IBM’s final slide:

In the first (did I mention it was rushed?) release, System S/InfoSphere Streams supports Intel architectures and Red Hat Linux only.


3 Responses to “IBM System S Streams, aka InfoSphere Streams, aka stream processing, aka “please don’t call it CEP””

  1. Microsoft announced CEP this week too | DBMS2 -- DataBase Management System Services on May 13th, 2009 11:58 pm

    […] it is more than coincidence that IBM rushed out its own announcement of an immature CEP technology — due to be more mature in a […]

  2. Followup on IBM System S/InfoSphere Streams | DBMS2 -- DataBase Management System Services on May 18th, 2009 2:46 pm

    […] posting about IBM’s System S/InfoSphere Streams CEP offering, I sent three follow-up questions over to Jeff Jones.  It seems simplest to just post the Q&A […]

  3. InfoSphere Streams Is A Game Changer « Market Strategies for IT Suppliers on May 22nd, 2009 7:50 pm

    […] seen skepticism about this from people I respect like Curt Monash (here and elsewhere), Neil Raden (here), and Peter Thomas (here), and it’s not surprising – […]

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.