August 30th, 2007 Curt Monash
The ratio of Philip Howard plaudits about Calpont to shipping products from Calpont has now doubled. Yet it also has remained the same. This is because it is a countably infinite number, namely a quotient whose denominator is zero. Last time around, he seemed to like their hardware strategy. This time around, he seems to like their lack of a hardware strategy. Be that as it may, the previously discussed nature of Calpont’s website hasn’t changed — one page, content-free, and misleading even so.
Oh, and it appears he broke the embargo on Paraccel. Bad Philip. Spank him, Kim.
Posted in Calpont, Data warehouse appliances, Data warehousing, Portability, transparency, and plug-compatibility, Relational database management systems | No Comments »
August 16th, 2007 Curt Monash
In the literal sense, that is. While the details on what I wrote about this a few weeks ago* are still embargoed, I’m at liberty to drop a few more hints.
*Please also see DATAllegro CEO Stuart Frost’s two comments added today to that thread.
DATAllegro systems these days basically consist of Dell servers talking to EMC disk arrays, with Cisco Infiniband to provide fast inter-server communication without significant CPU load. Well, if you decrease the number of Dell servers per EMC box, and increase the number of disks per EMC box, you can slash your per-terabyte price (possibly at the cost of lowering performance).
Read the rest of this entry »
Posted in DATAllegro, Data warehouse appliances, Data warehousing, Database compression, Relational database management systems | No Comments »
August 15th, 2007 Curt Monash
Robin Bloor is one of the best analysts around — which doesn’t say much about his eponymous firm, since he no longer works there, but I digress. Even so, he evidently got snookered by a Truviso spokesperson, as evidenced by this article.
Apparently, Truviso convinced him that other CEP firms execute one query at a time, while Truviso executes a bunch of queries at once. Well, the latter part of that is presumably true, but it’s hardly the big differentiatior for Truviso Robin would have one believe. That’s what everybody else — StreamBase, Coral8, Progress Apama, et al. — do too. I wouldn’t be surprised if Truviso had a somewhat different architecture for doing it (each vendor describes its approach in rather different language), or even if this were a particular focus and strongpoint of theirs. But fundamentally, all the CEP vendors are doing the same thing.
Posted in Complex event/stream processing (CEP), Memory-centric data management, Truviso | No Comments »
August 13th, 2007 Curt Monash
Coral8 at the time of a recent product release stated that it was improving the predictability of its queries. While this may sound like it has something to do with determinism, it doesn’t. Rather, it’s a matter of making what actually happens as a query result be more in line with what one would think will happen when one reads the query.
Coral8 CTO Mark Tsimelzon goes on to note:
But remember, we are really talking about a corner case — highly complex queries involving loops. We only had a couple of customers who were occasionally hitting queries that complex. The beauty of our SQL-based language is that the vast majority of queries, perhaps 99%, are very easy to understand, and their behavior is exactly what you’d expect based on your SQL experience.
Posted in Complex event/stream processing (CEP), Coral8, Memory-centric data management | No Comments »
August 12th, 2007 Curt Monash
The highest-profile applications for complex event/stream processing are probably the ones that require super-low latency, especially in financial trading. However, as I already noted in writing about StreamBase and Truviso, there are plenty of other CEP apps with less extreme latency requirements.
Commonly, these are data reduction apps – i.e., there’s a gushing stream of inputs, and the CEP engine filters and “enhances” it, so that only a small, modified subset is sent forward. In other cases, disk-based systems could do the job perfectly well from a performance standpoint, but the pattern matching and filtering requirements are just a better fit for the CEP paradigm.
Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Coral8, Hierarchies, networks, graphs, and trees, IBM and DB2, Memory-centric data management, Native XML, StreamBase | No Comments »
August 10th, 2007 Curt Monash
Complex event/stream processing vendors compete fiercely on the basis of low latency, down to the single-digit number of milliseconds, or even sub-millisecond levels. A question naturally springs to mind: When does this extreme low latency matter?
I think I’ve come up with a concise yet fairly accurate answer: Super-low latency matters when the application includes direct competition against a similarly fast opponent. The best example is automated stock trading – if you can exploit a market inefficiency 1 millisecond before your competition, you make money.
Other examples might arise in network security or battlefield systems, but I don’t know of any specific real-life cases. Instead, other applications for complex event/stream processing tend to be content with latencies that are easier to achieve. E.g., 100 milliseconds (1/10 of second) is likely to be plenty fast enough.
Keep getting great research about data management and related technologies. Get a FREE subscription by RSS/Atom or e-mail!
Posted in Complex event/stream processing (CEP), Memory-centric data management | No Comments »
August 10th, 2007 Curt Monash
Besides talking about what Coral8 and StreamBase (and other CEP vendors) have in common, Mark Tsimelzon and I talked quite a bit about what he sees as some of the important differences. There were a lot, of course, but three in particular stood out.
1. Mark believes Coral8 has significantly lower latency than StreamBase. E.g., the Wombat/Coral8 combo achieves sub-millisecond latency, with Coral8 itself consuming less than a tenth of that. The best comparable figures from StreamBase that I currently know of are almost an order of magnitude slower.
Top-end speed aside, Mark believes that Coral8 is fundamentally better suited for complex queries and pattern recognition, while StreamBase works well with simpler queries. For example, his other performance claims notwithstanding, he concedes that StreamBase is at least comparable to Coral8 in its throughput for huge numbers of simple queries. (The number he mentioned was ½ million queries/second.) Indeed, while we barely talked about customer/marketing issues, Mark asserts that the companies’ respective customer bases reflect this complex/simple distinction.*
Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Coral8, Memory-centric data management, Progress, Apama, and DataDirect, StreamBase | 4 Comments »
August 10th, 2007 Curt Monash
Last week, I complained that my first briefing with Coral8 wasn’t very technical. Wednesday I had a call with Mark Tsimelzon, CTO and founder of Coral8, and he made up for that in spades. In this post I’ll cover some of his general comments. Others will touch on more Coral8-specific topics, and his view of the Coral8/StreamBase comparison.
As Mark describes it, the big difference between a DBMS – even an in-memory DBMS – and a complex event processing engine is this: CEP engines do instantaneous incremental processing. He commonly refers to this as registering queries and operators for incremental evaluation. For example, suppose you need to maintain the sum of some data stream over the past 10 minutes. Then each second (or other short unit of time), the system adds in all the values that arrived in the past second, and subtracts all those that arrived 600-601 seconds ago. Voila! The sum is incrementally updated.
Now, rolling sums may not sound very interesting – but where you have rolling sums, you trivially also have rolling averages (just divide the sum by the count) and rolling standard deviations (same idea, with some squares and square roots mixed in). Those, of course, are primitives in Coral8 too. Ditto rolling maxima and minima. Ditto rolling joins (which are updated a lot like materialized views).
Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Coral8, Memory-centric data management | 1 Comment »
August 3rd, 2007 Curt Monash
For the most part, the vendors I talk with in complex event/stream processing like and speak well of each other (most of the exceptions seem to involve StreamBase). Even so, there are a lot of interesting competitive claims and counterclaims in this market. Prior posts and comment threads have covered Apama/StreamBase jousting on the subjects of who has more business and how many financial data feeds StreamBase supports. Other areas that generate interesting sparks are performance, parallelism, and determinism. Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Coral8, Memory-centric data management, Progress, Apama, and DataDirect, StreamBase | 1 Comment »
August 3rd, 2007 Curt Monash
My recent non-technical Apama briefing has now had a much more technical sequel, with charming founder and former Cambridge professor John Bates. He still didn’t fully open the kimono – trade secrets and all that — but here’s the essence of what’s going on.
Complex event/stream processing (CEP) is all about looking for many patterns at once. Reality – the stream(s) of data – is checked against these patterns for matches. In Apama, these patterns are kept in a kind of tree – they call it a hypertree — and John says the work to check them is only logarithmic in the number of patterns.
Since patterns commonly have multiple parts — and usually also take time to unfold — what really goes on is that partial matches are found, after which what’s being matched against is the REMAINDER of the pattern. Thus, there’s constant pruning and rebalancing of the tree. What’s more, a large fraction of all patterns – at least in the financial trading market — involve a short time window, which again creates a need for ongoing, rapid tree modification. Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Memory-centric data management, Progress, Apama, and DataDirect | 2 Comments »
August 3rd, 2007 Curt Monash
Complex event/stream processing vendor Coral8 raised its hand and offered a briefing – non-technical, alas, but at least it was a start. Here are some of the highlights: Read the rest of this entry »
Posted in Complex event/stream processing (CEP), Coral8, Hierarchies, networks, graphs, and trees, Memory-centric data management, Native XML, StreamBase | No Comments »