January 25, 2016

Kafka and more

In a companion introduction to Kafka post, I observed that Kafka at its core is remarkably simple. Confluent offers a marchitecture diagram that illustrates what else is on offer, about which I’ll note:

Kafka offers little in the way of analytic data transformation and the like. Hence, it’s commonly used with companion products. 

If we recognize that Jay’s interests are obviously streaming-centric, this distinction maps pretty well to the three use cases Cloudera recently called out.

Complicating this discussion further is Confluent 2.1, which is expected late this quarter. Confluent 2.1 will include, among other things, a stream processing layer that works differently from any of the alternatives I cited, in that:

The library will do joins, aggregations and so on, and while relying on core Kafka for information about process health and the like. Jay sees this as more of a competitor to Storm in operational use cases than to Spark Streaming in analytic ones.

We didn’t discuss other Confluent 2.1 features much, and frankly they all sounded to me like items from the “You mean you didn’t have that already??” list any young product has.

Related links


3 Responses to “Kafka and more”

  1. Notes from a long trip, July 19, 2016 | DBMS 2 : DataBase Management System Services on July 22nd, 2016 9:05 am

    […] Back in January I wrote that the leaders were mainly Spark Streaming, followed by Storm. […]

  2. Xi Liu on July 24th, 2016 2:14 am

    Do you know about distributedlog : http://distributedlog.io/ ? It would be interesting to compare Kafka with it.

  3. Introduction to data Artisans and Flink | DBMS 2 : DataBase Management System Services on August 24th, 2016 2:54 pm

    […] surveyed Spark Streaming, Storm et al. in […]

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.