September 7, 2014

An idealized log management and analysis system — from whom?

I’ve talked with many companies recently that believe they are:

At best, I think such competitive claims are overwrought. Still, it’s a genuinely important subject and opportunity, so let’s consider what a great log management and analysis system might look like.

Much of this discussion could apply to machine-generated data in general. But right now I think more players are doing product management with an explicit conception either of log management or event-series analytics, so for this post I’ll share that focus too.

A short answer might be “Splunk, but with more analytic functionality and more scalable performance, at lower cost, plus numerous coupons for free pizza.” A more constructive and bottoms-up approach might start with: 

Further, there would be numerous styles of business intelligence interface, at least including:

And there would be good support for quick-turnaround, easily-operationalized predictive analytics, of the sort that’s fairly central to the visions for Kiji and Spark.

The data management part of that is particularly hard, in that:

My thoughts on strengths and weaknesses of some obvious log data management contenders start:

In the interest of length, I’ll omit discussion of smaller vendors, except to say that Platfora’s integrated-stack event series analytics story deserves attention, and I’m disappointed that I never hear about Sumo Logic. And I don’t know a lot about companies positioned as SIEM (Security Information and Event Management), especially now that SenSage has left the scene.

Comments

12 Responses to “An idealized log management and analysis system — from whom?”

  1. Örjan on September 7th, 2014 9:03 am

    Also Tibco loglogic http://www.tibco.com/products/event-processing/loglogic-for-machine-data

    I think it is based on splunk though… (havent looked at it)

  2. Rajesh Nair on September 7th, 2014 9:39 am

    What do you think about the ELK stack?
    http://www.elasticsearch.org/overview/

    -Raj

  3. Joy-Paul Tharakan on September 15th, 2014 12:54 pm

    You may like to consider Nexthink from the list of smaller vendors. http://www.nexthink.com/

  4. The WibiWeekly: How To Save Petabytes in Hadoop, Why Customer Service is Dominating Retail & More | Data Wins on September 23rd, 2014 7:57 pm

    […] Industry analyst Curt Monash's overview and evaluation of log management & analysis […]

  5. The WibiWeekly: How To Save Petabytes in Hadoop, Why Customer Service is Dominating Retail & More | WibiData on September 24th, 2014 1:38 pm

    […] Industry analyst Curt Monash's overview and evaluation of log management & analysis […]

  6. Some stuff on my mind, September 28, 2014 | DBMS 2 : DataBase Management System Services on September 28th, 2014 8:21 pm

    […] The ability to mix traditional tabular data, JSON, and log data. […]

  7. Streaming for Hadoop | DBMS 2 : DataBase Management System Services on October 5th, 2014 4:57 am

    […] This also all fits with the importance I place on log analysis. […]

  8. Simone on October 7th, 2014 9:13 am

    SenSage is now HawkEye AP owned by Hexis Cyber Solutions, a KEYW company.

    HawkEye AP is perfectly positioned to lead the up and coming Security Analytics market. HawkEye AP continues to extend its core Log Management capability, further extending its lead as the world’s most efficient way to collect, store, and analyze mass quantities of Event Data.

    HawkEye AP is designed as a complete solution for security analytics with a large scale data warehouse, collection routines to bring in everything from your IT infrastructure, and a built-in reporting module. No single construct means HawkEye AP has virtually unlimited scalability.

  9. Luca Candela on May 19th, 2015 12:33 pm

    Your description describes very faithfully Treasure Data except for a couple bullet points that are not well developed yet.

  10. Rob Burton on October 30th, 2017 6:50 am

    There is one open source centralized log management software out there which provides scalable performance, it’s called NXLog: https://nxlog.co/products/nxlog-community-edition – it scales well event to thousands or ten thousands of servers while still providing high-performance. And it is a multi platform tool, so it can collect logs from Windwos, Linux, Android, etc. It definitely should be added to the list above.

  11. Maya Goodwin on December 22nd, 2019 9:22 pm

    Trying to find effective online promotion that isn’t full of crap? Sorry to bug you on your contact form but actually that’s exactly where I wanted to make my point. We can send your advertising message to sites through their contact pages just like you’re getting this ad right now. You can specify targets by keyword or just do bulk blasts to websites in any country you choose. So let’s assume you need to push through an ad to all the contractors in the USA, we’ll scrape websites for just those and post your ad text to them. As long as you’re advertising a product or service that’s relevant to that type of business then your business will get awesome results!

    Send a quick note to muhammad2435tay@gmail.com to get info and prices

  12. Kristan Ledesma on February 14th, 2020 11:12 pm

    Looking for powerful advertising that delivers real results? Sorry to bug you on your contact form but actually that was kinda the point. We can send your promotional copy to websites via their contact forms just like you’re getting this note right now. You can target by keyword or just fire off bulk blasts to sites in the country of your choice. So let’s assume you want to push through a message to all the plumbing companies in the US, we’ll scrape websites for just those and post your ad text to them. Providing you’re advertising something that’s relevant to that business category then you’ll receive awesome results!

    Write a quick note to poppy8542bro@gmail.com to find out more info and pricing

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.