November 19, 2015

The questionably named Cloudera Navigator Optimizer

I only have mixed success at getting my clients to reach out to me for messaging advice when they’re introducing something new. Cloudera Navigator Optimizer, which is being announced along with Cloudera 5.5, is one of my failures in that respect; I heard about it for the first time Tuesday afternoon. I hate the name. I hate some of the slides I saw. But I do like one part of the messaging, namely the statement that this is about “refactoring” queries.

All messaging quibbles aside, I think the Cloudera Navigator Optimizer story is actually pretty interesting, and perhaps not just to users of SQL-on-Hadoop technologies such as Hive (which I guess I’d put in that category for simplicity) or Impala. As I understand Cloudera Navigator Optimizer:

It grows out of Xplain.io, which started with the intention of being a general workload optimizer for Hadoop and wound up with this beta announcement of a tuning adviser for analytic SQL.

Right now, the Cloudera Navigator Optimizer service is:

Naturally, Cloudera’s intention — perhaps as early as at first general availability — is for the output to start including something that’s more like automation, e.g. hints for the Impala optimizer.

As Anupam Singh describes it, there are basically four kinds of problems that Cloudera Navigator Optimizer can help with:

Actually, all four of these cases can involve materializing tables so that they don’t need to keep being in part or whole recreated.

In essence, then, this is a way to add in more query pipelining than the underlying data store automagically provides on its own. And that seems to me like a very good idea to try. The whole thing might be worth trying out at least once, even if your analytic RDBMS installation has nothing to do with SQL at all.

Comments

4 Responses to “The questionably named Cloudera Navigator Optimizer”

  1. David Gruzman on November 21st, 2015 3:48 am

    It sounds like materialized views in Oracle terms (or indexed views in MS SQL terms). Is it comparison valid?

  2. David Gruzman on November 21st, 2015 3:51 pm

    I probably misunderstood in some extent. It is much more, but technically benefits described above are provided in “ideal” world of RDBMS by materialized views.

  3. Curt Monash on November 22nd, 2015 2:34 am

    It’s just an adviser about materialization, so I don’t think the comparison is good at all.

    Obviously I didn’t explain well enough. :(

  4. Early Week Readings: Nov 23, 2015 | TeqBiz on November 25th, 2015 8:01 pm

    […] Curt Monash’s blog about Xplain.io’s (a company I helped jumpstart and also served as VP of Product Management/marketing until we sold it to Cloudera) technology after it was released as “Cloudera Navigator Optimizer” (DBMS2). […]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.