If I had my way, the business intelligence part of investigative analytics — i.e. , the class of business intelligence tools exemplified by QlikView and Tableau — would continue to be called “data exploration”. Exploration what’s actually going on, and it also carries connotations of the “fun” that users report having with the products. By way of contrast, I don’t know what “data discovery” means; the problem these tools solve is that the data has been insufficiently explored, not that it hasn’t been discovered at all. Still “data discovery” seems to be the term that’s winning.
Confusingly, the Teradata Aster library of functions is now called “Discovery” as well, although thankfully without the “data” modifier. Further marketing uses of the term “discovery” will surely follow.
Enough terminology. What sets exploration/discovery business intelligence tools apart? I think these products have two essential kinds of feature:
- Query modification.
- Query result revisualization.*
Here’s what I mean.
*I’d wanted to call this re-presentation. But that would have been … pun-ishing.
The canonical form of query modification is:
- There’s a scatter plot or other graphical data visualization.
- You select a rectangular area on the graph.
- A new visualization is drawn.
That capability is much more useful in systems that allow you to change how the data is visualized, both:
- Before you select a subset of the results (so you can choose which visualization is easiest to select from).
- After you’ve made the selection (it would be silly to stay in a monthly bar chart if you’ve just selected a single month).
Other forms of query modification, such as faceted drill-down or parameterization, don’t depend as heavily on flexible revisualization. Perhaps not coincidentally, they’ve been around longer in some form or other than have the QlikView/Tableau/Spotfire kinds of interfaces. But at today’s leading edge, query modification and query result revisualization are joined at the hip.
What else is important for these tools?
- Good UI design, of course.
- Speed — split seconds matter.
- Most of the same features that matter for business intelligence tools with other kinds of UI.
Please note that speed is a necessary condition for exploratory BI, not a sufficient one; a limited UI that responds really fast is still a limited UI.
As for how the speed is achieved — three consistent themes are columnar storage, compression, and RAM. Beyond that, the details vary significantly from product to product, and I won’t try to generalize at this time.
- The importance of data exploration flexibility (July, 2012)
- QlikView architecture (June, 2010)
- A cool QlikView feature that isn’t particularly tied to data exploration (November, 2011)
- Endeca’s underlying technology (April, 2011)