Data exploration vs. data visualization
I’ve tended to conflate data exploration and data visualization, and I’m far from alone in doing so. But a recent Economist article is a useful reminder that they aren’t exactly the same thing.
The article makes the same conflation, but while reading it I noticed something interesting. The concrete examples cited are of clever consultants who crafted innovative data visualizations on the fly, to make conclusions patently apparent to even mathematically-challenged decision-makers. That kind of thing is important, and has been going on for over 140 years.*
*Yes, I’m trotting out the Florence Nightingale example again. I continue to be in awe of her.
What worries me is the article’s suggestion that the best data visualizations are done by visualization experts, as ways of making information apparent to other people. For as long as data visualization relies on hotshot visual-design experts doing one-off projects, its impact on enterprises overall will remain extremely limited. In other words, to the extent it is incorrect to conflate data visualization and data exploration, data visualization will remain a fringe technology.
To be fair, a primary decision support/business intelligence usage cycle has always been — where by “always” I mean “for at least the past 35+ years” —
- Data exploration. Power user uses technology to find something interesting.
- “Look what I found!” Power user then shows a report, chart, or other summary/representation to colleagues.
So to the extent modern interactive data exploration/visualization technology fits that paradigm, great. But to the extent that visualization experts are somehow integral to the technology’s use, it will remain stuck on the analytic fringe.
Comments
5 Responses to “Data exploration vs. data visualization”
Leave a Reply
What about tools like Trendly?
http://trendly.com/
The idea is to use sort sort of secret sauce (statistical processes obviously but which I have no clue; it’s by the guys who made DabbleDB so it obviously isn’t simply easy naive stuff) to analyze your traffic stats and provide a visualization of the exploration done behind the scenes by the algorithms.
Is this the kind of product that you figure will be necessary to bring data exploration/visualization our of the fringe?
I too would be worried about any assertion that “the best data visualizations are done by visualization experts, as ways of making information apparent to other people”.
I would contend that the best visualizations are done where Subject Matter Expert’s knowledge intersect with Visualization Design Experts; without the input of BOTH of these, any resultant visualization would be lacking in one of the core ingredients.
I also believe that there is confusion about exactly what defines a “Visualization” – in my book it is not just a collection of line/bar-graphs thrown together in a pretty manner (that is simply lipstick on a pig), IMHO it is about integrating large amounts of complex data, (often through models or calculations that the end user shouldn’t have to worry about) and then allowing the user to interact visually in multiple dimensions (visually and data-wise) in a very free-form manner to identify data that warrants further investigation.
It’s not unlike drilling for oil; Visualization can tell you where to look more deeply (and where to apply other seriously intelligent tools), but at the end of the day there’s no getting away from the fact that you still need some subject matter/domain knowledge to understand the true meaning and value of the data.
Visualization done properly will reduce the time taken to achieve this by up to an order of magnitude.
Hey, thanks for the link to my post! I also caught that Economist article and found it fascinating… there’s also this NYTimes one from 2 or 3 months ago: http://www.nytimes.com/2010/01/03/business/03unboxed.html?em …agreed that there is still a ton of potential in sifting out the meaningful bits of data. I wonder what industries would be interesting to bring into this; novelist? City planners?
Agree, data visualization is just the mean to representing finding from reporting or analytical applications. Graphs (not a word as sexy as data visualization) have been around forever, what has changed in recent year is the technology that offers more flexible and cool looking ways to show findings.
One thing remains constant though: a good analyst will be able to share his findings simply, elegantly and in a concise manner; a bad one will just inundate you with tons of data a graphs (good or bad looking).
Disclaimer: I work for Juice Analytics who does consider itself to be in the “hotshot visual-design experts” category.
You are exactly correct when you say “to the extent it is incorrect to conflate data visualization and data exploration, data visualization will remain a fringe technology.” People think visually; which is especially important when dealing with complex data. Without proper visualization, many important findings are lost.
The problem is that most people are never trained how to properly communicate visually. They’re best references are the defaults that come in Excel (yuk), or what they saw in somebody’s Power Point deck (yukkier).
As in any field, the innovation, quality and even perhaps the correctness of a product is certainly enhanced by folks who make it their profession to focus on that particular discipline. However, we (the visualization pundits) cannot simply require our insertion in order to accomplish simple tasks.
That’s why Juice Analytics produces many free tools in order to support the lay-visualizer. For instance, our white paper “Creating Dashboards People Love to Use” (http://www.juiceanalytics.com/registration/dashboard_design/) has gotten accolades on the openness and usefulness to the non-hotshot visual-design experts. Hopefully your readers will find it useful as well.