37 Ways To Get More From Analytics
I posted several stages of my thinking in connection with a February presentation on how to buy an analytic DBMS. The whole process seemed like a success, with good input early on, and at least one new client directly attracted by the uploaded slide presentation. So now I’m trying the same idea again, starting at an even earlier stage of the process.
I’m going to be speaking this September at six of the seven installments of Netezza’s 2009 traveling regional user conference, namely those in London, Milan, and the United States. (Edited for schedule changes.) The topic is going to be something like “N Ways to Get More From Analytics”, for N a decent-sized two-digit integer. The talk is meant to be more conceptual, upbeat, rah-rah, and/or inspirational than is my usual style, at the cost of perhaps being less complete, detailed, or carefully organized. Right now I’m at the point of sharing an initial list of ideas, and throwing open the question: What did I leave out?
The initial list is:
Analyzing more data
- De-anonymize transactions (e.g., retail store loyalty programs) — yes, this is mainly a 1990s idea, but not everybody has implemented it who could
- Capture information about tire-kickers (e.g., online sign-ups of various sorts)
- Make differentiated offers to test response (e.g., multiple versions of web pages)
- Text that you already have (e.g., incoming email)
- Text on the web
- Sensors in your equipment
- Factory floor
- GPS
- Utility networks
- Temperature/environmental?
- Sensors in your products — RFID
- Web logs
- Network event logs
- Keeping history you used to throw out — archive it if you can’t use it yet
Better decision support
- Adopt consistent enterprise KPIs
- Abolish or go beyond consistent enterprise KPIs
- Use new alerting technologies
- Data exploration & visualization UIs
- In-memory analytics
- Lower latency
- Show data to your customers
- Show data to your suppliers
- Geospatial data
Get and exploit better performance in data mining, statistics and machine learning
- On more columns of data than before
- Through more cycles of analysis than before
- To do better graph/network analytics
- Because of in-DBMS data mining
- Because of MapReduce
Different kinds of engines
- Better DBMS (many kinds)
- In-memory cache or OLAP
- Rules engines
- CEP
Applications
- Anti-fraud
- Other security
- Analytics-based price setting
- Interaction customization (e.g., personalized web pages, but it goes further)
- Quality diagnostics
- Production process improvement (the ghost of Wolfgang Deming waves Hi)
- Business process improvement (opportunity is not just on the factory floor)
- Mini trading floor (e.g., for energy supplies)
Comments
20 Responses to “37 Ways To Get More From Analytics”
Leave a Reply
the only thing that may be interesting to add here is some brief commentary about the kinds of people who are doing this work – just had a talk with hellerstein about a paper he put out with some of the greenplum guys (it’s very good) – looks a bit at the changing role of the analysts presiding over the mountains of data…sort of a, “so now you’ve got these loads of data, but what kinds of folks are able to most effectively bring analytics into the dbase at massive scale” (seems like traditional db brains are taking a backseat to the analysts within the enterprise, at least at first blush)…just my two cents…
oh, and if there’s gonna be the requisite “N Ways to..” then don’t you also have to crank out a bunch of Matrix slides? 😉
Dave,
I don’t think I’ll put an “N” in the title, as that will keep me from making last-minute changes to the talk. 🙂
But — given that my “ways” are overlapping — you’re suggesting that I have a new group for new kinds of users of the analytics (customers, suppliers, call-center workers, whatever) — that’s an excellent idea!
Thanks,
CAM
Curt,
A few things that I don’t see called out explicitly, although some are half-covered:
** Move heaven and earth to make ad-hoc queries run faster. **
– When we implemented Netezza it created an explosion in the kinds of analysis we could do and allowed us to just try things without knowing if they would be useful.
** Give your financial and stats users a very flat view of the data. **
– (Could be phrased better) These users are used to seeing very wide and flat representations of data. They don’t “think in SQL” like you do. Presenting your data in this way can result in a big increase in usage.
** Bring prospect data into the warehouse. **
– This can take a number of forms. Buy in population/census data, integrate click streams from the web, integrate sales force automation lead data (very noisy). Basically find any data you can about prospects before they become customers.
– Anecdote: I’ve seen sales projections (more than once) that require selling to significantly more than 100% of the addressable market.
Joe
I agree with the previous comments. I think analytics has been held back by the elitist assumption that it is just for a few clever people in head office. I want to see analytic capabilities used across the organization, and rendered as services for external customers and partners.
OK. Looks like there will be a broad category like “Serve more people” 😀
We’ll be at the roadshows also. Would love to catch up on in-memory analytics vs. in-database analytics. Looking forward to seeing you on the road.
@Dave: would you have a link to the Hellerstein paper?
Kind regards.
Curt,
Any chance you’ll be at VLDB 2009 in Lyon this August?
Jerome,
Wasn’t planning on it. Ditto XLDB. I view business travel as something that somebody ELSE should pay for. 😉
CAM
You know, Lyon to London is pretty easy — Since you’re in the UK early September, be a shame not to kill 2 birds with one stone 🙂
Jerome,
It seems as if the events are three weeks apart. So I’m not sure I’m seeing the travel synergies … 😉
CAM
Curt,
How about the shift from ETL to ELT and the shrinking (and in some cases elimination of) the nightly batch window?
Peter,
Hmm. I wonder whether I should have a whole category for data integration …
Thanks,
CAM
Definitely a good point made earlier about enabling experimentation over large data sets. This would fall under “Get and exploit better performance” although it’s not limited to statistics uses.
I would rephrase “Give your financial and stats users a very flat view of the data” to “customize views of data to the user” or “get to know how users think and provide them with views of data that they understand”. This is definitely not new in the DBMS world but is actually fairly new in the big data world. Many users of big data previously had to patch together many pieces of the data puzzle rather than getting a view that is customized to their needs.
Since I approach this from the view of a statistician, the first thing that comes to my mind is a cross cutting category: “Lower the technical barrier for implementing (and modifying) analytics”. That would be stuff ranging from improved ability to experiment, better views of data, different and better engines, centralized repositories of analytics logic, etc.
The paper @dave referred to is up at http://databeta.wordpress.com/2009/03/20/mad-skills/
Thanks for the pointer Joe!
How about the infusing of advanced analytics into operational applications to raise the bar throughout the enterprise? That makes analytics accessible and delivers value repeatedly.
@michele I think you’re over simplifying the situation. I usually warn people that it’s very easy to advocate for cutting edge analytics everywhere, it seems like something that couldn’t possibly be wrong. But it’s not so easy to make that work in practice.
Infusing advanced analytics is a slow process and with good reason. There are some well tested self-correcting analytics out there, but still quite often advanced analytics seem to either start producing bad results or be used in a completely wrong way, the moment you’ve decided they’re stable and pull the statistician off to another project.
Before infusing advanced analytics, any organization needs a clear path (process, people, senior management support) to maintaining a clear understanding of the meaning, monitoring effectiveness and maintaining them.
[…] I hoped, there were some very helpful responses to my post listing ways to improve analytic effectiveness. Here’s a second draft incorporating them. Comments continue to be very welcome. I need to […]
http://matt.wordpress.com/2010/01/20/dol-sot-bi-bim-bap/