Big Data hype?
A reporter wrote in to ask whether investor interest in “Big Data” was justified or hype. (More precisely, that’s how I reinterpreted his questions. 🙂 ) His examples were Splunk’s IPO, Teradata’s stock price increase, and Birst’s financing. In a nutshell:
- My comments, lightly edited, are in plain text below.
- Further thoughts are in italics.
- Of course I also linked him to my post “Big Data” has jumped the shark.
- Overall, my responses boil down to “Of course there’s some hype.”
1. A great example of hype is that anybody is calling Birst a “Big Data” or “Big Data analytics” company. If anything, Birst is a “little data” analytics company that claims, as a differentiating feature, that it can handle ordinary-sized data sets as well.
When I checked Birst’s website, “Big Data” was nowhere to be found. On the other hand, the term was all over its press pitch for the financing.
2. The great growth in database sizes is both caused and balanced out by Moore’s Law. The net effect is healthy but not enormous growth in the overall data management and analytics markets.
I’ve made versions of that point many times before.
3. Incumbent data and analytic technology vendors such as Oracle, IBM, and Microsoft are vulnerable, but are competing very hard. Favorable exits have ensued for companies such Netezza, DATAllegro, Vertica, and Aster Data.
The connection between those two points is that the big companies will hold a lot of share, but part of how they’ll hold it is through acquisitions. For example, IBM, Microsoft, HP, Teradata, and Greenplum all bought newish analytic RDBMS vendors, at an aggregate cost of several billion dollars. And SAP bought Sybase.
But while there have been billions of dollars in fairly recent analytics-related acquisitions, the pace of acquisition would have to accelerate much further yet to justify current valuations.
Upon reflection, I may have overestimated the acquisition/IPO total-value-created ratio somewhat. Even so, what’s the last enterprise technology vendor to create huge investor value by going public, continuing to prosper, and so on? Red Hat and Autonomy may be as good as it gets. VMware isn’t really an example, because of its ownership structure.
4. I’m worried that people may be overestimating the business benefit of accurate analytics, great though that value truly is. For example, it’s not plausible that all enterprises in the world use better analytics to all improve their respective market shares.
Yes, it’s great to be an arms dealer to all sides. But “Big Data” technology is just another chapter in the ever-growing importance of IT.
Comments
14 Responses to “Big Data hype?”
Leave a Reply
Database and related vendors are constantly calling themselves a “leading” force of next recurring hype in order to stay relevant, sell more to existing accounts and steal new accounts from competitor on becoming suddenly a “sexy” vendor
“For example, it’s not plausible that all enterprises in the world use better analytics to all improve their respective market shares.”
Business analytics is only a zero sum game if everyone’s equally as good at creating value. In practice it comes down to change management and, as everyone knows, shifting an organisation’s culture is no easy task. Business analytics is
It’s not plausible, but nor is it likely. Everyone will benefit, but those that do the best will lead. For everyone else, the acceptable minimum average level of performance will need to increase through being smarter.
In aggregate, the net gain will likely be enormous. The relative gain will largely depend on who is better at execution.
Evan,
You seem to be saying that games are zero-sum only if all the players are equally good, which doesn’t make any sense.
You go on to say that not all the players are equally good, which is of course true.
You further wave your hands and say things won’t really be zero-sum, with which I agree, but you say nothing to support your claim of “enormous”.
What am I missing?
CAM
Fair call – I can only blame my poor response on trying to respond faster than I should have! Now that I’ve got a bit more time, here’s what I was trying to say.
When you look purely at the stock of investment / market cap at any point in time, I agree with you. From both a market share and a market cap perspective, there’s only a finite amount of consumption / money to be invested during any single round. If all players are equally good, market share and investments will be equally across all participants – every participant’s gain is another’s loss. From a relative point of view, things don’t really change much – some people get better, some people lag. Overall, everything evens out; there’s only so much money. So, the leaders take the pot.
Where I was *definitely* confusing was that I was trying to say that it’s still a point of competitive differentiation and to underestimate the influence it can have is exceedingly dangerous. While the pool of money doesn’t change, it’s a point of leverage to move the leaders further away from the average. So, it’s dangerous to underestimate the impact it might have if you *don’t* do anything about it.
The flows over time are where it makes the difference and where I think things get interesting. To be clear, I’m shifting from considering an entity to considering the market here. Accuracy and efficiency drive productivity – they magnify return on investment. Operational analytics helps drives economies of scale and scope in internal production; technology drives economies of scale and common competencies drive economies of scope.
IMHO, saying it’s part of the on-going importance of IT is both true and over-simplistic. It’s no different to saying that it’s part of the ongoing influence of technological capital – we might as well link it back to Ford’s manufacturing lines. The big difference is that we’re at an inflection point. On one hand, our ability to apply (and manage) highly complex decisioning rules through statistics has advanced due to falling computation costs (EC2 is a prime example). On the other, the ongoing digitisation of business has passed a threshold in terms of the breadth and depth of data we have access to.
We now have the ability to automate processes *and* advanced insight / decisioning rules as well access to the structured and unstructured data we need to drive those insights and decisions. It’s impossible to argue that this is simply a minor change – what we’re doing would have been (generally) commercially impossible ten years ago. The CIA were doing it 20 years ago (based on what their ex-CIO told me), but that’s the CIA. In my mind, disagreeing is no different to arguing that the Apple II was just a fancy abacus and simply represented an ongoing improvement in calculators.
I believe that over time, this is going to help overcome the stagnant productivity growth we’ve seen over the last decade. And, that’s what’s going to drive aggregate net gain – the market as a whole will rise.
I don’t think the upper limits of this leverage are entirely clear yet. It’s hard to call – it’s still very much early days. In the long-run, it’s a positive-sum game – aggregate improvements in efficiency and accuracy drive productivity and economic gains which in turn drive further growth. So, while relative market shares might not change (as they still have to add up to 100%), the absolute market cap does.
Regarding my use of the word “enormous”, what can I say? The best I can do to say that anecdotally, when the organisations I work with successfully execute, they tend to have returns on investment that range from around 2:1 to 40:1, fully capitalised and realised. Those that don’t, obviously, don’t. NPV and IRR are better measures, but they need to take into account the opportunity cost of the investments that were sacrificed. From my perspective, that’s significantly better than I’m getting on either the stock market on in real-estate …
To flip it around, if I may, what do you mean when you say you’re worried that people may “overestimate the return of accurate analytics”? Overestimate in comparison to what – what’s the quantum you’re using? It’s sufficiently vague that it’s impossible to be wrong; of course some people are going to overestimate return. They always do – unrealistic expectations are the norm, not the exception.
That doesn’t mean that the returns aren’t real or large which, by my measure, is far more important. Look at Apple – the latest iPad sales were seen as disappointing by analysts despite their moving almost 12m units in a market that didn’t exist five years ago.
Hope that’s clearer. If not, I’ll blame the fact that I have a son who’s decided the best time to wake up is around 4:45am. And, it’s currently 8pm here, so I’m getting close to running on fumes. 🙂
And, that’s probably far more than I should have written. Anyway, food for thought – I enjoy what you write and there’s a reason I’m following your RSS feed. 🙂
I also should ask – when you say “people”, who are you talking about? Investors, or decision-makers *inside* the organisation?
If you’re talking about investors, I completely agree. I mean, look at the dot.com boom – rational expectations of future return are anything but when the hype machine rolls in!
It’s when you’re talking about internal decisions that I think things are more complicated than your statement implies. For example, it’s still possible to see great returns even when marketshare remains constant …
Curt,
Completely agree on point (4). Analysts go to great lengths to accomplish small improvements to predictive model accuracy in the sandbox — often at the expense of clarity and deployability — without the slightest understanding of how much extra precision is worth.
We can all agree that predictive accuracy is important in life-or-death matters, such as predicting whether or not a medical treatment will cure a deadly disease. But when you’re trying to predict whether a customer will respond to the pink creative or the blue creative, a simple model rapidly deployed wins.
TD
I’d like to suggest that improved intelligence through analytics is not a zero-sum game. A classic example from marketing; for generations now it is empirically verifiable that when Nestle launches an ad campaign for its chocolates, not only do its sales subsequently rise, but so do the sales of Hersheys and other chocolate brands.
Analytics can be viewed similarly. If analytics helps the business — and thereby the industry — to evolve to new products and markets then the size of the pie has increased, not just the proportion of the slice.
Attila,
I don’t think it’s zero-sum either. I’m suggesting, however, that the sum may not be as extremely high as the greatest optimists seem to believe.
>> 4. I’m worried that people may be overestimating the business benefit of accurate analytics, great thought that value truly is. For example, it’s not plausible that all enterprises in the world use better analytics to all improve their respective market shares.
Disagree with this statement. For being competitive organizations need to have sustained innovation – and the surviving organizations have innovated in the past on business models, value chains, et al. I see competing on analytics as the next big driver and I truly believe that it the leaders in this space will outperform others who will follow. Seen from that viewpoint, I do not think that business benefit is being overestimated. However, if you meant that people think it will be a panacea or provide sustainable advantages, I agree that this will become table-stakes in the near future, and the survivors will be ones who can innovate further.
One can agree or disagree with the proposition that “people are overestimating the business value of analytics”, but the reality is that predictive modelers and their business clients are rarely able to state the value of incremental model accuracy. They simply assume that more accuracy is better (even at the expense of deployability, latency or clarity).
Neural networks, for example, slightly outperform decision trees for many predictive use cases, but are much more difficult to build, deploy and interpret. Is the extra accuracy worth it? That question tends to stump predictive modelers and their clients alike.
What is “Big Data” anyway and how will “really big data” be called in 5 years or in 10 years? How big is today’s “small data” to “big data” 10 years ago? In the end, everything is hype…
Assuming that “Big Data” is a new thing and not purely hype… and assuming that for investors there has to be some sort of Enterprise Big Data (not just web-scale big data housed by a dozen Google-sized companies) then we might look at the edge of the data warehouse space to find examples of Big Data…
I suspect that Big Data is about fine-grained data like call detail records in a telco or click-stream data for an online retailer.
Further I would suggest that the value in this detail is exposed when you apply the graph analytics methods you suggest in your posts on relationship analytics…
[…] the widespread adoption of technology generally cannot be underestimated. This is echoed in a recent GIGAOM article by Curt Monash entitled “Big Data Hype?” where he states that “Big Data Technology is just another chapter in the ever-growing […]