January 22, 2011

Mega-trends driving data warehousing and business intelligence

Philip Russom opines (emphasis mine):

What’s driving change in data warehousing (DW) and business intelligence (BI)? There are obvious scalability issues, due to burgeoning data, reports, and user communities. Plus, end-users need more real-time and on-demand BI. For many organizations, integrating existing systems into DW/BI is a higher priority than putting in new ones. And the “do more with less” economy demands more BI at lower costs. Hence, most drivers of change in BI and DW concern four Mega-Trends: size, speed, interoperability, and economics.

Depending on which universe of enterprises and vendors you’re looking at, Philip’s claim of “most” may be technically true. But from where I sit, Philip omitted two other crucial trends: new kinds of data and increased analytic sophistication.

A year ago, I divided data into three kinds:

Most organizations on the planet could benefit from better understanding or exploiting their human-generated tabular data. But even so, many of the best opportunities to add analytic value come from capturing and analyzing fundamentally newer kinds of information.

I further would suggest that analytic sophistication is going up, for at least two reasons:

Some of the best examples of these trends, especially the second one, may be found in what I recently called analytic profiling.


4 Responses to “Mega-trends driving data warehousing and business intelligence”

  1. Conor O'Mahony on January 23rd, 2011 5:54 pm

    After having previously spent time working on products for nontabular and machine-generated data, I’ve got to say that I agree 100% with your statement regarding the value that organizations can derive from these kinds of data. I’ve worked with large organizations who are leveraging such data for new types of analysis (both automated and ad hoc) and the success has been quite startling. It really has provided those organizations with significant competitive differentiation within their fields. However, I wonder if this is a “trend”. For me, it still feels a little early in the adoption cycle to be a trend.

  2. Curt Monash on January 23rd, 2011 7:14 pm


    That’s a fair challenge. I’d say that the list of industry sectors where there’s no need to doubt whether these are trends yet is reasonably impressive — the whole online sector, the whole defense/intelligence sector, the telecommunications sector, the pharma research sector, scientific research in general (the fairness of Mike Stonebraker’s “zero billion dollar market” label notwithstanding, soon financial trading as well.

    In addition, they’re already trends wherever somebody is closely engaged with the internet, in sufficiently high volumes to justify the use of sophisticated analytic tools.

    OUTSIDE of those sectors and perhaps a couple more I’m forgetting at the moment, I would agree text and sensor data have been a bit slow to take off.

  3. Paul O'Hagan on January 24th, 2011 1:40 pm

    I agree Curt. There’s a set of industries/verticals where nontabular data is of a high potential business value, and it’s with those organizations where the trend has been apparent for many years.

    For example, I’ve worked with both a US city police department and a major European based international criminal investigation group who both have used nontabular data mining technologies integrated with a traditional data warehouse for advanced crime analysis – in production for 4 or 5 years now.

  4. Gartner: Data Warehousing at an Inflection Point « In(tegrate) the Clouds on February 23rd, 2011 1:06 pm

    […] Mega-trends driving data warehousing and business intelligence (dbms2.com) […]

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.