• In general, I regard Gartner Magic Quadrants as a bad use of good research.
• Illustrating the uselessness of — or at least poor execution on — the overall quadrant metaphor, a large majority of the vendors covered are lined up near the line x = y, each outpacing the one below in both of the quadrant’s dimensions.
• I find fewer specifics to disagree with in this Gartner Magic Quadrant than in previous year’s versions. Two factors jump to mind as possible reasons:
  • This year’s Gartner Magic Quadrant for Data Warehouse Database Management Systems is somewhat less ambitious than others; while it gives as much company detail as its predecessors, it doesn’t add as much discussion of overall trends. So there’s less to (potentially) disagree with.
  • Merv Adrian is now at Gartner.
• Whatever the problems may be with Gartner’s approach, the whole thing comes out better than do Forrester’s failed imitations.

*At the time of this posting, I don’t yet have a link. However, I expect that to change quickly, and I plan to edit this paragraph accordingly. If nothing else, I hope people will drop links into the comment thread.

Specific company comments, roughly in line with Gartner’s rough single-dimensional rank ordering, include:

• The Gartner Magic Quadrant’s comments on Teradata seem pretty fair. I don’t think I’m much in disagreement when I say:
  • Teradata has the richest, most mature analytic DBMS offering.
  • Teradata has an outstanding track record both for managing large data volumes and for high-concurrency mixed workloads.
  • Aster Data was a cool Teradata acquisition, even if Teradata/Aster synergies or integration have been nominal to date.
  • Teradata still needs to get out of its own way in marketing, positioning, packaging, and/or defining its premium-priced system vs. its more moderately-priced alternatives. Indeed, as necessary as this approach may have been to fending off encroachments by Netezza and others, what Teradata really needs to do is evolve to a more pick-your-own-node-combination mix-match kind of offering.
• Gartner has talked with a lot of Oracle Exadata users who say that the product works; Gartner has also stopped beating Oracle up for its previous policy of almost never doing onsite POCs (Proofs of Concept); both parts of that ring true with me. But Gartner also rightly dings Oracle for various issues in cost and cumbersomeness. Overall, while I agree there are organizations for which Oracle should indeed be a top-ranked choice, there are many others who shouldn’t put Oracle on their short list.
• Third in the Gartner MQ rankings is IBM.
  • Gartner gets so caught up in reciting the names of various IBM product offerings that it neglects to say much good about DB2 itself. (I tend to have a similar problem.)
  • But Gartner does mention concurrency as a strength. I agree, especially if we presume that that was a reference to DB2 rather than Netezza.
  • Gartner cites Netezza’s post-acquisition annual growth rate as 30%. Gartner seems to think this is a good number. I disagree, but in Netezza’s defense, it has had to endure IBM’s post-acquisition on-boarding process.
• Arguably fourth in the Gartner Data Warehouse Magic Quadrant rankings is EMC/Greenplum.
  • In general, Gartner likes the taste of Greenplum Kool-Aid.
  • Gartner neglects to ding Greenplum for concurrency challenges, which I view as an oversight given Gartner’s general stress on that area.
  • Gartner does ding Greenplum for support challenges.
  • Gartner neglects to praise Greenplum for true hybrid row/columnar data management, a feature shared by Teradata and Vertica, among others, but not by Oracle, DB2, or Netezza.
  • Gartner located a half-petabyte Greenplum database. This doesn’t surprise me, even though Greenplum has frequently made exaggerated claims about large-size database successes in the past.
  • Gartner reports a >400 figure for Greenplum customers, which is plausible.
• In its first deviation from strict one-dimensional rank ordering, the Gartner Magic Quadrant ranks Sybase ahead of Greenplum in completeness of vision but behind in “ability to execute”.
  • If that were the other way around, it might make more sense. Greenplum promises anything and everything you might ever want for analytic data management or the associated analysis; but Sybase has vastly more analytic DBMS users than Greenplum does, running a variety of demanding workloads.
  • Gartner appears to think that Sybase IQ requires less database administration than I do.
  • Gartner seems concerned that SAP will position HANA and Sybase ASE as, between them, the only DBMS you’ll ever need, casting doubt on Sybase IQ’s future. I wouldn’t worry about that if you have a problem you want to solve today.
• The Gartner Magic Quadrant for Data Warehouse Database Management Systems ranks Microsoft sixth overall, despite noting that there isn’t a single production reference for Microsoft’s Parallel Data Warehouse. In support of this ranking, it for example cites the compression feature, which distinguishes Microsoft SQL Server from no other product on the list except Kognitio. If you have such an undemanding data warehousing problem that many different analytic DBMS could meet your needs, there’s a good chance Microsoft SQL Server can also do the job; and if you’ve bought into the Microsoft technology stack, you might as well keep going down that path. Otherwise, I don’t know why somebody should adopt Microsoft’s offering at this time.
• Seventh along the main diagonal path in the Gartner Magic Quadrant is HP Vertica. I’d rank Vertica higher than that, but in fairness I note two execution concerns. First, HP has a lousy track record, both in acquisitions and in data warehousing/analytics. Second, Vertica is bad about answering my email. Anyhow, Gartner doesn’t seem to have given Vertica credit either for its full customer count or for the multiple petabyte-scale databases Vertica runs.
• 1010data is an outlier, with Gartner noting that it only partly fits in with other “Data Warehousing Database Management” companies, and hence kind of confessing that 1010data on the Magic Quadrant is somewhat arbitrary. Stuff like that is bound to happen, given the inherent difficulties of defining market categories. Anyhow, my thoughts on 1010data include:
  • I’m nervous about the fact that 1010data doesn’t actually control its own DBMS technology, but rather relies on old code from the small private company KX Systems.
  • There are three main reasons to consider 1010data:
    • You want to enter the data mart outsourcing business in a casual way, and you like its SaaS offering.
    • You want to engage in stakeholder-facing analytics in a casual way, and you like its SaaS offering.
    • You love 1010data’s particular set of interactive analytic features and performance.
• Back to the main path winding along the Gartner Magic Quadrant main diagonal — next up is ParAccel. While I question some of the peripheral comments, I agree with Gartner’s core messages that:
  • ParAccel, the product, is blazingly fast in certain use cases.
  • ParAccel, the company, is dangerously small.
• Eighth on the Gartner MQ’s main path is Kognitio. This is too high. Kognitio positions itself as offering in-memory DBMS, yet stubbornly refuses to do any kind of data compression. That’s an awful combination of choices. As for using Kognitio’s data warehousing SaaS offering — why would you do that, when more modern products are available on a SaaS/cloud basis as well?
• Ninth in the Gartner Magic Quadrant main rankings is SAND.
  • The SAND section is not a triumph of Gartner accuracy. For example:
    • Gartner completely missed the errors in SAND’s reported customer counts.
    • Gartner refers to SAND as being “in existence for approximately nine years”, which is too low by at least a factor of 2.
    • Gartner says “SAND is a privately held company”, even though Merv knows better than that.
  • Otherwise, Gartner’s opinion on SAND seems to boil down to “Interesting technology and ideas, but dangerously small company.” I agree.
• Tenth and too low in the Gartner MQ main rankings is Infobright.
  • At least by some metrics (e.g. customer count), Infobright isn’t as dangerously small as ParAccel, SAND, Kognitio, et al.
  • That said, Infobright is small and focused on machine-generated data. So I wouldn’t be confident in Infobright’s future technology path for human-generated data use cases.
  • Infobright’s performance is uneven — blazing in cases where the Knowledge Grid helps, but not necessarily stellar by analytic DBMS standards when full table scans are called for.
  • I agree with Gartner that the possibility of Oracle/MySQL future shenanigans is a concern. But while the energy behind MySQL forking efforts doesn’t seem too great right now, I’d expect them to revive and offer a successful escape path if it seemed Oracle was going to indeed play hardball.
  • Also, given that it’s already an open source vendor, there are various kinds of assurances Infobright could give that would also help alleviate customer concerns.
• Actian, formerly Ingres, took a big tumble in Gartner’s rankings versus last year, when I simply wrote “What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.” I’m even a little harsher about Ingres/Actian’s DBMS products and prospects than Gartner is, but at least now we’re in the same ballpark.
• Along with Infobright, ParAccel, and SAND, Exasol appears to be another of the “good columnar technology/small company” crowd. As with other such products, one should be careful about fit-and-finish features that are missing today, as there is no assurance they’ll be added in a timely manner going forward.
• illuminate Solutions, which was on last year’s Gartner list, now appears to be an ex-company.

Interesting quotes include:

WibiData is designed to store … transactional data side-by-side with profile and other derived data attributes.

… the ability to add new ad-hoc columns to a table enables more flexible analysis: output data that is the result of one analytic pipeline is stored adjacent to its input data, meaning that you can easily use this as input to second- or third-order derived data as well.

schemas can vary over time; you can easily add a field to a record, or delete a field. … But even though you start collecting that new data, your existing analysis pipelines can treat records like they always did; programs that don’t yet know about the new cookie are still compatible with both the old records already collected, and the new records with the additional field. New programs fill in default values for old data recorded before a field was added, applying the new schema at read time.

schemas for every column are stored in a data dictionary that matches column names with their schemas, as well as human-readable descriptions of the data.

Interesting aspects of the post that don’t lend themselves as well to being excerpted include:

• How the Produce-Gather “analysis calculus” — i.e. framework — works.
• How this all ties into Apache projects (and sub-projects) such as Hadoop, HBase, and Avro.

• The Forrester Wave’s relative vendor rankings are meaningless, in that the document compares apples, peaches, almonds, and peanuts. Apparently, it covers any vendor that includes a distribution of Apache Hadoop MapReduce into something it offers, and that offered at least two (not necessarily full production) references for same.
• The Forrester Wave for “enterprise Hadoop” contradicts itself on the subject of Hortonworks.
  • The Forrester Wave for “enterprise Hadoop” is correct when it says “Hortonworks … has Hadoop training and professional services offerings that are still embryonic.”
  • Peculiarly, the Forrester Wave for “enterprise Hadoop” also says “Hortonworks offers an impressive Hadoop professional services portfolio”. Hortonworks will likely win one or more nice partnership deals with vendors in adjacent fields, but even so its professional services capabilities are … well, a good word might be “embryonic”.
• Forrester Waves always seem to have weird implicit definitions of “data warehousing”. This one is no exception.
• Forrester gave top marks in “Functionality” to 11 of 13 “enterprise Hadoop” vendors. This seems odd.
• I don’t know why MapR, which doesn’t like HDFS (Hadoop Distributed File System), got top marks in “Subproject integration”.
• Forrester gave top marks in “Storage” to Datameer. It also gave higher marks to MapR than to EMC Greenplum, even though EMC Greenplum’s technology is a superset of MapR’s. Very strange. (Edit: Actually, as per a comment below, there is some uncertainty about the EMC/MapR relationship.)
• Forrester gave higher marks in “Acceleration and optimization” to Hortonworks than to Cloudera and IBM, and higher marks yet to Pentaho. Very odd.
• I’m not sure what Forrester is calling a “Distributed EDW file store connector”, but it sounds like something that Cloudera has provided via partnership to a number of analytic DBMS vendors.
• Forrester’s “Strategy” rankings seem to correlate to a metric of “We’re a large enough vendor to go in N directions at once”, for various values of N.
• Forrester is correct to rank Cloudera’s “Adoption” as being stronger than EMC/Greenplum’s or MapR’s. But Hortonworks’ strong mark for “Adoption” baffles me.

Three things that absolutely should NOT be obstacles to these ends are:

• Corporate DBMS standards.
• Corporate data governance processes.
• The difficulties of ETL.

Reasons they shouldn’t or don’t need to be obstacles include:

• Analytic DBMS are often vastly more cost-effective than general-purpose ones.
• In particular, analytic DBMS are often much easier to install and manage than general-purpose ones.
• Heavy data governance bureaucracy is often unnecessary because:
  • The department should know what the limitations on the data’s accuracy are.
  • The department should know how much data accuracy is required.
  • The side-effects on other departments of any data inaccuracy would be minimal.
• There are multiple good schemes for populating data marts, managed by cost-effective analytic DBMS, with data from integrated data warehouses.
  • ELT (Extract/Load/Transform) almost always works, because data cleaning/data quality was handled at or before the IDW level, and because the analytic DBMS has the processing power to pull it off.
  • ETL (Extract/Transform/Load) should be easy as well. (If isn’t, something may be lacking in your ETL set-up.)
  • Analytic DBMS are increasingly adding capabilities for easy spin-out of real or virtual data marts. Other kinds of technology (e.g. virtualization) are having their database spin-out capabilities upgraded as well.

One point to remember in support of departmental autonomy is that departments’ views of what data to use may be more expansive than central IT’s. One reason is that important data may be external to the company, outside IT’s natural realm  of concern. Examples of this include but are hardly limited to:

• Anything like “market data”.
• Anything like “sentiment analysis”.
• Data owned by supply chain partners.

Further, even the more innovative internal data sources are commonly departmental, for example various kinds of multi-structured data (text verbatims from customers, log file data, and so on).

Whatever is true of data management (and ETL) is true for metadata management, even if it’s done by some kind of business intelligence tool. What I mean by that is:

• Whoever manages data is also responsible for ingesting and emitting it …
• … and specifically for emitting it in understandable, well-organized, well-named formats, …
• … so that departments can take responsibility for what amounts to lightweight analytic application development.

As for the “application development” itself, I’m envisioning at least three things:

• Math.
• Sophisticated relational query.
• Data visualization.

I.e., I’m talking about what “analysts” and “quants” do. So to put the point even more simply:

• Analysts and quants should be able to consume data that’s organized in a friendly manner.
• Central IT should be friendly in how it serves data.

One corollary of this approach is that departments should try to adhere to corporate BI standards, at least for routine dashboard and reporting. Indeed, if a department brings in a business intelligence tool different from the corporate standard, there are three main possibilities:

• The tool is integrated with something else it makes sense to bring in, such as a third-party data supply or application.
• The tool has an important capability the corporate standard doesn’t have, such as more flexible visualization and drilldown.
• Central IT screwed up, making things much more difficult than they needed to be.

Reporter: [Care to comment]?
CAM: SQL Server is an adequate product if you don’t mind being locked into the Microsoft stack. For example, the ColumnStore feature is very partial, given that it can’t be updated; but Oracle doesn’t have columnar storage at all.

Reporter: Is the lock-in overall worse than IBM DB2, Oracle?
CAM: Microsoft locks you into an operating system, so yes.

Reporter: Is this release something larger Oracle or IBM shops could consider as a lower-cost alternative a co-habitation scenario, in the event they’re mulling whether to buy more Oracle or IBM licenses?
CAM: If they have a strong Microsoft-stack investment already, sure. Otherwise, why?

Reporter: [How about] just cost?
CAM: DB2 works just as well to keep Oracle honest as SQL Server does, and without a major operating system commitment. For analytic databases you want an analytic DBMS or appliance anyway.

Best is to have one major vendor of OTLP/general-purpose DBMS, a web DBMS, a DBMS for disposable projects (that may be the same as one of the first two), plus however many different analytic data stores you need to get the job done.

By “web DBMS” I mean MySQL, NewSQL, or NoSQL. Actually, you might need more than one product in that area.

• Many analytic challenges are inherently departmental.
• In many cases, central IT control of analytics isn’t needed.
• Departments move ahead without central approval or involvement because they can.

That said, arguments for centralizing analytic technology include:

• A lot of data is used by more than one department, for example:
  • Financial transactions (one or more affected departments and also the central accounting group).
  • Web logs (marketing and IT/web operations).
• Departments may not have the requisite technical expertise (and it may be redundant/cost-ineffective for them to acquire it).

What’s more, there are IT best practices to support department-level analytics. Some of the key ones boil down to:

• Be flexible in your analytic DBMS support.
• Be responsive to requests for ETL.

My conclusion is that central IT should encourage (and aid) departmental analytics. Let’s look at some details.

I think two huge categories of analytic problem are inherently departmental:

• Investigative analytics (pretty much all of it).
• Routine monitoring/dashboarding if the data is tracked just by one department.

Investigative analytics is a kind of research activity — you’re looking to discover previously unrecognized patterns. There are two approaches to this — you can do it in the department that has the relevant business knowledge, or you can outsource it to a special group of “discoverers” (commonly statisticians).* Either way, this is a small team/departmental kind of activity.

*Combining the two approaches is common — a department can have its own analytically adept discoverers, whether they’re call “quants” or just “business analysts”.

Reporting/monitoring BI at least has the potential to be enterprise-wide — but commonly it isn’t, as each department has its own operational data sources and metrics. Marketing departments may watch external data that the rest of the company doesn’t worry about. But it can be true across the board. Factory operations folks may track machine tool data the rest of us barely understand.

Even if a business need is strictly departmental, there can be at least two reasons to centralize technology implementation:

• The department doesn’t have the critical mass of IT expertise.
• Departmental IT has side effects on the rest of the company.

Whether those reasons hold up depends a lot on what kind of analytic scenario we’re talking about.

Let’s organize that part of this discussion in line with the taxonomy from my eight kinds of analytic database posts last July.

• Enterprise data warehouses fall under the purview of major IT organizations. That remains true even if we pivot to the more realistic concept of integrated data warehouse. However, less stuff needs to be protected in an EDW/IDW than some data authoritarians like to think.
• I wrote that the stresses on traditional data marts were “performance, concurrency, TCO.” This is a clue that the more demanding examples are right in IT’s wheelhouse. As for the less demanding cases — IT should be able to meet those needs without breaking a sweat.
• Agile investigative data marts are inherently departmental. If you have the talent to use one, you also have the talent to, for example, train into being a part time Netezza DBA. Who cares if you don’t have the expertise to do sophisticated tuning? Analytic DBMS are fast enough — and hardware is cheap enough — that you don’t that skill set anyway.
• Big investigative data marts can go either way. They’re technically challenging, so IT certainly has a claim on them. But in cases where the data, while big, is fairly homogeneous, it’s also not unrealistic for departments to handle the mart themselves.
• Bit buckets are often departmental today, with the department in question happening to be central IT. And central IT is where they’re likely to flourish, as the data they hold becomes ever more diverse.
• Archival data stores are a central IT matter. Nobody else is likely to care enough to do it right.
• Outsourced data marts, by definition, don’t live inside conventional enterprises. But they are often a way for business units to get access to data and analytics without relying on central IT.
• Operational analytics servers are likely to be sufficiently mission-critical that you want them handled by IT.

So in most cases I’d say: Departments can manage their own investigative data marts, and so of course can SaaS vendors and third-party data providers; other analytic databases should be run by central IT. (And of course, large departments with serious local IT can fuzz those distinctions up.) Beyond that, it would seem that whoever administers the database should administer the rest of the analytic stack as well.

That still leaves us with some practical questions, such as:

• Exactly what products should IT departments buy for which purposes? I hope a lot of posts in this blog are helpful in that consideration.
• How should development tasks be split between departments and central IT? It may take me a while to get a post together on that the subject, since in general the analytics-development picture is pretty complicated to lay out.
• How should departments and central IT work together to manage departmental investigative data marts? I hope to post on that subject soon.

• Splunk is first and foremost an analytic DBMS …
• … used to manage logs and similar multistructured data.
• Splunk’s DML (Data Manipulation Language) is based on text search, not on SQL.
• Splunk has extended its DML in natural ways (e.g., you can use it to do calculations and even some statistics).
• Splunk bundles some (very) basic, Splunk-specific business intelligence capabilities.
• The paradigmatic use of Splunk is to monitor IT operations in real time. However:
  • There also are plenty of non-real-time uses for Splunk.
  • Splunk is proudest of its growth in non-IT quasi-real-time uses, such as the marketing side of web operations.

As in any release, a lot of Splunk 4.3 is about “Oh, you didn’t have that before?” features and Bottleneck Whack-A-Mole performance speed-up. One performance enhancement is Bloom filters, which are a very hot topic these days. More important is a switch from Flash to HTML5, so as to accommodate mobile devices with less server-side rendering. Splunk reports that its users — especially the non-IT ones — really want to get Splunk information on the tablet devices. While this somewhat contradicts what I wrote a few days ago pooh-poohing mobile BI, let me hasten to point out:

• Splunk is used for a lot of (quasi) real-time monitoring.
• Splunk’s desktop user interfaces are, by BI standards, quite primitive.

That’s pretty much the ideal scenario for mobile BI: Timeliness matters and prettiness doesn’t.

Hmm. Maybe StreamBase LiveView needs a mobile option as well …

Splunk’s basic use is to take the text string that is a log and make sense of it. But Splunk now also supports JSON structures. It does this via something called spath, which as you might guess from the name has XPath similarities. That probably bore more discussion than we found the time to have.

By the way: If you’re interested in BI over XML, that’s what my former clients at Skytide were founded to do, before they pivoted a bit. I don’t think those capabilities have disappeared from the product.

Splunk has graciously allowed me to post a slide deck. More stuff in there, including quotes from a customer — Expedia — that has 2700 Splunk users.

• Bigness — Volume, Velocity, size
• Structure — Variety, Variability, Complexity

given that

• High-velocity “big data” problems are usually high-volume as well.*
• Variety, variability, and complexity all relate to the simply-structured/poly-structured distinction.

But the conflation should stop there.

*Low-volume/high-velocity problems are commonly referred to as “event processing” and/or “streaming”.

When people claim that bigness and structure are the same issue, they oversimplify into mush. So I think we need four pieces of terminology, reflective of a 2×2 matrix of possibilities. For want of better alternatives, my suggestions are:

• Relational big data is data of high volume that fits well into a relational DBMS.
• Multi-structured big data is data of high volume that doesn’t fit well into a relational DBMS. Alternative: Poly-structured big data.
• Conventional relational data is data of not-so-high volume that fits well into a relational DBMS. Alternatives: Ordinary/normal/smaller relational data.
• Smaller poly-structured data is data for which dynamic schema capabilities are important, but which doesn’t rise to “big data” volume.

Notes on all this include:

• “Relational big data” is commonly what you need a scalable analytic relational DBMS for. But there are non-analytic use cases as well.
• The paradigmatic example of “multi-structured big data” is log files. Thus, multi-structured big data is commonly what you need a big bit bucket for.
• One might want to equate non-analytic relational big data technology to “NewSQL”. However, I’m struggling to think of a database size range in which the entire NewSQL industry can match Oracle’s market share alone.
• One might want to equate non-analytic multi-structured big data technology to “NoSQL”. However:
  • “NoSQL” is also used to encompass not-so-big-data use cases, such as prototyping in MongoDB.
  • “NoSQL” has non-ACID/low(er)-data-integrity connotations that aren’t appropriate for all non-relational systems.
• Up to a point, you can analyze relational big data in a conventional relational DBMS, but an analytic RDBMS will usually win on TCO (Total Cost of Ownership). In particular, reasonable thresholds for moving an analytic database off Oracle might be:
  • 1-2 terabytes if you’ve never bought anything past Oracle Standard Edition.
  • 5-10 terabytes if you’re already paying for Oracle Enterprise Edition.
  • A lot higher than that if you actually find Oracle Exadata to be cost-effective.
• Depending on how big one acknowledges as “big”, the market share leader in “big bit bucket” use cases is either Splunk or Hadoop.
• If we look at multi-structured big data management overall, MarkLogic joins the list of market share contenders, as do various NoSQL alternatives.
• It is wrong to say that the large web companies invented “big data” technology. But it is more reasonable to say they invented much of “multi-structured big data” management. In particular (and this is just a partial list), Google, Amazon, Yahoo, Facebook, et al. can reasonably be credited with Hadoop, Cassandra, HBase and various predecessors to same.

I mean for the term “data mustering” to encompass at least three scenarios:

• Integrated (relational) data warehouse.
• Big bit bucket.
• Big bit stream.

Let me explain what I mean by each. 

“Integrated data warehouse” is a phrase Teradata has started using for enterprise data warehouses that, like approximately every other EDW in the entire history of data warehousing, aren’t truly enterprise-wide. In other words, it means “not just a data mart”. No category name is perfect, but I think that one works reasonably well.

I previously described the big bit bucket use case as

Users take a whole lot of data, often machine-generated data in logs of different kinds, and dump it into one place, managed by Hadoop, at open-source pricing.

and quickly added

Of course, there are various outfits who’d like to sell you not-so-cheap bit buckets. Contending technologies include Hadoop appliances (which I don’t believe in), Splunk (which in many use cases I do), and MarkLogic (ditto, but often the cases are different from Splunk’s). Cloudera and IBM, among other vendors, would also like to sell you some proprietary software to go with your standard Apache Hadoop code.

I think I’ll stand pat on that explanation.

By analogy, a big bit stream is various streams of data, assembled in the custody of a streaming engine. Sybase told me Wednesday that this scenario appears in both of the traditional markets for CEP/streaming — national intelligence, where it is a major use of streaming, and capital markets in some use cases as well. And it’s consistent with what I’ve heard from other CEP/streaming vendors as well.

As for where I got the word “mustering” — it’s a military term, for when you assemble your troops and their gear either for inspection or for actual use. The main modern usage I know of the word is as part of the phrase “pass muster”, which originally referred to the concept that the person being paid to put a regiment together should from time to time demonstrate that the regiment physically existed in the form that regimental records seemed to show.

• “Deliver products that really meet customers’ desires and needs.”
• “Successfully convince them that you’re doing so …”
• “… at an attractive overall cost.”

Vendors mentioned here are Oracle, SAP, HP, and IBM. Anybody smaller got left out due to the length of this post. Among the bigger omissions were:

• salesforce.com (multiple subjects).
• SAS HPA.
• The evolution of Hadoop.

A (lingering) issue for SAP and Oracle alike

As I noted in January of this year, integration of business intelligence into operational apps is making very slow progress. Even so, it’s a huge part of the apparent strategy at SAP and Oracle alike, as well it should be. Much of the benefit from automating routine desk work has already happened. The areas ripest for exploitation are the ones where analytics are part of the equation.

Given the lack of tangible progress, why do I think this is a genuine area of Oracle and SAP emphasis? Three reasons of many are:

• Why else did SAP buy Business Objects?
• If they’re not trying to integrate operational apps and analytics, why else does SAP’s emphasis on HANA make sense?
• Without business intelligence in the picture, how does Oracle’s integrated-stack story promise any direct user benefits?*

*As opposed to IT concerns — integration, administration, TCO (Total Cost of Ownership), etc.

After so many years of disappointment, I’m not going to forecast 2012 as a pivotal year for the integration of business intelligence into operational applications. But if one of SAP or Oracle ever does get a significant BI/operational app integration advantage over the other, it could be a major competitive advantage in those application market segments that are still up for grabs. It also is an opportunity for both vendors to gain BI market share in their respective application customer bases.

A more urgent issue for SAP

SAP has put huge amounts of credibility on the line for HANA, the integration of two different and not particularly mature in-memory database technologies. So far, it is difficult to find evidence that HANA is robust enough for widespread adoption. Whether or not SAP can fix that is a huge open question, which could have significant impact on the course of several technology areas: applications, business intelligence, in-memory DBMS, and maybe even hardware.

Based on current information, which is admittedly partial, I’m a short-term pessimist on HANA. Longer-term, I’m on record as saying that traditional databases will eventually wind up in RAM. SAP will surely get that technology right some day, whether or not the way it does so has anything to do with present-day HANA code.

Four more issues for Oracle

Oracle’s ambitions are near-endless, and so also therefore is its list of execution challenges. Four in the analytics area that I find particularly interesting are:

• True hybrid columnar DBMS. I was guessing that Oracle, like Teradata, would announce true hybrid columnar the week of Oracle OpenWorld. I was wrong. But if Oracle can’t bring out true hybrid columnar DBMS functionality relatively soon, Exadata will lose credibility as a competitor to more specialized analytic DBMS.
• Oracle Exalytics. With Exalytics in the mix, Oracle’s technology stack has HANA-like potential. But will Exalytics even ship in 2012? (I think so.) Will it be good for much in the first release? (I’m skeptical.)
• Oracle’s Big Data Appliance. I’m skeptical both about Oracle’s NoSQL product — a favorable InfoWorld review notwithstanding — and Hadoop appliances. But if I’m wrong, and Oracle can successfully embrace/extend the new non-relational paradigms, then it really might regain control over the evolution of data management.
• Oracle’s Endeca acquisition — will Oracle prove me wrong and integrate Endeca effectively into its overall analytic product line? If it does, we might finally see effective text (and eventually speech) navigation of enterprise software. (But as with all Oracle issues cited here, this is something that probably won’t amount to much in 2012 even if it does later go well.)

Three issues for IBM

Like Oracle, IBM is a huge company with many ambitions and hence many execution challenges. The biggest of those is surely: How effective can IBM be at selling outside its existing customer base? I don’t hear as much competitively about IBM DataStage, IBM SPSS or now IBM Netezza as I did when their vendors were independent companies. Even Cognos may not be much of an exception to the rule, although it has its own large customer base outside of IBM’s traditional one. (To lesser extents , the same is of course true of Netezza and numerous other IBM acquisitions.)

Another general issue for IBM is substantively integrating its various product lines, at least to the extent that makes sense. DB2/Netezza integration sounds good, but even that is a matter more of product marketing (the admirable part of that discipline) more than of actual technology. Other integrations (e.g. Cognos/DB2 in various bundles) have tended toward the dubious side.*

*I’m still waiting for IBM to get back to me with examples of how Cognos/DB2 joint tuning amounts to anything. It’s been more than a year, so I’m glad I didn’t hold my breath.

In a somewhat narrower vein, I wonder: Will IBM be able to gain traction for InfoSphere Streams? And if so, when and where will the traction be?

Will HP screw up Vertica?

Vertica has a very attractive product offering. It’s perhaps the most scalable analytic DBMS outside of Teradata, running on the hardware of your reasonable choice.  It’s also the one I recommend most often to clients in the 1-50 terabyte range.

So far HP doesn’t seem to have done much to leadfoot Vertica. (About all I’ve heard from competitors is that Vertica seems to have faded somewhat in the financial services market, and there could be multiple explanations if that is indeed true.) But if HP Vertica does somehow manage to botch things, opportunities will open up for a range of columnar analytic DBMS competitors.