• In general, I regard Gartner Magic Quadrants as a bad use of good research.
• Illustrating the uselessness of — or at least poor execution on — the overall quadrant metaphor, a large majority of the vendors covered are lined up near the line x = y, each outpacing the one below in both of the quadrant’s dimensions.
• I find fewer specifics to disagree with in this Gartner Magic Quadrant than in previous year’s versions. Two factors jump to mind as possible reasons:
  • This year’s Gartner Magic Quadrant for Data Warehouse Database Management Systems is somewhat less ambitious than others; while it gives as much company detail as its predecessors, it doesn’t add as much discussion of overall trends. So there’s less to (potentially) disagree with.
  • Merv Adrian is now at Gartner.
• Whatever the problems may be with Gartner’s approach, the whole thing comes out better than do Forrester’s failed imitations.

*At the time of this posting, I don’t yet have a link. However, I expect that to change quickly, and I plan to edit this paragraph accordingly. If nothing else, I hope people will drop links into the comment thread.

Specific company comments, roughly in line with Gartner’s rough single-dimensional rank ordering, include:

• The Gartner Magic Quadrant’s comments on Teradata seem pretty fair. I don’t think I’m much in disagreement when I say:
  • Teradata has the richest, most mature analytic DBMS offering.
  • Teradata has an outstanding track record both for managing large data volumes and for high-concurrency mixed workloads.
  • Aster Data was a cool Teradata acquisition, even if Teradata/Aster synergies or integration have been nominal to date.
  • Teradata still needs to get out of its own way in marketing, positioning, packaging, and/or defining its premium-priced system vs. its more moderately-priced alternatives. Indeed, as necessary as this approach may have been to fending off encroachments by Netezza and others, what Teradata really needs to do is evolve to a more pick-your-own-node-combination mix-match kind of offering.
• Gartner has talked with a lot of Oracle Exadata users who say that the product works; Gartner has also stopped beating Oracle up for its previous policy of almost never doing onsite POCs (Proofs of Concept); both parts of that ring true with me. But Gartner also rightly dings Oracle for various issues in cost and cumbersomeness. Overall, while I agree there are organizations for which Oracle should indeed be a top-ranked choice, there are many others who shouldn’t put Oracle on their short list.
• Third in the Gartner MQ rankings is IBM.
  • Gartner gets so caught up in reciting the names of various IBM product offerings that it neglects to say much good about DB2 itself. (I tend to have a similar problem.)
  • But Gartner does mention concurrency as a strength. I agree, especially if we presume that that was a reference to DB2 rather than Netezza.
  • Gartner cites Netezza’s post-acquisition annual growth rate as 30%. Gartner seems to think this is a good number. I disagree, but in Netezza’s defense, it has had to endure IBM’s post-acquisition on-boarding process.
• Arguably fourth in the Gartner Data Warehouse Magic Quadrant rankings is EMC/Greenplum.
  • In general, Gartner likes the taste of Greenplum Kool-Aid.
  • Gartner neglects to ding Greenplum for concurrency challenges, which I view as an oversight given Gartner’s general stress on that area.
  • Gartner does ding Greenplum for support challenges.
  • Gartner neglects to praise Greenplum for true hybrid row/columnar data management, a feature shared by Teradata and Vertica, among others, but not by Oracle, DB2, or Netezza.
  • Gartner located a half-petabyte Greenplum database. This doesn’t surprise me, even though Greenplum has frequently made exaggerated claims about large-size database successes in the past.
  • Gartner reports a >400 figure for Greenplum customers, which is plausible.
• In its first deviation from strict one-dimensional rank ordering, the Gartner Magic Quadrant ranks Sybase ahead of Greenplum in completeness of vision but behind in “ability to execute”.
  • If that were the other way around, it might make more sense. Greenplum promises anything and everything you might ever want for analytic data management or the associated analysis; but Sybase has vastly more analytic DBMS users than Greenplum does, running a variety of demanding workloads.
  • Gartner appears to think that Sybase IQ requires less database administration than I do.
  • Gartner seems concerned that SAP will position HANA and Sybase ASE as, between them, the only DBMS you’ll ever need, casting doubt on Sybase IQ’s future. I wouldn’t worry about that if you have a problem you want to solve today.
• The Gartner Magic Quadrant for Data Warehouse Database Management Systems ranks Microsoft sixth overall, despite noting that there isn’t a single production reference for Microsoft’s Parallel Data Warehouse. In support of this ranking, it for example cites the compression feature, which distinguishes Microsoft SQL Server from no other product on the list except Kognitio. If you have such an undemanding data warehousing problem that many different analytic DBMS could meet your needs, there’s a good chance Microsoft SQL Server can also do the job; and if you’ve bought into the Microsoft technology stack, you might as well keep going down that path. Otherwise, I don’t know why somebody should adopt Microsoft’s offering at this time.
• Seventh along the main diagonal path in the Gartner Magic Quadrant is HP Vertica. I’d rank Vertica higher than that, but in fairness I note two execution concerns. First, HP has a lousy track record, both in acquisitions and in data warehousing/analytics. Second, Vertica is bad about answering my email. Anyhow, Gartner doesn’t seem to have given Vertica credit either for its full customer count or for the multiple petabyte-scale databases Vertica runs.
• 1010data is an outlier, with Gartner noting that it only partly fits in with other “Data Warehousing Database Management” companies, and hence kind of confessing that 1010data on the Magic Quadrant is somewhat arbitrary. Stuff like that is bound to happen, given the inherent difficulties of defining market categories. Anyhow, my thoughts on 1010data include:
  • I’m nervous about the fact that 1010data doesn’t actually control its own DBMS technology, but rather relies on old code from the small private company KX Systems.
  • There are three main reasons to consider 1010data:
    • You want to enter the data mart outsourcing business in a casual way, and you like its SaaS offering.
    • You want to engage in stakeholder-facing analytics in a casual way, and you like its SaaS offering.
    • You love 1010data’s particular set of interactive analytic features and performance.
• Back to the main path winding along the Gartner Magic Quadrant main diagonal — next up is ParAccel. While I question some of the peripheral comments, I agree with Gartner’s core messages that:
  • ParAccel, the product, is blazingly fast in certain use cases.
  • ParAccel, the company, is dangerously small.
• Eighth on the Gartner MQ’s main path is Kognitio. This is too high. Kognitio positions itself as offering in-memory DBMS, yet stubbornly refuses to do any kind of data compression. That’s an awful combination of choices. As for using Kognitio’s data warehousing SaaS offering — why would you do that, when more modern products are available on a SaaS/cloud basis as well?
• Ninth in the Gartner Magic Quadrant main rankings is SAND.
  • The SAND section is not a triumph of Gartner accuracy. For example:
    • Gartner completely missed the errors in SAND’s reported customer counts.
    • Gartner refers to SAND as being “in existence for approximately nine years”, which is too low by at least a factor of 2.
    • Gartner says “SAND is a privately held company”, even though Merv knows better than that.
  • Otherwise, Gartner’s opinion on SAND seems to boil down to “Interesting technology and ideas, but dangerously small company.” I agree.
• Tenth and too low in the Gartner MQ main rankings is Infobright.
  • At least by some metrics (e.g. customer count), Infobright isn’t as dangerously small as ParAccel, SAND, Kognitio, et al.
  • That said, Infobright is small and focused on machine-generated data. So I wouldn’t be confident in Infobright’s future technology path for human-generated data use cases.
  • Infobright’s performance is uneven — blazing in cases where the Knowledge Grid helps, but not necessarily stellar by analytic DBMS standards when full table scans are called for.
  • I agree with Gartner that the possibility of Oracle/MySQL future shenanigans is a concern. But while the energy behind MySQL forking efforts doesn’t seem too great right now, I’d expect them to revive and offer a successful escape path if it seemed Oracle was going to indeed play hardball.
  • Also, given that it’s already an open source vendor, there are various kinds of assurances Infobright could give that would also help alleviate customer concerns.
• Actian, formerly Ingres, took a big tumble in Gartner’s rankings versus last year, when I simply wrote “What Gartner said in connection with Ingres is too inaccurate to deserve detailed attention.” I’m even a little harsher about Ingres/Actian’s DBMS products and prospects than Gartner is, but at least now we’re in the same ballpark.
• Along with Infobright, ParAccel, and SAND, Exasol appears to be another of the “good columnar technology/small company” crowd. As with other such products, one should be careful about fit-and-finish features that are missing today, as there is no assurance they’ll be added in a timely manner going forward.
• illuminate Solutions, which was on last year’s Gartner list, now appears to be an ex-company.

• Sumo Logic does SaaS (Software as a Service) log management.
• Sumo Logic is text indexing/Lucene-based. Thus, it is reasonable to think of Sumo Logic as “Splunk-like”. (However, Sumo Logic seems to have a stricter security/trouble-shooting orientation than Splunk, which is trying to branch out.)
• Sumo Logic has hacked Lucene for faster indexing, and says 10-30 second latencies are typical.
• Sumo Logic’s main differentiation is automated classification of events.
• There’s some kind of streaming engine in the mix, to update counters and drive alerts.
• Sumo Logic has around 30 “customers,” free (mainly) or paying (around 5) as the case may be.
• A truly typical Sumo Logic customer has single to low double digits of gigabytes of log data per day. However, Sumo Logic seems highly confident in its ability to handle a terabyte per customer per day, give or take a factor of 2.
• When I asked about the implications of shipping that much data to a remote data center, Sumo Logic observed that log data compresses really well.
• Sumo Logic recently raised a bunch of venture capital.
• Sumo Logic’s founders are out of ArcSight, a log management company HP paid a bunch of money for.
• Sumo Logic coined a marketing term “LogReduce”, but it has nothing to do with “MapReduce”. Sumo Logic seems to find this amusing.

What interests me about Sumo Logic is that automated classification story. I thought I heard Sumo Logic say:

• It’s largely unsupervised machine learning.
• It’s specific to a particular user/data set.
• It can be up and running and classifying things effectively almost instantly (i.e., on seconds’ or minutes’ worth of data).
• It’s informed by what different users tag as false positives. (Or maybe that is planned for future versions.)

I have a little trouble seeing how all those points fit exactly together, so perhaps I got some details wrong.

The payoff is that machine learning directly informs the Sumo Logic user interface. In particular, large numbers of events are bundled into a small number of categories, hopefully making it much easier for network operations types to scan the UI and pick out what’s important.

In general, the idea of machine-learning informing analytic UIs via some sort of classification is common in text-oriented technologies, notably in:

• Good ol’ text search.
• Text mining vendors’ approaches to clustering hits on words or phrases that say substantially the same thing.

But otherwise it seems kind of rare, if we stipulate that ad-serving/general internet personalization isn’t really an analytic UI — but I’d love to hear of any interesting examples I’ve overlooked.

Mobile business intelligence — real business value or just a snazzy demo?

I discussed some mobile BI use cases in July 2010, but I’m still not convinced the whole area is a legitimate big deal. BI has a long history of snazzy, senior-exec-pleasing demos that have little to do with substantive business value. For now, I think mobile BI is another of those; few people will gain deep analytic insights staring into their iPhones. I don’t see anything coming that’s going to change the situation soon.

BI-centric collaboration — real business value or just a snazzy demo?

I’m more optimistic about collaborative business intelligence. QlikView’s direct sharing of dashboards will, I think, be a feature competitors must and will imitate. Social media BI collaboration is still in the “mainly a demo” phase, but I think it meets a broader and deeper need than does mobile BI. Over the next few years, I expect numerous enterprises to establish strong cultures of analytic chatter (and then give frequent talks about same at industry conferences).  

Business intelligence for mid-market enterprises is problematic

Given the saturation of the large-enterprise BI market with supposed enterprise-standard BI systems, it would seem that smaller enterprises comprise a large part of the BI growth opportunity. However, the large-enterprise and mid-range BI markets are very different. For example:

• Large enterprises typically have tough challenges in data integration; smaller enterprises may truly start out with their data in only a few systems.
• There are many reasons for large enterprises not to do their BI in the cloud, such as bandwidth, internal politics, or the unsuitability of most cloud infrastructure for analytic DBMS scale-out. Smaller enterprises, however, may prefer SaaS (Software as a Service) BI.
• The BI market for smaller enterprises is heavily OEM. But unless you’re buying some kind of data/analytics bundle, the large enterprise BI market still seems overwhelmingly standalone.
• Large-enterprise BI tools incorporate much of a DBMS-like technology stack; at smaller enterprises, BI can often stick to its specialized-application-development-tool knitting. But on the other hand …
• … large enterprises almost always already have a data warehousing infrastructure. Mid-range BI buyers may not have a separate analytic DBMS. Therefore …
• … BI/DBMS bundles make more sense in the mid-market than they do at large enterprises.
• Each large enterprise has a unique infrastructure, and  commonly a unique competitive situation as well. Thus, the idea that you’ll pre-build most of an analytic application for a large enterprises — because you know what data model they need to do their BI — usually turns out to be silly. But smaller enterprises can be more homogeneous, and so for them pre-built analytic applications can actually work.

I don’t know of anybody who’s really cracked the code on mid-market BI. Crystal Reports (long owned by SAP Business Objects) has huge OEM share, but somehow hasn’t parlayed that into a comprehensive mid-market BI presence. Various SaaS or on-premise vendors have cool product ideas — e.g. Gooddata, Endeca, or my clients at PivotLink — but none seems to have set the world on fire to this point.

Departmental BI is doing better

The news is happier in a related market — business intelligence for departments of larger enterprises. However, this is a hard market to analyze, for at least two reasons. First — as is often the case — the distinction among large-enterprise-wise, smaller-enterprise-wide, and departmental BI is not a clear one.* Second, “departmental BI” has at least two major strains:

• Simple, pedestrian BI, implemented quickly.
• Investigative analytics.

*In particular, it has been the case since the 1990s that BI tools first get sold to departments, hopefully for fast implementations — think 4-6 weeks as a base case — and then spread out internally after their initial successes. I am frequently amused by vendors who claim to have pioneered that sales model sometime over the past decade, or even within the past few years.

That said, there are two main kinds of reason to do your BI departmentally, at arm’s length from central IT.

• Perhaps, for good reason or bad, IT is being insufficiently helpful at managing the data.
  • This can be a straightforward matter of politics and priorities — IT controls the data, but is slow about giving you access.
  • Also, you may want to include data that’s outside IT’s purview, be it third-party or just purely departmental.
• Further, you may want functionality that corporate-standard BI doesn’t offer. Potential examples include:
  • Cool analytic visualization.
  • “Real-time” data visualization.
  • The ability to play nicely with particular kinds of data sets.

I have a lot more to say about those points — but not in a post that’s already as long as this one.

This post is a moderately edited form of an actual interview. Two other posts cover analytic trends to watch (planned) and analytic vendor execution challenges to watch (already up).

Question: What do you think will happen next year with the Tableaus of the world?

Answer:

• I think adoption of flexible-visualization business intelligence tools will continue to be rapid.
• I think enterprise-friendly features will be increasingly important as a basis of competition.

Question: What do you mean by “enterprise-friendly”?

Answer: An example would be QlikTech no longer forcing you to use their native ETL, but rather working with Informatica and soon other third-party products. Also important can be:

• Database size.
• Concurrency.
• A full-featured development cycle for analytic applications.

Question: What does HP have to do to be relevant in analytics/data warehousing?

Answer: Avoid stupidity. HP Vertica is already relevant.

Question: OK. But what can HP do to build on Vertica?

Answer: HP — which botched Exadata 1 hardware — could do a good job with SAP HANA or other kinds of appliance products.

However:

• I don’t think trying to force Vertica beyond its natural growth — the way EMC is with Greenplum — is necessarily a good idea. Natural growth in Vertica’s case is plenty fast anyway.
• Obviously, making good Vertica hardware would be nice. But being hardware-independent is crucial to Vertica, not least because of cloud deployment, an option many buyers want to at least have in their hip pockets.

Question: You expressed some skepticism toward mobile BI/use cases. Why so?

Answer: The form factor hurts functionality a lot, so it’s only worthwhile in cases where timeliness is key.

And without more refined alert-setting functionality, it’s hard to think of that many cases.

Note: My views on mobile BI haven’t changed much since July, 2010.

Question: What about the idea of an enterprise being able to pay-per-drink to run jobs on an analytic cluster. Do you expect that concept to have any legs in 2012?

Answer: While other kinds of SaaS (Software as a Service) BI might make sense, remote computing BI that focuses on hardware cost sharing is problematic. Moving data in and out of the cluster is a big part of the overall cost, at least if you plan to process it only occasionally once it gets there. I haven’t seen a plan yet that gets around that point.

Top-level points included:

• Exasol’s technical philosophy is substantially the same as before, albeit not with as extreme a focus on fitting everything in RAM.
• Exasol believes its flagship DBMS EXASolution has great performance on a load-and-go basis.
• Exasol has 25 EXASolution customers, all in Germany.*
• 5 of those are “cloud” customers, at hosting providers engaged by Exasol.
• EXASolution database sizes now range from the low 100s of gigabytes up to 30 terabytes.
• Pretty much the whole company is in Nuremberg.

*That excludes some money from Hitachi. Exasol’s Hitachi partnership is still in limbo, an apparent casualty of the world economic crisis.

On the technical side:

• As noted in my 2008 post, EXASolution is a columnar, no-head-node MPP (Massively Parallel Processing) DBMS.
• The main way EXASolution compresses data is via dictionary/tokenization. 5:1 is a typical compression ratio before mirroring and so on, out of a 2-10:1 range.
• EXASolution writes data to blocks in memory that are smaller than what is otherwise its preferred size (1/2 to 5 megabytes). These are sent to disk, where merge eventually happens. Exasol insists that write performance has always been fully satisfactory to customers to date.
• EXASolution doesn’t have much in the way of performance tuning knobs. Exasol says they aren’t needed, and says that one really can start an EXASolution POC (Proof of Concept) in a day or so.
• EXASolution doesn’t have much in the way of workload management capabilities, except what’s automagic (e.g., short query bias). However, it does collect statistics you can query via your favorite BI tool.
• EXASolution doesn’t have much in the way of analytic platform capabilities, although there is some Lua-based scripting. However, there’s something NDA in the analytic platform area Coming Soon.*

In general, the whole thing sounds somewhat like ParAccel, at least at a high level.

*Exasol is not and never has been our client, but we can keep secrets for them even so.

Naturally, Exasol believes EXASolution has fine concurrency, with at least one customer routinely running 2000 concurrent users, 200 concurrent sessions (via connection pooling), and 5-10 concurrent queries. Another customer has 3500 Cognos users. 1-200 concurrent queries appears to be the record peak load. Anyhow, Exasol says that plans to offer real workload management could be accelerated if a need were discovered.

Exasol says it almost never loses POCs, but admits that it competes fairly rarely against Vertica and ParAccel, no doubt for reasons of geography. Exasol boasts one visible Sybase IQ replacement (Sony Music).

While Exasol’s sales to date have been in Germany, there are plans to change that soon. At least one sales cycle is well underway in Eastern Europe. Offices in other Germanic countries are planned. Existing customers are planning to deploy additional copies outside Germany. Discussions are underway regarding other geographies, e.g. English-speaking ones.

I’m not hugely optimistic about the Oracle Database Appliance. Rather, my thoughts — lightly edited from a chat with a reporter — include:

• This doesn’t solve Oracle’s SMB problems, which include:
  • Oracle software is too difficult and costly to administer. The appliance will make a modest dent in that one, but it’s not any kind of game-changer, because the issues relate to the antique design of the Oracle DBMS. (I.e., I think ongoing database administration is a bigger deal than, say, one-time system set-up.)
  • SMBs use third-party applications whenever they can, with an increasing preference for SaaS.  Application and SaaS vendors prefer non-Oracle alternatives when they are feasible.
• Thus, Oracle is not well positioned to thrive in the SMB market … except maybe through its MySQL subsidiary, but that has a long way to go too.
• Clayton Christensen’s The Innovator’s Solution teaches us that Oracle should focus on selling a thick stack of technology to its highest-end customers, and that’s exactly what Oracle does focus on.

• 100s of gigabytes of data at first, growing to >1 terabyte over time.
• High peak loads.
• Public cloud portability (but they have private data centers they can use today).
• Simple database design — not a lot of tables, not a lot of columns, not a lot of joins, and everything can be distributed on the same customer_ID key.
• Stream the data to a data warehouse, that will grow to a few terabytes. (Keeping only one year of OLTP data online actually makes sense in this application, but of course everything should go into the DW.)

So I’m leaning to saying:  

• They should go with a scalable, MySQL-based solution.
  • Lots of third-party software works with MySQL, in case that’s helpful.
  • Yes, any one vendor is small and not yet firmly established, but there are numerous vendors around with interesting MySQL scaling stories.
  • In a vendor emergency, just going with Oracle’s MySQL stuff would probably work …
  • … especially because there are these lovely things in the world called solid-state drives.
  • There’s also good escapability if one wants to move away from MySQL, because everybody knows how to handle MySQL data.
• The first product to look at is dbShards, because it meets all the topology needs:
  • Local scale-out (transparent sharding).
  • Local high availability.
  • Remote disaster recovery (details of that are underway).
• The first analytic DBMS to look at is Infobright.
  • Yes, I know Infobright is focused more on machine-generated data these days, but this client’s analytic needs are so straightforward Infobright should pass with flying colors.
  • The MySQL-to-MySQL aspect should make ETL dead simple.
  • Again, there’s escapability.

Mainly, this is all fine. But I’m getting pushback on the solid-state aspect, for fear that it will compromise public cloud portability.

Am I missing something here? As far as I’m concerned, if you’re planning an OLTP system with a many-year lifespan today, of course you should assume solid-state storage. Maybe you scale out just as far as you would with disk, striping indexes or entire databases across the RAM of multiple servers. It that case, having solid-state backing reduces the risk of bottlenecks. Maybe you don’t scale out as far as you would with disk. In that case, solid-state backing saves you money.

As for public-cloud support for solid-state storage, that’s coming fast, right? (Actually, I have data points in support of that theory, but they’re a bit tenuous.) A large fraction of web businesses with private data centers seem to be using solid-state storage — from Facebook on down — or so the NoSQL/NewSQL/short-request DBMS guys tell me. Surely a number of public cloud vendors are close behind.

• A long-ago marketing decision to not give infrastructure details, so as to convey a “Don’t worry; we’ll take care of everything” message.
• Even so, a long-ago and perhaps now-regretted marketing decision to disclose and even exaggerate salesforce.com’s reliance on Oracle, as part of an early-days attempt to prove salesforce was using enterprise-class technology.
• A desire to hide the recipe for salesforce.com’s secret sauce.
• Force of habit — I’m not sure salesforce even knows how to tell its technical story with any clarity.

Actually, salesforce.com has moved some kinds of data out of Oracle that previously used to be stored there. Besides Oracle, salesforce uses at least a file system and a RAM-based data store about which I have no details. Even so, much of salesforce.com’s data is stored in Oracle — a single instance of Oracle, which it believes may be the largest instance of Oracle in the world.

Salesforce did spell out some of its database story in a 2008 force.com white paper, which is good stuff, but potentially misleading in one important way. The paper tells of a level of abstraction, whereby what the application sees as logical “columns” are stored in a very different schema than one might assume. However, it doesn’t spell out a second level of abstraction, whereby that logical schema also isn’t how the database is actually laid out.

Another flaw in the paper is that it spins “We had to do this, to support multitenancy, so we did.” issues as “Because we’re multitenant, we can do this, while single-tenant systems can’t.” One example is the query optimization step around “user visibility” in Figure 11. Welcome to marketing.

At the first level of abstraction, data seems to be kept mainly in a single wide table, with hundreds of columns. What’s more, many of those are “flex columns”; a flex column can hold data of many different kinds and even datatypes. Notwithstanding the second level of abstraction, I imagine the idea of stuffing different kinds of thing into the same column has something to do with the fact that Oracle’s physical limit on columns falls far short of the number of logical columns salesforce wants to use.

If we imagine that the different kinds of data in a flex column were each in their own column instead, the whole thing might sound like BigTable/Cassandra/HBase-style column-group NoSQL. Thus, much as Workday uses MySQL to simulate a key-value store, salesforce.com can be said to use Oracle to simulate a different kind of NoSQL. In both cases, what’s going on seems to be a kind of object/relational mapping, but with the relational aspect strongly deemphasized. Or, if you take a more relational view, we could say that salesforce.com’s tables are a lot wider than any one user organization’s, because each user sees only its own custom columns (plus the standard ones common to all users).

The second layer of abstraction has a lot to do with multitenancy. If you want to stick data for many different user organizations into the same huge table, then you have to label it in some way to show who is permitted to see or update each part. Logically, this leads to a join, between one table carrying data plus a simple key showing which users/roles are entitled to see it, and a second table showing who actually is that kind of user/has that kind of role. But that join makes a lot of sense to store in a denormalized way, all the more because data is partitioned across the computer cluster in line with which user organization it actually belongs to.

Multitenant security isn’t the only reason for this denormalization, but it appears to be the biggest one.

The whole thing is doing 550 million or so transactions per day. salesforce.com thinks that fact should be regarded as evidence that it works.

• salesforce.com. This is the parent company, and the runaway leader in the SaaS (Software as a Service) enterprise application market, especially in the area of CRM (Customer Relationship Management).
• force.com. This is salesforce.com’s application development stack split out for other SaaS vendors to use, both inside and outside the CRM segment. It can be referred to as a PaaS offering (Platform as a Service). force.com relies on a proprietary salesforce.com language called APEX, which has a strong stored procedure/ database trigger orientation.
• database.com. This is the database part of force.com, spun out separately in general availability as of Dreamforce two weeks ago.
• data.com. Also launched at Dreamforce (and based, if I understand correctly, on an acquisition), this is a provider of 3rd-party data you might use as inputs to your CRM systems.
• Heroku. Another salesforce.com acquisition, Heroku is in essence a PaaS competitor to force.com. Heroku is focused on Ruby and Java, and supports a number of DBMS, SQL and NoSQL alike.
• AppExchange. This is a marketplace for things designed to integrate with salesforce.com (and perhaps also apps built on force.com). The latest claim is that there are 1200+ AppExchange offerings.
• The complete set of SaaS apps built on force.com. A 2008 white paper refers to 47,000 organizations being “supported” by force.com. Recently I’ve heard a figure just under 100,000. I’m not clear as to what that metric measures — aggregate users of SaaS apps built  via force.com? Clearly there are a lot of SaaS apps built on force.com, with actual customers, but I don’t know how big “a lot” is. (Perhaps a salesforce.com person could chime into the comment thread with some clarity.)

The pricing for force.com and database.com is clearly designed for enterprise SaaS applications whose users will be inside customer organizations. If you want to do something public-facing, prices are prohibitive without a special deal. That’s the bad news. The good news is that salesforce.com says, publicly and privately, that it’s indeed open to cutting such volume pricing deals.

When I talked with CTOs and the like at some Dreamforce-exhibiting SaaS vendors, on the whole they seemed very happy with force.com. The one repeated complaint was that force.com imposed unpleasant rate limits (e.g., number of API calls). Working around those limits involves unnatural acts of coding, phones calls to helpful salesforce.com staffers to get the limits raised, or both. When I talked with salesforce.com cofounder Parker Harris, he seemed painfully aware of the problem, and indicated that relaxing the limits is an important technical goal.

The other force.com weakness I uncovered was expected — while it may be great as long as your application matches its implicit assumptions, there are some things it can’t easily do. This has been a recurring issue since database-oriented 4GLs (Fourth-Generation Languages) came around in the 1980s. For example, one firm wanted a Flash UI — I’m not sure why — and went outside force.com for that part of the application.

Related link

• The database architecture of salesforce.com, force.com, and database.com

Context for any comments on customer traction includes:

• Membase went into limited production release in October, and full release in January. Similar things are true of CouchDB.
• Hence, most sales of Couchbase’s products have been made over the past 6 months.
• Couchbase (the merged product) is at this point only in a pre-production developer’s release.
• Couchbase has both a direct sales force and a classic open-source “funnel”-based online selling model. Naturally, Couchbase’s understanding of what its customers are doing is more solid with respect to the direct sales base.
• Most of Couchbase’s revenue to date seems to have come from a limited number of big-ticket “lighthouse” accounts (as opposed to, say, the larger number of smaller deals that come in through the online funnel).

That said,

• Most Membase purchases are for new applications, as opposed to memcached migrations. However, customers are the kinds of companies that probably also are using memcached elsewhere.
• Most other Membase purchases are replacements for the Membase/MySQL combination. Bob says those are easy sales with short sales cycles.
• Pure memcached support is a small but non-zero business for Couchbase, and a fine source of upsell opportunities.
• In the pipeline but not so much yet in the customer base are SaaS vendors and the like who use and may want to replace traditional DBMS such as Oracle. Other than among those, Couchbase doesn’t compete much yet with Oracle et al.
• Pure CouchDB isn’t all that much of a business, at least relative to community size, as CouchDB is a single-server product commonly used by people who are content not to pay for support.

Membase sales are concentrated in five kinds of internet-centric companies, which in declining order are:

• Social gaming
• Ad platforms
• Online retail
• Online business, including B2B  SaaS
• Social networking

Bob said that Couchbase often sees MongoDB competitively, but never Riak, HBase, or Redis. I got the impression Couchbase sees at least a little Cassandra. That would, of course, all pertain only to direct sales, rather than download/community kinds of usage.

Couchbase is also excited about the potential for the CouchDB-based Couchbase Mobile occasionally-connected offering. The hottest use cases, interestingly, seem to be non-consumer; Bob rattled off military, farming, and health care, and surely could have named more besides. However, the Couchbase Mobile sales effort still seems to be in early days, as is evidenced by the fact that Couchbase has not yet competitively encountered Sybase SQL Anywhere.

With all that said, I’ll go now to a separate post for a Couchbase technical update.