It’s blindingly obvious that Zynga is one of Vertica’s petabyte-scale customers, given that Zynga sends 5 TB/day of data into Vertica, and keeps that data for about a year. (Zynga may retain even more data going forward; in particular, Zynga regrets ever having thrown out the first month of data for any game it’s tried to launch.) This is game actions, for the most part, rather than log files; true logs generally go into Splunk.

I don’t know whether the missing data is completely thrown away, or just stashed on inaccessible tapes somewhere.

I found two aspects of the Zynga story particularly interesting. First, those 5 TB/day are going straight into Vertica (from, I presume, memcached/Membase/Couchbase), as Zynga decided that sending the data to some kind of log first was more trouble than it’s worth. Second, there’s Zynga’s approach to analytic database design. Highlights of that include:

• Data is divided into two parts. One part has a  pretty ordinary schema; the other is just stored as a huge list of name-value pairs. (This is much like eBay‘s approach with its Teradata-based Singularity, except that eBay puts the name-value pairs into long character strings.) About half the data is in each part, but I don’t think that’s by deliberate choice.
• Zynga adds data into the real schema when it’s clear it will be needed for a while. This isn’t a matter of query volumes, for the most part; rather, it’s when Zynga’s tests (e.g. of new games?) have determined that the data will keep being collected and used for a while.
• Zynga only adds columns to its analytic  database; it never goes through the more complex process of deleting them.

Just as Zynga is one of Vertica’s flagship accounts, LinkedIn is one of Aster Data’s. Specifically, before leaving LinkedIn for Aster, Jonathan Goldman built LinkedIn’s People You May Know feature in Aster nCluster. This was long ago, and I’m not sure how sophisticated his use of SQL and MapReduce would be in today’s terms; for example, I was told he didn’t use “nPath or anything like that.” (Edit: See the comments below for clarifications from Jonathan.) Anyhow, LinkedIn has replaced Aster for PYMK with Hadoop, and in my opinion is getting much better results.

That, from an Aster standpoint, is the bad news. The good news is that LinkedIn is happily using Aster nCluster for several other applications; LinkedIn folks doesn’t seem to regret throwing out* Greenplum for Aster; and they also seem to have a very high opinion of Jonathan and his work while he was there.

*And this time that is indeed the phrase that was used.

One thing that astonished me is that LinkedIn PYMK is based only on data innate to LinkedIn (as opposed to imported email addresses, the results of web crawls, and so on). Given that, I am at a loss to explain how it suggested a couple of old friends, to whom I have no discernable chain of connection. Yes, we were at Harvard at the same time, but if that’s all it was, there would be a huge number of false positives I’m not actually seeing.

Context for any comments on customer traction includes:

• Membase went into limited production release in October, and full release in January. Similar things are true of CouchDB.
• Hence, most sales of Couchbase’s products have been made over the past 6 months.
• Couchbase (the merged product) is at this point only in a pre-production developer’s release.
• Couchbase has both a direct sales force and a classic open-source “funnel”-based online selling model. Naturally, Couchbase’s understanding of what its customers are doing is more solid with respect to the direct sales base.
• Most of Couchbase’s revenue to date seems to have come from a limited number of big-ticket “lighthouse” accounts (as opposed to, say, the larger number of smaller deals that come in through the online funnel).

That said,

• Most Membase purchases are for new applications, as opposed to memcached migrations. However, customers are the kinds of companies that probably also are using memcached elsewhere.
• Most other Membase purchases are replacements for the Membase/MySQL combination. Bob says those are easy sales with short sales cycles.
• Pure memcached support is a small but non-zero business for Couchbase, and a fine source of upsell opportunities.
• In the pipeline but not so much yet in the customer base are SaaS vendors and the like who use and may want to replace traditional DBMS such as Oracle. Other than among those, Couchbase doesn’t compete much yet with Oracle et al.
• Pure CouchDB isn’t all that much of a business, at least relative to community size, as CouchDB is a single-server product commonly used by people who are content not to pay for support.

Membase sales are concentrated in five kinds of internet-centric companies, which in declining order are:

• Social gaming
• Ad platforms
• Online retail
• Online business, including B2B  SaaS
• Social networking

Bob said that Couchbase often sees MongoDB competitively, but never Riak, HBase, or Redis. I got the impression Couchbase sees at least a little Cassandra. That would, of course, all pertain only to direct sales, rather than download/community kinds of usage.

Couchbase is also excited about the potential for the CouchDB-based Couchbase Mobile occasionally-connected offering. The hottest use cases, interestingly, seem to be non-consumer; Bob rattled off military, farming, and health care, and surely could have named more besides. However, the Couchbase Mobile sales effort still seems to be in early days, as is evidenced by the fact that Couchbase has not yet competitively encountered Sybase SQL Anywhere.

With all that said, I’ll go now to a separate post for a Couchbase technical update.

10gen recommends solid-state storage in many cases. In some cases solid-state lets you get away with fewer total nodes. 10gen also likes Flashcache (Facebook-developed technology to put a flash cache in front of hard disks). But the 100-node example mentioned above uses spinning disk.

Use cases 10gen is proud of include:

• Lots of user profile maintenance, including at online ad companies. This includes full user ad impression data. (I’ve argued for a while that user profile information belongs in something like a NoSQL database.)
• A big-name web company that wants to inspect every packet that enters their network, and replaced Splunk with MongoDB for performance reasons.
• A big-name photo/video site whose metadata is all in MongoDB. (That’s the kind of thing that often makes for good MarkLogic use cases.)

But actually, the reason we had the call was to review cases where MongoDB’s schemaless nature was significant. Examples of those included:

• A couple of top examples were of the kind “A bunch of apps, similar but not the same.” For MTV, it’s a single content management system for a bunch of websites. For Disney Playdom, it’s different schemas for every game.
• For a wireless telco, the issue was a product catalog in which devices and service plans called for very different schemas, and which the telco felt had thus become unmanageable in Oracle.
• For Craigslist, the issue wasn’t programming so much as performance — ALTER TABLE operations took months in MySQL, and that’s not a typo, although I’ll confess to not understanding why this was the case.

The 10gen guys went on to claim that schemalessness is helpful for incremental development in general, the point being that you don’t have a database-modification step. To some extent, changes can even be rolled back more easily than if you actually changed your schemas.

Last week, Mike Stonebraker insulted MySQL and Facebook’s use of it, by implication advocating VoltDB instead. Kerfuffle ensued. To the extent Mike was saying that non-transparently sharded MySQL isn’t an ideal way to do things, he’s surely right. That still leaves a lot of options for massive short-request databases, however, including transparently sharded RDBMS, scale-out in-memory DBMS (whether or not VoltDB*), and various NoSQL options. If nothing else, Couchbase would seem superior to memcached/non-transparent MySQL if you were starting a project today.

*The big problem with VoltDB, last I checked, was its reliance on Java stored procedures to get work done.

Pleasantries continued in The Register, which got an amazing-sounding quote from Mike. If The Reg is to be believed — something I wouldn’t necessarily take for granted — Mike claimed that he (i.e. VoltDB) knows how to solve the distributed join performance problem. 

So, it’s Stonebraker against the web. And the difference of option is severe. In May, at a MongoDB developer conference in San Francisco, Mongo creator Dwight Merriman told his audience there was “no way” to do distributed joins in a way that really scales. “I’m not smart enough to do distributed joins that scale horizontally, widely, and are super fast. You have to choose something else. We have no choice but to not be relational,” he said

“You can do distributed transactions, but if you do them with no loss of generality and you do them across a thousand machines, it’s not going to be that fast.”

Stonebraker says precisely the opposite, and in typical fashion, he goes right for the jugular. “I reject what Merriman says out of hand,” he tells The Register. Merriman and his company, 10gen, declined to comment for this story. But Stonebaker says words don’t matter. As much as he likes to wield his opinions, he insists the debate will be decided elsewhere. “Let the bake-off begin,” he crows.

But when last I checked, VoltDB made nowhere near that claim. And well it shouldn’t have. In the fully general case, there’s no way to ensure super distributed join performance other than by throwing lots and lots of gear at the problem. But if you do that, many alternatives are fast. More specialized cases may be a different matter — but there are many fast alternatives for those too.

I imagine there will be use cases for which VoltDB sustains a lead as the truly fastest alternative, similarly-architected competitors perhaps excepted.* But what Mike supposedly said seems quite forward-leaning when compared to technical reality.

*The canonical VoltDB use case is e-commerce in virtual goods, the point of “virtual” being that physical inventory might necessitate costlier kinds of joins.

Sybase said that Sybase IQ had > 2000 direct customers and >500 indirect customers (i.e., end customers of OEMs). That’s counting by customers; I know from prior discussions that Sybase IQ is running at close to two installations per customer. I also believe that Sybase counts different divisions of the same large enterprise as separate customers.

Vertica cited a figure of 500 customers as of April (end Q1?), which is close to 600 now, about 40% or a little more direct. The difference between this and a 2010 year-end figure of 328 is not only new sales, but also slow reporting by OEMs.  One cool figure — a single OEM reported 82 end sales in a single (quarterly?) report. And a number of those direct customers are substantial; Vertica’s customer logo page features lots of telcos, lots of internet companies, and the national operation of Blue Cross/Blue Shield.

Pay no attention to small inconsistencies in the number of Vertica direct customers (250 at year-end, no more than that now); Colin Mahony just estimates these numbers for me from memory, and minor inaccuracies are quite excusable.

Even cooler — Vertica reports 7 customers with a petabyte or more of user data each. About 5 of the 7 are obvious-suspect big-name firms; but unsurprisingly, those big names are NDA. I did secure permission to say that there are 2 telecom companies, a mobile gaming vendor, another internet company, and 3 financial services outfits of various kinds.

SAND Technology reported >600 total customers, including >100 direct. Since SAND has been around since the 1990s, those aren’t great average annual figures, but they’re probably more than many people (including me) thought.

Infobright reported around 200 total paying customers, 130 direct. There are surely a lot more users of open source Infobright, but precise numbers are of course hard to come by.

If I asked ParAccel in the April go-round, I’ve misplaced their answer, but back in October the figure was >30 customers, 2 of them over 100 terabytes. I’ve seen published figures of 40+ for ParAccel since.

There’s something akin to “social graphs” and “social network analysis” that is more or less worthy of all the current hype – but graphs and network analysis are only a minor part of the whole story.

In particular, the most important parts of the Facebook “social graph” are neither social nor a graph. Rather, what’s really important is an aggregate Profile of Revealed Preferences, of which person-to-person connections or other things best modeled by a graph play only a small part.

Let me hasten to note that – even when viewed narrowly — the ideas of “social graph”and “social network analysis” do have significance. Nontrivial use cases to date for big data social network analysis include:

• Intelligence agencies identify and analyze terrorist networks. Corporations and civilian law enforcement do the same for fraud networks.
• Telephone companies use calling data to figure out which of their customers are most likely to influence which other customers in the decision to keep or change service providers. (Frankly, I find that rather creepy.)
• Social networks figure out which other members you’re likely to know, and encourage you to connect with them.

Epidemiologists aspire to add to that list, based on their success to date using much more micro forms of social network analysis. But after that, I’m running out of examples. Sure, graph analytics is good for a bunch of other things (e.g., biology at the genetic or molecular level), but those have little or nothing to do with “social graphs” or social network analysis as they are commonly understood.

Note: Of course, it is also the case that everything can be modeled by entity-attribute-value triples, and those can always be modeled by graphs. But so what?

Let’s consider what, in a marketer’s ideal world, would go into your Profile of Revealed Preferences. Raw data might include:

• Personally identifyING information. Duh. This is what makes everything else possible.
• Purchase transaction data. Lots of it. Like, everything on your credit card statements.
• Demographic and lifestyle information. Address, date of birth, educational history, race, household composition, and so on.
• Affiliations. Politics, religion, group membership of any kind. (OK, that’s partly social.)
• Explicitly stated opinions, preferences and desires, including:
  • Any surveys you have filled out.
  • Any recommendations you have made (e.g., through the Facebook Like feature).
  • The text of anything you’ve written and posted – and, very ideally, of your private emails as well.
  • Any wish lists you’ve filled in.
• Attention information. What you clicked on, what you looked at, and all that stuff website owners track.
• Your movements, to the extent they are tracked. (E.g., via Foursquare and the like.)
• Your gaming activities and the like. (This is social mainly to the extent it overlaps with other categories I’ve already mentioned.)
• Your medical information.
• Who you communicate with, and what you communicate with them about. (Hey! There’s something else social!)
• Similar information about the people you communicate with.

My core privacy thoughts about that data include:

• Individuals deserve the right to control all that information. At a minimum, they deserve total control over how the data (raw or derived) is passed from the service – e.g., website – where it naturally resides (e.g., where it is originated) to any other place.
• Given a chance, individuals would make fine-grained choices about what parts of their Profile of Revealed Preferences are available to which organizations. Reasons include:
  • Individuals have rather complex trust relationships with different kinds of merchants and marketers.
  • Consumers get different benefits from sharing information with different kinds of merchants and marketers. (Sometimes personalization is a large benefit. Sometimes it’s just creepy. And some companies actively bribe you to give them information they can use to sell to you.)

When one frame things this way, two rather difficult technological questions naturally arise.

1. Suppose, implausibly, that a single entity were allowed to control and use (for marketing) all of your Profile of Revealed Preferences information. How would they store and analyze it?
2. How does the answer to #1 change because control over the information will, in fact, be fragmented?

It’s tough enough to answer these questions for data about one person. Trying to include all but the simplest information about other people is and will for years remain quite infeasible. So, for the most part, this is not “social” information.

It’s also not naturally a “graph.” Similarly, it is not a good candidate for network analysis. To see why, let me outline why I used the name “Profile of Revealed Preferences”:

• The reason marketers want this data is, mainly, because they want to know what appeals to you, and how strongly you feel about it.
• The analytic process often entails taking explicit choices you have made, and inferring other preferences from them.
• The output of the analytic process is often one or more “scores” that then get plugged into various selection algorithms to determine what you should be shown or offered. At least implicitly, these algorithms are predicting what you will or won’t respond well to.

Not much graph-like there.

This post has gotten pretty long, so I’ll stop here without spelling anything else out. But questions I still hope to address down the road include:

• How should Profile of Revealed Preferences data be stored?
• Suppose we want to pass around derived results and not the raw data. How could we ever get to standards that would make such interchange realistic?
• If we only have raw data to pass around, what are the implications for privacy, liberty, and the structure of the online industries?

• VoltDB is based on the H-Store technology, which I wrote about in February, 2009. Most of what I said about H-Store then applies to VoltDB today.
• VoltDB is a no-apologies ACID relational DBMS, which runs entirely in RAM.
• VoltDB has rather limited SQL. (One example: VoltDB can’t do SUMs in SQL.) However, VoltDB guy Tim Callaghan (Mark Callaghan’s lesser-known but nonetheless smart brother) asserts that if you code up the missing functionality, it’s almost as fast as if it were present in the DBMS to begin with, because there’s no added I/O from the handoff between the DBMS and the procedural code. (The data’s in RAM one way or the other.)
• VoltDB’s Big Conceptual Performance Story is that it does away with most locks, latches, logs, etc., and also most context switching.
• In particular, you’re supposed to partition your data and architect your application so that most transactions execute on a single core. When you can do that, you get VoltDB’s performance benefits. To the extent you can’t, you’re in two-phase-commit performance land. (More precisely, you’re doing 2PC for multi-core writes, which is surely a major reason that multi-core reads are a lot faster in VoltDB than multi-core writes.)
• VoltDB has a little less than one DBMS thread per core. When the data partitioning works as it should, you execute a complete transaction in that single thread. Poof. No context switching.
• A transaction in VoltDB is a Java stored procedure. (The early idea of Ruby on Rails in lieu of the Java/SQL combo didn’t hold up performance-wise.)
• Solid-state memory is not a viable alternative to RAM for VoltDB. Too slow.
• Instead, VoltDB lets you snapshot data to disk at tunable intervals. “Continuous” is one of the options, wherein a new snapshot starts being made as soon as the last one completes.
• In addition, VoltDB will also spool a kind of transaction log to the target of your choice. (Obvious choice: An analytic DBMS such as Vertica, but there’s no such connectivity partnership actually in place at this time.)

I should also note that when Tim Callaghan described architectural options to get around 2PC performance issues, they sounded a lot like eventual consistency. Maybe tunable RYW consistency isn’t in the cards, but at least there’s a NoSQL-like possibility with VoltDB.

VoltDB’s open source strategy is:

• VoltDB will be open sourced.
• Community VoltDB will be GPLed. Professional Edition VoltDB has a non-GPL license.
• The VoltDB Professional Edition won’t start out with features beyond the Community Edition ones, but will gain such later on. I didn’t get the sense the plans for those features were completely baked yet, but ideas mentioned included:
  • Management/monitoring tools.
  • Integration with expense closed-source enterprise software products, such as ones in the management/monitoring area.
  • Yet more “extreme”/edge-case performance.
• Before VoltDB decided for sure that it wasn’t selling licenses, it sold a license to Getco, which also seems to be an investor in the company.

VoltDB had a beta test with about 150 participants. None is in production yet, although at least a few are clearly headed there. Most VoltDB beta testers are in some kind of online business, with a particular concentration in everybody’s new favorite market, online gaming. Most of the rest are in investment/trading — a major target market for at least three different Mike Stonebraker companies — and a few are in telecom. VoltDB assures me that some of the beta users are companies one actually has heard of before, but VoltDB is not in a position to name any of those.

VoltDB is not ideally suited for a classic order management system, since you’d want to partition both on CustomerID and SKU, the latter because you’d constantly updating inventory stock levels. However, this argument doesn’t apply in the case of virtual goods. Virtual goods that are sold for real money — and hence need ACID levels of transaction integrity — are thus a clear target market for VoltDB. (The example that came up was in, you guessed it, online gaming.) The other interesting use case that Tim highlighted was low-latency analytics/ELT. For reasons I didn’t totally grasp, Tim likes to call this “Stateful ELT.” (Given that the data goes into the VoltDB database before much else happens to it, I’m pretty sure I heard “ELT” correctly. But I guess I might have been mishearing “ETL”.)

VoltDB company highlights include:

• VoltDB has about a dozen employees, all but two of whom are technical. (However, I’m not sure they’re counting Andy Ellicott against the two. But then, last I heard he wasn’t full time at VoltDB.)
• VoltDB’s venture funding status is, if I may paraphrase, “Mumble mumble.”
• Although long separate from Vertica, VoltDB is still located in Vertica’s offices.

• Nothing in my original short post about Clustrix was actually incorrect.
• Clustrix plans to reveal actual production “name-brand” customers soon.
• The name of Clustrix’s software, or at least the guts thereof, is Sierra.
• Clustrix’s products have actually been in general availability since last quarter, with some versions at customer sites for 2 years. Development started 3 ½ years ago.
• Clustrix says its technology is for OLTP systems, which it calls “non-batch/non-analytic,” with mixed read/write workloads. All Clustrix’s example target markets are “internet verticals,” such as photo sharing, gaming, social media, e-commerce, etc.
• Clustrix’s heart is in SQL, as is most of its customer base. Clustrix Sierra’s key-value-store option has little or no performance advantage over Clustrix Sierra’s SQL option, nor any other advantage over SQL that came up in discussion.
• Clustrix Sierra is “wire-compatible” with MySQL, but doesn’t use MySQL code; Clustrix wrote all the code itself.
• Clustrix asserts that Clustrix Sierra supports the “vast majority” of MySQL features. Examples of MySQL features Clustrix doesn’t support at this time are full-text search and geospatial indexing.
• Indeed, Clustrix claims Clustrix Sierra can be used to replace MySQL with few or zero changes to existing applications.
• I specifically asked about referential integrity, which has a poor performance reputation in MySQL. Besides saying they supported it, Clustrix said that some customers actually use referential integrity in some of their less active tables.
• Clustrix Sierra is fully ACID-compliant, with no eventual consistency or RYW consistency story. The default number of copies of each datum is two, and they’re kept consistent via two-phase commit.
• Clustrix Sierra is fully parallel, with no “head” node. I forgot to ask how it was determined which queries would be addressed to and/or controlled by which nodes, but I presume there’s some sort of a load-balancing scheme.
• Clustrix says that because Clustrix Sierra uses MVCC (Multi-Version Concurrency Control), and thus reads and writes don’t block each other, global locks aren’t a major issue. (They’re rare or short or something – I have trouble seeing why they would be non-existent.)
• Clustrix says there’s a second class of locks and latches that are purely local and short-lived, for B-tree indexes and the like. (I didn’t drill down into those either.) I guess this means Clustrix Sierra is B-tree-centric, which makes sense for an OLTP-oriented system.
• Clustrix Sierra distributes data among nodes via consistent hashing (default), range partitioning, or “full distribution”(i.e., copying a – presumably small – table to each node). The choice of distribution plans is manual now; more automation is a future feature.
• Clustrix Sierra’s CBO (Cost-Based Optimizer) is, as one would hope, distribution-aware.
• Clustrix Sierra compiles query fragments and ships them off to the relevant nodes. A fragment might contain both instructions for SQL to be executed locally and for where data is to be sent next.
• Clustrix says that Clustrix Sierra does data migration and redistribution (e.g., when you add a node) transparently online, and further says that in practice this doesn’t cause a performance hit.
• As for Clustrix hardware:
  • Clustrix makes Type I appliances.
  • A Clustrix node contains 2 quad-core chips, 32 gigs of RAM, and 7 160 GB solid-state drives.
  • Specifically, Clustrix is using Intel SSDs, with a SAS interface.
  • Clustrix says solid-state memory isn’t really essential to the product design; it’s just cheap in terms of $/IOPS (I/O Per Second).
• A minimum Clustrix configuration is 3 nodes, for redundancy. After that you can add nodes one at a time. Clustrix says it built a 20-node system in-house, leading me to suspect that customers don’t have anything bigger than 20 nodes either.
• That 20-node Clustrix system was tested to show near-linear scalability. (In discussing this, Clustrix tends to forget to use the word “near”.)
• Clustrix has partnered with somebody to provide global 4-hour-response support. As of now Clustrix seems to be active mainly in North America and Europe.
• Clustrix is formed from the combination of two startups, which I’ve heard elsewhere were called Clustrix and Sprout. Exactly when the combination happened sounds a little different depending on who’s telling the story (one version has the predecessors still being separate well into 2008, but Clustrix implies the combination happened pretty much on Day 1).

• Vertica plans to build a massive, world-conquering sales force.
• If Vertica dips back into negative cash flow to do that and has to raise more venture capital, so be it.
• “Triple-digit” revenue growth is expected for this year.

*I’ve since heard more both from Ralph and his former colleagues, and I’m comfortable taking the move more or less at face value — for some reasons he doesn’t want to spell out, Ralph really wanted to move back home to South Africa.

While they were at it, Vertica also put out a press release reporting very good success in the social gaming market. The biggest and best known of the bunch is Zynga. Three months ago, Wayne Eckerson had figures of 3 TB/day added to the database, 200 nodes, and >40 million users. Now Zynga is using a figure of >65 million daily users and 230 nodes. More precisely, at Zynga:

• There are two Vertica databases with identical data.
• Each Zynga Vertica database runs on 115 nodes.
• Zynga’s two Vertica database clusters are used for different applications.
• It’s undisclosed exactly what Zynga runs on what Vertica cluster. But best practice would be to put mission-critical, fast-response stuff on one cluster, and use the other for longer-running or less-critical queries — plus have it be available as hot standby — given that I don’t see much reason to put data geographically close to users around the world for reasons of latency or whatever.
• An undisclosed amount of data, amounting to all of what Wayne earlier estimated at 3 TB, is added to each of Zynga’s Vertica databases daily.

In other news, Vertica now states its customer count as being >130.

• Computer, network, and other equipment logs
• Satellite and similar telemetry (whether for espionage or science)
• Location data such as RFID chip readings, GPS system output, etc.
• Temperature and other environmental sensor readings
• Sensor readings from factories, pipelines, etc.
• Output from many kinds of medical device, in hospitals and (increasingly) homes alike

The core idea here is that human-generated data can grow only as fast as human data-generating activities allow it to, but machine-generated data is limited only by capital budgets and Moore’s Law.  So machines’ ability to generate data is growing a lot faster than humans’.

Up to this point, I think there’s broad agreement, at least on the part of anybody who’s thought about it this way for very long. But that still leaves open questions as to which kinds of machine-generated data will matter first. The big five that matter right now are:

• Web logs (partially machine-generated, but tied to human actions)
• Call detail records (CDRs — ditto)
• Financial instrument trades (some purely machine-generated, some human-based)
• Network event logs (commonly associated with web logs)
• Telemetry collected by the government (especially for intelligence purposes)

A large fraction of all the 100 TB+ or petabyte+ data warehouse activity I know of falls into those areas.

Following along quickly are:

• Online game data. Since late last year, online game companies have come up over and over again as an important category of data warehousing/analytics users. Like most of the categories above, the gaming area actually features a hybrid between human- and machine-generated data.
• Genetic research data, although I don’t know to what extent the investment in data gathering is concentrated among the few obvious big pharmaceutical companies. Other health care data (research or clinical) will come along too, but doesn’t seem to be there yet.

Until recently I would have added:

• Energy exploration, energy production, energy refining, and/or utility network data

But while those areas seemed poised to get hot last year, I haven’t heard much about them the past few months, with a few exceptions:

• Accenture’s observation that new smart grids will generate up to eight orders of magnitude more data than old dumb grids do
• The recent article about the Terralliance fiasco (new kinds of oil exploration analytics, going beyond seismological data)
• Lots of  concern about security flaws in utility smart grids.

Finally, I’ve been assuming that a big area going forward is location data, especially personal movement data. The data volumes involved could be similar to or even greater than those of CDRs. But privacy concerns with that are obviously immense. (Of course, in the case of Foursquare, this sort of overlaps with freely-shared game data.)

If you want to make all this more tangible in your mind, one area to look for ideas is in the huge amount of news about various kinds of innovative sensors. Sources include:

• Somebody named Landon Cox, who maintains a couple feeds of sensor news.
• A Twitter feed, apparently associated with a Sensor Expo.
• Another Twitter feed, this one from Sun Labs. (I have no idea what Oracle is or isn’t doing with the Sun SPOT project that links to.)
• Yet another Twitter feed.