memcached is a way to cache data in RAM across a cluster of servers and have it all look logically like a single memory pool, extremely popular among large internet companies. The Membase product — which is what Couchbase has been selling this year — adds persistence to memcached, an obvious improvement on requiring application developers to write both to memcached and to non-transparently-sharded MySQL. The main technical points in adding persistence seem to have been:

• A persistent backing store (duh), namely SQLite.
• A change to the hashing algorithm, to avoid losing data when the cluster configuration is changed.

Couchbase is essentially Membase improved by integrating CouchDB into it, with the main changes being:

• Changing the backing store to CouchDB (duh). This will be in the first Couchbase release.
• Adding cross data center replication on CouchDB’s consistency model. This will not, I believe, be in the first Couchbase release.
• Offering CouchDB’s programming and query interfaces as an option. So far as I can tell, this will be implemented straightforwardly in the first Couchbase release, with elegance planned for later down the road.

Let’s drill down a bit into Membase/Couchbase clustering and consistency.

• When data is written to RAM in memcached, it immediately gets copied to another server. The same is of course true in Membase/Couchbase. The terminology on all this is confusing, but I think:
  • The portion of data that is stored as a primary copy on any given server is called a “shard”.
  • That would seem to make sense, as that data could correspond to what goes — non-transparently — into an instance of MySQL in a classical memcached/MySQL set-up.
• Updates are of course also banged to disk ASAP — but at times of heavy load, that can take a while. A few seconds to a couple of minutes is normal operation; if it takes an hour, you really should buy more hardware. (Or solid-state storage.)
• Similarly, the replication of data to a second machine’s RAM may not happen at times of heavy load — and that’s another sign you don’t have enough machines.
• Each Membase/Couchbase “shard” has lots of logical sub-shards.* (1024 for now, at least as default, although Dustin finds that number excessive and is looking to lower it.)  So if you add a node, some of the sub-shards get sent over to the new node. Unlike the case for straight memcached, no data is lost from cache (and of course not also from the persistent store). Blocking of operations from such a move only happens in narrow time windows, and then only in edge cases.

*Edit: They’re called vbuckets.

So if we consider Membase technology alone, Couchbase is CA in the CAP Theorem.  CouchDB, however, is gloriously AP in the CAP Theorem, in that it was written to assume an occasionally connected topology.* Based on that, Couchbase will allow AP operation between data centers (i.e. “stay synchronized if you can, to within the limitations of physics and so on, but don’t beat yourself up on the rare occasions that you can’t.”) I don’t know that that capability will quite be in the first release of Couchbase, but it’s coming soon.

*CouchDB also has other features friendly to occasionally-connected use cases, such as a lot of flexibility as to which parts of the database are or aren’t synced when you do reconnect. These are at the heart of the Couchbase Mobile offering.

memcached and Membase have a very simple key-value interface. CouchDB adds secondary indexes and so on. I think in the first release of Couchbase this is pretty much like having two different APIs for the same product; more elegant integration is planned down the road, and more language support as well.

The highest-performing way to use Couchbase will probably always be to just pretend it is Membase, which is to say memcached+. Dustin told me of Membase users who demanded 10-40 millisecond response times, and that not even for single queries but rather for sequences of several queries in succession. He further told me of customers asking for 1-200 microsecond response, and insisting on no worse than 1 millisecond. Frankly, the first requirement could be met by lots of technologies I can think of, at least if  you don’t rely on disk; the second is thoroughly impossible if you rely on disk, and pretty demanding no matter what kind of hardware and storage you have.

Couchbase performance against disk is a work in progress. CouchDB started out 8X slower than SQLite as a backing store, apples to apples, but Couchbase is fixing that before they roll the product out. (After all, they wouldn’t want to slow the product down in the course of an upgrade.) Beyond that, when you do exploit the indexing capability of CouchDB, performance of course slows down. Work is underway to lower the performance hit; I imagine much improvement can indeed be made, given how few resources CouchDB has been able to devote to date to Bottleneck Whack-A-Mole.

• Facebook et al. are in effect Software as a Service (SaaS) vendors, not enterprise technology users. In particular:
  • They have the technical chops to rewrite their code as  needed.
  • Unlike packaged software vendors, they’re not answerable to anybody for keeping legacy code alive after a rewrite. That makes migration a lot easier.
  • If they want to write different parts of their system on different technical underpinnings, nobody can stop them. For example …
  • …  Facebook innovated Cassandra, and is now heavily committed to HBase.
• It makes little sense to talk of Facebook’s use of “MySQL.” Better to talk of Facebook’s use of “MySQL +  memcached  + non-transparent sharding.” That said:
  • It’s hard to see why somebody today would use MySQL +  memcached  + non-transparent sharding for a new project. At least one of Couchbase or transparently-sharded MySQL is very likely a superior alternative. Other alternatives might be better yet.
  • As noted above in the example of Facebook, the many major web businesses that are using MySQL +  memcached  + non-transparent sharding for existing projects can be presumed able to migrate away from that stack as the need arises.

Continuing with that discussion of DBMS alternatives:

• If you just want to write to the memcached API anyway, why not go with Couchbase?
• If you want to go relational, why not go with MySQL? There are many alternatives for scaling or accelerating MySQL — dbShards, Schooner, Akiban, Tokutek, ScaleBase, ScaleDB, Clustrix, and Xeround come to mind quickly, so there’s a great chance that one or more will fit your use case. (And if you don’t get the choice of MySQL flavor right the first time, porting to another one shouldn’t be all THAT awful.)
• If you really, really want to go in-memory, and don’t mind writing Java stored procedures, and don’t need to do the kinds of joins it isn’t good at, but do need to do the kinds of joins it is, VoltDB could indeed be a good alternative.

And while we’re at it — going schema-free often makes a whole lot of sense. I need to write much more about the point, but for now let’s just say that I look favorably on the Big Four schema-free/NoSQL options of MongoDB, Couchbase, HBase, and Cassandra.

Last week, Mike Stonebraker insulted MySQL and Facebook’s use of it, by implication advocating VoltDB instead. Kerfuffle ensued. To the extent Mike was saying that non-transparently sharded MySQL isn’t an ideal way to do things, he’s surely right. That still leaves a lot of options for massive short-request databases, however, including transparently sharded RDBMS, scale-out in-memory DBMS (whether or not VoltDB*), and various NoSQL options. If nothing else, Couchbase would seem superior to memcached/non-transparent MySQL if you were starting a project today.

*The big problem with VoltDB, last I checked, was its reliance on Java stored procedures to get work done.

Pleasantries continued in The Register, which got an amazing-sounding quote from Mike. If The Reg is to be believed — something I wouldn’t necessarily take for granted — Mike claimed that he (i.e. VoltDB) knows how to solve the distributed join performance problem. 

So, it’s Stonebraker against the web. And the difference of option is severe. In May, at a MongoDB developer conference in San Francisco, Mongo creator Dwight Merriman told his audience there was “no way” to do distributed joins in a way that really scales. “I’m not smart enough to do distributed joins that scale horizontally, widely, and are super fast. You have to choose something else. We have no choice but to not be relational,” he said

“You can do distributed transactions, but if you do them with no loss of generality and you do them across a thousand machines, it’s not going to be that fast.”

Stonebraker says precisely the opposite, and in typical fashion, he goes right for the jugular. “I reject what Merriman says out of hand,” he tells The Register. Merriman and his company, 10gen, declined to comment for this story. But Stonebaker says words don’t matter. As much as he likes to wield his opinions, he insists the debate will be decided elsewhere. “Let the bake-off begin,” he crows.

But when last I checked, VoltDB made nowhere near that claim. And well it shouldn’t have. In the fully general case, there’s no way to ensure super distributed join performance other than by throwing lots and lots of gear at the problem. But if you do that, many alternatives are fast. More specialized cases may be a different matter — but there are many fast alternatives for those too.

I imagine there will be use cases for which VoltDB sustains a lead as the truly fastest alternative, similarly-architected competitors perhaps excepted.* But what Mike supposedly said seems quite forward-leaning when compared to technical reality.

*The canonical VoltDB use case is e-commerce in virtual goods, the point of “virtual” being that physical inventory might necessitate costlier kinds of joins.

• Human/Tabular data –i.e., human-generated data that fits well into relational tables or arrays.
• Human/Nontabular data — i.e., all other data generated by humans.
• Machine-Generated data.

I won’t now stand by every nuance in that post, which may differ slightly from those in my more recent posts about machine-generated data and poly-structured databases. But one general idea is hard to dispute:

Traditional database data — records of human transactional activity, referred to as “Human/Tabular data above” — will not grow as fast as Moore’s Law makes computer chips cheaper.

And that point has a straightforward corollary, namely:

It will become ever more affordable to put traditional database data entirely into RAM. 

Actually, there are numerous ways for OLTP, other short-request, and some analytic databases to wind up in RAM.

• SAP has some good ideas for how it could happen, banging transactions into what is essentially an in-memory analytic database. (I dispute SAP’s claims of transformational database technology leadership, but that doesn’t mean the underlying ideas aren’t good.)
• For those who can afford the associated technology disruption, memory-centric object-oriented DBMS could be appealing.
• Web scalability best practices commonly include keeping data in RAM (e.g., that’s pretty much the point of caching layer memcached).
• SaaS (Software as a Service) companies — such as Workday — often bring a particular tenant’s database entirely into RAM.
• QlikView highlights the benefits of doing business intelligence in RAM.
• SAS HPA makes the argument that even “big data analytics” should sometimes be done in RAM.
• I don’t have particularly favorable opinions at this time about marketing strategies or momentum at Oracle TimesTen, IBM solidDB, or VoltDB, but those examples at least serve to illustrate that memory-centric OLTP DBMS have existed for years.
• Actually, SAP has at least two good ideas, if you count Sybase as part of SAP.

And here’s the kicker: Intel told me last year that CPUs are headed to 46-bit address spaces around mid-decade. Indeed, they hired me to help figure out if that was enough.* That multiplies out to 64 terabytes of RAM on a single server, chip costs permitting. So most of what we now think of as operational databases — and many of the analytic ones too — will fit in-memory, even if they run very large businesses.

*And did so without putting the discussion under any kind of NDA.

Likely consequences of all this include:

• Legacy apps will (eventually) be consolidated and virtualized in-memory. Their underlying databases will grow so slowly that eventually the cost of putting them in RAM will be too low to worry about.
• Expensive storage systems will (continue to) be irrelevant to database processing. Databases that don’t fit in RAM will typically be big enough to require the attention of a lot of CPUs — and in those cases the DBMS software itself will handle all the storage tasks.
• Major OLTP DBMS vendors, such as Oracle, will need alternate in-memory code lines, because disk-centric architectures are sub-optimal in-memory. Well, that’s what they have those big R&D budgets for.
• SaaS vendors and web businesses may not rely on today’s major OLTP DBMS vendors. (I was going to say “won’t” rather than “may not” until I recalled the likely M&A endgame.) Traditional enterprises may blanch at migrating away from their legacy DBMS environments, but the trade-offs are different for technology companies using DBMS as subsystems.

Of course, the same trends that make data-storing chips cheaper will make data-generating chips cheaper too. So, just as there are huge amounts of machine-generated data that you’d never pay to store in RAM, the same will still be true 10 years from now; the data volumes involved will just be a lot bigger. And thus there will still be plenty of very large analytic databases using relatively cheap forms of storage, perhaps even disk.

But OLTP and other short-request processing are likely to wind up in-memory. And the same may be true for a considerable amount of analytics, especially but not only if the analytics have a low-latency requirement.

An object-oriented database management system (OODBMS, but sometimes just called “object database”) is a DBMS that stores data in a logical model that is closely aligned with an application program’s object model. Of course, an OODBMS will have a physical data model optimized for the kinds of logical data model it expects.

If you’re guessing from that definition that there can be difficulties drawing boundaries between the application, the application programming language, the data manipulation language, and/or the DBMS — you’re right. Those difficulties have been a big factor in relegating OODBMS to being a relatively niche technology to date.

Examples of what I would call OODBMS include: 

• Intersystems Cache’, the most successful OODBMS product by far, with good OLTP (OnLine Transaction Processing) capabilities and a strong presence in the health care market. Although it was designed around the specialized MUMPS/M language, Cache’ happily talks Java and SQL.
• ObjectStore, a well-pedigreed startup a couple decades ago, which wound up focusing on complex objects in markets such as computer-aided design. ObjectStore was eventually sold to Progress Software, which is positioning ObjectStore more as a distributed caching system than anything else (Amazon was an impressive reference for that use case). That said, Progress’ ObjectStore business is small, as is its ObjectStore level of effort. Both Cache’ and ObjectStore were at some point unsuccessfully targeted at the XML database market.
• Part of Workday’s technology stack. Very-well-pedigreed SaaS (Software as a Service) application vendor Workday decided to go with what amounts to an in-memory OODBMS. This makes all kinds of sense, and is a lot of what rekindled my interest in object-oriented database management.
• Objectivity, also from the 20-years-ago generation, and a poster child for the “DBMS toolkit as much as a DBMS” issue.
• McObject Perst, an embeddable memory-centric OODBMS.
• Versant. Actually, by now the Versant company has several different OODBMS; I’m not sure whether what it’s selling has much to do with the original Versant product. Anyhow, both the original and current Versant product seem to be positioned in OLTP. Versant has recently suffered from declining revenues, in license fees and maintenance alike.
• Forthcoming technology from Starcounter, in the area of high-performance memory-centric OLTP. According to my correspondents, Starcounter still needs to explain how its technology is different from what Versant and ObjectStore introduced 20 or so years ago. Interestingly, while ObjectStore shines as a distributed system, Starcounter’s developers have consigned scale-out to the “too hard to bother with” category.
• Gemstone, which seemed to be on an ObjectStore-like caching track until it was acquired by VMware.

Arguably, OODBMS have all the benefits of document-model DBMS, but with different language bindings. And if you’re going to write in an object-oriented language anyway, those language bindings can seem pretty appealing. In particular, they might be preferable to fighting your way through object/relational mapping.

Other than the double-edged language sword, the main criticism of object-oriented DBMS is that they include a whole lot of pointers. Intersystems and others have shown that, even in a disk-centric world, OODBMS can have excellent performance in OLTP and tolerable performance in simple reporting. As RAM gets cheaper, memory-centric operation becomes ever more viable, making the pointers even less problematic.

Bottom line: If I were starting a SaaS project today, I’d give serious consideration to memory-centric OODBMS technology.

• 95% scalability up to 64 machines
• 90% scalability up to 88 machines
• 89% scalability up to 112 machines
• 84% scalability up to 128 machines

More precisely, those are counts of cluster “members,” but the recommended configuration is one member per operating system instance — i.e. one member per machine — for reasons of availability. In an 80% read/20% write workload, scalability is less — perhaps 90% scalability over 16 members.

Several elements are of IBM’s DB2 pureScale architecture are pretty straightforward:

• There are multiple pureScale members (machines), each with its own instance of DB2.
• There’s an RDMA (Remote Direct Memory Access) interconnect, perhaps InfiniBand. (The point of InfiniBand and other RDMA is that moving data doesn’t require interrupts, and hence doesn’t cost many CPU cycles.)
• The DB2 pureScale members share access to the database on a disk array.
• Each DB2 pureScale member has its own log, also on the disk array.

Something called GPFS (Global Parallel File System), which comes bundled with DB2, sits underneath all this. It’s all based on the mainframe technology IBM Parallel Sysplex.

The weirdest part (to me) of DB2 pureScale is something called the Global Cluster Facility, which runs on its own set of boxes.  (Edit: Actually, see Tim Vincent’s comment below.)These might have 20% or so of the cores of the member boxes, with perhaps a somewhat higher percentage of RAM (especially in the case of write-heavy workloads). Specifically:

• The DB2 pureScale Global Cluster Facility maintains a buffer pool (cache) shared by all the DB2 pureScale members.
• Even so, the DB2 pureScale members themselves are in charge of disk access.

So what’s going on here is not an Exadata-like split between database server and storage processing tiers. The Global Cluster Facility also handles lock management, presumably because locking issues only arise when a page gets fetched into the buffer.

The other surprise is that every client talks to every member, usually through a connection pool from an app server. Tim Vincent assures me that DB2 connections are so lightweight this isn’t a problem. Clients have load-balancing code on behalf of the members, and route transactions to whichever pureScale member is least busy.

DB2 pureScale is designed to be pretty robust against outages:

• In the case of planned maintenance, a pureScale member can be “quiesced.” I.e., it stops being given new work; it finishes up its existing work; then maintenance happens; then the member starts being given work again.
• In the case of an unplanned outage, the redo log naturally comes into play. The pureScale twist on this is that a second small instance of DB2 is around — or is started up? — just to handled the redos.

Also, IBM believes that the DB2 pureScale locking strategy gives availability and performance advantages vs. the Oracle RAC (Real Application Cluster) approach. The distinction IBM draws is that any member can take over the lock on a buffer page from any other member, just by attempting to change the page — and the attempt will succeed; only row-level locks can ever block work.  Thus, if a node fails, I/O can merrily proceed on other nodes, without waiting for any recovery effort. IBM’s target is <20 seconds for full row availability to be restored.

Obviously, it’s crucial that the Global Cluster Facility machines be fully mirrored, with no double failure — but so what? Modern computing systems have double-points-of-failure all over the place.

• Given Oracle’s market penetration and share, it makes sense that Oracle is focused on selling add-on products to its installed base. Oracle’s three top such go-to-market emphases at the moment are:
  • Database consolidation, especially on Exadata.
  • Data warehousing, presumably on Exadata.
  • Database security, especially encryption. This is not Exadata-specific, but does exploit Intel Westmere on-chip encryption, which Oracle says allows encryption with minimal overhead. This seems to be via something called Oracle Advanced Security.
• Deleted*

*Oracle asked me to delete a point on pricing they went out of their way to make, because they are in quiet period — even though nobody said it was confidential at the time, we weren’t under NDA, and it looks like public information to me anyway. Frankly, I’m not sure I was right to comply.

Oracle also told me quite a bit about Exadata onsite POCs (Proofs of Concept) and Exadata references, but I’ll save those subjects for future posts. The same goes for workload management.

Oracle’s version names and numbers can get confusing, but it turns out that:

• Oracle 11.203 11.2.0.3 will come out this fall. Oracle 11.204 11.2.0.4 will come out a little more than a year later. After that I imagine it will be time for Oracle 12.
• The current versions of Oracle Exadata are Exadata X2-2 and Exadata X2-8.
  • Oracle Exadata 2-2 is evolutionary from prior Exadata versions, and has 8 moderately big servers per rack. It can be sliced into half- or quarter-racks.
  • Oracle Exadata 2-8, in lieu of those 8 servers, has 2 bigger SMP (Symmetric MultiProcessing) systems, each with a terabyte of RAM. You can’t slice Exadata 2-8 below full-rack size, as you’d lose redundancy among the servers.

I didn’t really understand the discussion as to why certain workloads and/or workload consolidations go better on the SMP boxes of Exadata X2-8 than the blades of Exadata X2-2, but Oracle assures me that some do. I also suspect that some Oracle customers prefer large SMP boxes for no good reason other than familiarity.

As for recent-release adoption:

• Oracle estimates that 40-50% of customers have Oracle 11g running somewhere in their shops, mainly Oracle 11g Release 2.
• All major ISVs (Independent Software Vendors) are certified on Oracle 11g, typically Oracle 11g Release 2.
• But Exadata certification is something different from Oracle 11g certification; for example, SAP certification on Exadata is still underway, targeted for some time this year.

Exadata obviously enjoys huge performance gains over existing Oracle installations for certain analytic queries, and therefore for some whole analytic workloads. Oracle has happily trumpeted these. But it turns out that Exadata’s OLTP (OnLine Transaction Processing) performance gains are less dramatic. This makes all kinds of sense, given that Oracle’s analytic query performance was in pretty bad shape pre-Exadata, while OLTP has been just fine. The range Oracle used was 2-3X OLTP performance gains vs. existing Oracle installations on several-year-old hardware. Oracle says somewhere over 50% of Exadata physical I/O* goes against flash cache in uses cases such as running Oracle’s application suite.

*Note that physical I/O may be only a small fraction of logical; e.g., SAP long ago said that >99% of SAP transactions never hit disk.

Finally, we talked about a variety of options or other related products. Highlights included:

• One piece of the Oracle security story is a new product called Oracle Database Firewall, released in January, based on an acquisition of a small startup last year. Targeted primarily at internal hackers, Oracle Database Firewall sniffs your SQL traffic for a week or so, observes what kinds of SQL statements can be expected, builds a white list accordingly, and casts a jaundiced eye on any other kind of SQL statements that come through.
• Edit: I have no idea why I was told the following, in view of a subsequent email. Oracle Active Data Guard, first introduced in the Oracle 11g code line, is the preferred way to do active-active Oracle replication. That said:
  • Not a lot of customers use Oracle Active Data Guard yet …
  • … but a considerable fraction of Exadata users are at least interested in it.
  • Some number of Oracle customers have other kinds of active-active implementation. One option is via GoldenGate.
• Oracle Cloud File Management System is an Oracle 11g feature/option that lets you managed non-Oracle data. It is related to ASM (Automatic Storage Management), which seems to have been the most popular Oracle 10g feature, and which is essential to Exadata. Oracle Cloud File Management Systems seems to be popular for consolidation uses. But it is not technically well suited to, for example, play the role of HDFS in a MapReduce implementation.
• For DBAs who care, Exadata now supports Solaris on the database server tier as well as Linux. (That would be Solaris on Intel, of course; Exadata doesn’t use Sparc.) The storage tier still runs only on a kind of embedded Linux.
• Oracle 11g Express Edition (free crippleware) just went into beta test.
• And finally, Oracle SQL Developer 3.0 features, among other things, a GUI for Oracle Data Mining, and migration tools. Sybase migration is in there now, and was enhanced for SQL Developer 3.0. Teradata migration is slated for the next release.

In essence, Couchbase will be CouchDB with scale-out. Alternatively, Couchbase will be Membase with a richer programming interface. The Couchbase sweet spot is likely to be: 

• Internet applications, especially ones that involve connectivity between a host and mobile devices.
• Delivery of data, content, and/or software across a network. (That’s a high-profile CouchDB use case today.)
• (Possibly) transactions for virtual goods that have no scarcity. (Once there’s actual inventory involved, the traditional relational database model starts looking pretty appealing.)

And now let’s go to the lists of bullet points.

Background to the Membase/CouchDB/Couchbase integration story:

• Membase is a key-value store with the memcached interface. Its strengths are memcached compatibility and performant scale-out. What it stores are in essence JSON documents.
• CouchDB is designed for ease of programming, and for built-in handling of occasionally-connected replication. (Not coincidentally, Damien Katz used to work on Lotus Notes.) CouchDB indexes individual data fields for reasonable query capability, although joins are problematic. What CouchDB stores are in essence JSON documents.

Highlights of how Membase works and is deployed today:

• Your API is Get/Set, just like in memcached.
• To a first approximation, Membase just persists memcached cache at every node. That said, it can certainly store more data per node than fits in cache.
• Most Membase installations are in Amazon EC2, where flash memory is not available. Most in-house Membase installations, however, use flash.

Business background on Couchbase predecessors:

• Membase raised $15 million, had 20 employees, and has a number of paying customers.
• CouchOne raised $2 million, had 16 employees, hadn’t focused much on traditional customer acquisition yet or on building an enterprise edition of the product, and had about 4 customers anyway …
• … except that CouchOne’s plans included CouchDB hosting, and there are around 4500 users of same in a free beta that’s on the verge of going non-free. Damien positions his hosting as being focused on high throughput and concurrency, while rival CouchDB host Cloudant is in his opinion more focused on big data.
• The apparent repositioning of CouchOne as being highly focused on mobile applications (with unreliable host connections) never really had time to take hold. Indeed …
• … Damien asserts that CouchDB has a lot more mission-critical enterprise deployments than MongoDB, whereas he concedes that MongoDB is doing great in a Ruby-centric market.

Happy talk around Membase/CouchDB/Couchbase product integration:

• Hey, both Membase and CouchDB talk JSON.
• Product strengths and weaknesses are synergistic. For example:
  • Membase started with caching technology (memcached). CouchDB doesn’t yet make much use of cache.
  • Membase’s back end is SQLite, used in a “dumb” way. CouchDB can presumably do everything the dumb implementation of SQLite can.
  • Membase’s scale-out is designed for a single data center, with strict consistency. CouchDB’s is designed for wide-area networks, with eventual consistency. At least one big internet company likes the idea of strict consistency within data centers, but eventual consistency among them.
  • The CouchDB interface takes the place of something Membase planned to build called Node Code, which was going to overcome the limitations of a simple key-value interface. Node Code development didn’t ever really get started, and indeed was deferred for a couple of months while CouchOne acquisition discussions were underway. However, Membase did build Node Code’s underpinnings, called the “TAP” interface.
  • And on the operations side: Membase has been in Mountain View, right by the CalTrain. CouchOne has been in Oakland, but with a lot of at-home workers. One option is to move the Oakland office to a San Francisco location that, you guessed it, is also right by the CalTrain.

Other technical notes:

• The only current API to CouchDB is http/https. memcached protocols will be added to Couchbase.
• CouchDB has design documents. These are used to tell you how to do indexes. They’re built on the fly if they don’t already exist. Then there are Javascript functions that update the indexes as documents are added/updated.
• In particular, CouchDB has a geospatial index, in a true R-tree. Damien fondly thinks it already has most albeit not all the features of PostgreSQL GIS. I gather CouchDB geospatial will be straightforwardly integrated into Couchbase.
• There’s also a CouchDB add-on project for full-text indexing. Damien seems less confident of how that will be integrated into Couchbase.

Finally, I’m curious about the relative performance of Couchbase/Membase and Schooner Membrain when using flash memory. I would guess that the comparison favors Schooner, because of Schooner’s extensive focus on flash optimization. I would also guess that Schooner’s edge is small, because I’d think it would be less than Schooner’s advantage vs. alternative Flash uses on the MySQL side, and Schooner’s MySQL performance advantage seems to be less than 2X even when Schooner is doing the benchmarks.

Membase announced a Cloudera partnership. I couldn’t detect anything technically exciting about that, but it serves to highlight what I do find to be an interesting usage trend. A couple of big Web players (AOL and ShareThis) are using Hadoop to crunch data and derive customer profile data, then feed that back into Membase. Why Membase? Because it can serve up the profile in a millisecond, as part of a bigger 40-millisecond-latency request.

And why Hadoop, rather than Aster Data nCluster, which ShareThis also uses? Umm, I didn’t ask.

When I mentioned this to Colin Mahony, he said Vertica had similar stories. However, I don’t recall whether they were about Membase or just memcached, and he hasn’t had a chance to get back to me with clarification.  (Edit: As per Colin’s comment below, it’s both.)

I wrote back in August about Membase, and that covers most of what I think, with perhaps a couple of exceptions: 

• One point I might put more weight on — by doing the same thing in memory (the well-known memcached caching system) and on disk, Membase may have a pretty slick high-performance memory-centric architecture.
• I’m getting more sympathetic to the idea of just banging objects to disk, whether via a key-value store in some other NoSQL kind of model. E.g., if somebody were to imitate Workday’s or salesforce.com’s technical architectures today — which while of course not identical are actually pretty similar — they might use a key-value store, rather than the actual choices of MySQL and Oracle respectively.

Generally, as per my recent NoSQL overview, I’m still negative about pure key-value stores. However, Membase:

• Has a roadmap to get beyond pure key-value.
• Is plug-compatible with the popular memcached.
• Has a high-performance memory-centric story.

So I can see some appealing Membase use cases.

And in case somebody is wondering why I don’t compare/contrast Membase with Basho’s key-value store Riak — beyond the obvious concerns raised by Basho’s VC down round — let me copy what I tweeted last night in response to a flamefest about same:

I’m not some symbolism-heavy figurehead that should be held responsible for all-inclusive nuance-balancing. I write about as much of what I think is interesting as I can get around to.

Related links

• A post I wrote about RYW consistency based in part on a great discussion with Basho’s Justin Sheehy.
• Two posts by Alex Popescu about Membase. The second has some technical details.