• Neil Raden points out that business intelligence dashboards can be dangerously misleading. His reasoning (sound) is that whatever you measure is apt to be distorted by the fact people know they’re being measured. His solution (implied) is to hire a good-looking consultant like himself to do it right.
• I’ve had my issues with Fred Holahan, who was VP of Marketing when I posted that EnterpriseDB was not to be trusted. (That said, Fred is long gone from EnterpriseDB and my opinion hasn’t changed.) But he’s put up a good series of posts on the basis of the open source “progressive engagement” marketing funnel, including this gem on why you shouldn’t count on monetizing your community/free users.
• Oracle plans to increase its acquisition budget. The figure given is $70 billion over the next 5 years. Edit: But see this funny Register followup.
• Clayton Christensen wrote a phenomenal article on how to live a good life, from a very business-y perspective. (Only in one anecdote was it too religiously-oriented for my tastes.) Takeaways include:
  • Your core goals probably revolve around something other than business success. (E.g., family.) Don’t lose sight of that.
  • To the extent you’re a manager or leader, you may have a huge impact on other people’s lives. Use that power in admirable ways.
  • Teach people how to fish for answers, rather than just giving them answers. They’ll probably come to better conclusions than you would have anyway. (This is a core principle in my own consulting.)
  • Take time to reflect. And by the way, the same techniques you use for strategic analysis in business can be applied to your life as well.
• Mark Suster has a pretty good post expanding on my first Christensen takeaway, highlighting a point too often missing from articles in that genre: It’s not just family; it’s also all the cool things around us.
• I haven’t gone through the Hadoop Summit archives yet, but it looks as if there’s a lot of insight there about current Hadoop application activity.
• If you’re a cat lover and don’t hate simple/traditional music, check out Marc Gunn’s cat filksongs, especially the infectious “What Shall We Do With a Catnipped Kitty?” and “Lord of the Pounce”, both playable from the right sidebar of that page (#7 and #10 respectively). Gunn is also a chief perpetrator of the justly (in)famous Do Virgins Taste Better? cycle of filksongs.
• Former SAP exec Dennis Moore offers a theory as to why SAP cares so much about in-memory DBMS. It’s to integrate business processes, because SAP has no other software layer good at doing same. Interestingly, Dennis originated SAP’s previous attempt at meeting a similar need via its composite applications initiative. However, in Dennis’ view this benefit would only be achieved by a major rewrite of SAP’s applications.

*Who, in Todd’s apt description, is “the sword wielding Johnny Appleseed of the database world”.

Todd wrote:

What matters to VoltDB is: speed at scale, speed at scale, speed at scale, SQL, and ACID. If that matches your priorities then you’ll probably be happy. Otherwise, as you’ll see, everything is sacrificed for speed at scale and what is sacrificed is often ease of use, generality, and error checking. It’s likely we’ll see ease of use improve over time, but for now it looks like rough going, unless of course, you are a going for speed at scale.

Indeed.

Todd’s list of interesting VoltDB features is also pretty good, namely

Main-memory storage.

Run transactions to completion –single threaded –in timestamp order. 

Replicas.

Tables are partitioned across multiple servers.

Stored procedures, written in Java, are the unit of transaction.

A limited subset of SQL ‘99 is supported.

Design a schema and workflow to use single-sited procedures.

Challenging operations model.

No WAN support.

OLAP is purposefully kept separate.

• VoltDB is based on the H-Store technology, which I wrote about in February, 2009. Most of what I said about H-Store then applies to VoltDB today.
• VoltDB is a no-apologies ACID relational DBMS, which runs entirely in RAM.
• VoltDB has rather limited SQL. (One example: VoltDB can’t do SUMs in SQL.) However, VoltDB guy Tim Callaghan (Mark Callaghan’s lesser-known but nonetheless smart brother) asserts that if you code up the missing functionality, it’s almost as fast as if it were present in the DBMS to begin with, because there’s no added I/O from the handoff between the DBMS and the procedural code. (The data’s in RAM one way or the other.)
• VoltDB’s Big Conceptual Performance Story is that it does away with most locks, latches, logs, etc., and also most context switching.
• In particular, you’re supposed to partition your data and architect your application so that most transactions execute on a single core. When you can do that, you get VoltDB’s performance benefits. To the extent you can’t, you’re in two-phase-commit performance land. (More precisely, you’re doing 2PC for multi-core writes, which is surely a major reason that multi-core reads are a lot faster in VoltDB than multi-core writes.)
• VoltDB has a little less than one DBMS thread per core. When the data partitioning works as it should, you execute a complete transaction in that single thread. Poof. No context switching.
• A transaction in VoltDB is a Java stored procedure. (The early idea of Ruby on Rails in lieu of the Java/SQL combo didn’t hold up performance-wise.)
• Solid-state memory is not a viable alternative to RAM for VoltDB. Too slow.
• Instead, VoltDB lets you snapshot data to disk at tunable intervals. “Continuous” is one of the options, wherein a new snapshot starts being made as soon as the last one completes.
• In addition, VoltDB will also spool a kind of transaction log to the target of your choice. (Obvious choice: An analytic DBMS such as Vertica, but there’s no such connectivity partnership actually in place at this time.)

I should also note that when Tim Callaghan described architectural options to get around 2PC performance issues, they sounded a lot like eventual consistency. Maybe tunable RYW consistency isn’t in the cards, but at least there’s a NoSQL-like possibility with VoltDB.

VoltDB’s open source strategy is:

• VoltDB will be open sourced.
• Community VoltDB will be GPLed. Professional Edition VoltDB has a non-GPL license.
• The VoltDB Professional Edition won’t start out with features beyond the Community Edition ones, but will gain such later on. I didn’t get the sense the plans for those features were completely baked yet, but ideas mentioned included:
  • Management/monitoring tools.
  • Integration with expense closed-source enterprise software products, such as ones in the management/monitoring area.
  • Yet more “extreme”/edge-case performance.
• Before VoltDB decided for sure that it wasn’t selling licenses, it sold a license to Getco, which also seems to be an investor in the company.

VoltDB had a beta test with about 150 participants. None is in production yet, although at least a few are clearly headed there. Most VoltDB beta testers are in some kind of online business, with a particular concentration in everybody’s new favorite market, online gaming. Most of the rest are in investment/trading — a major target market for at least three different Mike Stonebraker companies — and a few are in telecom. VoltDB assures me that some of the beta users are companies one actually has heard of before, but VoltDB is not in a position to name any of those.

VoltDB is not ideally suited for a classic order management system, since you’d want to partition both on CustomerID and SKU, the latter because you’d constantly updating inventory stock levels. However, this argument doesn’t apply in the case of virtual goods. Virtual goods that are sold for real money — and hence need ACID levels of transaction integrity — are thus a clear target market for VoltDB. (The example that came up was in, you guessed it, online gaming.) The other interesting use case that Tim highlighted was low-latency analytics/ELT. For reasons I didn’t totally grasp, Tim likes to call this “Stateful ELT.” (Given that the data goes into the VoltDB database before much else happens to it, I’m pretty sure I heard “ELT” correctly. But I guess I might have been mishearing “ETL”.)

VoltDB company highlights include:

• VoltDB has about a dozen employees, all but two of whom are technical. (However, I’m not sure they’re counting Andy Ellicott against the two. But then, last I heard he wasn’t full time at VoltDB.)
• VoltDB’s venture funding status is, if I may paraphrase, “Mumble mumble.”
• Although long separate from Vertica, VoltDB is still located in Vertica’s offices.

That said,  

• Raj said that Sybase’s Aleri acquisition was, if anything, tracking ahead of expectations.
• Raj didn’t seem the slightest bit focused on the Coral8/Aleri CEP-based BI strategy that John Morell had long championed.
• Raj reminded me that Sybase SQL Anywhere has numerous OEMs, not just on the true desktop/laptop or smaller, but also in a return to its server/workgroup roots. Sybase SQL Anywhere even added geospatial indexing recently.

Raj also spoke glowingly of SAP’s in-memory database technology and the potential for Sybase of same — until I asked a follow-up question. At that point, he confessed that he didn’t really know much about about SAP’s in-memory database technology yet. As I said before, I believe SAP is fairly sincere about its belief that its in-memory database technology will conquer the world — but this is a naive and poorly-founded opinion even so.

One tidbit I did get is that SAP’s in-memory database technology is not just son-of-T-REX. A Korean (Raj thinks) company SAP had acquired is also in the mix. Raj also had the impression SAP’s in-memory technology can do rows, columns, or hybrid structures. On the one hand, that makes sense. On the other, it’s not a perfect fit with what Hasso Plattner said last year.

• Sybase’s main businesses are:
  • Classic OLTP DBMS (Sybase ASE, for Adapative Server Enterprise, unless I’ve missed yet another name change).
  • Analytic technology — mainly Sybase IQ, but more generally Sybase RAP.
  • Mobile technology. (The frequently renamed small DBMS SQL Anywhere was the foundational product of and still is included in the mobile division.)
• SAP’s thoughts on in-memory database management are interesting. However, I think SAP’s oft-repeated claim that it has a lot of important in-memory database technology to bring to Sybase (or for that matter SAP customers) is mainly smoke and mirrors. Cool data access methods, good niche database products, and broadly applicable multi-domain DBMS innovations are three different things. Granting that SAP probably has the first and thinks it has the second is not the same as giving it much credence for having the third.
• SAP claims that, 15 years after its refusal to support Sybase turned Sybase into a DBMS also-ran, it by now is “relatively simple” to port SAP’s apps to Sybase ASE, and that they will make that happen. I actually believe that SAP’s apps will soon run on Sybase ASE, where by “soon” I mean “in a couple of years for no-apologies general availability.” (Certifying a DBMS for SAP is a long process.) The main missing features — e.g., row-level locking — were already put into Sybase back in the last millenium. Nor could there be fundamental architectural problems that keep SAP from supporting Sybase ASE, or else SAP couldn’t have supported Microsoft SQL Server (which, long ago, was a Sybase fork).
• I don’t see any market or competitive dynamics that would lead the SAP acquisition to hurt Sybase’s ASE or mobile businesses. General merger management mishegas is, of course, always a possibility.
• SAP Business Objects partners with Sybase IQ’s competitors. That could be a problem. However, coopetition is pretty strong in the business intelligence market. I don’t think any of SAP Business Objects, IBM Cognos, or Oracle Business Intelligence are much held back from partnering by competitive dislike of their parent companies.
• The rest of SAP might be able to drum up some extra business for Sybase IQ.
• It would be natural for IBM/Cognos to now buy a columnar DBMS of its own. Vertica is an obvious first choice. ParAccel would surely come much cheaper. Since ParAccel has little chance of surviving as an independent company — too immature and too little differentiation to overcome that — I’d expect ParAccel’s board to jump at the chance to sell out.
• It would be interesting if SAP Business Objects would revive the CEP-based BI idea.
• I gather Sybase’s AnswersAnywhere concept network/object model-based natural language/speech recognition technology never went anywhere. Unsurprising (it seemed like it needed too much hand-building to scale semantically), but regrettable even so.
• I don’t see anything in this acquisition that would revive PowerBuilder (Sybase’s Visual Basic competitor), Sybase’s CASE (Computer-Aided Software Engineering) tools, and so on.
• And on the personal side — I’ll probably lose Sybase as a customer due to this merger, but it could have been worse. A lot of vendors smaller than Sybase are bigger customers for Monash Research.

Edit: Right after I posted this, I saw email from Sybase clarifying that Sybase’s in-memory technology, while slightly influenced by some ANTs IP Sybase bought non-exclusive rights to, is essentially home-grown. That’s what I thought, but the call sounded like it was saying something different.

Further coverage of SAP/Sybase:

• SAP believes in database proliferation
• More quick reactions to SAP/Sybase
• Technical basics of Sybase IQ
  

• One of the most confusing things about Sybase ASE is its version numbering. In particular,
  • Sybase ASE 15.5 went GA in December, 2009. (But the clustered version is just coming out in March.)
  • The prior version of Sybase ASE was 15.03.
  • Sybase ASE 15.0 came out in September, 2005.
  • The version of Sybase ASE before that was 12.5.
  • And by the way, Sybase System 10 came out in 1994 or so.
• Sybase ASE 15.0 was a major rewrite. In particular, Sybase ASE 15.0 had a “brand new” optimizer and query processing engine, based on the Volcano model. The main driver of the rewrite was to make Sybase ASE suitable for mixing OLTP and some level of decision-support workloads. (Not on the order of what Sybase IQ can handle, but at least operational reporting and so on.)
• I haven’t looked up Volcano in more detail than to confirm that what I thought Raj said made sense, but as he characterized it, it’s a lot more modular than what Sybase had in ASE 12.5. For example, substantially the only join algorithm in Sybase ASE 12.5 was nested loop – no hash or sort/merge.
• As you might imagine, a lot of things one might regard as core modern DBMS features were only added to Sybase ASE once 15.0 came out. Examples include:
  • Various forms of partitioning at the storage level.
  • User-defined functions (UDFs).
  • A clustering offering that competes with Oracle RAC. (100 or so customers are on that so far.) Absent clustering, Sybase ASE is limited to a single SMP (Symmetric Multiprocessing) box.
  • Shared disk. Amazingly, it seems that before 2008, every node in an SMP box running Sybase ASE had its own private partition (maybe not the right word) of data.
• In Sybase ASE, you have lots of databases managed by one database server. You can write SQL statements that span multiple databases, but they have to reference database names as well as table names.
• There are several ways to get data from one place to another in Sybase’s technology and nomenclature, specifically including Replication Server, Incremental Data Transfer, and “proxy tables.” (Other than the fact that Replication Server is a separate, chargeable product, I don’t really have these straight.) In addition, there’s a hand-coded one in Sybase RAP, which will get a planned 5-6X performance improvement later this year when it is replaced by Incremental Data Transfer.

And in what basically sounds like a very cool approach, Sybase ASE has a lot of memory-centric aspects. That said, Sybase’s in-memory ASE story is still incomplete (wait until the next release) and confused (I think in part because of what’s missing in the current release).  Also, this is one area where the non-technical nature of the briefing got in my way. So here’s some of what I do and don’t know about Sybase’s memory-centric ASE strategy:

• Sybase lets you mix and match on-disk and in-memory databases under one instance of Sybase ASE. To a programmer, it all looks like ASE.
• I don’t know exactly what the limitations are on what you can do with in-memory databases, how you can use them in tandem with on-disk databases, etc.
• You can replicate data from disk to an in-memory Sybase ASE database today. (Hello caching, ala Oracle Times Ten or IBM DB2/solidDB.)
• Replicating from memory to disk is a near-term future capability. (So Sybase does not yet have a hybrid memory-centric story ala solidDB Classic.)
• I have no clue as to what kinds of in-memory data structures Sybase ASE uses.

To understand Sybase’s plans for Aleri and CEP, it helps to understand Sybase’s current CEP-oriented offering, Sybase RAP. So far as I can tell, Sybase RAP has to date only been sold in the form of Sybase RAP: The Trading Edition. In that guise, Sybase RAP has been sold to >40 outfits since its May, 2008 launch, mainly big names in the investment banking and stock exchange sectors. If I understood correctly, the next target market for Sybase RAP is telcos, for real-time network tuning and management.

In addition to any domain-specific applications, Sybase RAP has three layers:

• CEP (Complex Event Processing). Sybase RAP CEP is based on a version of the Coral8 engine Sybase licensed and has been subsequently developing.
• In-memory DBMS. Sybase’s IMDB is part of (but I guess separable from) and has the same API as Sybase’s OLTP DBMS Adaptive Server Enterprise (ASE, aka Sybase Classic).
• Sybase IQ. Actually, Sybase used the phrase “based on Sybase IQ,” but I’m guessing it’s just Sybase IQ.

In theory, there could be a DBMS other than Sybase IQ, such as Sybase ASE or even Oracle, because Sybase IMDB can talk to a variety of DBMS. I didn’t get the impression, however, that in practice there were any Sybase RAP installations whose persistent DBMS was anything other than Sybase IQ.

Aleri had all along had something called Project Ohio, to merge Coral8 with Aleri Classic. Now Sybase’s own CEP engineering team is being added to the mix, schedules are being reconsidered and haven’t been disclosed yet. (If one woman can produce one baby in nine months, how long does it take nine women to produce a baby?) Apparently Sybase has a dozen programmers in the CEP area, plus ~20 more on Sybase RAP, not counting QA, documentation, etc.; that represents a significant bump to the overall Aleri development team.

Sybase doesn’t seem to have decided what to do yet with the various business intelligence/real-time OLAP engine products and technologies it is inheriting from Aleri.

And finally, some metrics:

• The Sybase/Aleri guys estimate that 1/3 of of Aleri’s customers and even less of its revenue came from outside the financial services sector. They did say the non-financial-services business was “starting to pick up,” but not very convincingly.
• Sybase IQ is now up to >1800 customers, with >200 new ones in 2009.
• Sybase IQ indeed has users taking in market feeds up to 3 terabytes a day, so it probably matches Vertica in having at least several-hundred-terabyte databases in the financial sector.

*While a few use cases are exceptions, being “better than MySQL” for a DBMS is basically like being “better than Pabst Blue Ribbon” for a beer. Unless price is your top consideration, why are you even making the comparison?

Even worse, the press release, from its subhead and very first sentence, emphasizes the claim “the Groovy SQL Switch’s ability to significantly outperform relational databases.” As CEO Joe Ward quickly agreed by email, that’s not accurate.  As you would expect from the “SQL” in its name, the Groovy SQL Switch is just as relational as the products it’s being contrasted to.  Unfortunately for Joe, who I gather aspires to edit it to say something more sensible, the press release is out already in multiple places.

More favorably, Renee Blodgett has a short, laudatory post about Groovy, with some kind of embedded video.

My client actually signed a deal for EnterpriseDB’s Postgres Plus Advanced Server and GridSQL, but unwound the transaction quickly. (They say EnterpriseDB was very gracious about the reversal.) There seem to have been two main reasons for the flip-flop.  First, it seems that EnterpriseDB’s version of Postgres isn’t up to PostgreSQL’s 8.4 feature set yet, although EnterpriseDB’s timetable for catching up might have tolerable. But GridSQL apparently is further behind yet, with no timetable for up-to-date PostgreSQL compatibility.  That was the dealbreaker.

The current base-case plan is to use generic open source PostgreSQL, with scale-out achieved via hand sharding, Hibernate, or … ??? Experience and thoughts along those lines would be much appreciated.

Another option for OLTP performance and scale-out is of course memory-centric options such as VoltDB or the Groovy SQL Switch.  But this client’s database is terabyte-scale, so hardware costs could be an issue, as of course could be product maturity.

By the way, a large fraction of these updates will be actual changes, as opposed to new records, in case that matters.  I expect that the schema being updated will be very simple — i.e., clearly simpler than in a classic order entry scenario.