Oracle. On the plus side, you guys have been making progress against your reputation for untruthfulness. Oh, I’ve dinged you for some past slip-ups, but on the whole they’ve been no worse than other vendors.’ But recently you pulled a doozy. The analyst reports section of your website fails to distinguish between unsponsored and sponsored work.* That is a horrible ethical stumble. Fix it fast. Then put processes in place to ensure nothing that dishonest happens again for a good long time.

*Merv Adrian’s “report” listed high on that page is actually a sponsored white paper. That Merv himself screwed up by not labeling it clearly as such in no way exonerates Oracle. Besides, I’m sure Merv won’t soon repeat the error — but for Oracle, this represents a whole pattern of behavior.

Oracle. And while I’m at it, outright dishonesty isn’t your only unnecessary credibility problem. You’re also playing too many games in analyst relations.

HP. Neoview will never succeed. Admit it to yourselves. Go buy something that can. 

Smaller BI vendors. Analytic DBMS evaluations commonly include BI strategy and tool selection as well. If an analytic DBMS expert tells you he needs to learn more about your product line, don’t blow him off. In fact, you should be particularly embracing anybody who’s shown a fondness for small DBMS vendors; maybe he or his clients will like small BI vendors as well. That means (among others) you, Jaspersoft, Endeca, and Tableau.

Information Builders. Is there anything about your BI products that is in any way technologically differentiated? If so, you might want to mention some examples to somebody some time.

Kalido. I’ve said this to you before, but it bears repeating — your positioning translates to “I-CASE for analytics,” and that’s not a good thing. If your product is not as cumbersome and entrapping as that sounds, you need to do a much better job of explaining why not.

SenSage. You are what you are. Sell out while the selling is good. You don’t have the corporate personality to make it into the analytic DBMS mainstream on your own.

Ingres. You need to be more engaged with analysts than you are. Ingres navel-gazed too much 25 years ago, and evidently you haven’t outgrown it yet.

TIBCO. You probably have a lot of cool analytic technology, but I don’t know of an influencer who has much relationship with or trust in you. Rethink how you’re approaching influencer relations top to bottom.

Tableau. You had a lot of mindshare, but it’s fading. Do something.

MarkLogic, graph DBMS vendors, etc. You’re clinging too hard to the NoSQL label. Nobody is out there deciding among Cassandra, neo4j, and MarkLogic. They might be deciding between MongoDB and MarkLogic, I guess, but if you admit to yourself that’s all it is you’ll probably change your messaging somewhat.

Objectivity. Get real about marketing. Infinite Graph is a cool opportunity. But I didn’t even ping you for a meeting when I’m in your area next week, because I wouldn’t have known who to reach out to.

Everybody (especially Objectivity). “First X deployed in the cloud” is almost surely an inaccurate claim. Don’t make it. And by the way, even if it were true, it probably wouldn’t be interesting.

• Unemployed, underemployed, or otherwise desirous of having more commercial skills.
• Not a programmer, but desirous of having some technical skills.
• Astute enough to realize s/he will never be a serious techie.

I generally have two models in mind when guiding such a person:

• Analytics/business intelligence/stats.
• Website building.

Those are both useful skill sets for people who aren’t full-time techies, the first perhaps best for those who are more quantitative and big-company-friendly, the second perhaps better for the creative and/or rebellious types.

So what SPECIFICALLY should one guide them to do? My initial thoughts include:

• Learning Java is overkill for most of these people.
• Learning C++ is overkill for ALL of these people. If you’re not out to be a hardcore engineer, the “advantages” of C++ over Java are pointless.
• They all should learn some SQL.
• MySQL is the most accessible DBMS against which to learn SQL. They should download a (free) copy and install it on their PC.
• But I have no idea which books or websites they should go to to learn about SQL.
• While at first blush it sounds like overkill, downloading and installing the free version of Microstrategy 9 is a good way to learn about BI and also the analytic side of SQL.
• The first thing you learn in an app dev tool used to be and probably still is how to do a master-detail form. That would cover the other side of learning SQL. But what would be a good choice of tool? (Preferably free, as building serious OLTP apps is probably not what these people will want to do.)
• One idea I had is that the website-oriented ones should learn how to modify WordPress, by which I really mean modifying WordPress themes. That would involve learning PHP, SQL, and HTML/CSS, which seems like a great place to start.
• But I have no idea which books or websites they should go to to learn about PHP.
• I also have no idea which books or websites they should go to to learn about CSS — or for that matter even basic HTML.
• If they want to take the analytics route, I assume R is the way to go. Thoughts?
• Python isn’t the ideal language for much of anything, but it’s an easily accessible “first language”. Umm, is that a good way to go, or would PHP be a better choice?
• Any other ideas?

For anybody who pitches in — thanks!! I hope to get enough useful answers so as to keep editing this post with people’s ideas.

Edit: Suggestions have started to come in on Twitter. A couple of folks are saying that HTML is a good place to start. Hard to argue with that, although it’s hardly where one should finish. There also was a vote for Yahoo YQL, and of course for a vendor’s own product.

Some great points are in the comments below, including the idea that you should pick an actual, fun, small project to build to get you started. (A site built in WordPress or Mambo would be a pretty obvious choice for such a project, come to think of it.)

• Despite heavy emphasis on the word “sharding,” dbShards’s scale-out is transparent to the application programmer. E.g., in dbShards + MySQL, the APIs are more or less the same ones you’d expect for MySQL (JDBC, etc.)
• If the DBMS underneath is ACID-compliant (e.g., MySQL + InnoDB), then the dbShards version is ACID-compliant too.
• Beyond those basics, I forgot to check the fine details of dbShards’ MySQL (or PostgreSQL) syntax support. Todd Hoff, however, did not forget.
• dbShards keeps copies of each shard on two different servers, via asynchronous log-shipping. This allows for failover in both planned and unplanned outages.
• dbShards wants you to distribute big tables among shards via a “shard key,” which is a lot like the distribution key in MPP analytic DBMS. You’re encouraged to replicate small, low-update-volume tables across each shard.
• Cory says that dbShards has good join performance when – you guessed it! – everything being joined is co-located shard-by-shard, because the tables were distributed on the same shard key and/or replicated across each shard. Cory can’t imagine why you’d want to do an inner join under any other circumstances.
• The basic dbShards query execution model is: A query comes in; it’s parsed; a shard key is automagically detected (one hopes); the “global configuration file” is checked to see which shard to ship the work off too. I forgot to ask whether lookup was done via a hash table (the obvious guess) or something else. The programmer can put hints in the code comments to direct the sharding, but Cory asserts those aren’t needed very often.
• Cory says that insert performance with dbShards + MySQL + InnoDB is 1500-3000 inserts per shard per second, scaling almost linearly with the number of shards. I forgot to ask how many shards this had been tested for.
• If you want blazing dbShards performance, Cory’s base-case figure is 25 gigabytes of data per node, so that the most commonly used indexes can camp out in memory. (I forgot to ask what kind of hardware he was assuming per node.) This is if you’re going to be doing joins or aggregrations. If it’s just single-row inserts and updates, or if your performance requirements are lower, you can go with 10X that figure.
• Cory tells stories wherein going from an unsharded database to 4 or so shards took database re-indexing time down 50X or more. Apparently, such tasks can be exponential or even super-exponential with database size over InnoDB. (That said, I’d be surprised if all large InnoDB users suffered from that problem to the same degree.)
• dbShards’ customer workloads are all >= 50% reads. This is reflective of dbShards’ design priorities.
• As long as it can be in charge, dbShards is happy to interface to whatever kind of database backup software you want to use on a node by node basis. (dbShards wants to drive your backup software for you so that it can be sure the replicas are handled properly.)
• It’s “fairly common” for dbShards to be paired with memcached. I forgot to ask whether memcached typically lived on its own pool of servers, or on the same pool that runs dbShards.
• Future DBMS options under consideration for dbShards include Oracle and (unspecified) in-memory.

Business highlights for CodeFutures and dbShards include:

• dbShards’ price is $5000/server/year, including support and OEMed MySQL, with stated quantity discounts up to 40%.
• dbShards cloud pricing is different (on a usage basis).
• dbShards has 6 or so customers, half each on-premises and in the cloud. One of them is Facebook. (Those “100s” of customers mentioned on the dbShards website are for a fairly unrelated product.)
• CodeFutures has been at this 2 ½ years or so. There is no venture capital in the company.
• Early deals dbShards deals have evidently involved a fair amount of professional services.
• Counting contractors, Code Futures has 10-12 people, which has been as high as 15.
• Target dbShards customers are as you’d expect. Cory says he’s actually been more successful getting early-adopted money out of Web companies than Wall Street firms.
• There are a couple of dbShards PostgreSQL customers for greenfield applications. Most dbShards customers and prospects, however, are looking to scale out existing apps.
• Despite its connection to open source DBMS, there’s nothing open source about dbShards itself.

At this time, I have no idea who the lucky buyer is.

• Neil Raden points out that business intelligence dashboards can be dangerously misleading. His reasoning (sound) is that whatever you measure is apt to be distorted by the fact people know they’re being measured. His solution (implied) is to hire a good-looking consultant like himself to do it right.
• I’ve had my issues with Fred Holahan, who was VP of Marketing when I posted that EnterpriseDB was not to be trusted. (That said, Fred is long gone from EnterpriseDB and my opinion hasn’t changed.) But he’s put up a good series of posts on the basis of the open source “progressive engagement” marketing funnel, including this gem on why you shouldn’t count on monetizing your community/free users.
• Oracle plans to increase its acquisition budget. The figure given is $70 billion over the next 5 years. Edit: But see this funny Register followup.
• Clayton Christensen wrote a phenomenal article on how to live a good life, from a very business-y perspective. (Only in one anecdote was it too religiously-oriented for my tastes.) Takeaways include:
  • Your core goals probably revolve around something other than business success. (E.g., family.) Don’t lose sight of that.
  • To the extent you’re a manager or leader, you may have a huge impact on other people’s lives. Use that power in admirable ways.
  • Teach people how to fish for answers, rather than just giving them answers. They’ll probably come to better conclusions than you would have anyway. (This is a core principle in my own consulting.)
  • Take time to reflect. And by the way, the same techniques you use for strategic analysis in business can be applied to your life as well.
• Mark Suster has a pretty good post expanding on my first Christensen takeaway, highlighting a point too often missing from articles in that genre: It’s not just family; it’s also all the cool things around us.
• I haven’t gone through the Hadoop Summit archives yet, but it looks as if there’s a lot of insight there about current Hadoop application activity.
• If you’re a cat lover and don’t hate simple/traditional music, check out Marc Gunn’s cat filksongs, especially the infectious “What Shall We Do With a Catnipped Kitty?” and “Lord of the Pounce”, both playable from the right sidebar of that page (#7 and #10 respectively). Gunn is also a chief perpetrator of the justly (in)famous Do Virgins Taste Better? cycle of filksongs.
• Former SAP exec Dennis Moore offers a theory as to why SAP cares so much about in-memory DBMS. It’s to integrate business processes, because SAP has no other software layer good at doing same. Interestingly, Dennis originated SAP’s previous attempt at meeting a similar need via its composite applications initiative. However, in Dennis’ view this benefit would only be achieved by a major rewrite of SAP’s applications.

A consensus seems to have emerged that Thesis had actually copied beyond-fair-use amounts of WordPress code, which if true was Game Over. Beyond that, however, both sides of the strongly-viral-GPL debate scored some points.

• Public pressure, FUD, etc. in favor of the GPL clearly were successful.
• Mark Aquith carefully explained  how tightly integrated WordPress and WordPress themes are. So we don’t necessarily have much of a precedent for more hands-off integration.
• Previous precedents were dredged up on both sides. More on that below.
• Once again, the claims about the GPL were not tested in court. Various open source developers and lawyers have stated their opinions as to what their rights are, but the courts were not given the opportunity in this case to (dis)agree on the core issue …
• … which everybody seems to be correctly agreeing is: “When is one software program a ‘derivative work’ of another one, in the copyright-law sense of ‘derivative work’?”

The pro-GPL argument on that last point would probably boil down, colloquially, to “Well, is it one program or two? If it’s one, then it clearly is derivative from the big part that’s copyrighted under the GPL. If it’s two, then the creator of the second one is home free — but c’mon, now, it’s really just one.” That, in turn, would be supported by Dan Weinreb’s point from a previous MySQL storage engine/GPL comment thread (which also has a lot of applicability to the WordPress theme case) – you really don’t want to make the user do two installs, so you really do want to include the GPLed code in your package. And by the way — so far the product packaging by the MySQL storage engine vendors* is in line with Dan’s observation.

*Infobright, Akiban, Tokutek, Calpont, et al.

One point I haven’t seen discussed much yet is this:

Suppose a MySQL storage engine vendor integrated with a forked, GPLed MySQL, and then didn’t obey the GPL. Who would have standing to sue them? It’s obvious that the developers of the forked MySQL would. But it’s not at all obvious that Oracle would. A derivative of a derivative of a copyrighted work is NOT necessarily a derivative of the original. (Think about it.) Unless Oracle could prove that the MySQL storage engine really did happen to be a derivative of MySQL Classic, I don’t know why Boogeyman Oracle would have standing to sue.

Finally, those precedents.

• A couple of comments on my earlier post point out that the Linux community tends to be pretty tolerant of proprietary code that links tightly into Linux. Oh, they may not like — but in most cases they neither do sue nor believe they successfully can.
• Wikipedia cites some cases in which the GPL has been successfully enforced.
• One of the earliest GPL controversies was over what sounds like a MySQL storage engine — Progress Software’s NuSphere, developed in connection with MySQL AB. Litigation ensued, and before the case was settled, the judge wrote “After hearing, MySQL seems to have the better argument here, but the matter is one of fair dispute.“

Now a related can of worms has been opened in a related technology area — WordPress and WordPress themes. Since many bloggers use WordPress, this has gotten a lot of attention, and some interesting new insights have emerged.

Um, in case you didn’t know: WordPress is the software that runs blogs such as this, and it’s a GPLed open source project. However, the user interface — look, feel, and behavior alike — are determined by separate themes, that one usually gets from third parties (WordPress ships with a a few default choices).

It started when Matt Mullenweg went after the makers of an unfree theme Thesis, and wielding a legal opinion from the Software Freedom Law Center. The gist of the SFLC’s argument seems to be

They are derivative of WordPress because every part of them is determined by the content of the WordPress functions they call. As works of authorship, they are designed only to be combined with WordPress into a larger work.

And of course the point of the GPL is that if you create a derivative work of something GPLed, you have to GPL it yourself.

However, Perpetual Beta pointed out that, under the rules of copyright law as expressed in a court case known as Galoob, depending on another program does not make something a derivative work. This is actually blindingly obvious, as in the example of any program that runs on top of an operating system. Or for more examples see Dave Winer on the point.

Perpetual Beta further argued that, even if it were a derivative work, fair use would let one copy it anyway. I.e., if you’re engaging in “fair use,” you’re entitled to do what otherwise would be a copyright violation. Good point. The GPL license says in effect “You only are allowed to use this material (in certain ways) if you do as we say about your own work,” so that is defeated if the Fair Use Doctrine lets you say “Um, actually, I’m using this without your permission, so buzz off.”

GPL advocates can pontificate all they want about certain uses of GPLed code violating their license terms. But if either of these arguments holds up — and it looks to me like both do — a program that invokes GPLed code is not subject to the GPL just based on those invocations. And that, in turn, would more than imply that MySQL storage engine vendors could use GPLed MySQL-compatible front-ends without being under any GPL obligation themselves.

The first two things to understand about SQL Anywhere is that there actually are three products:

• Sybase SQL Anywhere, a mid-range relational DBMS.
• Sybase UltraLite, a DBMS for mobile devices.
• Sybase MobiLink, a replication/sync tool.

and also that there are three main deployment/use cases:

• Generic desktop or server computers. This was the original market for SQL Anywhere.
• Laptop/handheld computers. This was the original growth market for SQL Anywhere. In particular, Siebel Systems’ first growth spurt was selling sales force automation software on laptop computers with SQL Anywhere underneath.
• Specialized devices. Earlier this decade, Sybase thought SQL Anywhere’s big growth market was on specialized devices. (I recall a video featuring some kind of automated pill dispensing machine for hospitals.)

Meanwhile, terminological weirdness includes:

• At various times the product Sybase SQL Anywhere has had “Server” in its name, but that seems not be part of the naming convention at the moment. Sybase folks still talk about it that way, however.
• Is SQL Anywhere a product? A group of products? A division? Again, that depends on whom and when you ask.
• When Sybase speaks of SQL Anywhere being in the “embedded” database market, it’s talking of any OEM business, not just deployment on particular kinds of devices. (IDC has long used that confusing terminology, which is a regrettably good excuse for vendors to use it as well.)

I get the impression that there are two big markets for the Sybase SQL Anywhere family:

• ~1000 OEM partners for Sybase SQL Anywhere. Notable examples include:
  • Intuit (for QuickBooks)
  • Symantec (for NetBackup)
  • Sybase IQ, if you want to view it that way (a lot of Sybase IQ’s front-end is actually SQL Anywhere code)
• Direct enterprise customers for mobility-oriented combinations of Sybase UltraLite, Sybase SQL Anywhere, and Sybase MobiLink. Notable examples include:
  • Pepsi Bottling Group (repairmen, using handheld devices)
  • US 2010 Census (140,000 people address checkers – every address in the US was checked against GPS coordinates, it seems)

By way of contrast, the market for true embedded systems never took off as strongly for Sybase SQL Anywhere as it did for, say, pre-acquisition solidDB. Added together, this all seems to amount to ~ 10 million users overall, as per a slide deck Sybase graciously gave me permission to post.

Other market notes include:

• Sybase says that the main competitors for Sybase SQL Anywhere are Microsoft SQL Server, Oracle, and occasionally MySQL, with Progress coming up only rarely. I would have thought Progress OpenEdge RDBMS would be higher on the list (simple DBMS for the OEM market) – but come to think of it, it’s been quite a while since I’ve run into anybody who says they compete much with Progress OpenEdge. Perhaps Progress just isn’t selling its core product very hard these days.
• 80-90% of Sybase SQL Anywhere installations are on Windows, but the fact that SQL Anywhere also runs on Macintosh and ‘nix platforms is apparently pretty important to sales.
• The main platforms for Sybase UltraLite are Blackberry and Windows Mobile, although with iPhone support newly added in SQL Anywhere Version 12, that’s a good bet to change.

Besides the above, the key points about Sybase SQL Anywhere are:

• Sybase SQL Anywhere is designed for a no-DBA, “customers may not know there’s a DBMS” there kind of easy administration. (That’s one reason I compared it to Progress.)
• Sybase SQL Anywhere has a reasonable set of mid-range DBMS features, including:
  • Triggers
  • Stored procedures
  • Referential integrity
  • Geospatial datatype
  • Text datatype (with a CONTAINS clause), and you can index external documents
  • Multiprocessor/multicore operation

Sybase SQL Anywhere Version 12 seems to have a variety of performance improvements, of which the most interesting are:

• If you’re using an ORM (Object-Relational Mapping) layer such as Hibernate, the SQL Anywhere optimizer now works hard to rewrite some of the horrific SQL that can be generated, for example by flattening or eliminating a lot of subselects.
  • There’s also a “Hibernate dialect” of SQL Anywhere – I’m not sure what that means, but it clearly indicates that SQL Anywhere wants to be Hibernate-friendly.
• When the SQL Anywhere optimizer decides the statistics are wrong, it reruns them, piggybacking on a subsequent query if it can, doing a background job if it must. Somewhat pretentiously, Sybase refers to this as “self-healing statistics”.

Many SQL Anywhere features are of course missing in Sybase UltraLite, but geospatial datatypes are present.

Finally, it’s a natural conjecture that MobiLink is related to Sybase Replication Server – but that conjecture turns out to be wrong. That said, keeping Sybase’s various replication offerings straight is tricky.

• Ben says Greenplum wasn’t being shopped, by which he means Greenplum was out raising more capital and the fund-raising was going well.  Note: Half or so of Greenplum’s deals were subscription-priced, so it had weaker cash flow than it would have if it were doing equally well selling perpetual licenses.
• However, joint engineering was also going well with, e.g., Greenplum CTO Luke Lonergan spending time at EMC facilities in Cork, Ireland. And one thing led to another …
• Greenplum has ~ 140 customers, vs. ~65 five quarters ago, 100+ at year-end, and an acquisition rate of 12-15/quarter last fall.
• A typical “small” paying customer for Greenplum starts with 10-20 TB of data.
• Greenplum Chorus isn’t generally available yet, with rollout energy being focused on Greenplum 4.0. Note: As important as it is for overall industry direction, Greenplum Chorus is a product which won’t be a terribly big deal in Release 1 anyway.

Highlights looking forward include: 

• When I challenged him, Ben sounded quite optimistic that Pat Gelsinger will immunize Greenplum against and generally counteract some of EMC’s traditionally stifling bureaucracy. (My words, of course, not his.)
• The initial Greenplum/EMC product vision appears truly centered around “private cloud,” specifically including Greenplum, VMware, and EMC storage arrays.
• Some other areas of potential Greenplum/EMC technical synergy I think are cool obviously haven’t been seriously addressed yet.
• Based on what I heard from Ben about the aura around the deal and also on what I know of the individual executives at Greenplum, I think each of them is a good bet to stick around EMC for a while. (That’s on average. Of course, it would be surprising if 100% of them stayed around very long.) Basically, there’s at least a chance EMC/Greenplum will do some pretty cool stuff, and most of the guys will probably stick around to see if that actually starts to happen.*

*Also, when they do eventually leave, they’ll surely say things to the effect “The cool stuff is well underway; my work here is done.” That party line is almost guaranteed, no matter how things unfold in reality.

I think that even software-only analytic DBMS vendors should design their systems in an increasingly storage-aware manner

promising to explain what I meant later on. So here goes. 

There always have been good technical reasons to tailor hardware to analytic database software. Data moves through disk controller, network, RAM, CPU and more, each with its own data rate. Getting different kinds of parts into the right balance doesn’t completely eliminate bottlenecks – the Wonderful One-Hoss Shay is poetic fiction – but it certainly can help. As a result, every analytic DBMS vendor of any size offers at least one of:

• A Type 0 appliance
• A Type 1 appliance
• A “recommended hardware configuration”

And beyond performance, appliances and pre-specified hardware configurations offer at least the possibility of easing installation, administration, and support.

There also are marketing reasons to offer an appliance or something appliance-like.

• To various extents, Oracle, Teradata, Microsoft, IBM, Netezza, and EMC are all telling the world that your hardware should be optimized for your analytic DBMS.
• Smaller vendors such as Vertica and Aster Data also tend to cobble together some sort of appliance, in part so they don’t have to say they disagree.
• Thus, a “We don’t see any point in special hardware assembly at all” story would leave an analytic DBMS vendor pretty far out on a limb.

Finally, there are three overlapping technical trends that increase the need for storage-awareness in analytic DBMS. First and foremost is the rise of solid-state memory. For starters, I believe:

• Flash will be important for analytic DBMS soon.
• There are good technical reasons for this.
• Oracle’s marketing will make a big deal out of the Flash aspects of Exadata, so other analytic DBMS vendors will need a response. And of course, if Netezza or Teradata preemptively make a big deal of their Flash-based offerings, that just adds to the pressure for Flash adoption on everybody else.
• But it’s not just Flash – Flash, other solid-state memory, and disk will be combined in various ways.

But this move to Flash will require analytic DBMS vendors to be increasingly storage-aware for at least three reasons:

• It just adds another level of complexity to their hardware-balancing challenges.
• Flash overturns some of the fundamental assumptions of modern analytic DBMS, in particular:
  • Sequential reads are hugely better than random
  • The worst bottleneck is at the point where data comes out of storage.
• The Flash technology stack is still immature, and you have to pick your poison in how to deal with it. Vendors are making very different choices in this regard – and they do have to choose.

Another trend that could naturally lead analytic DBMS vendors to be more storage-aware is their incorporation of what could be viewed as hierarchical storage/ILM technologies. Different data is stored in different ways and/or on different kinds of storage hardware. (Vendors pursuing – you guessed it – different approaches to this include Teradata, Greenplum, Vertica, and Sybase.) The more automatic that process is, the more storage-aware the DBMS will need to be.

Finally, there are reasons to think that DBMS should be split between conventional servers and smart storage. This is, of course, the Exadata strategy. Netezza’s two-processor approach, while rather different, also somewhat validates the idea.