• Business is strong. Multiple vendors claim to be going gangbusters, with the happy sounds coming out of Vertica and Infobright being echoed by several competitors. Hearsay suggests some other companies in related businesses are doing well too. Depending on who you talk to, the business pickup dates back to Q4, give or take a quarter.
• Oracle Exadata has become a formidable competitor, on the strength of Exadata 2. Exadata 2’s positioning and perception among Oracle users seem to be pretty much in line with what Oracle portrayed to me.
• Teradata is portrayed as a weak competitor. Competitors don’t worry about Teradata nearly as much as they do about Oracle. That said, I suspect a bit of wishful thinking; Teradata is clearly still getting a lot of business the other vendors would dearly love to have.
• HP Neoview is reeling. (Almost) nobody sees Neoview competitively. The Walmart Neoview installation is said to have stayed small at best. JP Morgan is said to have completely thrown Neoview out (and a bunch of HP engineers with it).
• (Almost) nobody mentions competing against DB2 either. This continues to baffle me.

I just don’t see Oracle abandoning HPC entirely. I think it may call it by some other name or describe it differently, but it will be in the high throughput computing business for the foreseeable future.

There are some interesting angles for it to pursue. Many of its best commercial customers have sizeable HPC or HPC-like workloads that Oracle can now (with the addition of Sun) compete for. I don’t see it passing up those opportunities.

Oracle can also look to specialize on certain subsets of the market and provide more of a solution rather than piece parts. I wouldn’t be surprised to hear of it offering an Exadata-like system that is optimized for, say, seismic or financial services. In fact, Exadata as it stands today is a decent fit for financial service analytic workloads.

HPC can be a profitable business and, in a lot of organizations, it’s growing faster than traditional business processing. From Oracle’s perspective, what’s not to like?

Now, except for the Exadata-in-financial-services comment, that’s not directly an argument for the convergence of number crunching and data management.  However, I think Netezza and Aster Data are showing the way for that convergence. So, up to a point, is the scientific-research community. And of course the Hadoop guys think they have the best way to that convergent future.

But if Dan Olds is right that the best technologies for Oracle to pursue HPC and big-data processing with aren’t all that far apart, then the chances that Oracle will indeed pursue their convergence are pretty high. And that would amount to critical mass for the trend.

• Personally selected annual lists of 12 “Most influential” companies and 36 “Companies to watch” in analytics- and database-related sectors.
• Made it clear that these are his personal selections.
• Nonetheless has called it an Editors’ Choice list, rather than Editor’s Choice.

(Actually, he’s really called it an “award.”)

People advising Doug — who come to think of it actually are Contributing Editors to Intelligent Enterprise or something like that — included Cindi Howson, Seth Grimes, three others, and me.

And if past is prologue, I will now get a flood of PR emails calling my attention to this award that I already have both participated in and blogged about.

As usual, the sense:nonsense ratio on these lists was pleasingly high. Analytic DBMS vendors cited included IBM, Microsoft, Netezza, Oracle, Sybase, and Teradata in the “Most influential” group, with Aster, Greenplum, HP, Infobright, and Vertica among the “To watch” crowd. It’s tough to argue with those selections, whose most questionable element is probably the not-ridiculous supposition that HP could do something interesting over the coming year. Cloudera and Intersystems also made the list, deservedly.

All three of QlikTech, Tableau, and TIBCO made the list, which is appropriate given the potential for and interest in interactive data exploration technology.  The BI majors, independent or otherwise, were all on as well. In text mining, Doug included Attensity and Clarabridge, which I think is exactly right. (Plus OpenCalais.)  Upon reflection, I probably should have nominated Mark Logic, even though most of its business is non-enterprise; but hey, nobody’s perfect, and the same goes for lists. Open source was well represented, with Apache, Actuate, Jaspersoft, Eclipse, Infobright, Nuxeo and R all being cited (but not Ingres or Pentaho). Kalido made the list, with my endorsement, their silly I-CASE like marketing messaging notwithstanding.

Speaking of imperfections — there only are a few category names, and so category assignments can be pretty bizarre. (In an ideal world, middleware wouldn’t be included under “enterprise applications”.) Greenplum hasn’t really “extended” its DBMS with a “cloud” option. As much as I’d like Netezza to be more influential than SAP, that’s probably not the best way to rank them. And there are a number of “This company is on a roll!” kinds of comments that I wouldn’t necessarily endorse.

But those are all nitpicks. On the whole, it’s another nice job.

*Links to Gartner Magic Quadrants commonly break, but that one worked at the time of this posting.

• Gartner thinks that data warehouse appliances are on the rise, due to their simplicity.
• Gartner correctly says that Teradata has been a data warehouse appliance vendor from the getgo.
• Gartner characterizes IBM as being an appliance vendor as well.
• Gartner suggests that HP is having trouble living up to its technical promises for Neoview.
• Gartner further suggests — no surprise here — that HP Neoview has had very few new customers past its initial wave.
• Gartner notes IBM’s difficulties in selling data warehouse installations of DB2, despite what on paper is great-sounding technology.
• Gartner says — also no surprise — that illuminate “has seen little success in North America since opening its first office in the U.S. over two years ago.”
• Ingres has evidently gotten a few BI-centric “appliance” deals, e.g. with Jaspersoft. But basically Ingres isn’t doing well in data warehousing.
• Gartner does say Ingres has “the strongest open-source DBMS offering for data warehousing.” Being very literal about “open source,” that’s a defensible claim — but it’s pretty irrelevant in a world where Greenplum Single-Node Edition can be had for free. It also waves away all the data mart use cases in which Infobright Community Edition shines.
• Gartner says that Netezza is working out as a “complex workload” enterprise data warehouse provider, according to reference checks, in addition to its established success in data mart scenarios.
• Gartner says Oracle’s offering has finally become “accepted” in the market for databases >50 TB. I guess I can live with that fairly weak claim, but I wouldn’t go much further than that.
• Gartner asserts that, unlike software-only Oracle, Oracle Exadata isn’t significantly harder to administer than “other mixed OLTP/OLAP DBMS vendors,” because Exadata is fast enough you don’t need to jump through all those hoops any more to get tolerable performance. The money quote is “one reference reported reducing the number of indexes by a factor of 100 to fewer than five.” Note, however, that Gartner does not seem to assert that Exadata’s ease of use rivals that of the newer analytic DBMS specialists.
• Gartner confirms Oracle’s reluctance to do onsite Exadata POCs, but says it is not absolute. This is roughly compatible with what I’m hearing elsewhere, and indeed with Oracle own claims to be ramping up availability of Exadata POC hardware.
• Gartner’s criteria for inclusion include at least 10 different organizations having a product “in production.” Thus, the big surprise was ParAccel being included. The money quote there is “With approximately 20 customers in the pharmaceutical, retail, financial and media/advertising analytics sectors, ParAccel has a good reference base.” That assessment is difficult to reconcile with other information, but I’ve been told Gartner is sticking to its guns. That assessment would be even harder to believe if those 20 references were all alleged to be true production customers.
• Gartner notes that you basically can’t run a 1 TB+ MySQL data warehouse without sharding. (Of course, Infobright has an alternative, and up to a small number of terabytes so does Kickfire.)
• Gartner reports that at least some customers are pleased with Sybase IQ’s mixed workload/enterprise data warehouse capabilities.
• Gartner correctly notes that Oracle Exadata is a price-competition challenge for Teradata.
• Gartner notes that 20% of Vertica’s customers are outside the US. While not shocking, that’s more than I realized.
• Gartner notes something I don’t think I’ve posted yet, which is that Vertica has a customer with 300 TB of data. (The identity is a deep dark secret, but if I told you you probably wouldn’t recognize the name anyway.)

As does any such piece, the Gartner Data Warehouse DBMS Magic Quadrant also has outright errors.  For example:

• Aster Data isn’t really “the newest entrant to the DBMS data warehouse world.”
• Aster’s SQL/MapReduce was not new in Release 4.0.
• Greenplum isn’t yet pushing down code to the storage tier.
• I’m not sure what kind of database-tier parallelism Gartner is claiming is new in Oracle in 11g Release 2 — but I doubt it’s really new. Rather, what Oracle has done recently is make parallelism less administratively cumbersome.
• Vertica wasn’t really the first DBMS in the cloud. At most it was the first pure-play analytic DBMS to get there.

• Oracle thinks flash memory is the most important hardware technology of the decade, one that could lead to Oracle being “bumped off” if they don’t get it right.
• Juan believes the “bulk” of Oracle’s business will move over to Exadata-like technology over the next 5-10 years. Numbers-wise, this seems to be based more on Exadata being a platform for consolidating an enterprise’s many Oracle databases than it is on Exadata running a few Especially Big Honking Database management tasks.

And by the way, Oracle doesn’t make its storage-tier software available to run on anything than Oracle-designed boxes.  At the moment, that means Exadata Versions 1 and 2. Since Exadata is by far Oracle’s best DBMS offering (at least in theory), that means Oracle’s best database offering only runs on specific Oracle-sold hardware platforms.

*E.g., I was sitting upstairs in my parents’ apartment in Columbus, OH having the call while their doctor, who I’ve never met, was visiting downstairs. He offered to make a special trip back Saturday afternoon because he missed me Wednesday, but he’s notorious for not coming when he says he will. Update: He didn’t come Saturday. On Saturday he said he’d come Sunday. He didn’t do that either.

Other high- and lowlights of our conversation included:

• Flash is the main new hardware element in Exadata Version 2. Otherwise, Exadata 2 is just an annual refresh of Exadata Version 1 to include updated components (Nehalem chips, bigger disk drives, etc.)
• Juan thinks it’s suboptimal to use flash memory through the bottleneck of disk controllers, favoring PCIe cards instead. (I emphatically agree.)
• Juan resolutely ducked questions about actual Exadata production deployment. Literally the only fact he shared in that regard is that there are at least 2 Exadata production systems running that each have 2 or more racks cabled together.
• Juan stressed that Exadata runs apps written over Oracle DBMS unchanged.
• When making mixed-workload claims for Exadata 2, Juan stressed consolidation of multiple databases, some OLTP and some analytic. He didn’t really argue with my skepticism about integrating OLTP and analytics in the same database, with one exception:
• Juan pointed out that in major OLTP apps such as ERP systems, there often is actually more processing going on in reporting and other batch stuff than there is in true OLTP.
• Exadata 2’s flash memory is designed as a disk cache, smarter than LRU (Least Recently Used). The two examples Juan gave of “smarter than LRU” are that backups and table scans don’t flush the cache.
• I forget whether this is new in Exadata 2 (I think it is), but anyhow – Exadata has a “Storage Index” that’s a lot like a Netezza zone map. I.e., for each megabyte or so of data it stores the min and max value of every column; if a query predicate rules out those ranges, that megabyte is never retrieved.
• Oracle has long offered what sounds like flexible workload management capability, and this has now been extended to specifically include I/O resources on the storage tier.
• This isn’t Exadata-specific, but Oracle has built a file system on top of its DBMS, optimized for speed, which helps with, e.g., ELT (Extract/Load/Transform). Evidently, it’s not at all the same thing as Mark Benioff’s 1990s Microsoft-annoying IFS (Internet File System) project, which seems to have morphed into a content management SDK.

Highlights specifically in the area of parallelization included:

• Juan stressed that all databases consolidated onto an Exadata machine are/should be striped across all storage units.
• On the other hand, Juan said that different databases should be confined to specific cores or CPUs on the database tier.
• But on the third hand, Juan also stressed – in what could be called a “private cloud” pitch – that there’s great elasticity as to which databases are matched to which server CPUs.
• Contrary to what I thought he and/or his colleagues told me a year ago, Juan said RAC (Real Application Clusters) is a big part of Oracle’s data warehouse processing.
• However, Juan says that what I regard(ed) as a major objection to Oracle’s database-tier parallelization — the need to manually specify “degrees of parallelism” — has now been obviated by automation. Juan thinks that few data warehouse DBAs will now need to manually tune parallelism, with minor exceptions. One exception he cites is that if a nightly report really is non-urgent, it can just be forced to run on a single core with no chance to grab more resources. (However, Juan thinks manual tuning of parallelism will continue to play a greater role in OLTP.)

OK. That’s all I can get done tonight (see above re: inconvenience of timing). Follow-on subjects I’d like to and indeed plan to post about include:

• What Juan said about hybrid columnar compression
• Oracle’s delightfully non-confidential slide deck, and a few comments about same

Vertica offers a post on its 3.5 release, with a riff on the popular theme “We’ve fixed some weaknesses in our prior versions that we didn’t previously say we had.” More important, Vertica is pretty clear on the virtues of its hybrid columnar architecture.

Speaking of which — Oracle is going true hybrid columnar as well. I don’t have details or timing, however.

Dave Kellogg of Mark Logic wrote in to amusedly point out  Oracle’s anti-MarkLogic collateral. The very first charge Oracle levies is that MarkLogic goes beyond the emerging XQuery standard to add additional functionality. Considering Oracle’s approach to SQL standards, I tend to share Dave’s amusement.

Bill Conniff of Xponent LLC wrote in to tell of a vastly cheaper and less functional approach to XML management, apparently geared to looking at very large XML files one at a time.

Cayuga is a Cornell research project in complex event processing (CEP). There’s a Cayuga academic home page, a Sourceforge page for some open source Cayuga CEP code, and so on. Minsheng Hong, writing from a Vertica email address, tipped me off some months ago. The basic idea seems to be to do lots of queries very quickly, rather than a smaller number of queries over and over again. Whether this is an advance in anything but open-sourceness over Apama or Aleri I couldn’t say, but I do think it’s a different focus than that of StreamBase or pre-Aleri Coral8.

And finally, editor Doug Henschen listed his 15 favorite Intelligent Enterprise blog posts of 2009 — four each by Seth Grimes and Doug himself, three by Cindi Howson, two by me,* and one each by Mark Smith and Neil Raden.

*Doug selects up to three posts a month from here to republish.

1. Continued Availability of Storage Engine APIs. Oracle shall maintain and periodically enhance MySQL’s Pluggable Storage Engine Architecture to allow users the flexibility to choose from a portfolio of native and third party supplied storage engines.

MySQL’s Pluggable Storage Engine Architecture shall mean MySQL’s current practice of using, publicly-available, documented application programming interfaces to allow storage engine vendors to “plug” into the MySQL database server. Documentation shall be consistent with the documentation currently provided by Sun.

Well, duh.

2. Non-assertion. As copyright holder, Oracle will change Sun’s current policy and shall not assert or threaten to assert against anyone that a third party vendor’s implementations of storage engines must be released under the GPL because they have implemented the application programming interfaces available as part of MySQL’s Pluggable Storage Engine Architecture.

A commercial license will not be required by Oracle from third party storage engine vendors in order to implement the application programming interfaces available as part of MySQL’s Pluggable Storage Engine Architecture.

Oracle shall reproduce this commitment in contractual commitments to storage vendors who at present have a commercial license with Sun.

This is the biggie, lifting a major cloud from the MySQL storage engine business. It sounds like the third of four options I suggested as to how Oracle could legitimately earn antitrust approval of its MySQL takeover. Sure, Infobright, Kickfire, et al. already had what they saw as adequate safeguards or contingency plans vs. Oracle skullduggery. It’s still big even so.

(Quoted out of order.) The geographic scope of these commitments shall be worldwide and these commitments shall continue until the fifth anniversary of the closing of the transaction.

Not a disaster, but with respect to at least point #2 there should be no time limit whatsoever. I’d like to see the EC require that change as a further Oracle concession.

3. License commitment. Upon termination of their current MySQL OEM Agreement, Oracle shall offer storage vendors who at present have a commercial license with Sun an extension of their Agreement on the same terms and conditions for a term not exceeding December 10, 2014.

Oracle shall reproduce this commitment in contractual commitments to storage vendors who at present have a commercial license with Sun.

Actually, I don’t think this was ever enough of a problem to be a big deal.

4. Commitment to enhance MySQL in the future under the GPL. Oracle shall continue to enhance MySQL and make subsequent versions of MySQL, including Version 6, available under the GPL. Oracle will not release any new, enhanced version of MySQL Enterprise Edition without contemporaneously releasing a new, also enhanced version of MySQL Community Edition licensed under the GPL. Oracle shall continue to make the source code of all versions of MySQL Community Edition publicly available at no charge.

This one is too weasel-worded to matter much.

5. Support not mandatory. Customers will not be required to purchase support services from Oracle as a condition to obtaining a commercial license to MySQL.

Not clear how significant this is given that there are no price assurances about the license cost.

6. Increase spending on MySQL research and development. Oracle commits to make available appropriate funding for the MySQL continued development (GPL version and commercial version). During each of the next three years, Oracle will spend more on research and development (R&D) for the MySQL Global Business Unit than Sun spent in its most recent fiscal year (USD 24 million) preceding the closing of the transaction.

Oracle won’t shut down the MySQL business for at least 3 years. Duh. If there was any chance of Oracle doing so in the first place, it wouldn’t have let MySQL delay the whole merger for this long.

7. MySQL Customer Advisory Board. No later than six months after the anniversary of the closing, Oracle will create and fund a customer advisory board, including in particular end users and embedded customers, to provide guidance and feedback on MySQL development priorities and other issues of importance to MySQL customers.

8. MySQL Storage Engine Vendor Advisory Board. No later than six months after the anniversary of the closing, Oracle will create and fund a storage engine vendor advisory board, to provide guidance and feedback on MySQL development priorities and other issues of importance to MySQL storage engine vendors.

Two small gestures – but I bet the Oracle people tasked to interact with those boards will truly want to do the right thing for MySQL users. Engineers are like that – they want to make things that actually work well.

9. MySQL Reference Manual. Oracle will continue to maintain, update and make available for download at no charge a MySQL Reference Manual similar in quality to that currently made available by Sun.

10. Preserve Customer Choice for Support. Oracle will ensure that end-user and embedded customers paying for MySQL support subscriptions will be able to renew their subscriptions on an annual or multi-year basis, according to the customer’s preference.

Small stuff.

• RainStor expects to finish the year with > 50 users (overwhelmingly via partners)
• A big market for RainStor (at least in terms of signed partnerships and large deal activity) is retention of telecom records, for compliance purposes, typically for a 1-3 year period. This includes:
  • CDRs (Call Detail Records)
  • Mobile phone records including CDRs and missed calls
  • SMS (Short Message Service), including the complete text of same
• RainStor thinks a number of larger telcos have the need to store a billion records per day each. (I’m not sure how many subscribers such a telco would have to have).
• John further thinks that, for the same query performance, RainStor can handle such a database on 4 blades. More precisely, he says that’s what happened at a test conducted by a major technology firm. In the same test case, SenSage required 40 blades, and Oracle required 80 or more cores on a pair of big SMP machines. John further says that the Oracle solution required a new table and new tablespace every day, while RainStor’s took 3 days for initial installation and required no DBA afterwards. However, I’m in no position to verify this report independently.
• In a different kind of proof point, so extreme it gives even the RainStor folks pause, a user has retired 300 different applications and put their databases onto a single 2-core box. (Presumably, this is via RainStor’s OEM relationship with Informatica.)
• Coming Very Soon are some services tying RainStor’s DBMS to obvious-suspect SaaS offerings. The core positioning is “SaaS data escrow”.i.e., RainStor will help you ensure that, in a worst-case scenario, there’s a nice safe copy of your data you can get at. RainStor also encourages you to do basic reporting and BI against the RainStor copy of the data, if you choose.
• The idea I’ve been pushing lately of taking a heterogeneous replication offering like Continuent’s and having it feed an archiving store like RainStor’s has hit a rather basic snag. RainStor doesn’t actually consume change data capture kinds of information directly, at least as of yet, because of difficulties fitting such a stream into its guaranteed-data-immutability model.

*I coined that category description for John in the tea room of the Park Lane Hotel. He’s subsequently embraced it enthusiastically, and I kind of like it myself.

Related links

• RainStor’s approach to compression, as described by me and by RainStor itself

• Teradata ascribes considerable importance to a Virtual Storage technology whose main purpose is to allow mixing of heterogeneous storage devices in a single system. And the discussion rarely suggests that these parts will be in a rigid fixed relationship.
• Netezza — as Teradata keeps reminding me — often sells boxes with the expectation that they won’t be filled with data, so as to increase spindle count and hence performance.
• Oracle/Sun have dropped some comments about Exadata being more flexibly configured going forward.
• Kickfire’s new “high-end” appliance lets you attach fairly arbitrary amounts of external storage.
• And of course, software-only analytic DBMS vendors run their software in all sorts of hardware and storage environments.

What’s more, the claim never made a lot of sense anyway. With the rarest of exceptions, even a single data warehouse’s workload will contain different queries that strain different parts of the system in different ratios. Calculating the “ideal” hardware configuration for that single workload would be forbiddingly difficult. And even if one could calculate it, it almost surely would be different than another user’s “ideal” configuration. How a single hardware configuration can be “ideally balanced” for a broad class of use cases boggles the imagination.

]]> http://www.dbms2.com/2009/10/25/data-warehouse-balanced-hardware-configuration/feed/ 6 http://www.dbms2.com/2009/10/06/oracle-exadata-customers-presenting-at-oracle-open-world/ http://www.dbms2.com/2009/10/06/oracle-exadata-customers-presenting-at-oracle-open-world/#comments Tue, 06 Oct 2009 12:46:12 +0000 Curt Monash http://www.dbms2.com/?p=1052 Greg Rahn tweeted a list of Exadata-focused sessions at Oracle Open World next week. As Oracle employees and supporters have been foreshadowing, there will be Exadata users and user-like folks presenting. I identified what look like half a dozen (not counting any who, for example, will make surprise appearances at keynote addresses), specifically:

• Paragon Data GmbH (which seems to be a system integrator and internet services provider)
• Metavante (which seems to be a data mart outsourcer)
• LGR Telecommunications (ditto, and in the Exadata 1 beta program)
• The Hartford, which is an insurance company
• Giant Eagle, which is a supermarket chain
• Allegro Group, which is a Polish online auction company, and one of Oracle’s most accessible Exadata references

By way of comparison, users presenting at Teradata Partners the following week include (partial list):

• Bank of America
• Nationwide Insurance
• Volvo
• Amgen
• Travelers
• Paypal
• eBay
• American Airlines
• AT&T
• GlaxoSmithKline
• Toys R Us
• Nokia
• Intel
• Office Depot
• WellPoint
• Telefónica Chile
• 3M
• Defense Logistics Agency
• Verizon
• FedEx
• Sabre
• LockheedMartin
• Hershey
• Wells Fargo
• Saks Fifth Avenue

Related link

• The hunt for Oracle Exadata production references