… or something like that. I don’t intend to write seriously about NorthScale until I have a better idea of what MemBase is.

In the mean time,

• VentureBeat put up a solid post on NorthScale’s company history and so on
• Om Malik bought into the NorthScale memcached pitch
• TechCrunch has a low-quality post about NorthScale (although it wasn’t as error-riddled as the same author’s post about nStein, which Seth Grimes properly blasted)

• One of the most confusing things about Sybase ASE is its version numbering. In particular,
  • Sybase ASE 15.5 went GA in December, 2009. (But the clustered version is just coming out in March.)
  • The prior version of Sybase ASE was 15.03.
  • Sybase ASE 15.0 came out in September, 2005.
  • The version of Sybase ASE before that was 12.5.
  • And by the way, Sybase System 10 came out in 1994 or so.
• Sybase ASE 15.0 was a major rewrite. In particular, Sybase ASE 15.0 had a “brand new” optimizer and query processing engine, based on the Volcano model. The main driver of the rewrite was to make Sybase ASE suitable for mixing OLTP and some level of decision-support workloads. (Not on the order of what Sybase IQ can handle, but at least operational reporting and so on.)
• I haven’t looked up Volcano in more detail than to confirm that what I thought Raj said made sense, but as he characterized it, it’s a lot more modular than what Sybase had in ASE 12.5. For example, substantially the only join algorithm in Sybase ASE 12.5 was nested loop – no hash or sort/merge.
• As you might imagine, a lot of things one might regard as core modern DBMS features were only added to Sybase ASE once 15.0 came out. Examples include:
  • Various forms of partitioning at the storage level.
  • User-defined functions (UDFs).
  • A clustering offering that competes with Oracle RAC. (100 or so customers are on that so far.) Absent clustering, Sybase ASE is limited to a single SMP (Symmetric Multiprocessing) box.
  • Shared disk. Amazingly, it seems that before 2008, every node in an SMP box running Sybase ASE had its own private partition (maybe not the right word) of data.
• In Sybase ASE, you have lots of databases managed by one database server. You can write SQL statements that span multiple databases, but they have to reference database names as well as table names.
• There are several ways to get data from one place to another in Sybase’s technology and nomenclature, specifically including. Replication Server, Incremental Data Transfer, and “proxy tables.” (Other than the fact that Replication Server is a separate, chargeable product, I don’t really have these straight.) In addition, there’s a hand-coded one in Sybase RAP, which will get a planned 5-6X performance improvement later this year when it is replaced by Incremental Data Transfer.

And in what basically sounds like a very cool approach, Sybase ASE has a lot of memory-centric aspects. That said, Sybase’s in-memory ASE story is still incomplete (wait until the next release) and confused (I think in part because of what’s missing in the current release).  Also, this is one area where the non-technical nature of the briefing got in my way. So here’s some of what I do and don’t know about Sybase’s memory-centric ASE strategy:

• Sybase lets you mix and match on-disk and in-memory databases under one instance of Sybase ASE. To a programmer, it all looks like ASE.
• I don’t know exactly what the limitations are on what you can do with in-memory databases, how you can use them in tandem with on-disk databases, etc.
• You can replicate data from disk to an in-memory Sybase ASE database today. (Hello caching, ala Oracle Times Ten or IBM DB2/solidDB.)
• Replicating from memory to disk is a near-term future capability. (So Sybase does not yet have a hybrid memory-centric story ala solidDB Classic.)
• I have no clue as to what kinds of in-memory data structures Sybase ASE uses.

• This is a good idea, and many software vendors should do it if they aren’t already.
• It’s no big deal.

For reasons why, let me quote an email I just sent to an inquiring reporter:

• StreamBase sells mainly to the financial services and intelligence community markets. Neither group will share much in the way of core algorithms.
• But both groups are pretty interested in open source software even so. (I think for both the price and customizability benefits.)
• Open source software commonly gets community contributions for connectors, adapters, and (national) language translations.
• But useful contributions in other areas are much rarer.
• Linden Labs is one of StreamBase’s few significant customers outside its two core markets.
• All of the above are consistent with the press release (which quotes only one StreamBase customer — guess who?).

To understand Sybase’s plans for Aleri and CEP, it helps to understand Sybase’s current CEP-oriented offering, Sybase RAP. So far as I can tell, Sybase RAP has to date only been sold in the form of Sybase RAP: The Trading Edition. In that guise, Sybase RAP has been sold to >40 outfits since its May, 2008 launch, mainly big names in the investment banking and stock exchange sectors. If I understood correctly, the next target market for Sybase RAP is telcos, for real-time network tuning and management.

In addition to any domain-specific applications, Sybase RAP has three layers:

• CEP (Complex Event Processing). Sybase RAP CEP is based on a version of the Coral8 engine Sybase licensed and has been subsequently developing.
• In-memory DBMS. Sybase’s IMDB is part of (but I guess separable from) and has the same API as Sybase’s OLTP DBMS Adaptive Server Enterprise (ASE, aka Sybase Classic).
• Sybase IQ. Actually, Sybase used the phrase “based on Sybase IQ,” but I’m guessing it’s just Sybase IQ.

In theory, there could be a DBMS other than Sybase IQ, such as Sybase ASE or even Oracle, because Sybase IMDB can talk to a variety of DBMS. I didn’t get the impression, however, that in practice there were any Sybase RAP installations whose persistent DBMS was anything other than Sybase IQ.

Aleri had all along had something called Project Ohio, to merge Coral8 with Aleri Classic. Now Sybase’s own CEP engineering team is being added to the mix, schedules are being reconsidered and haven’t been disclosed yet. (If one woman can produce one baby in nine months, how long does it take nine women to produce a baby?) Apparently Sybase has a dozen programmers in the CEP area, plus ~20 more on Sybase RAP, not counting QA, documentation, etc.; that represents a significant bump to the overall Aleri development team.

Sybase doesn’t seem to have decided what to do yet with the various business intelligence/real-time OLAP engine products and technologies it is inheriting from Aleri.

And finally, some metrics:

• The Sybase/Aleri guys estimate that 1/3 of of Aleri’s customers and even less of its revenue came from outside the financial services sector. They did say the non-financial-services business was “starting to pick up,” but not very convincingly.
• Sybase IQ is now up to >1800 customers, with >200 new ones in 2009.
• Sybase IQ indeed has users taking in market feeds up to 3 terabytes a day, so it probably matches Vertica in having at least several-hundred-terabyte databases in the financial sector.

• The folks at Sybase still haven’t figured out when to prebrief me. (Edit: I’ve been briefed subsequently.)
• Sybase/Aleri is a potentially powerful combination, if they can effectively address the point I just made about integrating disparate latencies. That said, I’m not expecting a lot, because the CEP industry always disappoints me.
• Microsoft, IBM, and (somewhat less clearly) Oracle are all trying to do CEP inhouse. Sybase is making a good choice in having serious CEP inhouse itself
• Surely the main focus and financial justification for the Sybase/Aleri acquisition is the financial services market.
• Specifically, I expect the focus of technical integration between Aleri and Sybase’s DBMS products to start with Sybase IQ.
• Coral8 had some interesting ideas about how to integrate CEP with OLTP/operational BI, but I’m not aware that they got much traction.
• I bet there are use cases where Sybase tries and fails to sell Adaptive Server SQL Anywhere that CEP would be a better technical fit, but I don’t immediately see much practical business significance to that observation.
• While this deal could easily strengthen the Vertica/StreamBase partnership, I don’t see any reason why it would lead those two companies to actually merge.

Related link

• Thinking about analytic latency

One section covered recent/ongoing/near-future trends that I thought were particularly interesting, including:

Simpler database technology, by which I mean DBMS that are:

• Easier to administer than market-leading systems …
• … even if at the cost of being special-purpose
• E.g.,
  • MySQL and older mid-tier RDBMS such as Progress
  • Many analytic DBMS and appliances, most notably Netezza’s

For general purpose or OLTP uses, I’m not a big fan of MySQL (not enough progress in making it industrial-strength), PostgreSQL (no good company behind it – I’m a non-fan of EnterpriseDB), or Ingres (open source or not, it’s an antiquated system that hasn’t been invested in as much as Oracle, DB2 or SQL Server).

But I get the impression there are a lot of contenders among small startups, featuring very new architectures for OLTP or general-purpose database management. VoltDB comes to mind. NimbusDB is finally within range of getting funded. Dan Weinreb told me Friday he knows of a bunch of others as well. And that’s all before we even get into the NoSQL kind of alternative.

Flexible storage architectures. That’s starting out with an emphasis on hybrid columnar, as in the examples of Vertica and Greenplum. Oracle (to whom I’m under no NDA obligation) and other vendors (to whom I am) are going that way as well.

Multi-tier database architectures, by which I mean at least two things:

• The database tier/server tier split of Exadata
• Hybrid RAM/disk architectures, examples of which include
  • Vertica’s RAM-based write-optimized store
  • Sensage’s CEP-in-the-DBMS
  • This in-memory analytics stuff we keep hearing about from the BI vendors
  • Any true in-memory/disk hybrid, such as the regrettably sidelined solidDB
  • Smart thinking by numerous DBMS vendors about optimizing the use of RAM and/or Level 2 cache

Netezza is particularly interesting to watch in this regard because it:

• Had a pretty strict storage/other processing split in prior product generations and …
• … ditched that in its latest generation …
• … which however is focused on optimizing the use of RAM cache

Also noteworthy is Petascan, the stealth-mode –and therefore harder to watch right now – company I keep teasing about, which makes a strong case for carrying the database/storage tier split into the flash/solid-state memory technology generation. Calpont also has a server/storage tier split, but that’s of mainly theoretical interest unless and until Calpont actually ships an MPP version of InfiniDB.

Cheaper parts, which have of course been a huge trend for decades. Solid-state memory will soon conquer the world. Meanwhile, cheaper sensors drive that machine-generated data I keep talking about.

An ever-better understanding of scale-out technology, in several respects, including:

• Query, notably data movement for MPP DBMS
• Update, especially minimalistic DBMS approaches, be they sharded MySQL or more NoSQLish
• Number-crunching, especially via MapReduce and/or parallel analytic libraries integrated into DBMS

Cool trends I touched on more briefly include:

• More data being available for analysis. This was a core theme of my Enzee Universe keynote speeches; there are also some notes on it in my post based on my Boston Big Data Summit talk.
• More users being served by analytics. Ditto.
• Data exploration/visualization, ala QlikView, Spotfire, or Tableau, and also the faceted stuff.
• The democratization of data mining. But I’m not as sure of that one as of the others…

One area I flat-out forgot to mention is easy data mart spin-out.

Other posts based on my January, 2010 New England Database Summit keynote address

• Data-based snooping — a huge threat to liberty that we’re all helping make worse
• Flash, other solid-state memory, and disk
• Open issues in database and analytic technology

• Oracle thinks flash memory is the most important hardware technology of the decade, one that could lead to Oracle being “bumped off” if they don’t get it right.
• Juan believes the “bulk” of Oracle’s business will move over to Exadata-like technology over the next 5-10 years. Numbers-wise, this seems to be based more on Exadata being a platform for consolidating an enterprise’s many Oracle databases than it is on Exadata running a few Especially Big Honking Database management tasks.

And by the way, Oracle doesn’t make its storage-tier software available to run on anything than Oracle-designed boxes.  At the moment, that means Exadata Versions 1 and 2. Since Exadata is by far Oracle’s best DBMS offering (at least in theory), that means Oracle’s best database offering only runs on specific Oracle-sold hardware platforms.

*E.g., I was sitting upstairs in my parents’ apartment in Columbus, OH having the call while their doctor, who I’ve never met, was visiting downstairs. He offered to make a special trip back Saturday afternoon because he missed me Wednesday, but he’s notorious for not coming when he says he will. Update: He didn’t come Saturday. On Saturday he said he’d come Sunday. He didn’t do that either.

Other high- and lowlights of our conversation included:

• Flash is the main new hardware element in Exadata Version 2. Otherwise, Exadata 2 is just an annual refresh of Exadata Version 1 to include updated components (Nehalem chips, bigger disk drives, etc.)
• Juan thinks it’s suboptimal to use flash memory through the bottleneck of disk controllers, favoring PCIe cards instead. (I emphatically agree.)
• Juan resolutely ducked questions about actual Exadata production deployment. Literally the only fact he shared in that regard is that there are at least 2 Exadata production systems running that each have 2 or more racks cabled together.
• Juan stressed that Exadata runs apps written over Oracle DBMS unchanged.
• When making mixed-workload claims for Exadata 2, Juan stressed consolidation of multiple databases, some OLTP and some analytic. He didn’t really argue with my skepticism about integrating OLTP and analytics in the same database, with one exception:
• Juan pointed out that in major OLTP apps such as ERP systems, there often is actually more processing going on in reporting and other batch stuff than there is in true OLTP.
• Exadata 2’s flash memory is designed as a disk cache, smarter than LRU (Least Recently Used). The two examples Juan gave of “smarter than LRU” are that backups and table scans don’t flush the cache.
• I forget whether this is new in Exadata 2 (I think it is), but anyhow – Exadata has a “Storage Index” that’s a lot like a Netezza zone map. I.e., for each megabyte or so of data it stores the min and max value of every column; if a query predicate rules out those ranges, that megabyte is never retrieved.
• Oracle has long offered what sounds like flexible workload management capability, and this has now been extended to specifically include I/O resources on the storage tier.
• This isn’t Exadata-specific, but Oracle has built a file system on top of its DBMS, optimized for speed, which helps with, e.g., ELT (Extract/Load/Transform). Evidently, it’s not at all the same thing as Mark Benioff’s 1990s Microsoft-annoying IFS (Internet File System) project, which seems to have morphed into a content management SDK.

Highlights specifically in the area of parallelization included:

• Juan stressed that all databases consolidated onto an Exadata machine are/should be striped across all storage units.
• On the other hand, Juan said that different databases should be confined to specific cores or CPUs on the database tier.
• But on the third hand, Juan also stressed – in what could be called a “private cloud” pitch – that there’s great elasticity as to which databases are matched to which server CPUs.
• Contrary to what I thought he and/or his colleagues told me a year ago, Juan said RAC (Real Application Clusters) is a big part of Oracle’s data warehouse processing.
• However, Juan says that what I regard(ed) as a major objection to Oracle’s database-tier parallelization — the need to manually specify “degrees of parallelism” — has now been obviated by automation. Juan thinks that few data warehouse DBAs will now need to manually tune parallelism, with minor exceptions. One exception he cites is that if a nightly report really is non-urgent, it can just be forced to run on a single core with no chance to grab more resources. (However, Juan thinks manual tuning of parallelism will continue to play a greater role in OLTP.)

OK. That’s all I can get done tonight (see above re: inconvenience of timing). Follow-on subjects I’d like to and indeed plan to post about include:

• What Juan said about hybrid columnar compression
• Oracle’s delightfully non-confidential slide deck, and a few comments about same

• Over a 20 week period, I had travel in 13 of them.
• 3 of those were vacation in November.
• As travel finally wound down:
  • It was time to focus a bit on my own business
  • Elder care got serious; e.g., my parents went to the hospital on consecutive days, Christmas week, the first one on their 52nd wedding anniversary
  • Linda and I both got really nasty colds
  • The holidays were happening
  • I started helping out a really cool startup company (first time I’ve taken stock in a private company in years; more on that soon)
  • There was less industry news going on anyway than in some other recent months

But of course I plan to speed up the research/analysis/writing soon. Here, FYI, are a few things I have on my plate.

For a couple of years now, the center of what I’ve written about has been high-performance analytic data processing. You can expect me to keep pursuing that in all its aspects. But there are two specific areas I’ve identified in which I want to redouble my efforts.

First, almost every BI vendor has an effort in “in-memory analytics” and/or “interactive data exploration.” I suspect there’s a lot of difference in underlying technologies, but I’m having trouble getting details. QlikTech (the worst foot-dragger of the three), Microstrategy, and Jaspersoft all owe me follow-up conversations with the people who know what’s going on well enough to explain it. Tableau keeps promising me a briefing and then not delivering. And I’m even further behind with the behemoth companies — Oracle, Microsoft, IBM/Cognos (arguably) et al.

Second, solid-state memory is coming to data warehousing. The obvious reasons are that it’s obviously close, and Moore’s Law still applies to bring it closer. More specific reasons for believing in solid-state include:

• Teradata has made large strides in making solid-state memory useful.
• The stealth start-up I mentioned above is poised to make further strides.
• (I’m not totally sure yet about this part) The in-memory analytics mentioned above might wind up working better in solid-state memory than in DRAM.

I’m spending quite a few cycles thinking about this area.

I’d also like to look further at analytic applications and advanced analytic functionality. I foreshadowed some of that in my Aster webinars. There’s some good stuff to talk about at Teradata I should try to write up soon. I need to have a follow-up conversation with fascinating anti-fraud guy I met at Netezza’s London event. But that’s all just scratching the surface.

Both the MySQL and PostgreSQL communities are in some disarray. Other non-behemoth OLTP/general-purpose DBMS seem to be, at best, thriving niche products. (I see little in the way of innovative new use for, say, Progress, Cache’, Ingres, or anything multivalue.) But it feels as if there’s more opportunity out there than is being met. And at a minimum, I’d like to learn more than the almost nothing I know about OLTP NoSQL alternatives.

I’ve already said that I expect to give an industry-overview talk at MIT on January 28. I also have an overviewy press article and overviewy white paper under discussion. If those come to fruition, I’ll of course let you know.

Besides the above, I of course have a number of specific posts that I need to get around to researching and writing at some point, often on topics I’ve already written about before.  Three subjects fairly high on the priority list are scientific data management, machine-generated data, and Oracle Exadata.

And finally, I have some subjects queued up for a couple of my other blogs as well. If you don’t already take our multi-blog integrated feed, this might be a good time to switch over.

Vertica offers a post on its 3.5 release, with a riff on the popular theme “We’ve fixed some weaknesses in our prior versions that we didn’t previously say we had.” More important, Vertica is pretty clear on the virtues of its hybrid columnar architecture.

Speaking of which — Oracle is going true hybrid columnar as well. I don’t have details or timing, however.

Dave Kellogg of Mark Logic wrote in to amusedly point out  Oracle’s anti-MarkLogic collateral. The very first charge Oracle levies is that MarkLogic goes beyond the emerging XQuery standard to add additional functionality. Considering Oracle’s approach to SQL standards, I tend to share Dave’s amusement.

Bill Conniff of Xponent LLC wrote in to tell of a vastly cheaper and less functional approach to XML management, apparently geared to looking at very large XML files one at a time.

Cayuga is a Cornell research project in complex event processing (CEP). There’s a Cayuga academic home page, a Sourceforge page for some open source Cayuga CEP code, and so on. Minsheng Hong, writing from a Vertica email address, tipped me off some months ago. The basic idea seems to be to do lots of queries very quickly, rather than a smaller number of queries over and over again. Whether this is an advance in anything but open-sourceness over Apama or Aleri I couldn’t say, but I do think it’s a different focus than that of StreamBase or pre-Aleri Coral8.

And finally, editor Doug Henschen listed his 15 favorite Intelligent Enterprise blog posts of 2009 — four each by Seth Grimes and Doug himself, three by Cindi Howson, two by me,* and one each by Mark Smith and Neil Raden.

*Doug selects up to three posts a month from here to republish.

Roman’s latest venture is business intelligence firm Gooddata. Gooddata was founded in 2007 and has been soliciting and getting attention for a while, so I was surprised to learn that Gooddata officially launched just a few weeks ago. Anyhow, some less technical highlights of the Gooddata story include:

• Gooddata believes it makes BI easy to adopt, unlike every other BI vendor on the planet — not excluding the many other BI vendors who say the same thing about themselves.
• Gooddata is entirely cloud-based, specifically in the Amazon cloud. I.e., Gooddata is selling SaaS-based BI.
• Gooddata wants to sell to enterprises that are large enough to have more than a couple of BI users, and small enough not to be well served by the BI market leaders.
  • In revenue terms, this is the ever-popular $100 million – $1 billion market.
  • Specifically, Gooddata believes that those enterprises may have decent “back office” BI, but don’t have much in the front office. Gooddata wants to provide them with front office BI, which seems to basically mean CRM analytics. Gooddata sees this as a market in which QlikTech is the major player. Generally, Gooddata wants to emulate and go after QlikTech.
  • Even more specifically, Gooddata wants to sell to Salesforce.com customers, who it believes are not well-served by what passes for built-in analytics at Salesforce. Partnering with NetSuite didn’t work as well, since NetSuite’s customers turn out to be smaller firms than are in Gooddata’s target market.
• Something I heard from both Jaspersoft and Gooddata is that there’s a hot market in providing cloud-based BI to online gaming companies. I gather these are mainly games running on mass communication platforms such as Facebook or the iPhone. Surely not coincidentally, it seems likely that:
  • These are small companies whose success – and hence data intake – can suddenly explode.
  • The data originates in cyberspace, with no particular need ever to come to the game companies’ own premises.
• Gooddata has 50 production customers.
• Gooddata had 2500 “projects” at the end of beta in June, and is adding 100 more per month. (Those numbers look weird together.) A “project” is a lot like a database, with associated reports, security privileges, etc.
• Gooddata has close to 40 people, mainly in development.
• I didn’t detect much of a sales strategy, nor much of a marketing strategy beyond the impressive early buzz generation. Perhaps that’s a partial explanation as to why the rate of Gooddata adoption fell even before the company officially launched.
• I forgot to ask what those 50 customers were actually paying, but considering Gooddata’s price list, it appears a typical price range for Gooddata’s stuff would be $500-$2,000/month.

Gooddata technical highlights include:

• Gooddata is building an entire BI stack – reporting, dashboards, ETL, in-memory database management, everything. I doubt Gooddata would claim that the pieces are best-of-breed in many ways other than BI ease of adoption and use.
• So far I’ve seen three Gooddata ease-of-use features or feature groups that strike me as differentiated – reusability (of metrics and/or reports), collaboration, and tag clouds. More on those below. Gooddata is also building toward an agility pitch, but those features aren’t all baked yet.
• Gooddata is MySQL-based today, but plans to move to a memory-centric compressed column store in 2010. Roman doesn’t reject analogies to SAP’s BI/BW/whatever Accelerator. Yes, folks – Gooddata is yet another BI vendor doing some form of memory-centric OLAP. That’s a big trend.
• I’m guessing that a big reason Gooddata is reinventing so many technical wheels is to ensure that the Gooddata stack is seamlessly multi-tenant from top to bottom. (Hasso Plattner of SAP’s comments on a similar idea suggest a similar emphasis.)
• Gooddata has its own multidimensional query language called MAQL (the A doesn’t seem to stand for anything). Today MAQL generates SQL for MySQL. The future columnar memory-centric data store will — I think – understand MAQL natively.

Now we get to the good stuff. When I wrote about reinventing business intelligence back in May, I focused on some interesting developments I see as actually underway — at least on an experimental basis and/or from small vendors – namely:

• Text-search interfaces. Well, while I didn’t see true text search in the Gooddata demo, I did see tag clouds, which have some of the same benefits.
• Collaboration tools. Well, Gooddata has a nice-looking approach to BI collaboration, heavily reflected in its UI metaphors. (That said, I haven’t really compared Gooddata to Microsoft SharePoint or SAP’s Portal/Rooms/whatever.)
• Memory-centric analytics (for speed of exploration). As noted above, Gooddata has that coming soon.
• Data exploration that tries to ignore fixed relational schemas, ala Attivio or Splunk. Roman says Gooddata is interested in or working on that, but offers no timetable.

Meanwhile, something I’ve been seeking for years, but haven’t seen much progress on since enhancement stopped on Cognos Metrics Manager, is more user-friendly metrics management. Well, it doesn’t have a lot of bells and whistles, but at least Gooddata has the basics – a list of already-defined metrics, and a reasonable way of compounding them into other metrics. I think that kind of thing will be a major BI feature going forward, to the point that a few years from now we’ll be worrying about how to port them from one BI vendor’s tool from another.

Bottom line: If you’re interested in BI, you should look at a Gooddata demo.

]]> http://www.dbms2.com/2009/12/27/introduction-to-gooddata/feed/ 12