• Slide 2 has some market success figures and so on. (>3100 copies at >1800 users, >200 sales last year)
• Slides 6-11 give more detail on Sybase’s indexing and data access methods than I put into my recent technical basics of Sybase IQ post.
• Slide 16 reminds us that in-database data mining is quite competitive with what SAS has actually delivered with its DBMS partners, even if it doesn’t have the nice architectural approach of Aster or Netezza. (I.e., Sybase IQ’s more-than-SQL advanced analytics story relies on C++ UDFs  — User Defined Functions — running in-process with the DBMS.) In particular, there’s a data mining/predictive analytics library — modeling and scoring both — licensed from a small third party.
• A number of the other later slides also have quite a bit of technical crunch. (More on some of those points below too.)

Sybase IQ may have a bit of a funky architecture (e.g., no MPP), but the age of the product and the substantial revenue it generates have allowed Sybase to put in a bunch of product features that newer vendors haven’t gotten around to yet.

More recently, Sybase volunteered permission for me to preannounce Sybase IQ Version 15.2 by a few days (it’s scheduled to come out this week). Sybase IQ seems to be focused on large part on the government/intelligent market, with three major features being:

• A kind of data federation, querying external databases, that makes sense mainly in the context of rigorous security rules. (I find that confusing, since Sybase IQ’s indexes tend to hold all the information in the database, but I didn’t push the point.)
• An upgrade to Sybase IQ’s built-in text indexing. I doubt anybody would confuse this with best-of-breed text search, but evidently that intelligence community is satisfied with less. But even before 15.2, Sybase IQ could do both LIKE and WHERE CONTAINS searching.
• Improved LOB (Large OBject) management.

One part of my Sybase IQ conversations I haven’t blogged yet in much details is scale-out, concurrency, and “multiplexing.”

• Sybase feels that Sybase IQ’s competitive sweet spot, especially in terms of performance, is reached when there are 20 or more concurrent queries.
• In general, Sybase asserts that a shared-everything architecture is great for concurrency — just run different queries on different boxes, all against the same data.
• The ability to use a bunch of boxes run Sybase IQ is called “multiplexing.”  This is a chargeable option, without which one is limited to a single SMP box.
• Just under 20% of the top 250 Sybase IQ customers have multi-node scale-out configuration (vs. single-node SMP scale-up). And around 8% have it overall.
• Sybase IQ nodes can be heterogeneous (e.g., in compute power).
• Sybase IQ nodes can be dedicated to be read-only, or can be read-write. Indeed, Sybase IQ nodes can change roles dynamically, for example becoming write-only during nightly batch load. (I didn’t clarify whether all this applies just to nodes-as-boxes, or if some parts apply to specific processors or cores within the same box.)
• Sybase noted that data mart outsourcers can offer differentiated SLAs (Service Level Agreements) depending upon which nodes they give which customers access to.
• Most Sybase IQ installations start at 8 cores or more. The Sybase IQ Small Business Edition, limited to 4 cores, is not a big seller.
• Sybase IQ has a straightforward round-robin load-balancing story via third-party technology.

Finally, along the way in the discussions I picked up various tidbits about the Sybase IQ user base. Unfortunately, Sybase is pretty vague in discussing database sizes — are they user data? Are they compressed? What do the numbers mean? With that huge caveat:

• By some metric or other, a couple of classified customers are approaching petabyte scale.
• The largest commercial Sybase IQ customer — a credit card company — has a couple hundred terabytes or so.
• The largest financial services Sybase IQ databases are 50-70 terabytes. This sounds low, frankly, so maybe those are compressed figures, with user data being 200+ terabytes. But I’m just speculating there.
• Sybase IQ has a little less than 100 customers in the “data aggregator” market, which is a lot like what I call “data mart outsourcer.”
• Sybase IQ’s ILM technology is a chargeable option, with Sybase being “cautious” about sales. Compliance is a big market driver for it.
• Sybase IQ’s #1 vertical market is financial services. Other biggies are government, telecom, marketing services, and to some extent retail.
• As of February, there were 40-45 production users of Sybase IQ 15.0 and 15.1.

Analytic technology isn’t only for you. It’s also for your customers, citizens, and other stakeholders.

I am not referring here to what is well understood to be an important, fast-growing activity — providing data and its analysis to customers as your primary or only business — nor to the related business of taking people’s data, crunching it for them, and giving them results. That combined sector — which I am pretty alone in aggregating into one and calling data mart outsourcing — is one of the top several vertical markets for a lot of the analytic DBMS vendors I write about. Rather, I’m talking about enterprises that gather data for some primary purpose, and have discovered that a good secondary use of the data is to reflect it back to stakeholders, often the same ones who provided or created it in the first place.

For now I’ll call this category stakeholder-facing analytics, as the shorter phrase “stakeholder analytics” would be ambiguous.* I first picked up the idea early this decade from Information Builders, for whom it had become something of a specialty. I’ve been asking analytics vendors for examples of stakeholder-facing analytics ever since, and a number have been able to comply. But the whole thing is in its early days even so; almost any sufficiently large enterprise should be more active in stakeholder-facing analytics than it currently is.

*Comments as to what the category should be called are welcome below.

Examples of stakeholder-facing analytics include:

• Enterprises report back on the business customers do with them. For example:
  • Credit card companies provide reports on spending back to their credit card holders, especially small businesses.
  • So do office supply retailers.
  • Brokerage firms provide reporting back to their small-institution customers.
• Governments expose information to their citizens online.
  • In an early example, New York City restaurant ratings were put online.
  • Putting SEC filings online has has been a huge success.
  • The Obama Administration has committed to putting large amounts of information online.
• Regulated companies (such as utilities) could be required to put data online directly, without even using the government as an intermediary.
• Some part of Fox — perhaps MySpace Music? — offers free access to a PostgreSQL extract from its Greenplum database to each of its largest advertisers.
• Google Analytics offers some basic BI for free to website owners everywhere.
• Anybody from web hosting companies to public utilities could open their kimonos and allow their customers to track adherence to actual or implied SLAs (Service Level Agreements) in areas such as uptime, length of outage, responsiveness, and the like.

So what cool examples do you have of stakeholder-facing analytics?*

*Yes, this is an invitation to drop links to case studies into the comment thread below.

Highlights on the market share/sector side include:

• Infobright’s customer base grew 500% over the past year, to 120 paying customers.
• This included end users (60%), as well as ISVs and SaaS providers (40%) who embed Infobright’s DBMS in their application.
• During the same period, Infobright’s open source software was downloaded 35,000 times.
• The end user applications were heavily clustered around web and online analytics tracking, with a focus on understanding customer behavior on the web.
• Infobright also continues to see the growth of application-specific data marts.
• There is also continued interest and growth in using Infobright technology to analyze IT logs and telecom CDR (Call Detail Record) data, to identify fraud or security issues, to understand and improve network performance, and other purposes.

Product highlights include:

• Infobright be much more transparent in 2010 about its plans.
• Infobright will start posting and commenting on future releases and themes in March of this year. (However, they haven’t run much of that by me yet, and we’re past the middle of March.)
• Infobright expects to drop 3-4 interim releases for every major release, with at least two major releases in 2010.
• Some of Infobright’s major improvements this year will be:
  • Continued SMP performance improvements “without the need for complex hardware configurations or administrative effort”.
  • Extending the “hit rate” of the Knowledge Grid, which is central to Infobright’s performance story.
  • Better international support with UTF-8 extensions.

That said, highlights of my Netezza Skimmer briefing included:

• In essence, Netezza Skimmer is 1/3 of Netezza’s previously smallest appliance, for 1/3 the price.
• I.e., Netezza Skimmer has 1 S-blade and 9 disks, vs. 3 S-blades and 24 disks on the Netezza TwinFin 3.
• With 1 disk reserved as a hot spare, that boils down to a 1:1:1 ratio among CPU cores, FPGA cores, and 1-terabyte disks on Netezza skimmer. The same could pretty much be said of Netezza TwinFin, the occasional hot-spare disk notwithstanding.
• Netezza Skimmer costs $125K.
• With 2.8 or so TB of space for user data before compression, that’s right in line with the Netezza price point of slightly <$20K/terabyte of user data.
• That assumes Netezza’s usual 2.25X compression. I forgot to ask when 4X compression was actually being shipped.
• I forgot to ask, but it seems obvious that Netezza Skimmer uses identical or substantially similar components to Netezza TwinFin’s.
• Netezza Skimmer is 7 rack units high.
• In place of the SMP hosts on TwinFin Systems, Netezza Skimmer has a host blade.
• Netezza (specifically Phil Francisco) mentioned that when Kalido uses Netezza Skimmer for its appliance, there will be an additional host computer, but when it uses TwinFin for the same software, the built-in host will suffice. (Even so, I suspect it might be too strong to say that Skimmer’s built-in host computer is underpowered.)
• Netezza also suggested that more appliance OEMs are coming down the pike specifically focused on the affordable Skimmer.

Obviously, Netezza Skimmer isn’t breaking any new technical ground. If Netezza had just called Skimmer “TwinFin 1,” nobody should have objected. So the main news here is that you can buy a Netezza box for $125K, plug it in, load a few terabytes of data, and be good to go with a pretty solid data warehouse.  For enterprises and data mart outsourcers with databases of the appropriate size, that could be a pretty attractive deal.

Is Netezza Skimmer as cheap as buying your own hardware and putting (free) Greenplum Single-Node Edition software on it? Not even close, especially since Greenplum’s free option limits you to lower overall compute power. Does Netezza Skimmer have as high availability as more expensive alternatives? In some cases, surely not. Skimmer is neither the cheapest thing around nor an utterly high-end product.

But Netezza Skimmer belongs on a lot of short lists even so.

• Infobright now has >100 paying customers.
• Typical database size is from the low 100s of gigabytes to the low single-digit number of terabytes.
• Agile development is at or approaching two-week release cycles.
• Like Kickfire, Infobright  has a multi-year deal with MySQL that insulates it against many potential Oracle/MySQL shenanigans.
• From an industry perspective, Infobright’s customer base sounds a lot like other vendors’:
  • Data mart outsourcing/online analytics
  • Log files for websites
  • Telecommunications
  • Financial services
  • OEM, especially in the markets cited above
  • “Hey, we’re beginning to see the occasional energy deal”
  • A few random others
• Infobright is seeing some household-name customers, who surely have big-name analytic DBMS products, but who also have a policy that open source is the default choice, and if open source can get the job done then the favorite closed-source choices aren’t used.
• Infobright has the usual open-source community story — lots of involvement and engagement in the forums, but contributions are limited mainly to connectivity, utility scripts, etc. (Maybe some national language translation too; I’m not sure.)

• The Nielsen Company has over half a petabyte of data on Netezza in the US. This installation is growing.
• The Nielsen Company indeed has 45 terabytes or whatever of data on Oracle in its European (Customer) Information Factory. This is not particularly growing. Nielsen’s Oracle data warehouse has been built up over the past 9 years. It’s not new. It’s certainly not on Exadata, nor planned to move to Exadata.
• These are not single-instance databases. Nielsen’s biggest single Netezza database is 20 terabytes or so of user data, and its biggest single Oracle database is 10 terabytes or so.
• Much (most?) of the rest of the installations are customer data marts and the like, based in each case on the “big” central database. (That’s actually a classic data mart use case.) Greg said that Netezza’s capabilities to spin out those databases seemed pretty good.
• That 10 terabyte Oracle data warehouse instance requires a lot of partitioning effort and so on in the usual way.
• Nielsen has no immediate plans to replace Oracle with Netezza.
• Nielsen actually has 800 terabytes or so of Netezza equipment. Some of that is kept more lightly loaded, for performance.

• >1500 users
• >3000 installations (Sybase has variously cited 2.1 and 2.5+ as the installation/user ratio)
• At least ~50-60 installations with >5 terabytes of user data

Note that 98% of Sybase IQ installations are under 5 terabytes; the heart of Sybase IQ’s business is the sub-terabyte data warehouse market.*

*Unlike most other analytic DBMS startups, Kickfire seems to be increasingly pursuing. that market too.

Sybase IQ was traditionally sold mainly to users of Sybase’s core Adaptive Server Enterprise DBMS (whether or not they ran other DBMS such as Oracle as well). Sybase recently has become more aggressive about selling IQ into non-Sybase shops. More generally, Sybase seems to have repositioned IQ in 2005, decided it liked the results, and ramped up investment in Sybase IQ as of 2006.

The way Sybase breaks down its different target markets is somewhat confusing, but so far as I can tell:

• A whole lot of Sybase IQ installations are focused on straight reporting.
• Sybase is beefing up its efforts and penetration for IQ in “advanced analytics.” How advanced that is to date is a little unclear.
• Sybase claims 80-90+ Sybase IQ customers in the “data aggregator” business, counting fairly narrowly.
• Financial services is, unsurprisingly, a special-case market of particular focus.

Sybase IQ pricing is traditionally complicated; perhaps one of these months Sybase will clarify it for me. The latest iteration appears to be mainly per-core, but I don’t have a good sense for what kinds of workloads can be handled by what number of cores.

Related links

• Sybase IQ technical highlights

]]> http://www.dbms2.com/2009/08/25/sybase-iq-business-notes/feed/ 3 http://www.dbms2.com/2009/07/29/scaling-postgres-choices/ http://www.dbms2.com/2009/07/29/scaling-postgres-choices/#comments Wed, 29 Jul 2009 06:16:02 +0000 Curt Monash http://www.dbms2.com/?p=849 I have a client who wants to build a new application with peak update volume of several million transactions per hour.  (Their base business is data mart outsourcing, but now they’re building update-heavy technology as well. ) They have a small budget.  They’ve been a MySQL shop in the past, but would prefer to contract (not eliminate) their use of MySQL rather than expand it.

My client actually signed a deal for EnterpriseDB’s Postgres Plus Advanced Server and GridSQL, but unwound the transaction quickly. (They say EnterpriseDB was very gracious about the reversal.) There seem to have been two main reasons for the flip-flop.  First, it seems that EnterpriseDB’s version of Postgres isn’t up to PostgreSQL’s 8.4 feature set yet, although EnterpriseDB’s timetable for catching up might have tolerable. But GridSQL apparently is further behind yet, with no timetable for up-to-date PostgreSQL compatibility.  That was the dealbreaker.

The current base-case plan is to use generic open source PostgreSQL, with scale-out achieved via hand sharding, Hibernate, or … ??? Experience and thoughts along those lines would be much appreciated.

Another option for OLTP performance and scale-out is of course memory-centric options such as VoltDB or the Groovy SQL Switch.  But this client’s database is terabyte-scale, so hardware costs could be an issue, as of course could be product maturity.

By the way, a large fraction of these updates will be actual changes, as opposed to new records, in case that matters.  I expect that the schema being updated will be very simple — i.e., clearly simpler than in a classic order entry scenario.

• Large enterprise data warehouse projects are often being deferred or cut back. (My sense is that a little of this happened in 2008, but more is happening with new budget cycles in 2009.)
• Smaller projects with credible, quick ROIs are doing fine. In some cases, these are the scaled-back or nose-under-the-tent parts of bigger enterprise-wide efforts.
• Perhaps not coincidentally, the technical trend of having a variety of data marts inside an EDW – sometimes called “sandboxes” — is going strong. I’ve commented on that before in connection with Microsoft.  It’s also at the heart of eBay’s Teradata-based “analytics-as-a-service” strategy.
• Uses of data warehousing for security or compliance still seem strong. I guess “Keep us safe (and out of jail)” is still a strong motivator for buying.
• Oracle’s is the #1 installed base in which smaller vendors go fishing. Teradata’s is probably #2. Microsoft SQL Server and MySQL are also suffering data warehouse competitive replacements.
• I haven’t heard many more examples of enterprises doing more analysis because the bad economy has invalidated their prior models and assumptions. More’s the pity.
• Projects to provide data to one’s customers are going gangbusters. There are many flavors of this, from pure third-party analytics vendors to (for example) credit card companies who sell transactional data back to their merchants to governments that become more “transparent” by exposing information to their citizens.

Obviously, if the economy is bad enough, everything will be hurt. E.g., a lot of those data sellers have something to do with advertising, and the underlying business sector is in the tank. Ditto consumer or mass B-to-B marketing as well. But for now, the worst hits are being suffered by projects with large price tags, long lead times, and unclear benefits.

Less successful to date are efforts in the area of pure database software-as-a-service.  It seems that if somebody is going for SaaS anyway, they usually want a more complete, integrated offering. The most noteworthy exceptions I can think of to this general rule are Kognitio and Vertica, and they only have a handful of database SaaS customers each. To wit:

1.  Kognitio has built a lot of its marketing around database SaaS, which it calls DaaS for data-as-a-service, and runs primarily from its own facility.  On a small sample size, it reports a very roughly 50-50 split in new business activity (that’s customers/prospects, not revenue) between DaaS and conventionally licensed software.

2.  Vertica has expressed high hopes for its Amazon cloud offering. Actual production usage has so far only matched part of that, but it isn’t exactly zero either. Specifically, marketing chief Dave Menninger writes by email:

In addition to approximately a dozen POCs running on the cloud at any point in time we have five customers using the cloud on a regular basis. Three of these customers do short lived projects so they start up instances, run them for the duration of a project, and shut them down. They are three different types of orgs: govt agency, pharma consulting org and SaaS provider.

Two financial services companies use the cloud as spare resource/capacity.  When they need additional computing resource or capacity they will temporarily move some projects onto the cloud with the anticipation of moving them back off once the capacity constraint is relieved (new hardware arrives, other projects or systems come to an end, etc.

3.  1010data offers its data warehousing product by remote service only.  However, unlike Gartner I’m not totally convinced 1010data should be regarded as comparable to DBMS vendors; perhaps it’s more like a SaaS business intelligence provider.

Edits:

• A comment below says Gerry Cohen wrote Nomad too.
  
• Kognitio commented on Twitter that they actually use DaaS to mean Data warehouse As A Service.