Advance sound bites on the Microsoft/DATAllegro announcement

Microsoft said they’d prebrief me on at least the DATAllegro part of tomorrow’s SQL Server announcements, but that didn’t turn out to happen (at least as of 9 pm Eastern time Sunday night). An embargoed press release did just arrive, but it’s so concise and high-level as to contain almost nothing of interest.

So I might as well post sound bites in advance. Here goes:

• With the DATAllegro acquisition, Microsoft leapfrogged Oracle. But with Exadata, Oracle leapfrogged Microsoft back. Exadata is actually shipping.
• There’s no assurance that the first DATAllegro/Microsoft release will inherit SQL Server’s level of concurrency. After all, DATAllegro/Ingres wasn’t as concurrent as plain Ingres.
• Porting DATAllegro from Ingres to SQL Server is likely to be straightforward. If they screw up it will be because they tried to do too much else at the same time, not because the basic port failed.
• Porting DATAllegro from Linux to Windows should also be OK. DATAllegro doesn’t stress the operating system in the areas where Windows remains weak.
• Earlier this year, DATAllegro had exactly one customer known to be in production, but I’ve spoken with that one. It’s TEOCO, which has a multi-hundred terabyte DATAllegro installation. TEOCO is a very price-oriented buyer.
• DATAllegro reports that two more customers are in production with large systems now. Neither of those is believed by industry sources to be especially in love with DATAllegro. Otherwise, nobody seems able and willing to identify other DATAllegro customers.

I’m going to be pretty busy Monday anyway. Linda is having a bit of oral surgery. And if I get back from that in time, I have calls set up with a couple of clients.

Microsoft/DATAllegro time frame announced

Edit:  Actually, an email did eventually wend its way to me about a day later, which evidently had run into major congestion somewhere in the intertubes.

My resolve to eschew scathing sarcasm is being sorely tested tonight. The lastest trial is my discovery that nobody thought to so much as email me a press release, let alone brief me, on Microsoft’s announcement of a timetable for DATAllegro/SQL Server integration. Per Ina Fried — with a hat tip to anonymous commenter L.J. — Microsoft says:

The final version of that product is slated for the first half of 2010, though Microsoft said it will begin giving customers and partners access to early “community technology preview” releases within the next 12 months.

My current customer list among the data warehouse specialists

One of my favorite pages on the Monash Research website is the list of many current and a few notable past customers. (Another favorite page is the one for testimonials.) For a variety of reasons, I won’t undertake to be more precise about my current customer list than that. But I don’t think it would hurt anything to list the data warehouse DBMS/appliance specialists in the group. They are:

• Aster Data
• Calpont
• DATAllegro
• Greenplum
• Infobright
• Netezza
• ParAccel
• Teradata
• Vertica

All of those are Monash Advantage members.

If you care about all this, you may also be interested in the rest of my standards and disclosures.

Three happy 100 terabyte-plus customers for DATAllegro

Over on my Network World blog, I asked the question “So who are DATAllegro’s actual current customers?” As regular readers know, that’s a fairly hard question to answer. TEOCO is widely known as DATAllegro’s flagship reference, but after that the list gets thin in a hurry.

As a by-the-by to other discussions, DATAllegro Stuart Frost undertook to respond in part himself. Specifically, he gave me two names of two other happy customers that are or imminently will be running DATAllegro against 100+ terabytes of user data. Read more

Patent nonsense in the data warehouse DBMS market

There are two recent patent lawsuits in the data warehouse DBMS market. In one, Sybase is suing Vertica. In another, an individual named Cary Jardin (techie founder of XPrime, a sort of predecessor company to ParAccel) is suing DATAllegro. Naturally, there’s press coverage of the DATAllegro case, due in part to its surely non-coincidental timing right after the Microsoft acquisition was announced and in part to a vigorous PR campaign around it. And the Sybase case so excited a troll who calls himself Bill Walters that he posted identical references to it on about 12 different threads in this blog, as well as to a variety of Vertica-related articles in the online trade press. But I think it’s very unlikely that any of these cases turn out to much matter. Read more

Further thoughts on DATAllegro/Microsoft

My first, biggest thought about DATAllegro’s acquisition by Microsoft is “Why the ____ did it have to happen while I was trying to relax on my annual Cayman vacation???” Not coincidentally, I don’t plan to neatly cross-link all my posts and so on about DATAllegro/Microsoft until I get back to Acton this weekend.

One linking screwup is that I previously forgot to mention that — in addition to the numerous posts here — I also made several DATAllegro/Microsoft-related posts on my Network World blog A World of Bytes.  They include: Read more

Other early coverage of Microsoft/DATAllegro

• Here’s the official press release on DATAllegro’s site, and Microsoft’s.
• Doug Henschen of Intelligent Enterprise has a good article.  He got quotes from Microsoft claiming that SQL Server on its own would be able to handle 10s of terabytes of data in the next release, but DATAllegro was needed to get up to the 100s of terabytes.  That said, the quotes don’t say whether that’s user data or total disk usage — the latter frankly seems more plausible.
• James Kobielus of Forrester has a long post on the Microsoft/DATAllegro deal, emphasizing product packaging issues and glossing over technological differentiators. (Edit:  The post seems down as of Friday midday.)
• This is a few weeks old, but Kevin Closson is extremely skeptical of some of DATAllegro’s technical claims.  (Not that it matters much if he’s right — more nodes = more throughput, no matter how much Oracle folks rant.)
• Eric Lai of Computerworld gets it right.
• Larry Dignan thinks the acquisition is part of an overall strong Microsoft enterprise push.
• William McKnight thinks Microsoft usually does a good job of integrating acquisitions.
• DATAllegro CEO Stuart Frost is happy.
• David Hunter thinks Microsoft will blithely continue with DATAllegro’s limited-hardware-support strategy.  He’s almost certainly wrong.
• Philip Howard says almost nothing I agree with, although I can’t argue with the part

Conversely, it’s bad news for Ingres, bad news for Oracle, bad news for IBM, bad news for Teradata and bad news for HP, all for obvious reasons. As for the other appliance vendors: they will not be too happy either. In particular, we now have to consider who can survive on their own, who might be acquired, who might do the acquiring, and who is going to disappear.

DATAllegro could provide Microsoft with a true enterprise data warehouse sooner than you think

Jim Ericson of DM Review emailed the excellent questions:

Does DATAllegro give MSFT full-service high end data warehousing capability? If not, what is missing?

My quick answers are:

• No.
• Two things:
  • Hard-core multi-user concurrency.
  • Support for more esoteric analytic tools and functionality

Both are largely a matter of product maturity, and as a young company DATAllegro isn’t quite there yet.

That said, integration with Microsoft SQL Server is apt to be a big help in addressing both issues.

Read more

How will Oracle save its data warehouse business?

By acquiring DATAllegro, Microsoft has seriously leapfrogged Oracle in data warehouse technology. All doubts about maturity and versatility notwithstanding, DATAllegro has a 10X or better size advantage (actually, I think it’s more like 20-40X) versus Oracle in warehouses its technology can straightforwardly handle. Oracle cannot afford to let this move go unanswered.

It’s of course possible that Oracle has been successfully developing comparable data warehouse technology internally. But it’s unlikely. Oracle hasn’t done anything that radical, internally and successfully, for about 15 years, RAC (Real Application Clusters) excepted. (I.e., since the object/relational extensibility framework started in Release 7.) So in all likelihood, the answer will come via acquisition. I think there are four candidates that make the most sense: Teradata, Vertica, ParAccel, and Greenplum. Kognitio (controlled by former Oracle honcho Geoff Squire) might be in the mix as well. Netezza is probably a non-starter because of its hardware-centric strategy.

Here’s why I’m emphasizing Teradata, Vertica, ParAccel, and Greenplum:

Read more

Microsoft is buying DATAllegro

I’ve long argued that:

• Oracle and Microsoft are doomed in the data warehouse market unless they acquire MPP/shared-nothing data warehouse DBMS and/or data warehouse appliances.
• DATAllegro is the ideal acquisition for either of them.

Microsoft has now validated my claim by agreeing to buy DATAllegro. As you probably know, we’ve been covering DATAllegro extensively, as per the links listed below.

Basic deal highlights include:

Read more

Three cartoons from DATAllegro

Related links:

• Humor from Netezza
• Another gerbil-based solution

DATAllegro on compression

DATAllegro CEO Stuart Frost has been blogging quite a bit recently (and not before time!). A couple of his posts have touched on compression. In one he gave actual numbers for compression, namely:

DATAllegro compresses between 2:1 and 6:1 depending on the content of the rows, whereas column-oriented systems claim 4:1 to 10:1.

In another recent post, Stuart touched on architecture, saying:

Due to the way our compression code works, DATAllegro’s current products are optimized for performance under heavy concurrency. The end result is that we don’t use the full power of the platform when running one query at a time.

Read more

Data warehouse appliance power user TEOCO

If you had to name super-high-end users of data warehouse technology, your list might start with a few retailers, credit data processors, and telcos, plus the US intelligence establishment. Well, it turns out that TEOCO runs outsourced data warehouses for several of the top US telcos, making it one of the top data warehouse technology users around.

A few weeks ago, I had a fascinating chat with John Devolites of TEOCO. Highlights included:

• TEOCO runs a >200 TB DATAllegro warehouse for a major US telco. (When we hear about a big DATAllegro telco site that’s been in production for a while, that’s surely the one they’re talking about.)
• TEOCO runs around 450 TB total of DATAllegro databases across its various customers. (When Stuart Frost blogs of >400 TB “systems,” that may be what he’s talking about.)
• TEOCO likes DATAllegro better than Netezza, although the margin is now small. This is mainly for financial reasons, specifically price-per-terabyte. When TEOCO spends its own money without customer direction as to appliance brand, it buys DATAllegro.
• TEOCO runs at least one 50 TB Netezza system — originally due to an acquisition of a Netezza user — with more coming. There also is more DATAllegro coming.
• TEOCO feels 15-30 concurrent users is the current practical limit for both DATAllegro and Netezza. That’s greater than it used to be.
• Netezza is a little faster than DATAllegro on a few esoteric queries, but the difference is not important to TEOCO’s business.
• Official price lists notwithstanding, TEOCO sees prices as being in the $10K/TB range. DATAllegro’s price advantage has shrunk greatly, as others have come down to more or less match. However, since John stated his price preference for DATAllegro as being in the present tense, I presume the price match isn’t perfect.
• Teradata was never a serious consideration, for price reasons.
• In the original POC a few years ago, the incumbent Oracle — even after extensive engineering — couldn’t get an important query down under 8 hours of running time. DATAllegro and Netezza both handled it in 2-3 minutes. Similarly, Oracle couldn’t get the load time for 100 million call detail records (CDRs) below 24 hours.
• Applications sound pretty standard for telecom: Lots of CDR processing — 550 million/day on the big DATAllegro system cited above. Pricing and fraud checking. Some data staging for legal reasons (giving the NSA what it subpoenas and no more).

DATAllegro finally has a blog

It took a lot of patient nagging, but DATAllegro finally has a blog. Based on the first post, I predict:

• DATAllegro’s blog will live up to CEO Stuart Frost’s talent for clear, interesting writing.
• Like a number of other vendor blogs — e.g., Netezza’s — DATAllegro’s will have infrequent but usually long posts.

The crunchiest part of the first post is probably

Another very important aspect of performance is ensuring sequential reads under a complex workload. Traditional databases do not do a good job in this area - even though some of the management tools might tell you that they are! What we typically see is that the combination of RAID arrays and intervening storage infrastructure conspires to break even large reads by the database into very small reads against each disk. The end result is that most large DW installations have very large arrays of expensive, high-speed disks behind them - and still suffer from poor performance.

I’ve pounded the table about sequential reads multiple times — including in a (DATAllegro-sponsored) white paper — but the point about misleading management tools is new to me.

Now if I could just get a production DATAllegro reference, I’d be completely happy …

Positioning the data warehouse appliances and specialty DBMS

There now are four hardware vendors that each offer or seem about to announce two different tiers of data warehouse appliances: Sun, HP, EMC, and Teradata. Specifically:

• Sun partners with both Greenplum and ParAccel.

• HP sells Neoview, and also is partnered with Vertica.

• EMC (together with Dell in North America and Bull in Europe) sells DATAllegro. Now EMC is also entering a partnership with ParAccel.

• Teradata is pretty far down the road toward releasing a low-end product.

Read more

Intelligent Enterprise’s list of 12/36/48 vendors

I’m getting a flood of press releases today, because many of the companies I write about were selected to Intelligent Enterprise’s list of 12 most influential vendors plus 36 more to watch in the areas Intelligent Enterprise covers (which seems to be pretty much the analytics-related parts of what I write about here and on Text Technologies). It looks like a pretty reasonable list, although I think they forced the issue in some of the small analytics vendors they selected, and of course anybody can quibble with some of the omissions.

Among the companies they cited, you can find topical categories here for IBM (and Cognos), Informatica, Microsoft, Netezza, Oracle, SAP/Business Objects (both), SAS, and Teradata; QlikTech; Cast Iron, Coral8, DATAllegro, HP, ParAccel, and StreamBase; and Software AG. On Text Technologies you’ll find categories for some of the same vendors, plus Attensity, Clarabridge, and Google. There also are categories for some of these vendors on the Monash Report.

A quick survey of data warehouse management technology

There are at least 16 different vendors offering appliances and/or software that do database management primarily for analytic purposes.* That’s a lot to keep up with,. So I’ve thrown together a little overview of the analytic data management landscape, liberally salted with links to information about specific vendors, products, or technical issues. In some ways, this is a companion piece to my prior post about data warehouse appliance myths and realities.

*And that’s just the tabular/alphanumeric guys. Add in text search and you run the total a lot higher.

Numerous data warehouse specialists offer traditional row-based relational DBMS architectures, but optimize them for analytic workloads. These include Teradata, Netezza, DATAllegro, Greenplum, Dataupia, and SAS. All of those except SAS are wholly or primarily vendors of MPP/shared-nothing data warehouse appliances. EDIT: See the comment thread for a correction re Kognitio.

Numerous data warehouse specialists offer column-based relational DBMS architectures. These include Sybase (with the Sybase IQ product, originally from Expressway), Vertica, ParAccel, Infobright, Kognitio (formerly White Cross), and Sand. Read more

Vertica update – HP appliance deal, customer information, and more

Vertica quietly announced an appliance bundling deal with HP and Red Hat today. That got me quickly onto the phone with Vertica’s Andy Ellicott, to discuss a few different subjects. Most interesting was the part about Vertica’s customer base, highlights of which included:

• Vertica’s claim to have “50” customers includes a bunch of unpaid licenses, many of them in academia.
• Vertica has about 15 paying customers.
• Based on conversations with mutual prospects, Vertica believes that’s more customers than DATAllegro has. (Of course, each DATAllegro sale is bigger than one of Vertica’s. Even so, I hope Vertica is wrong in its estimate, since DATAllegro told me its customer count was “double digit” quite a while ago.)
• Most Vertica customers manage over 1 terabyte of user data. A couple have bought licenses showing they intend to manage 20 terabytes or so.
• Vertica’s biggest customer/application category – existing customers and sales pipelines alike – is call detail records for telecommunications companies. (Other data warehouse specialists also have activity in the CDR area.). Major applications are billing assurance (getting the inter-carrier charges right) and marketing analysis. Call center uses are still in the future.
• Vertica’s other big market to date is investment research/tick history. Surely not coincidentally, this is a big area of focus for Mike Stonebraker, evidently at both companies for which he’s CTO. (The other, of course, is StreamBase.)
• Runners-up in market activity are clickstream analysis and general consumer analytics. These seem to be present in Vertica’s pipeline more than in the actual customer base.

Read more

DATAllegro discloses a few numbers

Privately held DATAllegro just announced a few tidbits about financial results and suchlike for the fiscal year ended June, 2007. I sent over a few clarifying questions yesterday. Responses included:

• Yes, the company experienced 330% year-over-year annual revenue growth.
• The majority of DATAllegro customers have bought systems in the 25-100 terabyte range.
• One system over 250 terabytes has been in production for months (surely the one I previously wrote about); a second is being installed.
• DATAllegro has “about 100” employees. By way of comparison, Netezza reported 225 full-time employees for the year ended January, 2007 – which probably means as of January 31, 2007.

All told, it sounds as if DATAllegro is more than 1/3 the size of Netezza, although given its higher system size and price points I’d guess it has well under 1/3 as many customers.

Here’s a link. I’ll likely edit that to something more permament-seeming later, and generally spruce this up when I’m not so rushed.

Gartner 2007 Magic Quadrant for Data Warehouse Database Management Systems

It’s early autumn, the leaves are turning in New England, and Gartner has issued another Magic Quadrant for data warehouse DBMS. The big winners vs. last year are Greenplum and, secondarily, Sybase. Teradata continues to lead. Oracle has also leapfrogged IBM, and there are various other minor adjustments as well, among repeat mentionees Netezza, DATAllegro, Sand, Kognitio, and MySQL. HP isn’t on the radar yet; ditto Vertica. Read more

Three ways Oracle or Microsoft could go MPP

I’ve been arguing for a while that Oracle and Microsoft are screwed in high-end data warehousing. The reason is that they’re stuck with SMP (Symmetric Multi-Processing) architectures, while Teradata, Netezza, DATAllegro, and many others enjoy the benefits of MPP (Massively Parallel Processing). Thus, Teradata and DATAllegro boast installations in the hundreds of terabytes each, while Oracle and Microsoft users usually have to perform unnatural acts of hard-coded partitioning even to reach the 10 terabyte level.

That said, there are at least three ways Oracle and/or Microsoft could get out of this technical box:

1. They could buy or just partner with MPP vendors such as Dataupia, who offer plug-compatibility with their respective main DBMS.

2. They could buy whoever they want, plug-compatibility be damned. Presumably, they’d quickly add a light-weight data federation front-end to give the appearance of integration, then merge the products more closely over time.

3. They could develop or buy technology like DATAllegro’s, which essentially federates instances of an ordinary SMP DBMS across nodes of an MPP grid (Greenplum does something similar). I imagine that, for example, ripping Ingres out of DATAllegro and slotting in Oracle instead would be a pretty straightforward exercise; even without dramatic change to any of the optimizations, the resulting port would be something that ran pretty quickly on Day 1.

Bottom line: Oracle and Microsoft are hemorrhaging at the data warehouse high end now. But there are ways they could stanch the bleeding.

Hot buzzword — multidimensional partitioning

Teradata finally announced multidimensional range partitioning in Version 12, not that they kept their plans in that regard a big secret. DATAllegro has also shipped multidimensional partitioning to at least one customer. Other vendors — well, I’ll stop there, given my ongoing atttitude problems about vendors’ self-defeating NDAs.

Whether or not multidimensional partitioning is a big improvement over single-dimensional will of course depend a great deal on the details of a particular database. Teradata used a figure of 30% performance improvement, but that’s surely just an example. Certainly in some extreme cases one could have a rather large reduction in the amount of data retrieved, and correspondingly a many-times-X improvement in the performance of certain important queries. Read more

Oracle sincerely flatters DATAllegro

Actually, I’m kidding with the post title; I doubt that Oracle’s new deal with DATAllegro partners Dell and EMC has much to do with DATAllegro at all. Rather, I think it’s an example of a trend I’m also sensing* from other major hardware vendors — doing deals with multiple data warehouse software suppliers to cover different hardware size ranges. This just happens to be the first one to be announced.

*How’s that for a nice, vague euphemism?

DATAllegro is targeted at warehouses sized, at a minimum, in the tens of terabytes of user data. Oracle’s technology works well enough up into at least the multi-terabyte range — unless you’re looking to get the best possible price and/or performance on your system — but then things start getting dicey. So there isn’t a lot of overlap between the two Dell/EMC offerings. Read more

Four anonymous Netezza fans

I just found a blog post asking about Netezza that elicited quite a few responses, including at least four that purported to be from people whose companies had selected Netezza in a POC (Proof Of Concept) bake-off. One says Netezza was super-fast, even over DATAllegro, and DATAllegro’s professional services were lacking. One says Netezza is 50X faster than traditional alternatives on some queries, but up to 2X slower on some others. Two others just expressed love (or at least commitment) without giving details.

I haven’t yet looked through the rest of the responses in the thread.

Keep getting great research about database management systems, business intelligence, and related technologies. Get a FREE subscription by RSS/Atom or e-mail!

Technorati Tags: Netezza, data warehousing, data warehouse appliance, proof of concept

Notes from the Netezza user conference

EDIT: Big whoops, and apologies to Philip. I didn’t check the date, and what I linked to was last year’s article. That said, it read as if it could have been this year’s, which tells us something about the pace of Netezza’s information disclosure. Resulting errors of mine are left in place.

Netezza perennially annoys me by the secrecy with which it surrounds its information disclosure, especially at the annual user conference (just concluded). Essentially, except for what has also been separately disclosed, the whole thing is under NDA beyond the generality “We told you that we intend to improve our product by making more use of the FPGA.” Blech. That said, Philip Howard* has a long and — no surprise there! — upbeat article. So I’ll link to that, saving me some worries about what I myself am or am not allowed to say. E.g., I wouldn’t dare suggest — as Philip does — that Netezza’s zone maps (essentially, one-dimensional partitioning) could be enhanced going forward. And while I think Netezza has made strong efforts to tell the marketing stories Philip describes as being “hidden under a bushel,” I agree that — largely because of its self-defeating mania for secrecy — Netezza hasn’t done nearly as good a job of getting those messages accepted as it could have.

*Just to be clear — notwithstanding how much I tweak him for his exuberance, Philip seems to be a great guy, both in his publications and in person.

In general, much of what Philip wrote I would agree with. That said, let me hasten to point out some exceptions, including: Read more

Next Page →

Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

• DBMS2 covers database management, analytics, and related technologies.
• Text Technologies covers text mining, search, and social software.
• Strategic Messaging analyzes marketing and messaging strategy.
• The Monash Report examines technology and public policy issues.
• Software Memories recounts the history of the software industry.

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Recent white paper

The Explosion in DBMS Choice August, 2008

Recent webcast

What leading database vendors don't want you to know Originally broadcast April 9, 2008

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.

 

• Recent posts

  • Interpreting the results of data warehouse proofs-of-concept (POCs)
  • MySQL Query Analyzer
  • Silly website tricks
  • Graphjam: I can haz BI
  • When people don’t want accurate predictions made about them

• Categories

  • About this blog
  • Analytic technologies
    • Business intelligence
    • Data mart outsourcing
    • Data warehousing
    • MOLAP
  • Application areas
    • Games and virtual worlds
    • Investment research and trading
    • Scientific research
    • Telecommunications
    • Web analytics
  • Companies and products
    • 1010data
    • Ab Initio Software
    • Alpha Five
    • Amazon and its cloud
    • ANTs Software
    • Aster Data
    • Business Objects
    • Calpont
    • Cast Iron Systems
    • Cogito and 7 Degrees
    • Cognos
    • Coral8
    • CouchDB
    • DATAllegro
    • Dataupia
    • Elastra
    • EMC
    • EnterpriseDB and Postgres Plus
    • Exasol
    • Facebook and Cassandra
    • FileMaker
    • Google
    • Greenplum
    • H-Store
    • HP and Neoview
    • IBM and DB2
      • pureXML
    • illuminate Solutions
    • Infobright
    • Informatica
    • Ingres
    • Intel
    • Intersystems and Cache'
    • Jaspersoft
    • Kalido
    • Kickfire
    • Kognitio
    • Mark Logic
    • McObject
    • Microsoft and SQL*Server
    • MySQL
    • Netezza
    • Oracle
      • Exadata
    • Oracle TimesTen
    • ParAccel
    • Pervasive Software
    • PostgreSQL
    • Progress, Apama, and DataDirect
    • QlikTech and QlikView
    • SAP AG
    • SAS Institute
    • ScaleDB
    • Software AG
    • solidDB
    • StreamBase
    • Sybase
    • Talend
    • Teradata
    • Truviso
    • Vertica Systems
  • Data integration and middleware
    • Application servers
    • EAI, EII, ETL, ELT, ETLT
  • Data types
    • GIS and geospatial