• Workday has forward-thinking ideas about SaaS enterprise applications and the integration of business intelligence into same.
• Workday has highly innovative ideas in how it manages data.
• Companies founded by Dave Duffield tend to feature smart, likeable people who talk to one pleasantly and forthrightly. Workday is no exception; CTO Stan Swete and the other Workday folks present were a delight to talk with.
• I’d invited Merv Adrian to come along with me. He asked great questions, and I could gather myself a bit despite how sleep-deprived I was for the first part of that trip.

Workday kindly allowed me to post this Workday slide deck. Otherwise, I’ve split out a quick Workday, Inc. company overview into a separate post.

The biggie for me was the data and object management part. Specifically: 

• Workday’s applications run entirely in-memory, in a highly object-oriented structure. Persistence is mainly for the sake of data safety …
• … but not entirely. In earlier releases, Workday kept absolutely everything in RAM. However, certain things are kept only on disk, such as:
  • Audit files.
  • Certain documents (notably resumes).
• Workday’s whole database – data and metadata alike – is persisted to disk in <10 MySQL/InnoDB tables. MySQL is basically just being used as a key-value store, albeit one with ACID transactional support.
  • There are 3 main tables: attributes, relationships, instances.
  • When I suggested this might be like an entity-attribute-value model, Workday said it would be even better to think in terms of instanceID-attribute-value.
  • As you might expect for a database that simple, its schema doesn’t change much.
  • By way of comparison, Workday estimates that if its software were written relationally, there would be 1000s of tables, which would take up 10-100X as much disk space.
• All write transactions are banged immediately into the MySQL database. I.e., RAM and disk are never allowed to get out of sync.
• Workday’s database is append-only. This is exploited for effective dating (pretty heavily, it seems, perhaps because that’s a useful concept in human resources) and snapshotted reporting.
• Workday’s built-in BI doesn’t have a lot of choice but to do scans, traversing the object model. This turns out to be fast enough.

Other notes on Workday’s data and object management strategy include:

• Workday is object-oriented through and through – no object-relational mapping – turtles all the way down. On average, a class has about 2 attributes.
• 94% of requests are reads, traversing the object hierarchy.
• Workday databases are pretty small.
  • The biggest database Workday supports uses 17 gigabytes of RAM.
  • Workday databases are much smaller on disk than in RAM.
• Workday’s “dream” is to move from disk to solid-state memory.
• Workday uses GPLed MySQL/InnoDB. So there’s no software license reason to ever move away (e.g., to a pure key-value store).
• Disaster recovery is based on local and remote MySQL slaves.

Obviously, serious apps have been built before in object-oriented and/or key-value ways, with the resulting objects then being banged to disk (or in some cases kept in memory). Examples include:

• Numerous applications are built on object-oriented DBMS. Generally they go against disk, although memory-centric implementations can save a lot of pointer-chasing. Often they’re queried via SQL.
• Basho’s website says that its key-value store Riak was originally conceived in connection with a planned salesforce automation product, but I don’t think that the application part of that plan ever got built.
• SAP has longstanding doubts about relational dogma, although not nearly to Workday’s extreme.
• Obviously, some major internet applications just bang data into key-value stores.

Still, perhaps because it wholly object-oriented yet doesn’t even bother with anything like a real object-oriented DBMS, Workday’s approach seems particularly cool.

Other highlights of Workday, Inc.’s technical story include:

• Workday has settled into a schedule of three releases per year, and has pretty much lived up to that for >2 years.
  • Every user is always on the latest Workday release.
  • You can delay turning on significant new Workday software functionality if you want to.
  • Pure UI changes to the Workday software are handled much as they are on various websites today. Sometimes you have no choice but to live with them; sometimes the prior version of the UI remains available to you for a while.
• Workday’s navigational approaches look pretty cool.
  • The core concept is a list of actions you can perform now, rather than more standard menus.
  • Roles/permissions are of course central to this.
  • Reports have lots of actionable links in them. (More than just drilldown, although specific examples have slipped my memory.)
  • Alternatively, you can navigate via a search box, searching both on names of objects (e.g. users, divisions) or on names of tasks. This is somewhat reminiscent of an approach SAP was considering a few years ago.
• Workday says it has four key design premises:
  • Web-Familiar Experience. I’d say that’s true to to the extent it makes sense. In many ways, the web needs to catch up to Workday.
  • Enterprise Reporting. The idea is that you get a report, then take actions based on it. Hence the report-centric options for navigation.
  • Integration On-Demand. That’s a fancy way of saying “Plays nicely with others.”
  • Configurable Business Processes. Duh. That’s pretty essential if you want to do serious SaaS applications.
• Workday maintains a strong separation between application logic and UI development. Developer do no screen layouts. Instead, Uis are automatically generated for:
  • Flash/FLEX
  • iPhone
  • Mobile HTML
  • PDF export
  • Excel export
• Workday only talks to the outside world via web services.
  • Workday is heavily into SOAP (Simple Object Access Protocol).
  • The acquisition of OEM partner CapeClear gave Workday an Integration Service (i.e., enterprise service bus) that translates SOAP into whatever else might be needed for integration, and also does reliable delivery.
  • All that said, Stan Swete sees integration among various SaaS offerings as an area needing significant future attention.
• Workday’s business intelligence ideas are interesting, but I think there’s a long way for that technology still to go.
  • Workday’s BI seems to be focused on report/drilldown kinds of functionality.
    • You can slice by up to 2 dimensions at once.
    • Then you can keep slicing, however, by more dimensions, as many times as you like.
  • While you can take actions straight from reports, some of the specific BI/app integration ideas we discussed are still futures. (E.g., analyzing spend at the time of expense report data entry or approval.)
  • Of course, Workday’s web services interface lets you export Workday data into 3rd-party tools. Indeed, if you want to integrate data from Workday and some other source(s), that’s your only choice.
• Workday offers a clever metaphor to illustrate that your data may be more secure offsite than on – the bank vault. (I have no idea whether that’s a SaaS industry standard, but I hadn’t heard it before.) Of course, that metaphor does beg some issues specific to the remote data case, such as:
  • When your data is on premises, you know whether the government has insisted on looking at it.
  • More than cash, data keeps traveling back and forth to the remote location, which creates at least a theoretical risk of interception.
• Workday says the toughest part of globalization is the issue of which personal data is or is not maintained. For example, in the US you’re not allowed to not ask a job applicant’s religion, but in the UK you’re not only permitted but indeed required to.

This post is part of a three-post series

• Workday Inc. company overview (brief)
• Workday Inc. technology overview (detailed)
• Workday Inc. CTO Stan Swete’s comments on database strategy

That said, I think I found four sensible points in Pentaho’s Hadoop strategy, namely:

1. If you use an ETL tool like Pentaho’s to move things in and out of HDFS, you may be able to orchestrate two more steps in the ETL process than if you used Hadoop’s native orchestration tools.
2. A lot of what you want to do in MapReduce is things that can be graphically specified in an ETL tool like Pentaho’s. (That would include tokenization or regex.)
3. If you have some really lightweight BI requirements (ad hoc, reporting, or whatever) against HDFS data, you might be content to do it straight against HDFS, rather than moving the data into a real DBMS. If so, BI tools like Pentaho’s might be useful.
4. Somebody might want to use a screwy version of MapReduce, where by “screwy” I mean anything that isn’t Cloudera Enterprise, Aster Data SQL/MapReduce, or some other implementation/distribution with a lot of supporting tools. In that case, they might need all the tools they can get.

The first of those points is, in the grand scheme of things, pretty trivial.

The third one makes sense. While Hadoop’s Hive client means you could roll your own integration with your own favorite BI tool in any case, having somebody certify it for you themselves could be nice. So if Pentaho ships something that works before other vendors do, good on them. (Target date seems to be October.)

The fourth one is kind of sad.

But if there’s any shovel-meet-pony aspect to all this — or indeed a reason for writing this blog post — it would be the second point. If one understands data management, but is in the “Oh no! Hadoop wants me to PROGRAM!” crowd, then being able to specify one’s MapReduce might be a really nice alternative versus having to actually code it.

The immediate hook here is that I had a short conversation with Scott Gnau of Teradata yesterday, triggered by Teradata’s acquisition of Kickfire’s assets. Takeaways from that part included:

• The acquisition is all about Kickfire’s data pipelining technology.
• Scott (in my opinion rightly) thinks that isn’t particularly tied to Kickfire’s choice of particular DBMS architecture (fairly vanilla columnar).
• No decision has been made about whether the right vehicle for this technology is an FPGA (Field Programmable Gate Array), conventional Intel CPU, RAM, etc.

If you want to handicap Teradata’s future data pipelining strategy, you might note that:

• Kickfire’s own choice – and hence its existing implementation – is an FPGA.
• VectorWise’s approach to pipelining is Intel-based, apparently at the cost of being closely tied to specific generations of Intel CPUs.
• XtremeData’s approach to pipelining is FPGA-based.
• Teradata has a lot more development resources than any of those other companies, as well as important existing products, and hence has both means and motive to shoehorn new technology into older system designs.

While I had Scott on the phone, I brought up a few other subjects too. Highlights included:

• Teradata’s Flash-based appliance is doing just fine in beta test and customer POCs (Proofs of Concept).
• Other kinds of Teradata appliance are not inconceivable.
• Scott thinks Michael McIntire’s condemnation of Active-Active architectures is overstated. That said,
  • Scott does acknowledge a need for greater Active-Active scalability, and suggests that the reason Xkoto’s current products are being discontinued is their lack of scaling.
  • Scott seems quietly confident the scaling will get done.
• Scott is emphatic that Teradata is not going to go to a two-tier architecture. In particular, the point of splitting storage/lightweight database processing and heavyweight database processing on separate tiers is generally to save bandwidth, and Teradata’s BYNET is typically less than 10% loaded.
• Scott didn’t dispute my claim that this all suggests Teradata Virtual Storage is the future, at the expense of a rigid delineation among specific use-case-focused product lines.
• Unlike Netezza or Aster, Teradata doesn’t seem to plan analytic capability that works outside the UDF (User Defined Function) framework. However, Scott noted that Teradata has long had the capability that Aster and Netezza now also have of letting you run analytic code either in “protected mode” (if the process fails the whole database doesn’t crash) or in the database kernel (best performance, if you’re sufficiently confident in the code’s stability to take the risk). Scott also spoke of the release later this quarter of Teradata FastPath, which will offer yet better performance (however, there’s a gotcha to Teradata FastPath that’s still NDA).

Putting all that together with the rest of what we know about Teradata, I’m going to call out three pillars of Teradata’s long-term product strategy:

• Same fundamentals as always. Teradata’s core product strategy is:
  • Single DBMS, capable of meeting all analytic needs while running in a single instance, usually running on …
  • … proprietary hardware …
  • … built from conservatively-chosen parts.
• Selective vertical application stack. No matter how horizontally-oriented they are, many companies that have been in the analytic technology business for a while wind up with some vertical applications. It sort of just happens. Teradata is no exception. Teradata also likes to sell services to its product customers, and some of those are quite vertical-aware.
• Mutable, modular platform. This is what I highlighted above. Note that it’s philosophically attuned with the one-system-does-everything approach Teradata prefers. More subtly, please also note that it goes well with customer-by-customer price customization, which is almost a must for Teradata given the Innovator’s Dilemma kind of pricing box it finds itself in.

So far, that’s not too exciting, except in the details of how Teradata’s engineers make that all work. But there’s a fourth pillar to Teradata’s technical strategy as well, and it’s a wild card: tight partnerships. Every time I talk with Teradata hardware chief Carson Schmidt, he seems excited about some particular version of a part or other – sometimes from a reasonably established vendor (once it was LSI Logic), sometimes from a tiny one (notably the “stealth” start-up on which Teradata bet its first solid-state product.) In the future, I expect tight business intelligence partnerships as well. Cognos BI will be increasingly integrated with IBM’s DBMS and hardware; Business Objects’ BI will increasingly be integrated with SAP’s applications; and Oracle’s BI will eventually be integrated with everything. How do you compete with that if you‘re Microstrategy? Well, you try to have superior product, of course – but you also partner as closely with DBMS vendors as you can, an approach Microstrategy has already started. Predictive analytics stalwart SAS, of course, is on a partnership binge as well.

Teradata has a larger installed base than almost all its competitors, and enjoys richer third-party software and service support as a result. But I suspect that going forward, for Teradata to remain a leading competitor at price points it is willing to accept, Teradata’s “ecosystem” advantages will need to ratchet up one or several notches.

Most of what I get paid for is in some form or other consulting. (The same would be true for many other analysts.) And so I can be a bit stingy with my advice toward non-clients. But my non-clients are a distinguished and powerful group, including in their number Oracle, IBM, Microsoft, and most of the BI vendors. So here’s a bit of advice for them too.

Oracle. On the plus side, you guys have been making progress against your reputation for untruthfulness. Oh, I’ve dinged you for some past slip-ups, but on the whole they’ve been no worse than other vendors.’ But recently you pulled a doozy. The analyst reports section of your website fails to distinguish between unsponsored and sponsored work.* That is a horrible ethical stumble. Fix it fast. Then put processes in place to ensure nothing that dishonest happens again for a good long time.

*Merv Adrian’s “report” listed high on that page is actually a sponsored white paper. That Merv himself screwed up by not labeling it clearly as such in no way exonerates Oracle. Besides, I’m sure Merv won’t soon repeat the error — but for Oracle, this represents a whole pattern of behavior.

Oracle. And while I’m at it, outright dishonesty isn’t your only unnecessary credibility problem. You’re also playing too many games in analyst relations.

HP. Neoview will never succeed. Admit it to yourselves. Go buy something that can. 

Smaller BI vendors. Analytic DBMS evaluations commonly include BI strategy and tool selection as well. If an analytic DBMS expert tells you he needs to learn more about your product line, don’t blow him off. In fact, you should be particularly embracing anybody who’s shown a fondness for small DBMS vendors; maybe he or his clients will like small BI vendors as well. That means (among others) you, Jaspersoft, Endeca, and Tableau.

Information Builders. Is there anything about your BI products that is in any way technologically differentiated? If so, you might want to mention some examples to somebody some time.

Kalido. I’ve said this to you before, but it bears repeating — your positioning translates to “I-CASE for analytics,” and that’s not a good thing. If your product is not as cumbersome and entrapping as that sounds, you need to do a much better job of explaining why not.

SenSage. You are what you are. Sell out while the selling is good. You don’t have the corporate personality to make it into the analytic DBMS mainstream on your own.

Ingres. You need to be more engaged with analysts than you are. Ingres navel-gazed too much 25 years ago, and evidently you haven’t outgrown it yet.

TIBCO. You probably have a lot of cool analytic technology, but I don’t know of an influencer who has much relationship with or trust in you. Rethink how you’re approaching influencer relations top to bottom.

Tableau. You had a lot of mindshare, but it’s fading. Do something.

MarkLogic, graph DBMS vendors, etc. You’re clinging too hard to the NoSQL label. Nobody is out there deciding among Cassandra, neo4j, and MarkLogic. They might be deciding between MongoDB and MarkLogic, I guess, but if you admit to yourself that’s all it is you’ll probably change your messaging somewhat.

Objectivity. Get real about marketing. Infinite Graph is a cool opportunity. But I didn’t even ping you for a meeting when I’m in your area next week, because I wouldn’t have known who to reach out to.

Everybody (especially Objectivity). “First X deployed in the cloud” is almost surely an inaccurate claim. Don’t make it. And by the way, even if it were true, it probably wouldn’t be interesting.

Highlights of the in-memory BI discussion included:

• Microstrategy’s in-memory BI data structure is some kind of simple array, redundantly called a “vector array.” A more precise description was not available.
• While early versions of the capability have been around since 2002, Microstrategy’s in-memory BI capability only got serious with Microstrategy 9, which was released in Q1 of 2009. In particular, Microstrategy 9 was the first time in-memory BI had full security.
• Mark says a core reason for having their own in-memory BI is because Microstrategy has more smarts to predict which aggregates will or won’t be needed. Strictly speaking, that can’t be argued with. Vendors like Infobright would argue they come close enough to that ideal as to make little practical difference – but I’m also cheating by naming Infobright, which is particularly focused in that direction.
• Microstrategy in-memory BI compresses data by about 2X. Mark didn’t know which compression algorithm was used.
• The limitation on what’s in-memory is, of course, how much RAM you can fit on an SMP box. Microstrategy has seen up to ½ terabyte deployments.
• In-memory Microstrategy data structures are typically built during the batch window, for performance reasons. This is not, strictly speaking, mandatory, but I didn’t get a sense that Microstrategy was being used for much that resembled real-time business intelligence.
• Mark said Microstrategy has no interest in using solid-state memory to expand the reach of its in-memory BI. Frankly, if Microstrategy doesn’t change that stance, it’s in-memory BI capabilities are unlikely to stay significant for too many years.

Another key subject we discussed was Microstrategy’s view of dashboards.

• Microstrategy thinks that what customers really want is to have a whole lot of navigational drilldown options into a few big reports. (“50 pages, 50 columns” was mentioned as an example of “big”.) This has been Microstrategy’s approach for three years or so.
• Microstrategy even offers a version of this in Flash, which can be drilled down on with no calls to the server whatsoever.
• This is also Microstrategy’s paradigm on the iPhone and iPad, where it would seem to make particular sense since you aren’t exactly going to tile a portal page into 6 different charts anyway.
• On the iPhone/iPad, this is all native code, with a simple local data structure. In a parallel project, Microstrategy is researching HTML 5.
• Microstrategy would rather call all this “microapps” than “dashboards.”

We also discussed Microstrategy’s approach to alerting. Highlights included:

• Microstrategy 9 introduced an alerting capability that Microstrategy sees as differentiated enough to emphasize in lots of sales cycles.
• Microstrategy’s alerting capability lets you set “thresholds”.
• A typical Microstrategy threshold would be a percentage change in a variable vs. another time period. You get to specify the variable (duh), the percentage, and the time comparison.
• When a threshold is crossed, Microstrategy sends you an alerting email. (There’s something native to Apple that’s an alternative for Apple platforms.)

We discussed one other subject as well, kicked off by my question “So why does Microstrategy spawn all those temporary tables anyway?” Mark and I more or less agreed:

• Microstrategy tries to do bigger queries than some of its competitors like to handle, by relying more on the DBMS for query execution.
• Not coincidentally, Microstrategy is often the favorite BI vendor of analytic DBMS vendors (and even some Hadoop folks) who specialize in very large data sets.

• Unemployed, underemployed, or otherwise desirous of having more commercial skills.
• Not a programmer, but desirous of having some technical skills.
• Astute enough to realize s/he will never be a serious techie.

I generally have two models in mind when guiding such a person:

• Analytics/business intelligence/stats.
• Website building.

Those are both useful skill sets for people who aren’t full-time techies, the first perhaps best for those who are more quantitative and big-company-friendly, the second perhaps better for the creative and/or rebellious types.

So what SPECIFICALLY should one guide them to do? My initial thoughts include:

• Learning Java is overkill for most of these people.
• Learning C++ is overkill for ALL of these people. If you’re not out to be a hardcore engineer, the “advantages” of C++ over Java are pointless.
• They all should learn some SQL.
• MySQL is the most accessible DBMS against which to learn SQL. They should download a (free) copy and install it on their PC.
• But I have no idea which books or websites they should go to to learn about SQL.
• While at first blush it sounds like overkill, downloading and installing the free version of Microstrategy 9 is a good way to learn about BI and also the analytic side of SQL.
• The first thing you learn in an app dev tool used to be and probably still is how to do a master-detail form. That would cover the other side of learning SQL. But what would be a good choice of tool? (Preferably free, as building serious OLTP apps is probably not what these people will want to do.)
• One idea I had is that the website-oriented ones should learn how to modify WordPress, by which I really mean modifying WordPress themes. That would involve learning PHP, SQL, and HTML/CSS, which seems like a great place to start.
• But I have no idea which books or websites they should go to to learn about PHP.
• I also have no idea which books or websites they should go to to learn about CSS — or for that matter even basic HTML.
• If they want to take the analytics route, I assume R is the way to go. Thoughts?
• Python isn’t the ideal language for much of anything, but it’s an easily accessible “first language”. Umm, is that a good way to go, or would PHP be a better choice?
• Any other ideas?

For anybody who pitches in — thanks!! I hope to get enough useful answers so as to keep editing this post with people’s ideas.

Edit: Suggestions have started to come in on Twitter. A couple of folks are saying that HTML is a good place to start. Hard to argue with that, although it’s hardly where one should finish. There also was a vote for Yahoo YQL, and of course for a vendor’s own product.

Some great points are in the comments below, including the idea that you should pick an actual, fun, small project to build to get you started. (A site built in WordPress or Mambo would be a pretty obvious choice for such a project, come to think of it.)

• “Push” alerting technology could be much more granular and useful, but is being held back by the problem of false positives.
• Metrics passed down from on high didn’t work too well in Stalin’s USSR, and haven’t improved sufficiently since.
• A large, necessary piece of the solution to both problems is a great engine for setting and modifying metrics definitions.

I shall explain. 

False positives on alarm systems are a huge problem. Alarms were ignored (indeed, turned off) on the Deepwater Horizon due to too many being false. When I had a fire in my house, the smoke alarm wasn’t on because of its penchant for earsplitting false positives. And false positives held back the software category of intrusion detection systems until it was eventually subsumed by related kinds of security appliance.

Similarly, false positives are an under-appreciated problem that restrains the advance of business intelligence software. It’s no big inconvenience when, out of your 3 biggest problems this week, your dashboard calls attention to 17 of them. But suppose you want to make alerts more granular, take them up in quantity a couple of orders of magnitude, and have your smartphone pinged each time the alert condition is met. Whoa. Now those bogus warnings start to be more of a consideration.

Why would you want some kind of “push” alerting? Use case circumstances include but are not limited to anything that might herald:

• A machine malfunctioning or reaching capacity limits (many kinds of machine).
• A critical shortage (inventory or supply).
• A sudden market trend change (if you’re in the kind of consumer market where significant developments an occur in hours or less)
• An investment opportunity or threat, or a financial risk threshold being breached (if you’re an investor, if you buy or sell commodities, or if you’re a corporate treasurer).
• An attitude change on the part of a specific important customer (sales contact, service contact, news development, whatever).
• The advisability of rerouting or rescheduling something.

Why would these alerts ideally be granular and high-volume? Well, you might have lots of machines, lots of trucks, lots of customers, lots of potential investments, and so on. I could list many more examples, but these probably suffice to at least start the discussion.

I’m agnostic about what the UI would be. Sounds going off on your mobile phone? Email on your mobile device? An app more like a chat window? Something more like a mobile dashboard? Whatever it is, false positives would really screw up your work day.

Classical enterprise dashboards suffer from a similarly fundamental problem — dashboard technology is optimized for a screwed-up enterprise analytic methodology. That is:

• Somebody tells you what you’re supposed to care about.
• Somebody tells you how to measure what you’re supposed to care about.
• The simpler and more top-down all this is, the better it creates enterprise-wide unity of vision/focus/purpose/blahblahblah.

Yes, you can do drilldown or even data exploration, QlikView-style or otherwise. You can communicate around the BI tools, whether via general portal technology or something built-in. Vendors keep trying to make it easier for you to build your own reports, charts, and dashboard elements. But you can’t very easily define and track your own metrics, and I don’t see a lot of effort toward making that better.

Perhaps vendors don’t try to provide strong metrics management because doing so might lead to – gasp! — more than “one version of the truth.” If so, such thinking is regrettably misguided. Examples of formulas – i.e., metrics — that should and can not be cast in stone* include:

• Profitability of a customer
• Expected lifetime value of a customer
• Anything else which boils down to “importance of a customer”
• Profitability, value, or importance of a lead or prospect action
• Significance of a social media flare-up
• Cost of a schedule slip
• Risk of a portfolio

*At least as a general rule – we surely can think of some simple cases or exceptions where extreme cookie-cutterness is just fine.

It’s not that there shouldn’t be preferred or default metrics for these quantities. Rather, my point is that individuals should be allowed to:

• Question those metrics
• Consider and test alternatives to those metrics
• Make decisions based on those alternatives (as long as they also keep the results and implications of the approved/default versions of the metrics in mind)

Why? Because the person closest to the situation may be the one best able to assess it, and also because recognizing that fact is a really good idea from the standpoints of employee morale, engagement, and even development. (Or you might say – shiny buzzword! – employee “empowerment”.) In particular, when a boss and her subordinate disagree about the ideal way to calculate a metric, they should be able to track both versions side-by-side. When there’s a significant difference between the two, that could be a fine time for an out-of-the-norm decision.* If that ability were available, portal-style BI collaboration would almost automatically come into its own.

*What kind of decision that is of course depends on the specifics of the case. If only one of two competing metrics says you should bother giving extra care to a specific customer, the answer is easy – you give it. In other cases, an actual meeting or other conversation may be in order to decide what to do.

A metric is just a function. To offer BI users the flexibility I want, it should be very easy to use these functions as inputs into other, custom functions. I.e., I’m talking about something with the flexibility of a stripped down Excel, although the UI would be more like that for a rules engine. And if you think about it, that’s exactly the same functionality needed to personalize your alerts and weed out false positives.

One last point – what I’m calling for could greatly increase the BI performance burden, perhaps even by three orders of magnitude. Well, that’s what we have all this great new super price-performance analytic database technology for. And anyhow the throughput hit won’t come overnight. It will be interesting to see where the boundaries end up between DBMS and inside-the-BI-tool analytic processing. But overall, the analytic DBMS industry – and the hardware vendors backing them up – should be able to handle anything the BI vendors throw at them.

Related links

• Gooddata may be a little more oriented toward what I’m talking about here than some bigger vendors are.

• Use cases for mobile business intelligence

• Neil Raden points out that business intelligence dashboards can be dangerously misleading. His reasoning (sound) is that whatever you measure is apt to be distorted by the fact people know they’re being measured. His solution (implied) is to hire a good-looking consultant like himself to do it right.
• I’ve had my issues with Fred Holahan, who was VP of Marketing when I posted that EnterpriseDB was not to be trusted. (That said, Fred is long gone from EnterpriseDB and my opinion hasn’t changed.) But he’s put up a good series of posts on the basis of the open source “progressive engagement” marketing funnel, including this gem on why you shouldn’t count on monetizing your community/free users.
• Oracle plans to increase its acquisition budget. The figure given is $70 billion over the next 5 years. Edit: But see this funny Register followup.
• Clayton Christensen wrote a phenomenal article on how to live a good life, from a very business-y perspective. (Only in one anecdote was it too religiously-oriented for my tastes.) Takeaways include:
  • Your core goals probably revolve around something other than business success. (E.g., family.) Don’t lose sight of that.
  • To the extent you’re a manager or leader, you may have a huge impact on other people’s lives. Use that power in admirable ways.
  • Teach people how to fish for answers, rather than just giving them answers. They’ll probably come to better conclusions than you would have anyway. (This is a core principle in my own consulting.)
  • Take time to reflect. And by the way, the same techniques you use for strategic analysis in business can be applied to your life as well.
• Mark Suster has a pretty good post expanding on my first Christensen takeaway, highlighting a point too often missing from articles in that genre: It’s not just family; it’s also all the cool things around us.
• I haven’t gone through the Hadoop Summit archives yet, but it looks as if there’s a lot of insight there about current Hadoop application activity.
• If you’re a cat lover and don’t hate simple/traditional music, check out Marc Gunn’s cat filksongs, especially the infectious “What Shall We Do With a Catnipped Kitty?” and “Lord of the Pounce”, both playable from the right sidebar of that page (#7 and #10 respectively). Gunn is also a chief perpetrator of the justly (in)famous Do Virgins Taste Better? cycle of filksongs.
• Former SAP exec Dennis Moore offers a theory as to why SAP cares so much about in-memory DBMS. It’s to integrate business processes, because SAP has no other software layer good at doing same. Interestingly, Dennis originated SAP’s previous attempt at meeting a similar need via its composite applications initiative. However, in Dennis’ view this benefit would only be achieved by a major rewrite of SAP’s applications.

In essence, mobile BI seems to be about small, portable dashboards. Now, I’ve been critical of dashboard technology for years, because of myriad ways in which it fails to live up to the potential of decision support. Some (not all) of those criticisms are being addressed by more recent dashboard technology developments. But with one exception, those criticisms are of little direct relevance to the mobile case.

What’s going on in mobile BI is not so much general decision support as it is quick information retrieval and navigation. Use cases include:

• Getting information about your prospect right before a sales call
• Checking on your customer’s order status during a sales call
• Checking up on a logistics or maintenance issue, in areas such as:
  • Airline (re)scheduling
  • Truck/warehouse dispatching
  • Medical procedure availability
  • Heavy equipment maintenance and repair
• Comparing against benchmarks the performance of something at your current location before you drive away, where “something” could be:
  • Customer purchases
  • Machine uptime
  • Floor cleanliness

Get the UI right on a handheld device is of course a big challenge, due to the small form factor. But any analysis being done in these cases is very simple,* and the same can be said for query execution. So a number of the usual issues in BI and analytic data management fall by the wayside. Other than the small — ahem — matter of squeezing things onto iPhones and the like, what’s going on in mobile BI is pretty straightforward stuff.

*I’m sure sophisticated and flexible handheld analysis is a nice-to-have. But I expect use cases in which it is a must-have will remain scarce for at least the next several years.

But there’s also something that’s evidently not going on, which highlights one of the most perplexing areas of non-progress in all of software — alerting. Alerting technology is clunky and inflexible. Hence it is plagued with intrusive false positives. Hence people don’t want to use it much. And so they miss out on some of the greatest potential advantages of analytic and mobile technologies alike.

I last wrote up my vision for alerting technology 11 years ago. It’s probably time for me to do so again. But you know what? When I do, very little will have moved from the “Should do” to the “Already done” ledger in the intervening decade.

Edit: Here’s some of where I think alerting technology needs to go.

• Are concise.
• Are priced in terms units of work, yet do not require me to meter services at precise hourly or daily rates.
• Have a minimum scope that allows me to feel comfortable I’m spending enough time with a client to do good work.
• Extend over time, mimicking the subscription model of analyst services.*
• Do not contain any concept of “work for hire,” transfer of intellectual property, or “we own your brain.”
• Don’t have any other features that are stunningly inappropriate for our business.

By way of contrast, the user services portion of our site is only a few lines long, and that’s beginning to hurt. When users do ask for consulting services, I have to define each project from scratch. I just suffered through a painful process in which a user insisted on writing their own contract. Even worse, I suspect that I’m missing out on a lot of potential user relationships I could have if I were a little more encouraging of them.

So I want to fix things. Most of the principles above carry over just fine to the user case, but there’s one big difference. While the “extended over time” aspect is great for vendors because:

• There’s always another release to plan and position
• They’re used to paying analysts out of subscription-oriented budget lines anyway

users — at least the ones I see — seem more focused on particular projects, generally along the lines of a once-every-several-years rearchitecting.  More specifically, when users approach me for advice, they generally are:

• Looking to significantly upgrade their analytic capabilities.
• Looking to adopt a new brand of analytic DBMS.
• Looking for architectural advice too.
• Interested in advice on BI and/or predictive analytics strategies as well.
• Correctly anticipating that, once the project is underway, we’ll probably uncover issues and opportunities (not always in the analytics area) they didn’t know they had.

The model I’ve come up with to address all this is to define everything in terms of …

… Service Units, of which there are three kinds:

• Session Units. A Session Unit comprises a meeting, telephone or in person, of 1 ½ – 2 hours, plus appropriate preparation (commonly preliminary phone calls and reading to acquaint us with specifics of your situation and needs).
• Document Review Units. When you are preparing a planning document of some kind (for example, technical architecture or vendor short list), a Document Review Unit consists of our feedback and advice through several iterations, starting from your initial draft.
• Retainer Units. A Retainer Unit allows unmetered quick inquiries for a period of time, typically 1 ½ – 3 months.

Our minimum project size is two Service Units – a Session Unit to start off and a Retainer Unit to help address follow-on and ancillary issues. This lets us advise you with the care and insight you expect. Beyond that minimum, project structure is flexible, and can be tailored to your specific needs, working style, and budget.

While on first reading that’s somewhat vague, it’s as concrete as I know how to get while maintaining the flexibility to:

• Reflect clients’ preferred ways of working.
• Adapt to surprises as the project unfolds.
• Volunteer advice and tangential thoughts without worrying about racking up fees for things the client didn’t actually ask for.

The price per Service Unit is $5,000, which means minimum project size is $10,000, with an approximate target of $25,000 per phase.

This kind of pricing — which has been validated in several recent user contacts and engagements — reflects my position at an extreme end of the freemium spectrum, in that I:

• Spend a large majority of my time doing research and giving it away for free.
• Charge high rates for the time I do actually sell.

One last thing — I like to talk to users, and hence want to have a low-priced, single-conversation offering as well. My current thinking about that part is:

• One hour phone call.
• Limited email follow-on afterwards, along the lines of:
  • “Are you SURE you were right to say Product X doesn’t have Feature Y?”
  • “Please remind me — why were you opposed to Product Z?”
  • “Yes, we’d love an email introduction to a marketing exec at Company W.”

The price for this low-end offering would be $1,995, limit 1/customer/year. I.e., it’s a bargain version of the Session Unit described above.  And there would be a special simplicity rule — absolutely, positively no paperwork except our standard NDA agreement, an invoice, and perhaps a W-9 form. If I can do 10 of those a year — well, it would pay like one medium-sized vendor relationship, and it would give me the opportunity to talk with 10 more users who are interested in talking with me.

And so we finally get to the question that is my reason for making this post:

Do the plans outlined above match the way users want to buy analyst consulting services such as ours?

Opinions and suggestions will be greatly appreciated.