Three things that absolutely should NOT be obstacles to these ends are:

• Corporate DBMS standards.
• Corporate data governance processes.
• The difficulties of ETL.

Reasons they shouldn’t or don’t need to be obstacles include:

• Analytic DBMS are often vastly more cost-effective than general-purpose ones.
• In particular, analytic DBMS are often much easier to install and manage than general-purpose ones.
• Heavy data governance bureaucracy is often unnecessary because:
  • The department should know what the limitations on the data’s accuracy are.
  • The department should know how much data accuracy is required.
  • The side-effects on other departments of any data inaccuracy would be minimal.
• There are multiple good schemes for populating data marts, managed by cost-effective analytic DBMS, with data from integrated data warehouses.
  • ELT (Extract/Load/Transform) almost always works, because data cleaning/data quality was handled at or before the IDW level, and because the analytic DBMS has the processing power to pull it off.
  • ETL (Extract/Transform/Load) should be easy as well. (If isn’t, something may be lacking in your ETL set-up.)
  • Analytic DBMS are increasingly adding capabilities for easy spin-out of real or virtual data marts. Other kinds of technology (e.g. virtualization) are having their database spin-out capabilities upgraded as well.

One point to remember in support of departmental autonomy is that departments’ views of what data to use may be more expansive than central IT’s. One reason is that important data may be external to the company, outside IT’s natural realm  of concern. Examples of this include but are hardly limited to:

• Anything like “market data”.
• Anything like “sentiment analysis”.
• Data owned by supply chain partners.

Further, even the more innovative internal data sources are commonly departmental, for example various kinds of multi-structured data (text verbatims from customers, log file data, and so on).

Whatever is true of data management (and ETL) is true for metadata management, even if it’s done by some kind of business intelligence tool. What I mean by that is:

• Whoever manages data is also responsible for ingesting and emitting it …
• … and specifically for emitting it in understandable, well-organized, well-named formats, …
• … so that departments can take responsibility for what amounts to lightweight analytic application development.

As for the “application development” itself, I’m envisioning at least three things:

• Math.
• Sophisticated relational query.
• Data visualization.

I.e., I’m talking about what “analysts” and “quants” do. So to put the point even more simply:

• Analysts and quants should be able to consume data that’s organized in a friendly manner.
• Central IT should be friendly in how it serves data.

One corollary of this approach is that departments should try to adhere to corporate BI standards, at least for routine dashboard and reporting. Indeed, if a department brings in a business intelligence tool different from the corporate standard, there are three main possibilities:

• The tool is integrated with something else it makes sense to bring in, such as a third-party data supply or application.
• The tool has an important capability the corporate standard doesn’t have, such as more flexible visualization and drilldown.
• Central IT screwed up, making things much more difficult than they needed to be.

• Pundits tend to sketch castles in the air.
• Vendors tend to confuse part of the story — generally the part they happen to offer — with the whole.
• Different use cases give rise to different kinds of issues.

At least three of the generic arguments for agility apply to predictive analytics:

• Doing the correct thing soon is usually better than doing the same correct thing later.
• If it doesn’t take much time to do something, hopefully it doesn’t take that much expense (labor and so on) either.
• It’s hard to get new stuff completely right on the first try. Often, the best strategy is to come close fast, then fix what’s still not ideal.

But the reasons to want agile predictive analytics don’t stop there.

Not only is it hard to get stuff right on the first try for a given information set, but the available information can also change quickly. For example:

• If you’re a consumer marketer, consumer tastes can change quickly, due to news (of many different kinds), seasonal trends, and so on. The most recent data you have contain information unavailable in your historical data sets. Also …
• … if you change your offers, prices, ad placement, ad text, ad appearance, call center scripts, or anything else, you immediately gain new information that isn’t well-reflected in your previous models.
• If you’re in capital markets, and you figure something out, probably so will rival investors. So whatever you knew three weeks ago may already be partially obsolete.

What’s more, often you deliberately don’t want to test, model, or tune all your variables at once. First you determine whether the ad text should be “Would you be so kind as to allow us to supply you with our wares?” or “Buy it, dude!”; only afterwards do you decide whether the color scheme should rely on red or green.

With that as backdrop, how can you make your predictive analytics more agile? Let’s start by breaking predictive analytics into four pieces:

• Data mustering for the analysts.
• Actual analysis.
• Data mustering for deployment.
• Deployment.

Only the second of those has much excuse for being an agility bottleneck; the other three are well addressed by technology you can buy (or straightforwardly build) today.

The deployment part of the story can be pretty simple, at least technically — spit out some PMML (Predictive Modeling Markup Language), and if you’re deploying to a DBMS with good enough PMML support, you’re good to go. Any vendor who doesn’t offer that degree of simplicity had better be working toward it fast. That said, your applications that are infused with predictive analytics need to be modular enough to accommodate model changes; if not, some refactoring lies ahead. And the same can be said for the work processes that surround them.

The data mustering parts should be pretty straightforward too. Setting up a relational data mart tuned for investigative analytics isn’t all that hard or costly (perhaps unless your database is enormously large), and the same actually goes for a Hadoop cluster. Beyond that, if you can model and deploy from the same database, that’s great; if not, you have an ETL (Extract/Transform/Load) need. I guess you could have data quality/MDM (Master Data Management) issues as well, but offhand I’m not seeing why you wouldn’t push their solutions back to analysis time. And any decent analytic technology stack can give sub-hour latency; while that may not suffice from all standpoints, it’s plenty fast enough for analysis-time agility.

With those preliminaries out of the way, now let’s turn to the heart of the agile predictive analytics challenge.

• As one might expect, QlikView 11 contains fairly leading-edge stuff, but also some “better late than never” features.
• The leading-edge stuff is concentrated in the general area of “collaboration”.
• Additionally, QlikTech is always pushing the QlikView user interface ahead in various ways.
• The “Well, it’s about time!” feature list starts with the ability to load QlikView via third-party ETL tools (Informatica now, others coming).
• QlikTech is generally good at putting up pretty pictures of its product. You can find some in the “What’s New in QlikView 11″ document via a general QlikView resource page.*
• Stephen Swoyer wrote a good article summarizing QlikView 11.

*One confusing aspect to that paper:  non-standard uses of the terms “analytic app” and “document”.

As QlikTech tells it, QlikView 11 adds two kinds of collaboration features:

• Integration with social media, which QlikTech calls “asynchronous integration.”
• Direct sharing of the QlikView UI, which QlikTech calls “synchronous integration.”

I’d add a third kind, because QlikView 11 also takes some baby steps toward what I regard as a key aspect of BI collaboration — the ability to define and track your own metrics. It’s way, way short of what I called for in metric flexibility in a post last year, but at least it’s a small start.

That direct sharing of user interfaces is a cool feature, which every business intelligence vendor should offer. In an era of distributed workforces, when people can’t be assumed able to huddle around the same desk, it has value even for use among close coworkers. But it also should prove useful in a variety of more naturally remote use cases, multiple examples of which can be found in each of the areas of:

• Support (internal or external).
• Faceoffs — I mean collaborations — between two or more enterprise departments. Examples might include: manufacturing and purchasing, manufacturing and sales, or accounting and anybody else.

As for social media being used for BI collaboration — that’s generally in the air. For example:

• salesforce.com is pushing enterprise social media use broadly, and will surely increase its emphasis on the social media/BI intersection now that Dave Kellogg is there.
• Spotfire has announced similar features in its latest release.
• The more cumbersome side of the feature set (portal-based collaboration, emailing of individual reports) has been available from multiple vendors for years.
• eBay open-sourced a more dataset-centric version of the idea, just as Oliver Ratzesberger left the firm.*

*Umm — does anybody have a link to the project, or at least a name for it?

BI has been a communication tool since the first green paper report was dumped on the first desk. And there’s been collaboration in doing analysis at least since it’s been possible to email .XLS file attachments. Still, BI is too often used as bludgeon rather than binocular. Hopefully, the current generation of technology will finally serve to change that.

Most of what’s confusing about the MarkLogic Hadoop Connector lies in two pairs of options it presents you:

• Hadoop can talk XQuery to MarkLogic. But alternatively, Hadoop can use a long-established simple(r) Java API for streaming documents into or out of a MarkLogic database.
• Hadoop can make requests to MarkLogic in MarkLogic’s normal mode of operation, namely to address any node in the MarkLogic cluster, which then serves as a “head” node for the duration of that particular request. But alternatively, Hadoop can use a long-standing MarkLogic option to circumvent the whole DBMS cluster and only talk to one specific MarkLogic node.

Otherwise, the whole thing is just what you would think:

• Hadoop can read from and write to MarkLogic, in parallel at both ends.
• If Hadoop is just writing to MarkLogic, there’s a good chance the process is properly called “ETL.”
• If Hadoop is reading a lot from MarkLogic, there’s a good chance the process is properly called “batch analytics.”

MarkLogic said that it wrote this Hadoop connector itself.

When I realized MarkLogic was claiming the ability to seamlessly integrate short-request and batch analytic processing, I asked about workload management. I gathered that:

• MarkLogic believes that MarkLogic 5 does a great job of granular workload monitoring.
• However, MarkLogic doesn’t have a strong workload management administrative interface. Rather, you may have to do workload management programmatically.

Overall, I think the MarkLogic Hadoop connector could prove pretty useful. The first question I ask somebody who wants to process relational data in Hadoop is “Why not just an analytic RDBMS?” But the natural use cases for MarkLogic are often ones in which you might as well do your analytics in Hadoop, including a 4 billion Word/PDF/image document insurance-industry example I recently encountered, and for which I favor MarkLogic over MongoDB or straight Hadoop alike.

Stefan reports that these capabilities are appealing to a significant fraction of enterprise or other commercial Hadoop users, especially the EtL and the BI. I don’t doubt him.

Bit bucket

• Kinds of data likely to be included: Logs, other technical/external
• Likely use styles: Staging/ETL, investigative
• Canonical example: Log files in a Hadoop cluster
• Stresses: TCO, scale-out, transform/big-query performance, ETL functionality

With the explosion of machine-generated data has come the need for a place to put it all, sometimes called the big bit bucket. This is like the investigative data mart for big databases, but more poly-structured. In some cases it is focused on data staging and transformation; but it can also be used for analysis in place.

The list of candidate technologies to run your bit bucket starts with Hadoop and Splunk.

Archival data store

• Kinds of data likely to be included: Operational, CDR (call detail record), security log
• Likely use styles: Archival, reporting (for compliance), possibly also investigative
• Examples: Any long-term detailed historical store
• Stresses: TCO, compression, scale-out, performance (if multi-use)

Analytic DBMS vendors have been insulting each other with the claim “that’s just an archival data store,” dating back at least to the first time Greenplum was deployed on an underpowered Sun Thumper system. Perhaps only Rainstor truly embraces the archival positioning, and I’ve become pretty dubious about their technical claims and their company alike.

Still, there’s a legitimate need for data stores — especially relational analytic DBMS that:

• Store data cheaply, with high rates of compression.
• Have decent performance if you do want to query the data.
• May have archiving/compliance-specific features as well.

Along with Rainstor, SAND and SenSage have at least partially targeted that use case. In addition, appliance vendors such as Teradata and Netezza try to have an archive-oriented product version in their lineups.

Outsourced data mart

• Kinds of data likely to be included: All
• Likely use styles: Traditional BI, investigative analytics, staging/ETL
• Examples: Advertising tracking, SaaS CRM
• Stresses: Performance, TCO, reliability, concurrency

Much of what happens in analytic database management can also be outsourced. Some applications that run via SaaS (Software as a Service) are analytic. I’ve had three different clients whose main business is picking marketing targets in various vertical segments; others who wanted to add analytics to what were historically OLTP applications; and others yet who just offered online business intelligence. Also, if your fundamental business is gathering data and reselling it to a variety of user organizations, that’s an analytic data management challenge. The possibilities expand from there.

Data outsourcers are in the IT business, and so their IT development is — hopefully! — more serious and less politically encumbered than at many conventional enterprises. Thus, legacy systems and master data management issues are commonly less prevalent, or at least more aggressively disposed of. The same, up to a point, goes for vendor politics.*  Multitenancy is commonly an issue, as is running in the cloud.

*Even so, there’s often That Guy who doesn’t want to migrate away from Oracle, no matter what.

Vertica gets the nod in a number of these cases; it’s cloud-friendly, and often the problem is naturally columnar. Other columnar products can be good choices too, with added brownie points for Infobright if the shop is MySQL-oriented anyway. Running Netezza or other appliances makes sense mainly if you’re pretty sure you want to keep operating your own data centers, but some data outsourcers are just fine with that assumption.

Operational analytic(s) server

• Kinds of data likely to be included: Customer-centric, log, financial trade
• Likely use styles: Advanced operational analytics
• Examples:
  • Lower latency: Web or call-center personalization, anti-fraud
  • Higher latency: Customer profiling, Basel 3 risk analysis
• Stresses: Performance, reliability, analytic functionality, perhaps concurrency

Even with eight different choices, I need a “catch-all” category; this is it.

Suppose you want to do reasonably sophisticated analytics, then use the results in operations. This is the classical challenge in integrating short-request and analytic processing. There are multiple ways to tackle it, embodying different trade-offs in cost, convenience, or analytic accuracy. If the platform on which you want to run your investigative analytics also has the reliability and concurrency appropriate for mission-critical operations, you’re set. Otherwise, you may want to pipe derived data into a more “industrial-strength” DBMS, ideally the one that runs your operational apps anyway

Another option is to integrate a limited amount of analytics immediately into your short-request processing system. For example, as bad as they are at the kinds of queries that require joins, NoSQL systems are often fast at simple aggregations. As MapReduce/NoSQL integrations mature, that option may not require pumping the data anywhere else for deeper analytics; even if it does, at least you’re starting out with the data in a convenient bit bucket.

Streaming/CEP-centric architectures could come into play as well. And it goes on from there. The possibilities in this last category are just too varied to generalize about.

So did I get them all? Or are there yet other analytic data management use cases that I don’t fit into my eight categories?

In almost any IT decision, there are a number of environmental constraints that need to be acknowledged. Organizations may have standard vendors, favored vendors, or simply vendors who give them particularly deep discounts. Legacy systems are in place, application and system alike, and may or may not be open to replacement. Enterprises may have on-premise or off-premise preferences; SaaS (Software as a Service) vendors probably have multitenancy concerns. Your organization can determine which aspects of your system you’d ideally like to see be tightly integrated with each other, and which you’d prefer to keep only loosely coupled. You may have biases for or against open-source software. You may be pro- or anti-appliance. Some applications have a substantial need for elastic scaling. And some kinds of issues cut across multiple areas, such as budget, timeframe, security, or trained personnel.

Multitenancy is particularly interesting, because it has numerous implications. If you’re a SaaS vendor supporting multiple customers, you must keep each customer’s data inaccessible to other users* — even if you offer high levels of flexibility or customization. You probably also want to keep data logically partitioned by user, in a way that the DBMS recognizes; you may also want that partition to hunt as a pack for caching purposes, especially if no one customer occupies a large part of your database. Administratively, you need a way to measure customer-specific metrics of the sort that might go into SLAs (Service-Level Agreements).

*Of course, there are exceptions. One of my clients is a SaaS vendor facilitating commerce; the whole point of their app is to let two different customers see and update the same records.

Getting more specific now, I’m usually called upon to advise users in two categories — those that already know they want to upgrade analytic functionality, and those that quickly realize they do once I remind them of it. Even so, many organizations struggle with the question “What do you want to do analytically?” It’s tough to blame them, for the question is distressingly circular; a big part of analytics is figuring out which kinds of analytics are worth doing. Also, SaaS vendors often struggle with the same question for a different reason, responding “Well, we know we’ve only been giving them basic stuff to date. What else do you think they would like?”

There’s no perfect solution to those difficulties, but a good way to start the evaluation is by assessing:

• The nature and value of your decisions that analytics could reasonably affect.
• Your realistic scope for automation of analytic decisions.
• The number and training of your “full-time analysts” — statisticians, SQL jocks who can program, SQL jocks who can’t really program, full-time users of BI tools, whatever.
• The number and training of your “part-time analysts” — normal business users who can get something out of a dashboard, and perhaps even drill down into it.

That should at least tell you which broad categories of analytics you want to engage in, and roughly how advanced in those areas you should try to be.

Basic business intelligence/dashboarding? Surely. Visualization-centric BI? If nothing else, it demos well. Basic predictive modeling? Hmm, are you sure nobody will want that? Advanced predictive modeling? Um, are you sure your users can handle that, or that the results will be worth the investment?

When I talk with users, there’s usually a data management problem in the mix too. In such cases, I quickly ask about data-related metrics, starting with database size, ingest volumes (batch, if relevant, but especially continuous), and simultaneous query load /concurrent user count. Similarly important are requirements for various kinds of latency, the big two being query response time and how long it takes for data to first be available for query. Less numeric questions in a similar vein boil down to “What kinds of requests will you make against the database, in what volume?”

And this loops back to the analytic-user inventory. Suppose you had a near-real-time dashboard — would anybody actually look at it minute to minute?

Specialized metrics I request when considering analytic DBMS include “How many columns are there in your widest table?” and “How many joins — or lines of SQL — are there in your most complex query?”, both of which are tools for assessing “Is your use case naturally columnar?”. Another, more general “natural structure of data” kind of consideration is what structure the data is in before it gets to the database being discussed; candidates include relational batch, XML stream, log file, and many more.

Also crucial are requirements for consistency, availability, and data integrity. Those tell you your needs in high availability and disaster recovery, and perhaps even how picky you have to be about your brands of hardware, software, or cloud/hosting provider. They also indicate how much you should care about relational or ACID properties, and where you should come down on CAP Theorem trade-offs.

I could go on even longer, but those seem like a pretty good set of initial questions with which to start discussions of data management, data integration, and analytic tools and architectures. What do you think I left out? And what do you think I could make substantially clearer by just adding a few more words? Any comments will be much appreciated.

• I’m posting draft slides.
• I’m encouraging comment (especially in the short time window before I have to actually give the talk).
• I’m offering links below to more detail on various subjects covered in the talk.

The talk concept started out as “advanced analytics” (as opposed to fast query, a subject amply covered in the rest of any Netezza event), as a lunch break in what is otherwise a detailed “best practices” session. So I suggested we constrain the subject by focusing on a specific application area — customer acquisition and retention, something of importance to almost any enterprise, and which exploits most areas of analytic technology. Then I actually prepared the slides — and guess what? The mix of subjects will be skewed somewhat more toward generalities than I first intended, specifically in the areas of investigative analytics and derived data. And, as always when I speak, I’ll try to raise consciousness about the issues of liberty and privacy, our options as a society for addressing them, and the crucial role we play as an industry in helping policymakers deal with these technologically-intense subjects.

Slide 3 refers back to a post I made last December, saying there are six useful things you can do with analytic technology:

• Operational BI/Analytically-infused operational apps: You can make an immediate decision.
• Planning and budgeting: You can plan in support of future decisions.
• Investigative analytics (multiple disciplines): You can research, investigate, and analyze in support of future decisions.
• Business intelligence: You can monitor what’s going on, to see when it necessary to decide, plan, or investigate.
• More BI: You can communicate, to help other people and organizations do these same things.
• DBMS, ETL, and other “platform” technologies: You can provide support, in technology or data gathering, for one of the other functions.

Slide 4 observes that investigative analytics:

• Is the most rapidly advancing of the six areas …
• … because it most directly exploits performance & scalability.

Slide 5 gives my simplest overview of investigative analytics technology to date: 

• Fast query
  • Persistent storage (any data volume)
  • RAM (10s -100s of gigabytes, or more)
• Fast analytics
  • Predictive modeling
  • Transformation/tagging
  • Graph

Slide 6 points out that this is all supported by cheap data creation and acquisition, specifically in the area of machine-generated data, which gets the full benefit of Moore’s Law.

Slides 7-13 point out how the example problem domain involves lots of analytic tasks performed on lots of kinds of data. Specific examples cited include text analytics and graph/relationship analytics.

Slide 14 contains the punch line, so I’ll quote it in full:

Derived data

• You can’t keep re-analyzing all that in raw form …
• … so don’t.

If you have one takeaway from this session, let it be the utter importance of derived data.

Slide 16 lists kinds of derived data that are important in the single application of reducing telco churn:

• Normalized data
  • Parsed/sessionized logs
  • Text/sentiment highlights
  • Social network graph(s)
  • Web de-anonymization
  • Household matching
• Scores and buckets
  • Demographic
  • Psychographic
  • Offer hot buttons
  • (Dis)satisfaction
  • Credit/fraud risk
  • Lifetime customer value
  • Influence on others!

And finally, Slide 17 is my first pass at best practices for dealing with derived data:

• Evolving data warehouse schema
• Data marts
  • Physical or virtual
  • Inputs/outputs to “EDW”
• “Data science”
  • Research != production
• Multiple processing pipelines
  • Log parsing
  • Text
  • Predictive analytics
  • Generic ETL
  • Streaming “ETL”

That last list looks like a starting point for a whole set of interesting future posts.

That did it. I tweeted, in succession:

• Actually, I vote for Hadoop as the lungs of the EDW — first place of entry for essential nutrients.
• Data integration can be the heart of the EDW, pumping stuff around. RDBMS/analytic platform can be the brain.
• iPad-based dashboards that may engender envy, but which actually are only used occasionally and briefly … well, you get the picture.*

*Woody Allen said in Sleeper that the brain was his second-favorite organ.

Of course, that body of work was quickly challenged. Responses included: 

• Re: your metaphor, oxygen is used in the combustion process that turns nutrients into energy, so Hadoop is more mouth-ish* (Seth Grimes)
• Data Quality/Governance can be the liver, filtering out toxins (Josep di Paolantonio)
• what’s nice is that BI (DSS) used to be the colon, but now it’s the Krebs Cycle (Neil Raden)

*Linda agrees with me that oxygen is a nutrient, and she’s taught both physiology and English at the college level.

But seriously, folks — I disagree with Jim’s follow-up post a lot less than I do the first one. He’s still overstating the case, of course, and he still seems confused about how some of the pieces of technology work. Even so, I agree that Hadoop is likely to play an important role in many enterprises’ analytic ecosystems, both for its big bit bucket and parallel analytics capabilities.

Meanwhile, in unrelated puns, Chris Kanaracus and I had something of a pizza party:

“Too many [midsized] businesses’ ideas for slicing data don’t get far past the pie-chart level,” said analyst Curt Monash of Monash Research. “‘Big data’ isn’t even the issue for them; they could get much more value than they do now just from personal-sized data sets.”

That’s not the case with Northeast pizza chain Papa Gino’s, which is using IBM analytics software to crunch business data in many ways, said Martha Lieber, director of business systems for the 280-restaurant company, which also includes a number of D’Angelo sandwich shops.

This work has resulted in some delicious insights …

Most Hadoop-related inquiries from Forrester customers come to me. These have moved well beyond the “what exactly is Hadoop?” phase to the stage where the dominant query is “which vendors offer robust Hadoop solutions?”

What I tell Forrester customers is that, yes, Hadoop is real, but that it’s still quite immature.

So far, so good. But I disagree with almost everything Jim wrote after that.

Jim’s thesis seems to be that Hadoop will only be mature when a significant fraction of analytic DBMS vendors have own-branded versions of Hadoop alongside their DBMS, possibly via acquisition. Based on this, he calls for a formal, presumably vendor-driven Hadoop standardization effort, evidently for the whole Hadoop stack. He also says that

Hadoop is the nucleus of the next-generation cloud EDW, but that promise is still 3-5 years from fruition

where by “cloud” I presume Jim means first and foremost “private cloud.”

I don’t think any of that matches Hadoop’s actual strengths and weaknesses, whether now or in the 3-7 year future. My reasoning starts:

• Hadoop is well on its way to being a surviving data-storage-plus-processing system — like an analytic DBMS or DBMS-imitating data integration tool …
• … but Hadoop is best-suited for somewhat different use cases than those technologies are, and the gap won’t close as long as the others remain a moving target.
• I don’t think MapReduce is going to fail altogether; it’s too well-suited for too many use cases.
• Hadoop (as opposed to general MapReduce) has too much momentum to fizzle, perhaps unless it is supplanted by one or more embrace-and-extend MapReduce-plus systems that do a lot more than it does.
• The way for Hadoop to avoid being a MapReduce afterthought is to evolve sufficiently quickly itself; ponderous standardization efforts are quite beside the point.

As for the rest of Jim’s claim — I see three main candidates for the “nucleus of the next-generation enterprise data warehouse,” each with better claims than Hadoop:

• Relational DBMS, much like today. (E.g., Teradata, DB2, Exadata or their successors.) This is the case in which robustness of the central data store matters most.
• Grand cosmic data integration tools. (The descendants of Informatica PowerCenter, et al.) This is the case in which the logic of data relationships can safely be separated from physical storage.
• Nothing. (The architecture could have several strong members, none of which is truly the “nucleus.”) This is the case in which new ways keep being invented to extract high value from data, outrunning what grandly centralized solutions can adapt to. I think this is the most likely case of all.