October 5, 2015

Consumer data management

Don’t plan to fish in your personal data lake.

Perhaps the biggest mess in all of IT is the management of individual consumers’ data. Our electronic data is thoroughly scattered. Most individual portions are poorly managed. There’s no integration. The data that’s on paper is even worse. For example:

For the most part, the technology community is barely trying to solve those problems. But even when it does try, success is mixed at best. For example:

And those are some of the most successful names.

There are numerous reasons for this dismal state of affairs. 

The toughest problem, I think, is in my middle bullet point — people hate organizing their own information. That’s true, by the way, of consumers and individual employees alike. Canonical examples on the enterprise side include knowledge management, taxonomy building,* or getting salespeople to properly fill in sales force automation software forms. On the consumer side, personal computers were pitched in their very early days as a way to store recipes; how did that ever work out? Thus, the standard for usability for people to actually like personal data management technology is very high, and very difficult to meet.

*Well, canonical at least among text search geeks. 🙂

Despite all this negativity, I think there are two areas in which it is inevitable that consumers will wind up with access to well-organized online data stores — health and money. The first reason is simply perceived value. Health and money are both important, and people know it, and so those have always been the two areas in which consumers have willingly paid quite a bit for information and advice.

I happen to have picked up that truism in the 1990s, when I published a subscription newsletter, and the only categories in which consumer newsletters sold well were health and money. But if you don’t believe me, you could note:

My more precise reasons for believing consumer financial data management will eventually be straightened out start:

As for health care:

But that’s most of the good news. Oh, I do think Apple will one of these decades come up with a decent way to manage what’s on your Apple devices. A few other point solutions will be similarly competent. But personal data lakes, or anything like that? I don’t think those are going to happen in any kind of foreseeable time frame.


One Response to “Consumer data management”

  1. clive boulton on October 12th, 2015 3:09 pm

    Curt et al, you might like to try out Camlistore your Personal Data Store for Life. Brad Fizpatrick’s 20 percent time at Google (project has many contributors). More on Wikipedia.

    With a FUSE filesystem, Camlistore includes importers for social data. Tweets, Flickr, Picasa and more. And is designed for new importers to be open source contributed. Health and Money importers could be added providing online services provide a way to get your personal data (most like AMEX / Intuit seem to).

    Scroll down to see video
    (Content-Addressable Multi-Layer Indexed Storage)

    What I really like, you can host your Camlistore on your PC at home or in the cloud.

    Local version for developers (free)

    Hosted version. https://www.scaleway.com/imagehub/camlistore/

Leave a Reply

Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:


Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.