February 6, 2012

WibiData, derived data, and analytic schema flexibility

My clients at Odiago, vendors of WibiData, have changed their company name simply to WibiData. Even better, they blogged with more detail as to how WibiData works, in what is essentially a follow-on to my original WibiData post last October. Among other virtues, WibiData turns out to be a poster child for my views on derived data and the corresponding schema evolution.

Interesting quotes include:

WibiData is designed to store … transactional data side-by-side with profile and other derived data attributes.

… the ability to add new ad-hoc columns to a table enables more flexible analysis: output data that is the result of one analytic pipeline is stored adjacent to its input data, meaning that you can easily use this as input to second- or third-order derived data as well.

schemas can vary over time; you can easily add a field to a record, or delete a field. … But even though you start collecting that new data, your existing analysis pipelines can treat records like they always did; programs that don’t yet know about the new cookie are still compatible with both the old records already collected, and the new records with the additional field. New programs fill in default values for old data recorded before a field was added, applying the new schema at read time.

schemas for every column are stored in a data dictionary that matches column names with their schemas, as well as human-readable descriptions of the data.

Interesting aspects of the post that don’t lend themselves as well to being excerpted include:

Comments

3 Responses to “WibiData, derived data, and analytic schema flexibility”

  1. Introduction to Continuuity | DBMS 2 : DataBase Management System Services on November 1st, 2012 7:14 am

    […] intelligence apps — I presume in human real-time —  much as might be the case for WibiData (to which Continuuity views itself as potentially complementary rather than competitive). This […]

  2. Should you offer “complete” analytic applications? | DBMS 2 : DataBase Management System Services on February 22nd, 2013 1:51 am

    […] WibiData is essentially on the trajectory: […]

  3. Some notes on new-era data management, March 31, 2013 | DBMS 2 : DataBase Management System Services on April 1st, 2013 4:45 am

    […] smart folks at WibiData felt the need for schema-definition tools over […]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.