<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: What needs to be updated anyway?</title>
	<atom:link href="http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/</link>
	<description>Choices in data management and analysis</description>
	<lastBuildDate>Tue, 07 Sep 2010 21:54:37 -0400</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Curt Monash</title>
		<link>http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/#comment-300</link>
		<dc:creator>Curt Monash</dc:creator>
		<pubDate>Wed, 01 Feb 2006 20:43:08 +0000</pubDate>
		<guid isPermaLink="false">http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/#comment-300</guid>
		<description>Eric,

If we leave out what is called &quot;operational BI,&quot; and also leave out planning, it is overwhelmingly the case that analytic data is used to identify ratios, trends, statistical correlations, and the like.  Small errors simply aren&#039;t important to those uses, because they don&#039;t change the outcome in any detectable way.

In most non-planning analytics, operational or otherwise, a lot of historical data is being examined.  Small latency is rarely an issue.

Planning applications are almost by definition non-urgent.  A small amount of latency is not a big problem.

There are a few application areas that we call &quot;analytic&quot; which truly require the same near-instantaneous data integrity that an order processing system would.  For example, fraud prevention comes to mind, in a variety of industries.  (Interestingly, that really is an aspect of order processing.)

But most analytics are done today with a latency of at least a day, and often a week or more.  I rarely find analytic applications that truly need sub-day latency.  And it&#039;s very rare to find ones where, to use the most common figure cited to me for &quot;real-time&quot; analytics, 15 minute latency would be a problem.

Hell, most individuals who dabble in the stock market do so off of price data with 20 minute delays ...</description>
		<content:encoded><![CDATA[<p>Eric,</p>
<p>If we leave out what is called &#8220;operational BI,&#8221; and also leave out planning, it is overwhelmingly the case that analytic data is used to identify ratios, trends, statistical correlations, and the like.  Small errors simply aren&#8217;t important to those uses, because they don&#8217;t change the outcome in any detectable way.</p>
<p>In most non-planning analytics, operational or otherwise, a lot of historical data is being examined.  Small latency is rarely an issue.</p>
<p>Planning applications are almost by definition non-urgent.  A small amount of latency is not a big problem.</p>
<p>There are a few application areas that we call &#8220;analytic&#8221; which truly require the same near-instantaneous data integrity that an order processing system would.  For example, fraud prevention comes to mind, in a variety of industries.  (Interestingly, that really is an aspect of order processing.)</p>
<p>But most analytics are done today with a latency of at least a day, and often a week or more.  I rarely find analytic applications that truly need sub-day latency.  And it&#8217;s very rare to find ones where, to use the most common figure cited to me for &#8220;real-time&#8221; analytics, 15 minute latency would be a problem.</p>
<p>Hell, most individuals who dabble in the stock market do so off of price data with 20 minute delays &#8230;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Eric</title>
		<link>http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/#comment-295</link>
		<dc:creator>Eric</dc:creator>
		<pubDate>Wed, 01 Feb 2006 13:41:47 +0000</pubDate>
		<guid isPermaLink="false">http://www.dbms2.com/2005/12/27/what-needs-to-be-updated-anyway/#comment-295</guid>
		<description>Hi Curt - I found this interesting:&lt;blockquote&gt;Analytic data usually doesn’t need to be updated with full transactional integrity; slight, temporary errors do little harm.&lt;/blockquote&gt;Perhaps true (difficult to say with precision), but the terms &quot;usually,&quot; &quot;slight,&quot; and &quot;little&quot; give me pause. It would seem that constraints on derived analytical data would follow from the needs which drove the data to be ETLed; wouldn&#039;t you want to know whether your ETL process actually produced the structures you thoguht it would produce?</description>
		<content:encoded><![CDATA[<p>Hi Curt &#8211; I found this interesting:<br />
<blockquote>Analytic data usually doesn’t need to be updated with full transactional integrity; slight, temporary errors do little harm.</p></blockquote>
<p>Perhaps true (difficult to say with precision), but the terms &#8220;usually,&#8221; &#8220;slight,&#8221; and &#8220;little&#8221; give me pause. It would seem that constraints on derived analytical data would follow from the needs which drove the data to be ETLed; wouldn&#8217;t you want to know whether your ETL process actually produced the structures you thoguht it would produce?</p>
]]></content:encoded>
	</item>
</channel>
</rss>
