July 15, 2012

Memory-centric data management when locality matters

Ron Pressler of Parallel Universe/SpaceBase pinged me about a data grid product he was open sourcing, called Galaxy. The idea is that a distributed RAM grid will allocate data, not randomly or via consistent hashing, but rather via a locality-sensitive approach. Notes include:

The whole thing is discussed in considerable detail in a blog post and a especially in a Hacker News comment thread. There’s also an error-riddled TechCrunch article.

In the areas I cover, “error-riddled TechCrunch article” is pretty much a redundant phrase — but that post looked particularly bad.

Meanwhile, I just noticed a May, 2009 blog post out of Progress Apama. The idea was that event streaming technology could be used to track moving objects, something I heard directly from the CEP (Complex Event Processing) vendors in the 20072009 period as well.

My tentative opinions on all this start:

Related link

Comments

2 Responses to “Memory-centric data management when locality matters”

  1. SpaceBase Introduces Memory-Centric Data Management | SiliconANGLE on August 7th, 2012 1:35 pm

    [...] SpaceBase Introduces Memory-Centric Data Management Tavis J. Hampton | August 7th READ MORE Tweet It is difficult enough to find data in a large database, but imagine if you also needed to locate the data and map its location in relation to other pieces of data. A distributed memory system needs to be able to place data on a specific node and also do so without putting an enormous amount of stress on the nodes in a cluster. Rather than randomly allocating data using consistent hashing , this type of distributed RAM grid would need to use locality-sensitive distribution. [...]

  2. SpaceBase Introduces Memory-Centric Data Management | DevOpsANGLE on August 7th, 2012 3:36 pm

    [...] SpaceBase Introduces Memory-Centric Data Management Tavis J. Hampton | August 7th READ MORE Tweet It is difficult enough to find data in a large database, but imagine if you also needed to locate the data and map its location in relation to other pieces of data. A distributed memory system needs to be able to place data on a specific node and also do so without putting an enormous amount of stress on the nodes in a cluster. Rather than randomly allocating data using consistent hashing , this type of distributed RAM grid would need to use locality-sensitive distribution. [...]

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.