June 19, 2010

Objectivity Infinite Graph

I chatted Wednesday night with Darren Wood, the Australia-based lead developer of Objectivity’s Infinite Graph database product. Background includes:

Infinite Graph is an API or language binding on top of Objectivity that:

The main point of the Infinite Graph beta test is to see whether Objectivity got the API right. By way of contrast, Objectivity is still just researching the DBMS optimization side of things. According to Darren, what makes that so hard is that if you partition the graph in some smart way, probably through some kind of costly algorithm to determine “least connectedness,” a bit more additional data can thoroughly invalidate your results. Thus, Darren is focused more on ensuring that performance is good even if data is distributed around the network in annoying ways.

One performance win that Infinite Graph seems to get (almost?) for free from being built on top of Objectivity is lots of prefetching. Specifically, graph nodes and their edges are stored together, just like objects and their pointers are in traditional Objectivity — and if a node is retrieved, the nodes it’s connected to might also get retrieved as a background operation, before they’re even needed. More generally, Objectivity has always tried to be fast about traversing pointers, and that is a whole lot like traversing graph edges.

As a future, Infinite Graph is looking at ideas from Google’s Pregel. As Darren characterizes it, in Pregel you wrap up information about a graph node and ship it off to another computing node if the next graph node you need is over there. Darren suspects that the extreme form of this strategy would not be ideal. (I gather from Darren that Google has realized the same thing from the getgo.) Instead, he’s pinning his hopes more on smarts about when to do that (costly) shipping, and when to just fetch the information back to the compute node currently being used.

The most interesting part of our discussion, in my opinion, was about applications and application functionality. In a nutshell, Darren seems to think that it’s all about the edges, rather than the nodes themselves. (My words, not his.) In particular:

Comments

9 Responses to “Objectivity Infinite Graph”

  1. Dan Weinreb on June 20th, 2010 9:54 am

    Objectivity’s basic system is an “object-oriented database” in the sense that was meant by that phrase in 1980. I was involved in a company that competed with them but did a database system that was very similar to theirs. So most of what I say in the following blog post applies to Objectivity as well:

    http://danweinreb.org/blog/object-oriented-database-management-systems-succeeded

    I think this kind of DBMS is great, and I truly think they’re on their way back. Not quite in the same form as in the 1980’s, of course: computer technology marches on. But the basic ideas are still great. I wish Objectivity the best of luck.

  2. Curt Monash on June 21st, 2010 6:21 am

    @Dan,

    I was hoping you’d comment on this post! :)

    Thanks,

    CAM

  3. Curt Monash interviews Darren Wood on InfiniteGraph « InfiniteGraph on June 21st, 2010 11:10 am

    [...] Curt Monash interviews Darren Wood on InfiniteGraph Curt Monash has written a great, in-depth review of InfiniteGraph. He spoke with InfiniteGraph’s lead architect, Darren Wood, last week and his write-up is available here. [...]

  4. Jonah H. Harris on June 21st, 2010 1:43 pm

    Hey Curt,

    I’ve done some investigation with Infinite Graph as well, and I believe that being based on their well-tested, distributed OODBMS gives them quite an edge over the current graph-based database systems.

    First, given that several OODBMSes are already quite optimized at performing fast lookups and traversals on large, complex object relationships, tailoring a distributed OODBMS engine for graph-specific functionality is a great base on which to start. Second, while several of the graph-oriented systems out there are focused on trying to solve the, “optimal cuts on which to partition/distribute the graph” problem, Infinite Graph is just moving ahead with a standard object-oriented approach of keeping edges/relationships right with the vertices/nodes/objects themselves. Sure, they still have to optimize object placement, but at least they’ll be off-and-running with a distributed platform and good locality for edges (at the cost of additional space requirements).

    Having done quite a bit of internals research on all of the current open-source graph systems, I believe Infinite Graph will already be more scalable. Neo4j and most others were not designed to be distributed, and some don’t even make good use of locality on a single node. The only concern I have is from a performance perspective–Infinite Graph is, as are most of its competitors, Java-based.

    Should be interesting to see how the graph database market plays out.

  5. My talk this morning | DBMS2 -- DataBase Management System Services on June 23rd, 2010 7:33 am

    [...] Graph [...]

  6. Some thoughts on the announcement that IBM is buying Netezza | DBMS 2 : DataBase Management System Services on September 27th, 2010 5:13 am

    [...] that’s before we even mention XML documents, graph, [...]

  7. Whither MarkLogic? | DBMS 2 : DataBase Management System Services on April 5th, 2011 9:16 pm

    [...] and the technology isn’t obviously needed in single-server cases. (But see my post on Objectivity’s Infinite Graph.) Even so, graph-oriented apps are exploding, and MarkLogic should think about playing in the graph [...]

  8. InfiniteGraph Gets Support for Common Graph Database Language and More | SiliconANGLE on February 21st, 2012 7:01 am

    [...] Objectivity was founded in 1988 and launched Objectivity/DB, an object database (a precursor to graph databases). It found early success as a non-relational database solving what we’d now call “big data” problems for a number of clients, particularly in science and government. It launched InfiniteGraph in 2010. You can find out more about its architecture in Curt Monash’s write-up here. [...]

  9. http://daisyjens.momaroo.com/ on May 6th, 2013 12:28 pm

    Ridiculous quest there. What happened after?

    Take care!

Leave a Reply




Feed: DBMS (database management system), DW (data warehousing), BI (business intelligence), and analytics technology Subscribe to the Monash Research feed via RSS or email:

Login

Search our blogs and white papers

Monash Research blogs

User consulting

Building a short list? Refining your strategic plan? We can help.

Vendor advisory

We tell vendors what's happening -- and, more important, what they should do about it.

Monash Research highlights

Learn about white papers, webcasts, and blog highlights, by RSS or email.