[vivace-graph-devel] transactions on top of triples, scalability, and another storage alternative

[Dan Lentz]

On the subject of billions of triples, one thing that comes to mind is that
true scalability comes from the ability to operate in a federated model, on
a possibly distributed store.  This requires  a transaction model that
operates in a shared, multi-user scenario.  One way to implement such a
thing is *on-top* of triples. Do we have interest in this?  I have in mind
de.setf.resource, (do i sound like a broken record?) which defines such a
methodology an implements it in such a way as to abstract over the
differences between single repo and distributed repo models.

By the way, as far as distributed stores, REDIS comes to mind as a far
better alternative to cassandra. Now, of course, this does introduce a
non-lisp component...

<pause to let all the booing and hissing die down>

...BUT provides near infinite scalability and provides the capability of
both remote and local storage configurations.  Eg, press a button, deploy a
simple REDIS server to EC2 and have near infinitely scalable graph storage
with all the benefits of the hosted EC2.

I've done some work with this and have found REDIS to be very good to work
with via cl-redis.  The downside is some per-query latency, and non-lispy
backend.  The upsides are many and also include bonuses such as pubsub
queues and sorted sets, upon which it is easy to build many other structures
out of triples, which, in turn, makes VG more broadly useful and the graph
model an easier foundation to build on.

you may now resume your normally scheduled booing and hissing :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/vivace-graph-devel/attachments/20110915/c0f06e42/attachment.html>