[elephant-devel] Retiring

[Ben]

On Mon, 14 Mar 2005, Edi Weitz wrote:

> On Mon, 14 Mar 2005 13:35:07 -0500 (EST), Ben <ben at medianstrip.net> wrote:
>
>> SQLLite (seems to not be that great.)
>
> Would you care to comment on that?  I just looked at it the other day
> and at first sight it seems nice.  What problems to you see?  (I
> understand that it always locks the whole file but for a certain class
> of projects I don't think that's a big problem.)

i was mostly responding to the "locking everything" thing, and perhaps
misconceived feelings that it wasn't ready for prime-time in the same
way postgresql or sleepycat is.  not that elephant is either.....

>> Or, write an All-Lisp backend.  Copy what ZODB does (maybe
>> DirectoryStorage instead of FileStorage.)  Ask me for details.
>
> OK, I'll bite: If you could provide some details about what you think
> is good in ZODB and how it should be ported to Lisp I'd be interested.

woo hoo!

here's an overview of the different ZODB backends:

http://cvs.zope.org/*checkout*/ZODB3/Doc/storages.html?rev=1.8.4.2&content-type=text/plain

the first thing to do is to choose between filestorage and
directorystorage.  one big difference between elephant and FileStorage
is that elephant stores slots in btree pairs while zodb FileStorage
stores objects at offsets.  hence the FileStorage way of doing things
would require more elephant massaging.  directorystorage on the other
hand uses the underlying filesystem's btrees implicitly.  i could be
convinced of either way though.  some thoughts:

filestorage: simple and fast.  requires an in-memory oid+slot->offset
index.  to mesh with elephant, either implement btrees and then store
slots in a btree as we do now, or (more likely) modify the MOP
implementation to use offsets rather than keys.

directorystorage: also pretty simple.  well-documented.  uses the
filesystem as a btree.  no index needed.  should be easier to
implement for elephant.  requires posix filesystem access on the lisp
side (which requires getting your hands dirty, IIRC.)  i'd have to
think about this more, but it seems possible for multiple lisps to
talk to the store without a server (which is definitely not possible
for filestorage.)  implementing btrees might be trivial (i'm not sure
about this though.)

http://dirstorage.sourceforge.net/technical.html

has good documentation on the implementation of directorystorage.

this probably goes without saying, but on the first pass make the
simplest implementation possible (no caching!)

for either of these systems, packing should be combined with GC.

>> Impossible projects:
>>
>> Also serialization of packages.
>
> Maybe look at how Plob! does it.

i will.

take care, B