[elephant-devel] Add indexed persistent class slots to elephant?

Tue Jan 24 18:54:04 UTC 2006

I think, in particular with respect to the oid issue, it is worth
reposting 
the note that I have added to the project as "BEN-RETIRING-TODO.txt".

Some of the things he lists have actually been done (whoopy!)

Hi all,

Thanks for your continued interest in Elephant.  Unfortunately I have
some bad news.  My day job has become all-consuming (and it
unfortunately does not involve working with Elephant at all.)  As such
I feel I need to step down as maintainer.  Andrew has volunteered to
maintain this project, but he will also probably have limited cycles
to contribute coding too.

While it seems Elephant is in a semi-useful state, it does need more
work.  So, if you like Elephant and would like to make it better,
there are plenty of things to work on!  I will stay on these email
lists and try to contribute what knowledge I have.  Andrew will try to
coordinate.  I've outlined here various projects in priority order, as
I see them.

At some point in the future I hope to come back!

High-priority projects:

1) we are storing persistent objects incorrectly.  they should be
stored only as OIDs, and we should have a separate OID->class table.
this way change-class can be handled correctly.

2) GC.  Gabor has started on this -- thank you!

3) New release.  Update docs, webpage to reflect new stuff.
(Sleepycat 4.3, SBCL+unicode, new MOP stuff.)

4) Metering and understanding locking issues.  Large transactions seem
to use a lot of locks.  In general understanding how to use Sleepycat
efficiently seems like a good thing.

5) Tests tests tests!

6) LispWorks, Allegro 7.0 and OpenMCL compatibility.  There are some
patches which have been sent in (check the logs.)  Also someone sent
in Solaris patches a while ago.

Difficult, but important projects:

7) callbacks for btree sorting: right now the sorted btree sorting
function is a bit of nasty C code.  i have code somewhere which did
callbacks instead, i can try to revive that.  the issue is that the
sorter needs to know the underlying lisp representation.  this is a
screw for unicode strings and bignums.  the C code is a hack for 16
bit unicode strings for allegro / lispworks (using IBMs ICU stuff),
and for the sbcl unicode stuff (using glibc wchar stuff.)  also bignum
sorting is by approximation via floats, which fails after some point.
but in hindsight maintaining all this is probably a lose.

8) Other backends.  Sleepycat is nice but has license issues.
Calling to C is nice but not nice.  Elephant is fairly modular:
serializer + MOP stuff + backend.  Therefore: write different
backends.  Possibly: Embedded Firebird, SQLLite (seems to not be that
great.)  Or, write an All-Lisp backend.  Copy what ZODB does (maybe
DirectoryStorage instead of FileStorage.)  Ask me for details.  With
SQL backends, there are possibility of using SQL types to solve some
of the btree sorting problems (and break others!)

9) init/reinit object protocol: Dan Knapp suggests (or I infer) an
initialization / reinitialization protocol might be nice, since there
is a difference between objects which are freshly created and those
which are loaded from the DB.

Med-priority projects:

10) equality joins have to be done on the lisp side, since many lisp
btrees share the same sleepycat btree.

11) single-user-mode: SUM = cache all slot values.  used to be high
priority, but i no longer want it, so unless someone wants it...

Low-priority projects:

12) class / slot to ID performance hack.  create a table which
contains slot definitions, probably cache this in memory on the class.
store slots as OID + slot ID -- avoid the symbol lookup.

13) typed arrays.  some lisps can pass things like arrays of 32 bit
signed integers directly to C.  right now we destructure and
restructure.  invalidated by 11).

14) bignum fixes.  OpenMCL: check that ldb is non-consing (i think it
is), look at %ldb-fixnum-from-bignum.  profile %bignum-ref on CMUCL /
SBCL.  or, figure out a great way to serialize bignums!

15) peter b's patch to non-persistent objs.  check the list.

16) dynamic-extent for transactions on SBCL/CMUCL.  i've tried to get
with-transaction to not cons.....

Impossible projects:

17) Serialize lambdas, closures, functions.  Seems to require
implementation support.  Also serialization of packages.

Take care, B

Seems to require
implementation support.  Also serialization of packages.

Take care, B

[read at localhost elephant]$

On Tue, 2006-01-24 at 12:13 -0500, Zach Beane wrote:

> On Tue, Jan 24, 2006 at 11:43:16AM -0500, Ian Eslick wrote:
> 
> > That reminds me.  Without GC and/or renaming oids we'll eventually run 
> > out of OID address space, again when the DB gets enough use.  I haven't 
> > seen any provisions for fixnum wrap-around.  Am I missing something or 
> > is this correct?
> 
> This isn't a direct comment, but I recently came across this book:
> 
>    http://www.amazon.com/gp/product/0136298338/102-6097685-4704118
> 
> It has a concise but interesting chapter on the design and
> implementation of the Statice database. You can also find it on
> Abebooks for $1.
> 
> I don't have it in front of me, but IIRC it described Statice OIDs as
> 96 bit and intended to be unique across space and time. It also said
> that the database was *never* garbage collected.
> 
> Zach
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060124/8bf9af4c/attachment.html>