[elephant-cvs] CVS update: elephant/NOTES

[blee at common-lisp.net]

Update of /project/elephant/cvsroot/elephant
In directory common-lisp.net:/tmp/cvs-serv13389

Modified Files:
	NOTES 
Log Message:
more changes than i can count

Date: Fri Aug 27 10:31:08 2004
Author: blee

Index: elephant/NOTES
diff -u elephant/NOTES:1.1 elephant/NOTES:1.2
--- elephant/NOTES:1.1	Thu Aug 26 19:59:46 2004
+++ elephant/NOTES	Fri Aug 27 10:31:08 2004
@@ -6,7 +6,12 @@
 this has been optimized for use with CMUCL.  it has been
 tested and somewhat optimized for allegro.  SBCL and OpenMCL
 are definitely also desired targets.  Lispworks is a target
-as well but less so; i can't test it.
+as well but less so: i don't have access to it.
+
+Theoretically one can port this to any lisp with a decent
+FFI and MOP.  However since those are two of the less
+standardized bits of Lisp, in practice this might be
+difficult.
 
 From top to bottom, here are the implementation layers:
 
@@ -24,6 +29,52 @@
 database / serializer, specials are needed.  The store
 controller (see below) is also a special.
 
+-----------------------
+CLASSES AND METACLASSES
+-----------------------
+
+Persistent classes which the user defines are declared and
+instrumented by using the persistent-metaclass.
+
+-----------
+COLLECTIONS
+-----------
+
+While we support serializing and persisting a wide class of
+Lisp data types, there are problems with aggregate types
+(conses, lists, arrays, objects, hash-tables...)
+
+1) not automatic: there's no way for elephant to know when
+you've changed a value in an aggregate object, so you have
+to manually restore it back into the slot to get it saved.
+
+slot-1 of obj A contains a cons.  you change the car of the
+cons.  this is not reflected into the database unless you
+re-set slot-1 of obj A with the cons.
+
+2) merge-conflicts: changing one value and saving an
+aggregate will write out the whole aggregate, possibly
+blowing away changes other threads have made behind your
+back.  this is not protected by transactions!
+
+3) consing, non-lazy and expensive (de)serialization: you
+have to serialize/deserialize the entire aggregate every
+time you save it or restore it.  This is pretty fast all
+things considered, but it's probably better to use
+persistent collections.
+
+4) you have to store the entire collection in memory,
+whereas one of the points of the database to store large
+collections of objects.....
+
+For these and other reasons, we provide a hash-table-like
+interface to Berkeley BTrees.  These have many advantages
+over ordinary hash-tables from the point of view of
+persistence.
+
+TODO: programmatic way to create secondary indicies
+(probably Lisp-level, since FFI callbacks are nasty.)
+
 ----------
 CONTROLLER
 ----------
@@ -57,6 +108,8 @@
 determines liveness.  The root object is a btree collection
 with a special OID 0.
 
+TODO: write the garbage collector.
+
 ------------
 TABLE LAYOUT
 ------------
@@ -157,32 +210,6 @@
 SERIALIZER: AGGREGATE TYPES
 ---------------------------
 
-problems, or Do You REALLY Want to Persist Aggregates?
-
-0) not automatic: there's no way for elephant to know when
-you've changed a value in an aggregate object, so you have
-to manually restore it back into the slot to get it saved.
-
-1) merge-conflicts: changing one value in the aggregate will
-write out the whole aggregate, possibly blowing away changes
-other threads have made behind your back.  this is not
-protected by transactions!
-
-2) object identity / circularity: what are the right
-semantics for things not tagged with OIDs?  certainly can't
-maintain identity across slots.
-
-obj A has slots slot-1 and slot-2.  puts same cons in each
-slot.  no way to keep track that they are supposed to be the
-same cons!
-
-3) consing, non-lazy and expensive (de)serialization
-
-"solution": in the FEW BIG TABLES schema, it is cheap to
-create and use persistent collection types, which will
-preserve identity et al.  as with all persistent objects,
-persistent collections are lazy-loading.
-
 if you really must store non-persistent aggregates, we support:
 
 conses (lists): 1 + car + cdr
@@ -238,6 +265,8 @@
 default accessed through a special variable
 *current-transaction*.  The special *auto-commit* (which
 defaults to nil!) is also a special.
+
+TODO: do transactions memory leak?
 
 UFFI is used as much as possible.  Because some
 implementations can't call function pointers, and because we