[elephant-cvs] CVS elephant
ieslick
ieslick at common-lisp.net
Sun Feb 25 09:12:47 UTC 2007
Update of /project/elephant/cvsroot/elephant
In directory clnet:/tmp/cvs-serv20086
Modified Files:
TODO
Log Message:
Fix SBCL struct serialization; cleanup TODO after Trac conversion; remove persistant aggregate stubs
--- /project/elephant/cvsroot/elephant/TODO 2007/02/25 03:40:18 1.60
+++ /project/elephant/cvsroot/elephant/TODO 2007/02/25 09:12:47 1.61
@@ -1,7 +1,10 @@
-Last updated: February 4, 2007
+Last updated: February 25th, 2007
-Ongoing release plan notes:
+Ongoing release plan notes.
+
+(Note: use of this file is deprecated after 0.6.1, see Trac site at
+ http://trac.common-lisp.net/elephant)
0.6.1 - performance, safety and portability
--------------------------------------------
@@ -75,7 +78,8 @@
Feature tweaking:
x Orthogonal feature addition: map-index, map-class and map-instances to avoid consing (Ian)
-x Tests to validate new map interfaces on top of existing tests
+x Tests to validate new map interfaces on top of existing tests (Ian)
+x Added support and tests for serializing structure objects on all supported platforms (Ian)
DEVELOPMENT CHECKINS:
@@ -156,151 +160,6 @@
x Remove sleepycat name. Change sleepycat to db-bdb to reflect oracle ownership and avoid
confusion for new users
-0.7.0 - Advanded work, low-hanging fruit (Summer '07)
------------------------------------------------------
-
-Migrate code base to Darcs and create feature/bug tickets in TRAC
-
-Storage and Indexing:
-- Add :inverse-reader to slot options to create a named method that indexes into objects
- based on slot values. Is this a GF or defun? Do we dispatch on class name or bake it in?
-- If a class inherits an indexed slot, is it also indexed for that class? This means a
- proliferation of indexes, or requires user to explicitly add an index as a derived slot.
- (Proposal: any subclass must have an :index specifier that is the same as the base class)
- (Imp. option: one index for all instances of primary and subclasses; or index per set?)
- (Perhaps slot indices should not be secondary so they can point to different class types?)
-- What if we want an index to index into a range of different subclasses or objects sharing
- a generic function? (roll your own?)
-- Reclaim table storage on index drop? It's nice to be able to reconnect sometimes!
- Perhaps an API command that allows explicit dropping of tables for a class and a policy
- parameter that determines if this is the default?
-- Delete persistent slot values from the slot store with remove-kv to ensure that
- there's no data left lying around if you define then redefine a class and add
- back a persistent slot name that you thought was deleted and it gets the old
- value by default?
-- Can we do automatic join cursors?
-- Add lazy deserialize to map functions?
-- First-cut query interface
-
-Performance:
-- Improve SQL base-64 serializer performance?
-- Implement unicode performance hacks for various lisps; validate UTF8 works everywhere
-- Metering and understanding locking issues. Large transactions seem
- to use a lot of locks. In general understanding how to use Berkeley DB
- efficiently seems like a good thing. (From Ben)
-- Add dependency information into secondary index callback functions so that
- we can more easily compute which indices need to be updated to avoid the
- global remove/add in order to maintain consistency (Ian)
-- Track derived indices across classes
-
-Design:
- - Move secondary index maintenance to backend; decison on how to call update fn's
- Will make lisp backend cheaper due to ability to life tree manipulation ops
- - Use SWIG and CFFI to better track changes in defconstant? (too expensive to be useful)
- - Evaluate porting elephant to closer-to-MOP to make it easier to
- support additional lisps and to seriously clean up
- metaclasses.lisp and classes.lisp protocols (no love on first attempt)
- - Work to integrate a proper condition system and potential restarts
- for various errors (especially recoverable ones) while accessing db data
- - deserialize fubar / diagnose & return value
- - missing package / add package/symbol translation
- - others?
- (log these in Track; not part of 0.6.2?)
-
-Features:
- - Backup function: allow users to specify a backup function to archive the database contents
- and checkpoint any active functions (how to lock out other threads?) In BDB this means
- running checkpoint and copying the DB files and any active log files.
- - Class option MOP add-on to support declared persistent baseclass slots for standard base classes
- - A wrapper around migration that emulates a stop-and-copy GC
-
-Documentation:
- - Tutorial example rethink: update the blog tutorial using indexed
- objects to create different views as well as integrating something
- like logging for admin or version control purposes.
- - Finish serious update and review of users manual (building on 0.6.0 update)
- - A guide to dealing with transactions
- - A guide to dealing with multiple open stores
- - A guide to performance
- - An overview of licensing issues...
-
-0.8.0: Native Lisp Backend (beta), Fast In-Memory Operations
-------------------------------------------------------------
-
-Major features:
- - Native Lisp Backend
- - All in common-lisp
- - Page-based architecture
- - Cheap copy-on-write transaction policy w/ concurrent transaction commits
- in non-conflicting transactions
- - Fast in-memory operations / prevalence like features
- - Make storage policy decisions on per-class (or per-instance) basis
- - Concurrent mode
- - Current default
- - For backends that allow multiple processes to connect
- - Full ACID functionality
- - Single-threaded objects
- - Cache values in instance slots for fast reads
- - Writes update slots and write to disk as normal
- - Violates consistency and isolation; users must enforce single operator
- - Prevalence mode
- - Standard object model (user enforced ACID properties)
- - Read/write to normal slots
- - Backup slot values on object creation and explicit synch calls on class or instances
- - In-memory slot indexing, write-through disk class indexing
- - NoSynch controller switch
- - Violates durability
- - Offline garbage collection (via migration)
- - Class schemas
- - Improve synchronization, support class and instance versioning
- - POBJs encoded by OID/CID and CID's are cached in working memory
- This way we can issue conditions if an object is out of date so the
- user can determine how/if to upgrade the reference to the current schema
- - From Ben's e-mail:
- We are storing persistent objects incorrectly. They should be stored only as OIDs,
- and we should have a separate OID->class table. This way change-class can be
- handled correctly. This also non-trivially compresses storage in the
- database as we only need to store oids rather than serialized class names.
- [Ian comment: only problem with this is an extra access to oid table each time a
- class is deserialized and overall storage is constant. Would make it easy to
- invalidate objects though!]
- - Persistent variables (abstraction that allows compound lisp objects at the cost of
- full serialization after each write that indirects through the API). Can this be done
- with clean semantics or should we punt it?
- - Support a simple object query language
-
-Details:
- - Revisit duplicate sorting on primary key (artifact of btree index storage order)
- - Usage model examples for new features
- - 64-bit oids / 64-bit file sizes
-
-0.8.1 - Elephant BDB/SQL/Lisp Pre-production Release
---------------------------------------------------
- - More work on testing, examples and documentation
- - Intent is for this to be a major, long-term supported release prior
- to work on the new backend (i.e. patches against this release for
- bugs rather than only available in latest development tree)
- - Online GC for lisp backend?
-
-
-0.9.0 - Supporting Tools Release
---------------------------------------------------
- - Richer query language and query compiler
- - Repository browser - a simple REPL tool like the Slime inspector
- to see what classes are in a repository and what state they're in...useful
- for long-lived repositories or if you've forgotten a variable name
- - Add special support (if any) for persistent graph structures & queries (ala AllegroCache)
- - Support for cheap persistent sets (medium? can do on SQL?)
- - Persistent aggregates for better conceptual integration with lisp?
- - pcons, parray, pstruct, etc
- - push/pop elements from a list defined on a persistent slot?
-
-1.0 - Final Production release (1st long-term version since 0.7.1)
-------------------------------------------------------------------------
- - Significant work on test cases & testing framework
- - Final pass of performance enhancements and review
- - Invite community review and testing
-
========================================================
========================================================
More information about the Elephant-cvs
mailing list