[elephant-cvs] CVS elephant
ieslick
ieslick at common-lisp.net
Thu Feb 8 23:07:18 UTC 2007
Update of /project/elephant/cvsroot/elephant
In directory clnet:/tmp/cvs-serv2451
Modified Files:
TODO UPGRADE-BDB
Log Message:
Checkpoint checkin of db-lisp
--- /project/elephant/cvsroot/elephant/TODO 2007/02/05 19:33:10 1.45
+++ /project/elephant/cvsroot/elephant/TODO 2007/02/08 23:07:18 1.46
@@ -9,12 +9,12 @@
ALPHA RELEASE TASKS
Bug and feature fixes:
+- Validate migration 0.6.0->0.6.1
+- Fix cur-del2 failure under SBCL (robert to reproduce and fix)
~ Resolve duplicate sorting guarantee in btree interface; currently supported
- by BDB but not SQL and it is not tested in the regression suite (Robert)
+ by BDB but not SQL and it is not tested in the regression suite (Robert to comment)
- Fix *dbconnection-spec* to support multiple controllers for multiple threads
- for CLSQL backend
-- Validate migration 0.6.0->0.6.1
-- Fix cur-del2 failure under SBCL
+ for CLSQL backend (Robert)
BDB Features/Cleanup:
- Trace all paths to db-put or db-delete and ensure that there is a check or a
@@ -57,7 +57,6 @@
- Ensure that variable length UTF-8 is automatically stored as UTF-16
Documentation:
-- Migrate code base to Darcs and create tickets in TRAC
- Add notes about with/ensure-transaction usage (abort & commit behavior on exit)
- Add notes about optimize-storage
- Add notes about deadlock-detect
@@ -128,6 +127,9 @@
0.6.2 - Advanded work, low-hanging fruit (Summer '07)
--------------------------------------------------
+
+Migrate code base to Darcs and create feature/bug tickets in TRAC
+
Storage and Indexing:
- Add :inverse-reader to slot options to create a named method that indexes into objects
based on slot values. Is this a GF or defun? Do we dispatch on class name or bake it in?
@@ -175,7 +177,22 @@
0.7.0: Native Lisp Backend (beta), Fast In-Memory Operations
------------------------------------------------------------
- Full support for DCM or integration of DCM functionality
- - Integrate prevalence-like in-memory database system for single image, multiple-thread operation
+ - Integrate prevalence-like in-memory database system for single image,
+ multiple-thread operation
+ - Richer set of policy decisions on per-class basis
+ - Concurrent mode (for backends that allow multiple processes to connect, current default)
+ - Single-user mode (cache values in instance slots for fast reads, write-through)
+ - Backing store mode (read/write to normal slots except on object creation or synch)
+ (in-memory slot indexing, on disk class)
+ (works for any backend)
+ - Backing-store mode
+ - Controller 'switches'
+ - NoSynch - allow transactions to be lost on failure but maintains consistency
+ instead of performance
+ - Upgrade overall functionality
+ - Solid garbage collection strategy
+ - 64-bit oids / 64-bit file sizes
+ - class templates stored and cached
- (From Ben's e-mail) We are storing persistent objects incorrectly. They should be
stored only as OIDs, and we should have a separate OID->class table. This way
change-class can be handled correctly. This also non-trivially compresses storage
@@ -183,24 +200,16 @@
[Ian comment: only problem with this is an extra access to oid table each time a
class is deserialized and overall storage is constant. Would make it easy to
invalidate objects though!]
- - Richer set of policy decisions on per-class basis
- - Concurrent mode (for backends that allow multiple processes to connect, current default)
- - Single-user mode (cache values in instance slots for fast reads, write-through)
- - Backing store mode (read/write to normal slots except on object creation or synch)
- (in-memory slot indexing, on disk class)
- (works for any backend)
- - Backing-store mode
- - Controller 'switches'
- - NoSynch - allow transactions to be lost on failure but maintains consistency instead of performance
- Usage model examples
-0.7.1 - Elephant/BDB/SQL Production Release
+0.7.1 - Elephant BDB/SQL/Lisp Production Release
--------------------------------------------------
- More work on testing, examples and documentation
- Intent is for this to be a major, long-term supported release prior
to work on the new backend (i.e. patches against this release for
bugs rather than only available in latest development tree)
+
0.8.0 - Supporting Tools Release
--------------------------------------------------
- Add special support (if any) for persistent graph structures & queries
--- /project/elephant/cvsroot/elephant/UPGRADE-BDB 2006/09/04 00:09:11 1.1
+++ /project/elephant/cvsroot/elephant/UPGRADE-BDB 2007/02/08 23:07:18 1.2
@@ -1,46 +1,42 @@
-Version 0.6.0 of Elephant using the BDB backend depends on Berkeley DB 4.3. As for September 3rd, 2006,
-the current CVS is now dependant on Berkeley DB 4.4. If you already have a database based on 0.6.0 or a HEAD prior to September 3rd, you'll need to take the following steps to upgrade your BDB databases to work with my latest checkin.
+Elephant 0.6.1 depends on Berkeley DB 4.5.
-(NOTE: Allegro users may have additional work to do, please check e-mail logs on this topic)
+------------------------------------
+Upgrading from 0.6 / Berkeley DB 4.3
+------------------------------------
-1) Install BDB 4.4.20 or later just as you installed BDB 4.3
+1) Install BDB 4.5 (keep 4.3 around for now)
-2) Pull the latest HEAD from CVS
+2) Update my-config.sexp to point to the appropriate BDB 4.5 directories
-3) Update config.lisp and Makefile in elephant root to point to the appropriate directories
+3) Upgrade your database directory to 4.5
-4) Rebuild elephant C libraries
+3a) Run db43_recover in your 0.6 database
+3b) Optional: run db43_archive -d to remove all logs not part of a checkpoint
+ This will make catastrophic recovery impossible, but reduces the amount of data you
+ have to backup.
+3c) Backup your db files and remaining logs
+3d) Run db45_checkpoint -1 in your 0.6 database directory
-In the root directory:
-> make clean
-> make
-> make bdb
+4) Migrate 0.6 data to a new 0.6.1 database
-5) Upgrade your database directory (only log files need updating)
+4a) Open your old database: (setf sc (open-store '(:BDB "/Users/me/db/ele060/")))
+4b) Run upgrade: (upgrade sc '(:BDB "/Users/me/db/ele061/"))
-From Sleepycat documentation:
+5) Test your new application and report any bugs that arise to elephant-devel at common-lisp.net
- 1. Shut down the old version of the application.
- 2. Run recovery on the database environment using the DB_ENV->open method or the db_recover utility.
- 3. Remove any Berkeley DB environment using the DB_ENV->remove method or an appropriate system utility.
- 4. Archive the database environment for catastrophic recovery. See Archival procedures for more information.
- 5. Recompile and install the new version of the application.
- 6. Force a checkpoint using the DB_ENV->txn_checkpoint method or the db_checkpoint utility.
- 7. Restart the application.
+NOTE 1: close-store may fail when closing the old 0.6 database, this is OK
+NOTE 2: 64-bit lisps will not successfully upgrade 32-bit 0.6 databases. Use a 32-bit
+ version of your lisp to update to 0.6.1 and then open that database in your 64-bit
+ lisp. There should be no compatibility problems. Best to test your application on
+ a 32-bit lisp if you can, just to be sure.
-A known good procedure:
+------------------------------------
+Upgrading from 0.5 / Berkeley DB 4.3
+------------------------------------
- 1. Cleanly exit lisp/elephant application
- 2. Run 'db_recover' in database directory using 4.3 tools
- 3. This will remove the environment
- 3.b (optional) Run 'db_checkpoint -1' and 'db_archive -d' to checkpoint and update db files to latest log. This will
- snapshot the DB and allow you to backup less data, but it makes catastrophic recovery to any time
- before the snapshot impossible as you are deleting history with the '-d' option so exclude that if you
- are conservative or aren't sure what you're doing.
- 4. Copy your database files and all log files to a backup
- 5. Run 'db_checkpoint -1' using 4.4 tools (ignore error message)
- 6. Restart lisp, reload application and ensure that the latest elephant source has been fully recompiled
+Follow the upgrade procedures outlined in Elephant 0.6.0 to migrate your database
+from 0.5 to 0.6. Then follow the above procedures for upgradeing from an 0.6 database.
-6) Connect to your DB, all should be well!
-
-This procedure worked for my Mac OS X upgrade from BDB 4.3 using the latest HEAD on a very large, complex DB.
+NOTE: It may not take much work to make 0.6.1 upgrade directly from 0.5.0. However
+there are so few 0.5.0 users that it wasn't deemed worth the work given that
+there's an upgrade path available.
\ No newline at end of file
More information about the Elephant-cvs
mailing list