[elephant-devel] Berkeley DB 4.4 Patch in HEAD + improvements

[Ian Eslick]

Recent CVS updates encompass:
- I added Vladimir's BDB 4.3->4.4 patch to the repository
- Fixed the bugs Vladimir reported; current state is bug free on
Allegro/Mac OS/BDB
- Re-organized sleepycat.lisp a bit to accommodate the BDB 4.4
convention changes
- Added a new method to the backend API 'optimize-storage'.
- Made serialization properly re-entrant (circularity-hash was not
re-entrant)
- BUGFIX: add-index with the populate option created incomplete
secondary indicies due to a non-local exit within with-transaction. 
This was introduced by an earlier bugfix to add-index to address out of
memory errors on very large indices.
- Updated the TODO list for 0.6.1

BDB 4.4 upgrade notes:

Existing users of the Berkeley DB backend using v4.3 MUST upgrade to
Berkeley DB v4.4.20 or later.  Upgrade is painless and quite safe. 
Instructions are included in the file "UPGRADE-BDB" in the elephant
root.  I will also send an e-mail update to the list shortly.

New API command (exploit new feature in BDB 4.4):

'optimize-storage' is currently implemented only on BDB backends for now
and takes the two flags described in Berkeley API docs.  I haven't
figured out a clean way to map lisp start/stop keys to database keys so
do not use the stop/start keywords for now.  Using this API function
with the default flags and store-controller will cause all empty pages
to be released back to the file system.

I did not add a test case for this command as yet, but I did run a test
DB to confirm that it reclaims storage.  Create a bunch of large
persistent objects, check out the DB file sizes, delete them using
delete-instances, call optimize-storage and then check out the file
sizes again.  If you've been working with really large DB's, this is a
sweet feature.

Thanks again to Vladimir for taking the time to look into this and
submit the patch!

Cheers,
Ian