[elephant-devel] SQLite locking error / BDB multi-image support

[Ian Eslick]

I just pushed a patch that should automatically manage two processes  
connecting to the same environment.  You typically have to coordinate  
who is the master user and who is the slave so that you don't try to  
recover an environment that is already in use by another process.   
This causes that first process to fail, I believe, because it  
discovers it's pointers into the environment are 'corrupted'.

I don't fully understand all the BDB particulars here, but I'm short  
on time to fix these things.  Let me know if this works for you.  The  
short story is that BDB is unlikely to 'just work' for a real system  
using multiple processes.  You have to run deadlock detection  
(probably in a parallel process), you have to think about how to  
coordinate processes to clean up in the event of an error, you may  
have to sequence your startup, you have to think about how parallel  
use of the DB effects performance.  No free lunches here.

Anyway, for the details of why I added the patch see info on  
DB_REGISTER option to DBENV->open (db-env-open in lisp via open- 
controller in bdb-controller which is called from open-store - I've  
made DB_REGISTER the default).

Berkeley DB considerations when using multiple processes:
http://www.oracle.com/technology/documentation/berkeley-db/db/api_c/env_open.html
http://www.oracle.com/technology/documentation/berkeley-db/db/ref/transapp/app.html

Some discussions of multi-processor use of BDB:
http://forums.oracle.com/forums/thread.jspa?messageID=1751065
http://forums.oracle.com/forums/thread.jspa?messageID=1436656

I'm not going to be able to track down the CL-SQL/SQLite issues  
anytime soon.

Ian


On Feb 25, 2008, at 8:22 AM, Leslie P. Polzer wrote:

>
>> To isolate the problem, can you verify the base case, connect two
>> images to a fresh db?
>
> For BDB:
>
> (asdf:oos 'asdf:load-op 'elephant)
>
> (defpackage #:ele-test (:use :cl :elephant))
>
> (in-package :ele-test)
>
>
> (defpclass myclass ()
>  ((testslot :accessor testslot :initarg testslot :index t)))
>
> (open-store '(:BDB "/tmp/db1") :recover t)
>
> (setf item (make-instance 'myclass))
>
> (setf (testslot item) 5)
>
>
> I load this in two images. When I then try to access TESTSLOT in the
> former, I get the mentioned error.
>
> I'll try to make up something for the SQLite case, too.
>
> FYI, Postmodern doesn't show any problems with my setup.
>
>  Leslie
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel