<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.3.2">
</HEAD>
<BODY>
I have had similar problems. My own solution has been to put mutual exclusion locks into<BR>
DCM. Since my own app only accesses the database through DCM, this works for me;<BR>
I think it would be inconvenient but not impossible if you are using Elephant directly.<BR>
DCM is a "director-based" system that is in the contrib directory, that aims more toward<BR>
a prevalence based system.<BR>
<BR>
There are/were at least three problems that have to be dealt with:<BR>
<BR>
1) The serializer is definitely not thread-safe. In the next release, we need to improve this,<BR>
but for the time being I have added mutexes (these are not checked in yet.)<BR>
<BR>
2) CLSQL is not threadsafe unless you use a separate connection in each thread.<BR>
Elephant's basic use of CLSQL reuses the connections, and so you can have <BR>
big problems even at the CLSQL level in using Elephant in that way.<BR>
<BR>
So, based on what you have described, I can suggest a few solutions, neither of <BR>
which are great:<BR>
<BR>
1) If your access to Elephant is in some relatively narrow place, that is, you can find<BR>
a few pieces of code where all of the elephant requests occur, you can put mutexes around<BR>
those. (Code to define and use these mutexes is included at the end of this message. It <BR>
defines a macro, "defemthodex", which is just like defmethod but takes an extra argument<BR>
used to define the mutex. However, this is HIGHLY SBCL specific, and would not <BR>
be trivial to adapt to anything other than DCM --- this is (in additio to being busy) why <BR>
I have not put the thread-safe version of DCM in the contrib directory yet. It is, however,<BR>
an example of how to use mutexes in SBCL, which you can easily copy.)<BR>
<BR>
2) Depending on the demands of your application, you might be able to put in a smaller<BR>
number of higher-level mutexes (that is, that operate around larger bodies of code), and <BR>
still serve pages sufficiently rapidly.<BR>
<BR>
3) If you use BDB, it will seem better, but the current version of Elephant is still not<BR>
thread-safe, because of the serializer, although it will take you a lot longer to hit a <BR>
problem than under CLSQL.<BR>
<BR>
I think really Ian and I, and whoever else is interested, need to plan on solving this<BR>
problem in the next major release. To my knoledge, this would mean:<BR>
<BR>
1) Writing a small connection-pool manager that is threadsafe in Elephant,<BR>
2) Making a reentrant serializer. (I know Ian is planning this already - AMEN!)<BR>
3) I'd be happy to publish my current version of DCM to anyone who wants it,<BR>
but it is SBCL specific, and I personally don't know how to do mutexes under <BR>
any other LISP.<BR>
<BR>
* * *<BR>
I know that may not be terribly helpful ---- but let's continue this discussion until<BR>
we work out a solution. I am extraordinarily busy launching my business, but am <BR>
committed to at least managing patches provided by others or getting out of the way<BR>
if someone else (such as Ian) wants to be the official maintainer.<BR>
<BR>
<BR>
<BR>
(defvar *dcm-mutexes* (make-hash-table :test 'equal))<BR>
<BR>
(defvar *a-mutex* (sb-thread::make-mutex :name "my lock"))<BR>
<BR>
(defun insure-mutex (name)<BR>
(let ((mtx (gethash name *dcm-mutexes*))<BR>
)<BR>
(or mtx (setf (gethash name *dcm-mutexes*) (sb-thread:make-mutex :name name)))<BR>
)<BR>
)<BR>
<BR>
;; This assumes that the the variable "dir" is being defined and that we can can <BR>
;; create <BR>
(defmacro defmethodex (mname dir args &body body)<BR>
`(defmethod ,mname ,(cons dir args)<BR>
;; (format t "Thread ~A running ~%" sb-thread::*current-thread*) <BR>
(sb-thread:with-mutex ((insure-mutex (format nil "mutex-~A" ,(car dir))))<BR>
;; (format t "Thread ~A got the lock~%" sb-thread::*current-thread*)<BR>
(let ((ret<BR>
,@body))<BR>
;; (format t "Thread ~A dropping lock~%" sb-thread::*current-thread*)<BR>
ret<BR>
)<BR>
)<BR>
)<BR>
)<BR>
<BR>
<BR>
<BR>
On Tue, 2006-10-31 at 14:43 +0100, Pierre THIERRY wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
<FONT COLOR="#000000">I may have a problem with concurrency and elephant. I'm developing a web</FONT>
<FONT COLOR="#000000">application using SBCL and Araneida (with it's threaded listener).</FONT>
<FONT COLOR="#000000">Whereas araneida by irself doesn't seem to have problems dealing with</FONT>
<FONT COLOR="#000000">many requests at a time (I tested it with apache's benckmark tool, with</FONT>
<FONT COLOR="#000000">thousands of request, with from 1 to 16 concurrent requests to see how</FONT>
<FONT COLOR="#000000">it reacts), my application just collapses when 3 requests come in a very</FONT>
<FONT COLOR="#000000">short time.</FONT>
<FONT COLOR="#000000">I had an error I did not copied, which was about a persistent slot of an</FONT>
<FONT COLOR="#000000">object being unbound (but I know it should be bound when accessed at</FONT>
<FONT COLOR="#000000">this time), and when trying to access one more time my app, I end up</FONT>
<FONT COLOR="#000000">with this in the debugger:</FONT>
<FONT COLOR="#000000">While accessing database #<POSTGRESQL-DATABASE localhost/efco-immo/pierre OPEN {A80FB99}></FONT>
<FONT COLOR="#000000"> with expression "SELECT value FROM keyvalue WHERE ((clctn_id = 0) and (key = 'FSAAAAAyAAAANAAAACAAAABUAAAASQAAAFQAAABSAAAARQAAAA=='))":</FONT>
<FONT COLOR="#000000"> Error NIL / </FONT>
<FONT COLOR="#000000"> has occurred.</FONT>
<FONT COLOR="#000000"> [Condition of type CLSQL-SYS:SQL-DATABASE-DATA-ERROR]</FONT>
<FONT COLOR="#000000">The database itself doesn't seem corrupted, as restarting the</FONT>
<FONT COLOR="#000000">application from scratch, with the same store, works well. For the</FONT>
<FONT COLOR="#000000">moment, I had to shield my app behind an HTTP cache so that requests are</FONT>
<FONT COLOR="#000000">basicaly served to the user by the cache instead of my app...</FONT>
<FONT COLOR="#000000">Concurrently,</FONT>
<FONT COLOR="#000000">Nowhere man</FONT>
<FONT COLOR="#000000">_______________________________________________</FONT>
<FONT COLOR="#000000">elephant-devel site list</FONT>
<FONT COLOR="#000000"><A HREF="mailto:elephant-devel@common-lisp.net">elephant-devel@common-lisp.net</A></FONT>
<FONT COLOR="#000000"><A HREF="http://common-lisp.net/mailman/listinfo/elephant-devel">http://common-lisp.net/mailman/listinfo/elephant-devel</A></FONT>
</PRE>
</BLOCKQUOTE>
</BODY>
</HTML>