[elephant-devel] run-elephant-thread

[Gábor Melis]

On Saturday 20 January 2007 22:54, Ian Eslick wrote:
> There should be an e-mail about this in the archive somewhere, but to
> summarize...
>
> I think that requiring client code to worry about wrapping every
> thread in run-elephant-thread is unrealistic, so that interface is
> now deprecated.  The reason for this is simple, when you are using a
> multithreaded web server the main thread launches client threads
> inside the server code and there is no way to wrap it in an elephant
> thread without modifying the web server and this is unrealistic.

For my web application it is entirely realistic: the webserver need not 
be modified, elephant using code was wrapped in run-elephant-thread - 
that only bound a few specials - within the _client code_ and all was 
well.

> A Thread-Safe Serializer:
>
> The answer to this problem is a little more important than the
> others.  The serializer is called all the time and is a performance
> critical part of the system.  The serializer is thread safe except
> for it's use of a hash table used to detect circular data
> structures.  I've used elephant in a multi-threaded setting for quite
> some time without worrying about the serializer because in practice I
> would never hit the case where an object I was serializing would
> accidentally lookup a duplicate object/id in the circularity hash.
> The way to avoid this case in general is to have a queue of hash
> tables that each thread can grab when it wants to serialize an object
> (you don't want to allocate a hash table in an inner loop - reuse by
> clearing is also costly, but about 50% as much).  So only this queue
> needs to be protected.
>
> I tried using standard locks in the various EXCL-like extended
> packages but the performance is atrocious for frequently called
> routines like serialize.  Instead, I use without-interrupts (a common
> lisp primitive) to block interrupts for the duration of a vector-pop
> command that grabs a new hash.  This gives much better performance.
> The only other variables that need to be so protected are:

Without-interrupts on allegro prohibits multitasking. It is between hard 
and impossible to implement on a true multithreaded lisp that runs on 
multiple cpus (and it kills performance anyway). You won't find an 
equivalent for without-interrupts in sbcl. Actually there is a 
without-interrupts macro in sbcl, but it does something else.

So without-interrupts may work fine on allegro, but on sbcl you need to 
use either mutexes, spinlocks or use thread local specials.

> Thread Safety in Backends:
>
> I'm pretty sure that the current behavior of BDB is thread-safe.  I
> researched this earlier, but only remember that I concluded it was
> safe so if anyone remembers the details feel free to contradict me.
>
> A quick Google investigation says that CL-SQL requires, at a minimum,
> that each thread have it's own connection object to be thread safe,
> so each thread needs to reconnect to the cl-sql database plus have a
> thread-local clsql:*default-database* binding.  (I don't think this
> works for SQLite though)
>
> These are pretty easy and will be handled in my next checkin:
> -------------------------------------------------------------
>
> Global variables (infrequently written):
> *elephant-controller-init*
> *dbconnection-spec*
>
> Store-controller slots that need infrequent write-protection:
> - instance-cache
> - symbol-cache (0.6.1+)
>
>
> The following elephant variables are a little tricky:
> -----------------------------------------------------
>
> Thread-local global vars (frequently accessed):
> *store-controller* (if different threads use different controllers)
> *transaction-stack*
> *current-transaction*
> *resourced-byte-spec*
> (errno handling in uffi?)
>
> Deprecated thread-local vars:
> *auto-commit* (BDB 4.4 no longer pays attention to auto-commit
> arguments so we can remove this from elephant)
>
> 1) You can use the macro with-elephant-variables in 0.6.1 to create
> new, thread-local dynamic bindings of the above variables, but that
> is still a manual solution for when you have access to the thread
> creation code and can create thread-local specials.

As I said above one only needs to be in control somewhere above in the 
dynamic context of any elephant usage. It's error prone but doable.

> 2) A more consequential is to excise required dependency on these
> variables entirely.  The implication of this is a potentially
> significant API change where an application can always provide the
> store controller on calls to collection accessors, cursor operations,
> etc and it defaults to *store-controller* for environments where
> there is only one store or where the user is managing the binding of
> *store-controller* in each thread.  I think this is already
> accommodated in much of the API, but I haven't investigated this to
> see how much work it is.
>
> We can further require that all transactions be wrapped in 'with-
> elephant-transaction' so that the appropriate specials are

Do you mean with-elephant-variables?

> dynamically bound within the stack.  I think this actually would be
> pretty easy.  We could document the internals of with-elephant-
> transaction for anyone who wants to do something sophisticated and is
> willing to manage the thread issues themselves.

Sounds reasonable to me. We really don't want multiple threads to use 
the same values thus the global value of some specials should never be 
used. This situation comes up often and there is a patch on sbcl-devel 
that implements thread local vars that are much like defvars that 
cannot have a global value. This is a side note only as elephant needs 
a portable solution.

> Does anyone have a better suggestion here?  For example is there a
> portability layer that can detect the current thread ID and use that
> to index the default global values?

No, I think what you described above is better and portable. There is 
already a mechanism for thread local storage is multithreaded lisps and 
that should be used.

> Regards,
> Ian

Thank you for the detailed answer.

Gabor