[elephant-devel] Serialization

[Ian Eslick]

Elephant users and developers,

I've made the serializer thread safe for 0.6.1, but face an design
tradeoff.  The serializer is a critical bottleneck for  Elephant's
throughput and by adding locking into the equation I significantly slow
down the calls to serialize that operate on a small amount of data (all
symbols, integers and short strings which are typical slot values). 
There are several possible solutions, in rough order of performance from
fastest to slowest:

1) Special start-elephant-thread macro which creates dynamic bindings
for the new process of a global special variable that maintains the
circularity buffer; this will allow us to be very efficient but requires
elephant-specific code on thread creation for safe threading

2) Bind a C-library which provides a CPU independent compare-and-swap
instruction and/or a lock-free queue so we can share a pool of
circularity buffers among processes in a lock-free way

3) It occurs to me that most lisps do not allow C code to be run
concurrently from multiple threads - perhaps just implementing a mutex
in C would be enough to provide a cheap way of updating head and tail
pointers in the queue.  Capturing and releasing would be atomic.  Is
this true for all lisps?  I know it's true for Allegro.  It would be a
hack, but a fairly harmless one given our existing dependence on the
memutil library.

4) Stick with the portable locking approach already implemented

Thoughts?
Ian