[elephant-devel] Serialization

[Robert L. Read]

Dear Ian,
    Since your doing all the work I'm a little hesitant to suggest a
solution, but it seems to me 
the best solution is to use an all-lisp buffer pool, and use portable
locking to atomically associate
a serializing thread with a buffer.  (This is very similar to solution
#1 you propose, I think.)
I'm actually not sure how your doing portable locking now; I just stuck
some SBCL mutexes 
in there, but that is makes my personal branch SBCL dependent.)
    In the previous version (approximately v. 1.6 of the file
serializer.lisp) the circularity hash 
is implemented as a hash-table.  It's not clear to me why one couldn't
just allocate this 
hashtable on every invocation, (perhpas lazily, when the first non-
atomic type which might 
lead to a circularity is serialized.)  Perhaps this allocation cost
would dominate the serialization
cost; but I would personally not be sure of that until a profiler gave
such information.

    (However, I'm starting to worry that I might simply be terribly
lazy, and covering up
for the fact by chanting the mantra: no premature optimization.  I trust
your instincts and 
knowledge more than my own on this matter.)

      I'd like to stay away from any C-coding as much as possible; just
as a matter of principle
I can't imagine that that makes us more robust.

    Thanks for your work Ian ---- a better serializer will be a big
performance boost for the 
whole project.


On Mon, 2006-11-20 at 09:56 -0500, Ian Eslick wrote:

> Elephant users and developers,
> 
> I've made the serializer thread safe for 0.6.1, but face an design
> tradeoff.  The serializer is a critical bottleneck for  Elephant's
> throughput and by adding locking into the equation I significantly slow
> down the calls to serialize that operate on a small amount of data (all
> symbols, integers and short strings which are typical slot values). 
> There are several possible solutions, in rough order of performance from
> fastest to slowest:
> 
> 1) Special start-elephant-thread macro which creates dynamic bindings
> for the new process of a global special variable that maintains the
> circularity buffer; this will allow us to be very efficient but requires
> elephant-specific code on thread creation for safe threading
> 
> 2) Bind a C-library which provides a CPU independent compare-and-swap
> instruction and/or a lock-free queue so we can share a pool of
> circularity buffers among processes in a lock-free way
> 
> 3) It occurs to me that most lisps do not allow C code to be run
> concurrently from multiple threads - perhaps just implementing a mutex
> in C would be enough to provide a cheap way of updating head and tail
> pointers in the queue.  Capturing and releasing would be atomic.  Is
> this true for all lisps?  I know it's true for Allegro.  It would be a
> hack, but a fairly harmless one given our existing dependence on the
> memutil library.
> 
> 4) Stick with the portable locking approach already implemented
> 
> Thoughts?
> Ian
> 
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20061120/bc4a276e/attachment.html>