[Ecls-list] Mutex & fairness

[Matthew Mondor]

On Tue, 27 Mar 2012 21:25:38 -0400 (EDT)
Daniel Herring <dherring at tentpost.com> wrote:

Hopefully Juan Jose can clarify if I'm wrong in the following answers,
as I'm not aware of all issues involved with locking and interrupts in
ECL.

> Can't you use pthread_mutex_trylock to query whether a thread is locked? 
> Not super efficient, but it should work.

If I understand, this would be possible, but if obtaining the lock
fails, the thread must still sleep in a way to be quickly awaken once
the lock has been unlocked, so it's very similar to using the custom
system.

> Similarly, might it be possible to use pthread_mutex_timedlock, and use it 
> in a polling loop to check whether any interrupts have been delivered?

One of the issues I can see here is that the latency of interrupts
would be higher, because they can only be processed after the
pthread_mutex_timedlock() timeout occurred.  This is similar to the
issue the new locking system experienced before including a custom
wakeup mechanism.

> Another idea.  I've seen a people advocate using UNIX sockets as a 
> synchronization mechanism in some circumstances.  Write a byte as a 
> signal.  The receiving process can use a blocking read() and still be 
> woken by interrupts.  Different efficiency tradeoffs than the polling 
> loop.  Harder to get lost between 1 and 2, but introducing other problems.

The libpth library, unless it changed (I've not used it in a while),
uses this for simplicity.  I believe it used pipes with a main
select(2) loop, but it'd also be possible for slightly less overhead to
use SOCK_DGRAM socketpair(2)s. This involves syscall overhead, so
performance is affected, but at least it allows to centralize all
events around a simple FD polling loop.  PTH allows to both poll on
descriptors and custom interthread messages via their ring, i.e. using
pth_poll_ev().  I'm not sure if pth now does this, but it'd also be
possible to use the most efficient OS-specific polling primitive, some
of which can also embed signal delivery as an event (i.e. kqueue(2) on
BSD), but there are libraries like libevent to do this portably...

> Random idea; I don't like it; but maybe it will spark a good one. If ECL 
> spawned a dedicated "kernel" process, and other processes sent it messages 
> to coordinate scheduling, then each of the other threads could have a 
> single dedicated locking mechanism, and lock arbitration would be 
> coordinated under a single thread under ECL's control.  So instead of the 
> futex "compare-and-swap, then syscall" this would be "compare-and-swap, 
> then ask the root thread".

I once had to implement a library with slimiar semantics to libpth
which needed to poll both for FD events and pthread condvar-based
interthread messages.  In this case I had opted for a single polling
thread, which is awaken by a user signal (which is the only thread to
not block) when a message is sent to it to add/remove FD sets, so the
main system simply had to deal with fast interthread-messages, which
the polling thread talked.  I think that this is similar to this last
approach you're describing?  For reference, that library is at
http://cvs.pulsar-zone.net/cgi-bin/cvsweb.cgi/mmondor/mmsoftware/pthread_util/

I personally don't know which of those methods would fit best ECL's
needs.  It's also necessary to take into consideration that ECL might
be embedded in another C application, and make this as simple as
possible...
-- 
Matt