[slime-devel] Re: Partial multiprocessing support on CMUCL

[Peter Seibel]

Luke Gorrie <luke at bluetail.com> writes:

> Helmut Eller <e9626484 at stud3.tuwien.ac.at> writes:
> 
> > Hmm, not sure if I understand the problem.  I think, if every thread
> > has it's own connection with a separate state machine and associated
> > buffers, we can do almost everything like we do now.  We just have to
> > make sure the we do it in the right buffer, something like per-session
> > variables.  And of course, we need a way to allow the thread to
> > initiate the new connection.
> 
> Perhaps I didn't understand the idea. I was thinking of "if ten
> threads hit the debugger at once, we don't want the user to be
> preempted with ten different debugger buffers".
> 
> But now I see the light! We could create ten debugger buffers, but not
> actually pop them up unless it's from the nominated current/foreground
> thread. Then the whole notion of foreground/background threads is just
> a window-management issue, and doesn't involved any fancy mutex
> hacking or any of that road to hell I embarked on over the weekend :-)
> 
> Sounds good so far :-)

Yes, this sounds (after being passed through my own biases) like what
I was thinking of. To be clear, I was thinking of something like this:

  Have one (or maybe two) threads dedicated to talking to emacs. (Two
  if the thread that reads expressions *from* emacs is going to
  blocked on IO). When an expression comes in from emacs it is
  dispatched to the appropriate "execution" thread by putting it on a
  queue associated with that thread. (These queues are implemented as
  monitors since it is going to have at least two threads touching
  it--one producer and one consumer). Similarly there is a queue for
  messages bound for emacs.

  Other threads go about their business. Any thread that needs to
  communicate with emacs creates the queue mentioned above. When it
  expects something from emacs (say it's entered the debugger) it does
  a blocking GET on its queue. Eventually the user does whatever thing
  is required to cause emacs to send the event addressed to that
  thread that will allow that thread to take a step in the protocol.
  That event will be dropped on the thread's queue by the IO thread,
  the GET will return, and the thread does whatever it does. When it
  wants to send something *to* emacs, it puts an event on the to-emacs
  queue which the outbound-IO thread picks up and sends to emacs.

  On the emacs side a similar dispatching is happening except instead
  of messages being handed off to threads via queues they are handed
  to a per-buffer statemachine.

The nice thing about this approach is that it allows arbitrary threads
to send events to emacs just be throwing them on the outbound (toward
emacs) queue. Thus I can spin up a thread that periodically sends a
message to emacs to be dropped in the associate buffer even if that
thread never gets any events *from* emacs.

And the bits that need to be thread-safe are limited to the queues.
The only tricky bit (depending on the primitives for blocking or
non-blocking IO) is coordinating the input-from and output-to emacs
threads. In the worst case scenario you simply have two, one that
blocks on reads and dispatches events to the appropriate thread queues
and another that blocks in a GET on the to-emacs queue and then writes
on the socket when an event is available. Depending on how the socket
data structure is implemented that may require another mutex to keep
those two threads from stomping on each other. Or not. I'd actually
expect not but who knows.

-Peter

P.S. I'm only contributing this stuff on the hope that it will be
useful or at least will spark some ideas among the folks actually
cutting code--until I have some time to put my hacking where my mouth
is I realize I'm a pure kibitzer.

-- 
Peter Seibel                                      peter at javamonkey.com

         Lisp is the red pill. -- John Fraser, comp.lang.lisp