[slime-devel] Re: Partial multiprocessing support on CMUCL

[Luke Gorrie]

Helmut Eller <e9626484 at stud3.tuwien.ac.at> writes:

> > Also, by allowing more things to happen asynchronously, it seems like
> > we're getting more race-conditions and it could complicate the state
> > machine quite a lot. I'm starting to wonder if we could move some of
> > the state out of Emacs and into Lisp alone, so that it doesn't have to
> > be synchronized. I haven't got any good examples yet.
> 
> Yes, if we have only one state machine on the Emacs side, it's
> probably better to move the protocol checking to the Lisp side.

Been thinking about this a bit.

Even in the single-threaded case the state machine is becoming awkward
as we get more ambitious. Originally our protocol push-down automaton
was very simple, with just these states and transitions:

  IDLE:       push into EVALUATING when Emacs makes an RPC
  EVALUATING: pop on result (or abort) arriving from Lisp
              push into DEBUGGING if Lisp enters the debugger
  DEBUGGING:  pop when restart causes exit from debugger
              push into EVALUATING when Emacs makes an RPC

Very simple, and it has been very good during debugging that illegal
transitions are nicely detected.

The trouble is that it didn't cope with Lisp initiating "asynchronous"
transitions, as does happen in real life. In general, we end up either
disallowing these things, or introducing protocol-level race
conditions to permit them with extra transitions.

Some examples:

  IDLE -> DEBUGGING (added for asynchronously occuring errors in Lisp)
    Introduces a protocol race condition between Emacs sending an RPC
    and Lisp entering the debugger.

  Lack of: DEBUGGING   -> DEBUGGING
           DEBUGGING   -> READ-STRING
           IDLE        -> READ-STRING
           READ-STRING -> DEBUGGING
    Without these transitions, certain things that can actually happen
    due to asynchronous activities (such as SERVE-EVENT, signal
    handlers, etc) will cause a protocol violation panic.

  :ONEWAY-EVALUATE event in IDLE, DEBUGGING, READ-STRING
    More of a code factoring issue perhaps, but adding a new way to
    evaluate expressions meant adding transitions separately to three
    states.

One trend I see is that states increasingly have a lot of transitions
in common, e.g. anything could transition into the debugger. We may
end up with so many common transitions that it's easier to say which
transitions _aren't_ allowed in each state -- in which case maybe we
end up rewriting the state machine as a single function.

The other thing is that there are fundamental race conditions, because
in reality both Emacs and Lisp can initiate transitions, potentially
at the same time. We need to cook up a scheme for resolving these
races in a sane way.

-Luke