[slime-devel] Re: OpenMCL

[Luke Gorrie]

Ahoy again,

Before going further, let me just note that all the problems I've been
discussing are hypothetical -- they don't exist in our current
single-threaded SLIME. They are problems that we would get if we
decided to add an `eval-in-thread' operation -- which is one of many
options for doing multiprocessing.

Peter Seibel <peter at javamonkey.com> writes:

> > But FOO causes an error and invokes the debugger, so Emacs enters a
> > recursive edit (i.e. calls the event loop recursively) to do the
> > debugging, hoping to debug the problem and then get its result:
> 
> Just to make sure I understand this, it can do that because it
> actually got back a response from the Common Lisp which it grokked is
> an event that told it to invoke the debugger. Otherwise it would still
> be blocked, right. Does that mean if I call (eval-in-thread (sleep
> 300) t1), my emacs is going to hang for 5 minutes?

Right. However, we also have an asynchronous programming interface,
which anything that could take time (including evaluating user-entered
forms) uses. So if you do (sleep 300) in the REPL, for example, Emacs
won't be blocked and won't be in a recursive edit. It will have a
continuation-function stored away that knows to insert the result in
the buffer once the reply arrives asynchronously.

> If there's only one emacs stack, regardless of how many buffers you
> have--which is what I'm gathering--then I'd say eventually the way
> to go is to make all interactions with Common Lisp be
> asynchronous--

There are many advantages to this approach, and I think it should be
used for most things. However, some things are more tedious to do
asynchronously.

Suppose you press `M-.' while Lisp is busy somehow and can't process
your request for a little while. Do you want to carry on editing, and
then be preempted at some "random" time when Lisp tells Emacs where
the definition is? How does Emacs decide what to do when the result
arrives?

Another example is completion. If you press M-TAB to complete a
symbol, but the answer is not immediately forthcoming, what should
happen when it arrives? There are several factors -- has the user
switched buffers, moved the point, done editing -- since pressing
M-TAB? In which cases would he like the completion to be done, and in
which should it be ignored?

There are a small but significant number of cases like this. I believe
it's both easier to code and nicer for the user to make synchronous
calls. If you press M-TAB, you either wait/block until the completion
is done (usually very fast), or you decide to give up and abort it
with C-g.

So I like our current programming interface: we offer both
asynchronous and synchronous evaluation, and use the most appropriate
one for the job -- which is almost-but-not-quite-always asynchronous.

I'm not certain of Helmut's opinion. SLIME was originally (way back
when it was SLIM) fully synchronous, and the first thing he did was
rewrite the guts to asynchronous'ify everything that takes time. He
did the current programming interface. I think it's just right, but
perhaps he's been thinking of killing off synchronous evals? :-)

> Well, you're the one hacking on it, not me. But I'd be worried that if
> you build an synchronous API and everybody builds on it, you'll soon
> have a code base that prevents you from switching to an asynch API.
> And then you'll never be able to provide the good
> debug-two-threads-in-interleaved-fashion UI. 

It is something to keep in mind. There are many trade-offs -- we need
a nice programming interface, predictable user interface, genuinely
correct protocol, and nice implementation. Our approach is to design
is pretty experimental -- the whole protocol has been rewritten at
least half a dozen times (and counting) -- so we'll just have to see
what works out best.

In the long run the future is probably bright for concurrently
debugging two threads. Practically speaking it's very rare to actually
have synchronous evaluations on the stack while running your own code,
since our evaluation commands all use the async interface.

> Will do, it just finished printing out.
> 
> >   Distel: Distributed Emacs Lisp (for Erlang)
> >   http://www.bluetail.com/~luke/distel/distel-euc.pdf

Sadly, it's probably gibberish unless you know some Erlang. I believe
you've read CSP, so it might be sufficient to think of Erlang as CSP
but with asynchronous messages -- but it might still be gibberish ;-)

(I'll never become a famous researcher with these hacks that only a
couple of dozen "Emacs and Erlang hacker" people appreciate :-))

Cheers,
Luke