[slime-devel] slime-streams overhead

[Helmut Eller]

* Tobias C Rittweiler [2010-06-27 05:39] writes:

> Compiling swank.lisp via
>
>   (time (compile-file "swank.lisp"))
>
> vs
>
>   (time (compile-file "swank.lisp" :verbose nil :print nil))
>
> reveals a major overhead.
>
> First command takes ~14secs, whereas the second 2.3 secs.
>
> The first command at the terminal, without Slime, takes
> roughly 5.5 secs.
>
> Using :spawn, without dedicated output stream. Dedicated
> output stream probably has a much better performance,
> but where does /that/ much overhead come from in the default
> implementation?

COMPILE-FILE with *COMPILE-PRINT* flushes the output buffer very often
that means that lots of small messages are sent.  With :spawn each
message is first sent to the control thread than through the printer to
the socket; Emacs needs to parse it and copy it to another buffer.  So
lots of copying&garbage collecting and the encoding is also not
efficient for this use; but there is also plenty of room for the OS
kernel to make bad scheduling decisions.  On Linux the situation seems
to be bad when the Lisp process produces output at about the same rate
that Emacs can consume; the overhead is much less if the Lisp process is
faster than that or if Emacs gets slower (inserting an artificial delay
on the Emacs side can reduce the overhead).  That's my hypothesis
anyway; it's hard to measure those things.

Sending lots of small messages will never be efficient.  The only cure
would be to add a buffer somewhere to reduce the number of messages;
possibly combined with a timeout.

Helmut