[Ecls-list] Problems with slime

[Ram Krishnan]

On Tue, Feb 9, 2010 at 2:18 PM, Tobias C. Rittweiler <tcr at freebits.de> wrote:
>
> Ram Krishnan writes:
>
> > Tobias,
> >
> > I ran into this issue on MacOSX, and it turned out to be because SLIME
> > has multiple threads (control and reader threads) which want to read
> > and write the connection socket stream. The issue (on MacOSX atleast)
> > was that the standard buffered I/O library (which handles the FILE*
> > data structure and functions), has been made thread safe, and locks
> > the file stream when any thread uses it. The tragedy is that there
> > seems to be only one lock, for both read and write, which means if one
> > thread locks the stream to read then another thread cannot do anything
> > with it, not even write. Anyway, the hack I came up with was to modify
> > the swank-ecl backend to return a two-way-stream with two independent
> > file-streams for the two directions:
> >
> > (defun socket-make-stream (socket &rest args)
> >   (let ((stream (apply 'sb-bsd-sockets:socket-make-stream socket args)))
> >     (setf (slot-value socket 'sb-bsd-sockets::stream) nil)
> >     stream))
> >
> > (defun make-socket-io-stream (socket)
> >   (case (preferred-communication-style)
> >     (:fd-handler
> >      (sb-bsd-sockets:socket-make-stream socket
> >                                         :output t
> >                                         :input t
> >                                         :element-type 'base-char))
> >     (:spawn
> >      (let* ((input (socket-make-stream socket
> >                                        :direction :input
> >                                        :element-type 'base-char))
> >             (output (socket-make-stream socket
> >                                         :direction :output
> >                                         :element-type 'base-char))
> >             (stream (make-two-way-stream input output)))
> >        (setf (slot-value socket 'sb-bsd-sockets::stream) stream)
> >        stream))))
> >
> > Admittedly this hack is ugly, as it knows way too much about what the
> > underlying socket implementation is doing. The right place to fix this
> > is probably in SOCKETS::SOCKET-MAKE-STREAM, based on a DIRECTION
> > keyword arg or something.
>
> Hi Ram,
>
> thanks for commenting. I saw your patch earlier, and I actually
> copy&pasted the above implementation of MAKE-SOCKET-IO-STREAM into my
> swank-ecl.lisp to see whether it makes a difference, but it does _not_.
>
> I'm on Ubuntu 9.10, FWIW.
>
> How did you find out about the locking?
>
>
> > Here's a link to the complete set of patches:
> >
> >   http://github.com/kriyative/ecl-iphone-builder/blob/master/swank-ecl-patches.txt
> >
> > Anyway, hope this is useful in your efforts.
>
> Thanks, I'll try to get the :FD-HANDLER communication style into the
> official swank-ecl when I find time. Why did you change AUTO-FLUSH-LOOP?
>
>  -T.

Well, I'm not too surprised that the patch didn't work. I'll try
resurrecting an older Ubuntu (8.4 I think) VMWare image that I have,
and seeing if I can reproduce ECL, SLIME, and this problem on it.

I found the blocking issue through a combination of dtrace/dtruss and
gdb -- not pretty. I couldn't find any official Apple documentation
confirming the behaviour of their thread-safe stdio, but all symptoms
pointed to it.

The AUTO-FLUSH-LOOP change may not be necessary any more; as late as
ECL 9.12.3, open-stream-p was raising a TYPE-ERROR condition
(something like "<a SLIME-OUTPUT-STREAM> is not of type STREAM"), when
testing a fundamental-character-output-stream derivative stream. I
didn't have a chance to chase this down, but I suspect the
implementation of cl_open_stream in file.d is excluding gray streams.

-ram