[Ecls-list] Problems with slime

[Ram Krishnan]

On Tue, Feb 9, 2010 at 3:16 PM, Ram Krishnan <kriyative at gmail.com> wrote:
> On Tue, Feb 9, 2010 at 2:18 PM, Tobias C. Rittweiler <tcr at freebits.de> wrote:
>>
>> Ram Krishnan writes:
>>
>> > Tobias,
>> >
>> > I ran into this issue on MacOSX, and it turned out to be because SLIME
>> > has multiple threads (control and reader threads) which want to read
>> > and write the connection socket stream. The issue (on MacOSX atleast)
>> > was that the standard buffered I/O library (which handles the FILE*
>> > data structure and functions), has been made thread safe, and locks
>> > the file stream when any thread uses it. The tragedy is that there
>> > seems to be only one lock, for both read and write, which means if one
>> > thread locks the stream to read then another thread cannot do anything
>> > with it, not even write. Anyway, the hack I came up with was to modify
>> > the swank-ecl backend to return a two-way-stream with two independent
>> > file-streams for the two directions:
>> >
>> > (defun socket-make-stream (socket &rest args)
>> >   (let ((stream (apply 'sb-bsd-sockets:socket-make-stream socket args)))
>> >     (setf (slot-value socket 'sb-bsd-sockets::stream) nil)
>> >     stream))
>> >
>> > (defun make-socket-io-stream (socket)
>> >   (case (preferred-communication-style)
>> >     (:fd-handler
>> >      (sb-bsd-sockets:socket-make-stream socket
>> >                                         :output t
>> >                                         :input t
>> >                                         :element-type 'base-char))
>> >     (:spawn
>> >      (let* ((input (socket-make-stream socket
>> >                                        :direction :input
>> >                                        :element-type 'base-char))
>> >             (output (socket-make-stream socket
>> >                                         :direction :output
>> >                                         :element-type 'base-char))
>> >             (stream (make-two-way-stream input output)))
>> >        (setf (slot-value socket 'sb-bsd-sockets::stream) stream)
>> >        stream))))
>> >
>> > Admittedly this hack is ugly, as it knows way too much about what the
>> > underlying socket implementation is doing. The right place to fix this
>> > is probably in SOCKETS::SOCKET-MAKE-STREAM, based on a DIRECTION
>> > keyword arg or something.
>>
>> Hi Ram,
>>
>> thanks for commenting. I saw your patch earlier, and I actually
>> copy&pasted the above implementation of MAKE-SOCKET-IO-STREAM into my
>> swank-ecl.lisp to see whether it makes a difference, but it does _not_.
>>
>> I'm on Ubuntu 9.10, FWIW.
>>
>> How did you find out about the locking?
>>
>>
>> > Here's a link to the complete set of patches:
>> >
>> >   http://github.com/kriyative/ecl-iphone-builder/blob/master/swank-ecl-patches.txt
>> >
>> > Anyway, hope this is useful in your efforts.
>>
>> Thanks, I'll try to get the :FD-HANDLER communication style into the
>> official swank-ecl when I find time. Why did you change AUTO-FLUSH-LOOP?
>>
>>  -T.
>
> Well, I'm not too surprised that the patch didn't work. I'll try
> resurrecting an older Ubuntu (8.4 I think) VMWare image that I have,
> and seeing if I can reproduce ECL, SLIME, and this problem on it.
>
> I found the blocking issue through a combination of dtrace/dtruss and
> gdb -- not pretty. I couldn't find any official Apple documentation
> confirming the behaviour of their thread-safe stdio, but all symptoms
> pointed to it.
>
> The AUTO-FLUSH-LOOP change may not be necessary any more; as late as
> ECL 9.12.3, open-stream-p was raising a TYPE-ERROR condition
> (something like "<a SLIME-OUTPUT-STREAM> is not of type STREAM"), when
> testing a fundamental-character-output-stream derivative stream. I
> didn't have a chance to chase this down, but I suspect the
> implementation of cl_open_stream in file.d is excluding gray streams.
>
> -ram
>

Interestingly, I was able to compile ecl 9.12.3, with threads and run
SLIME with my patches successfully on an Ubuntu 8.4 VMWare
image. Here's what I ran:

  ./configure --prefix=/mnt/ecl \
	--enable-threads \
	--enable-unicode \
	--enable-longdouble \
	--enable-gengc \
	--with-asdf=yes \
	--with-tcp=yes
  make
  make install

Then using the following script in ~/bin/ecl.sh:

  #!/bin/sh
  export LD_LIBRARY_PATH=/mnt/ecl/lib:${LD_LIBRARY_PATH}
  exec /mnt/ecl/bin/ecl "$@"

I loaded slime (with my patches), with the following command:

  (defun ecl ()
    (interactive)
    (let ((slime-lisp-implementations
           `((ecl ,(list (expand-file-name "~/bin/ecl.sh"))))))
      (slime)))

  M-x ecl

And, it just worked (tm).

Cheers,

-ram