[cl-plus-ssl-devel] (and :cl+ssl :clisp :windows)

Sat Mar 24 19:49:56 UTC 2007

Attempting to use cl+ssl (via drakma) with clisp on Win32 quickly resulted
in errors about streams ending, rather than encrypted connections (I didn't
keep a copy of the error around, so I can't tell you exactly; sorry). A
little digging revealed cl+ssl::stream-read-byte was returning nil rather
than a character.

;; from cl+ssl/streams.lisp
(defmethod stream-read-byte ((stream ssl-stream))
  (or (ssl-stream-peeked-byte stream)
      (let ((buf (ssl-stream-input-buffer stream)))
        (handler-case
            (cffi-sys::with-pointer-to-vector-data (ptr buf)
              (ensure-ssl-funcall (ssl-stream-socket stream)
                                  (ssl-stream-handle stream)
                                  #'ssl-read
                                  5.5
                                  (ssl-stream-handle stream)
                                  ptr
                                  1)
              (elt buf 0))
          (ssl-error-zero-return ()     ;SSL_read returns 0 on end-of-file
            :eof)))))

Sure doesn't look like it returns nil, does it?

A little more digging revealed this is caused by CFFI.
with-pointer-to-vector-data, on clisp, does a copy-in, copy-out of the
vector. Arguably, it's CFFI's fault for not ensuring the return value of
<body> is returned by with-pointer-to-vector-data. That's easy enough to
change (additions in [square brackets]):

;; from cffi/src/cffi-clisp.lisp
(defmacro with-pointer-to-vector-data ((ptr-var vector) &body body)
  "Bind PTR-VAR to a foreign pointer to the data in VECTOR."
  (with-unique-names (vector-var size-var)
    `(let ((,vector-var ,vector))
       (check-type ,vector-var shareable-byte-vector)
       (with-foreign-pointer (,ptr-var (length ,vector-var) ,size-var)
         [(prog2]
           ;; copy-in
           (loop for i below ,size-var do
                 (%mem-set (aref ,vector-var i) ,ptr-var :unsigned-char i))
           [(progn] , at body[)]
           ;; copy-out
           (loop for i below ,size-var do
                 (setf (aref ,vector-var i)
                       (%mem-ref ,ptr-var :unsigned-char i)))[)]))))

But it doesn't actually solve the problem of broken abstraction: That (elt
buf 0) in stream-read-byte looks at buf before data is copied out, so rather
than returning the first line of an HTTP response #(#\H #\T #\T #\P ...), it
returns the first line of that HTTP response one character late #(#\Null #\H
#\T #\T ...), which makes drakma unhappy. This is still arguably CFFI's
fault for having a broken abstraction, however it /is/ an experimental API
and I don't see how CFFI could plug this particular abstraction leak (which
appears to have been discussed before[1]); meaning it's up to cl+ssl to work
around the limitation.

By moving (elt buf 0) out of the with-pointer-to-vector-data call like so
(changes in [square brackets] again), stream-read-byte returns the right
characters at the right time on clisp.

;; in cl+ssl/streams.lisp
(defmethod stream-read-byte ((stream ssl-stream))
  (or (ssl-stream-peeked-byte stream)
      (let ((buf (ssl-stream-input-buffer stream)))
        (handler-case
            [(progn]
              (cffi-sys::with-pointer-to-vector-data (ptr buf)
                (ensure-ssl-funcall
                  (ssl-stream-socket stream)
                  (ssl-stream-handle stream)
                  #'ssl-read
                  5.5
                  (ssl-stream-handle stream)
                  ptr
                  1)[) #| ends w/pointer, not the new progn |#]
              (elt buf 0))
          (ssl-error-zero-return ()     ;SSL_read returns 0 on end-of-file
            :eof)))))

This obviates the need for the above changes to CFFI and I /think/ retains
correct behavior on other lisp implementations--buf is still around, so I
don't see why it wouldn't--but that's not a theory I can test.

[1] http://common-lisp.net/pipermail/cffi-devel/2006-January/000495.html

-pixie                   If you think I'm nuts, feel free to call me on it.