[usocket-devel] SBCL/Win32: finalizer problem, etc.

[Anton Kovalenko]

Greetings to all USOCKET developers and contributors.

First, thank you for the wonderful package. But of course, I'm posting
here to report some problems (on a particular platform: #+(and sbcl win32))

1. Event handles are leaking in current SBCL backend implementation,
   because of SBCL-unfriendly usage of finalizers.

   SBCL never calls a finalizer that closes over a finalized object: a
   reference from that closure prevents its collection forever. That's
   the case with USOCKET in %SETUP-WAIT-LIST.

   I use the following redefinition of %SETUP-WAIT-LIST:

(defun %setup-wait-list (wait-list)
  (setf (wait-list-%wait wait-list) (sb-alien:make-alien ws-event))
  (setf (os-wait-list-%wait wait-list) (wsa-event-create))
  (sb-ext:finalize wait-list
		   (let ((event-handle (os-wait-list-%wait wait-list))
			 (alien (wait-list-%wait wait-list)))
		     #'(lambda ()
			 (wsa-event-close event-handle)
			 (unless (null alien)
			   (sb-alien:free-alien alien))))))

   Of course it may be rewritten with more clarity, but you can see the
   core idea: I'm closing over those components of WAIT-LIST that I need
   for finalization, not the wait-list itself. With the original
   %SETUP-WAIT-LIST, hunchentoot stops working after ~100k accepted
   connections; it doesn't happen with redefined %SETUP-WAIT-LIST.


2. SB-BSD-SOCKETS:SOCKET-ACCEPT method returns NIL for EAGAIN/EINTR,
   instead of raising a condition. It's always possible for
   SOCKET-ACCEPT on non-blocking socket to fail, even after the socket
   was detected to be ready: connection might be reset, for example.

   I had to redefine SOCKET-ACCEPT method of STREAM-SERVER-USOCKET to
   handle this situation. Here is the redefinition:

(defmethod socket-accept ((socket stream-server-usocket) &key element-type)
  (with-mapped-conditions (socket)
    (let ((sock (sb-bsd-sockets:socket-accept (socket socket))))
      (if sock
	  (make-stream-socket
	   :socket sock
	   :stream (sb-bsd-sockets:socket-make-stream
		    sock
		    :input t :output t :buffering :full
		    :element-type (or element-type
				      (element-type socket))))
                                      
          ;; next time wait for event again if we had EAGAIN/EINTR
          ;; or else we'd enter a tight loop of failed accepts
          
	  (setf (%ready-p socket) nil)))))   


P.S. For some months I'm working on threading support for
SBCL/Windows (X86 and AMD64)
[ http://github.com/akovalenko/sbcl-win32-threads/wiki ], continuing the
work of Dmitry Kalyanov, who implemented threading support initially.
Traditionally, I use HUNCHENTOOT as a kind of smoke test for my builds,
and it's a wonderful choice.

Two issues described above, however, are universal -- it's not something
that we meet (only) on my experimental threads builds. #1 applies to
_any_ SBCL/Windows, #2 might not apply to some earlier versions that are
still in use; both deserves fixing.

Here is another issue with USOCKET that _can't_ affect upstream SBCL
until Windows/AMD64 support is accepted:

3. SOCKET is defined as intptr_t in Windows headers; however, WS-SOCKET
   is defined as unsigned-int, i.e. 32-bit even on 64-bit platform.  It
   seems to be a good thing to redefine WS-SOCKET as SB-ALIEN:SIGNED,
   which is always machine word-sized (exactly as intptr_t;
   N.B. as of Windows/x64, long and signed-long are 32-bit, and thus not
   enough -- potentially).

Fortunately, this last issue doesn't cause any actual damage now:
HUNCHENTOOT works on my x64 build (when redefinitions mentioned above
are loaded).  However, it happens just because SBCL does more than MS
ABI specification requires, i.e. doesn't leave garbage in the upper 32
bits of the argument.  I'm afraid that some future optimization (and/or
reimplementation) of SBCL FFI callout may eventually break this behavior
-- so it still makes sense to define WS-SOCKET according to Windows
headers.

-- 
Regards, Anton Kovalenko
+7(916)345-34-02 | Elektrostal' MO, Russia