[usocket-devel] SBCL/Win32: finalizer problem, etc.

[Chun Tian (binghe)]

Hi, Anton!

Thank you very much.  Actually I just realized that you're the author of sbcl-win32-threads.  Among other things, I want to say how important your packages are, because I know there's a commercial Lisp-based product (Gensym G2) which heavily depends on SBCL/win32 (for development, they use Lisp->C translator to build final product), but it's so big and memory-costs that cannot continuously running on 32-bit SBCL. And its threading version can be tested on SBCL now (used to be LispWorks 6).

I've installed your every binary release and just yesterday night was I thinking that I should have a test of USOCKET with Hunchentoot on the new threading SBCL ... and your mail come in! But, obviously, I can never figure out what you've already found.  I thought the SB-EXT:FINALIZE I wrote was already working, but your test confirm it wasn't.

I think I cannot understand why the closure must close over those components of WAIT-LIST but WAIT-LIST itself, but since this is a fact (as you confirmed), I'd like to adopt your patches and quote all your explanations and put with the new code together.  I hope, with your SBCL and USOCKET work, Hunchentoot (and other Lisp-based servers) could have a beautiful future on Windows platform.

Anyway, I merged all your work, as r588 on USOCKET 0.5.x branch [1].  There're other fixes after 0.5.0 was releasd, and Zach's Quicklisp can automatically test/accept new USOCKET releases now, I'll do rest of scheduled fixes quickly and make a new 0.5.1 release soon.

Best Regards,

Chun Tian (binghe)

[1] svn://common-lisp.net/project/usocket/svn/usocket/branches/0.5.x

在 2011-3-22，02:31， Anton Kovalenko 写道：

> 
> Greetings to all USOCKET developers and contributors.
> 
> First, thank you for the wonderful package. But of course, I'm posting
> here to report some problems (on a particular platform: #+(and sbcl win32))
> 
> 1. Event handles are leaking in current SBCL backend implementation,
>   because of SBCL-unfriendly usage of finalizers.
> 
>   SBCL never calls a finalizer that closes over a finalized object: a
>   reference from that closure prevents its collection forever. That's
>   the case with USOCKET in %SETUP-WAIT-LIST.
> 
>   I use the following redefinition of %SETUP-WAIT-LIST:
> 
> (defun %setup-wait-list (wait-list)
>  (setf (wait-list-%wait wait-list) (sb-alien:make-alien ws-event))
>  (setf (os-wait-list-%wait wait-list) (wsa-event-create))
>  (sb-ext:finalize wait-list
> 		   (let ((event-handle (os-wait-list-%wait wait-list))
> 			 (alien (wait-list-%wait wait-list)))
> 		     #'(lambda ()
> 			 (wsa-event-close event-handle)
> 			 (unless (null alien)
> 			   (sb-alien:free-alien alien))))))
> 
>   Of course it may be rewritten with more clarity, but you can see the
>   core idea: I'm closing over those components of WAIT-LIST that I need
>   for finalization, not the wait-list itself. With the original
>   %SETUP-WAIT-LIST, hunchentoot stops working after ~100k accepted
>   connections; it doesn't happen with redefined %SETUP-WAIT-LIST.
> 
> 
> 2. SB-BSD-SOCKETS:SOCKET-ACCEPT method returns NIL for EAGAIN/EINTR,
>   instead of raising a condition. It's always possible for
>   SOCKET-ACCEPT on non-blocking socket to fail, even after the socket
>   was detected to be ready: connection might be reset, for example.
> 
>   I had to redefine SOCKET-ACCEPT method of STREAM-SERVER-USOCKET to
>   handle this situation. Here is the redefinition:
> 
> (defmethod socket-accept ((socket stream-server-usocket) &key element-type)
>  (with-mapped-conditions (socket)
>    (let ((sock (sb-bsd-sockets:socket-accept (socket socket))))
>      (if sock
> 	  (make-stream-socket
> 	   :socket sock
> 	   :stream (sb-bsd-sockets:socket-make-stream
> 		    sock
> 		    :input t :output t :buffering :full
> 		    :element-type (or element-type
> 				      (element-type socket))))
> 
>          ;; next time wait for event again if we had EAGAIN/EINTR
>          ;; or else we'd enter a tight loop of failed accepts
> 
> 	  (setf (%ready-p socket) nil)))))   
> 
> 
> P.S. For some months I'm working on threading support for
> SBCL/Windows (X86 and AMD64)
> [ http://github.com/akovalenko/sbcl-win32-threads/wiki ], continuing the
> work of Dmitry Kalyanov, who implemented threading support initially.
> Traditionally, I use HUNCHENTOOT as a kind of smoke test for my builds,
> and it's a wonderful choice.
> 
> Two issues described above, however, are universal -- it's not something
> that we meet (only) on my experimental threads builds. #1 applies to
> _any_ SBCL/Windows, #2 might not apply to some earlier versions that are
> still in use; both deserves fixing.
> 
> Here is another issue with USOCKET that _can't_ affect upstream SBCL
> until Windows/AMD64 support is accepted:
> 
> 3. SOCKET is defined as intptr_t in Windows headers; however, WS-SOCKET
>   is defined as unsigned-int, i.e. 32-bit even on 64-bit platform.  It
>   seems to be a good thing to redefine WS-SOCKET as SB-ALIEN:SIGNED,
>   which is always machine word-sized (exactly as intptr_t;
>   N.B. as of Windows/x64, long and signed-long are 32-bit, and thus not
>   enough -- potentially).
> 
> Fortunately, this last issue doesn't cause any actual damage now:
> HUNCHENTOOT works on my x64 build (when redefinitions mentioned above
> are loaded).  However, it happens just because SBCL does more than MS
> ABI specification requires, i.e. doesn't leave garbage in the upper 32
> bits of the argument.  I'm afraid that some future optimization (and/or
> reimplementation) of SBCL FFI callout may eventually break this behavior
> -- so it still makes sense to define WS-SOCKET according to Windows
> headers.
> 
> -- 
> Regards, Anton Kovalenko
> +7(916)345-34-02 | Elektrostal' MO, Russia
> 
> _______________________________________________
> usocket-devel mailing list
> usocket-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/usocket-devel