[cl-plus-ssl-devel] ssl crashing in hunchentoot. A solution?

[Anton Vodonosov]

I tested also with CLISP. 

It wasn't easy BTW, required to fix usocket, and also cl+ssl function
stream-write-sequence had strange assumption, that the sequence
to written is always (simple-array (unsigned-byte 8) (*)). The first line was:

  (check-type thing (simple-array (unsigned-byte 8) (*)))

Also it didn't take into account tat the parameters START and END
can be null.

Example where this fails: hunchentoot function start-output calls 
write-sequence with a list, and cl+ssl:ssl-stream implementation of 
stream-write-sequence receives this list as a parameter (on CLISP,
probably other Lisps convert the list to an array).

Anyway, back to the original problem. CLISP doesn't crash. Sometimes
it print errors like this:

An unhandled error condition has been signalled: An I/O error occurred: undocumented reason. (return code:  5)SSL error queue: 
   
<1/333> #<SYSTEM-FUNCTION SHOW-STACK> 3
<2/326> #<COMPILED-FUNCTION SYSTEM::PRINT-BACKTRACE>
<3/322> #<COMPILED-FUNCTION TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM>
<4/316> #<COMPILED-FUNCTION TRIVIAL-BACKTRACE:PRINT-BACKTRACE>
<5/311> #<COMPILED-FUNCTION #:|280 297 (DEFMETHOD PROCESS-CONNECTION :AROUND ...)-17-1-1-1-1|>
<6/307> #<COMPILED-FUNCTION #:|280 297 (DEFMETHOD PROCESS-CONNECTION :AROUND ...)-17-1-1|>
<7/302> #<SYSTEM-FUNCTION SIGNAL> 1
<8/303> #<SYSTEM-FUNCTION ERROR>
<9/292> #<COMPILED-FUNCTION CL+SSL::SSL-SIGNAL-ERROR>
<10/280> #<COMPILED-FUNCTION #:|97 112 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1-2|>
<11/266> #<COMPILED-FUNCTION #:|97 112 (DEFMETHOD STREAM-READ-BYTE (#) ...)-10-1-1|>
<12/266> #<STANDARD-GENERIC-FUNCTION STREAM-READ-BYTE>
<13/265> #<SYSTEM-FUNCTION READ-BYTE>
<14/261> #<COMPILED-FUNCTION CHUNGA:READ-CHAR*>
<15/251> #<COMPILED-FUNCTION CHUNGA:READ-LINE*>
<16/248> #<COMPILED-FUNCTION HUNCHENTOOT::READ-INITIAL-REQUEST-LINE-2-2>

This means that SSL_read() failed and the SSL_get_error() reports SSL_ERROR_SYSCALL (== 5)

This means some underlying planform IO call failed.

But this is not too big problem (probably platform call really failed, I don't know). 
The server handled thousands of requests from Apache Bench,
and remains hanging browser requests correctly.

CLISP is single threaded.

I tested CCL with hunchentoot:single-threaded-taskmaster. It crashes.

Not immediately, but after client closes the connection. For example you open
the URL in browser, refresh the page serveral times. Everything is OK. 

But 10-20-30 seconds CCL crashes.

This delay is interesting. I have impression/assumption, that OpenSSL calls the
locking-callback from it's internal thread (which was not started by lisp). 

I posted a question to cffi-devel, whether callbacks should work from when called
from non-lisp threads.

But on the other hand, if the cause was the impossibility to invoke callbacks
from non-lisp threads, it should fail on Linux as well. Therefore, probably,
this is not a valid assumption.

Best regards,
- Anton