[mod-lisp-devel] Socket leaks, again

[Marc Battyani]

Edi wrote:

>[Again, I'm in a train and sending via cell phone and GPRS. Let's see
>if it works better this time. Also, I'm not sure if SquirrelMail will
>be able to keep indentations.]
>
>I'm prepared to make myself a fool in public again.

Heh?? Reporting and fixing problems is not really what I would call making
oneself a fool in public. ;-)

> I've debugged the
>mod_lisp/Apache interaction (by inserting log statements in strategic
>places) a bit and here's what I found out (Linux x86, mod_lisp 2.33,
>Apache 1.3.29, CMUCL 18e):
>
>1. Whenever OpenLispSocket is called, cfg->LispSocket is 0. This can
>   be best seen if you start Apache with the -X option so that you
>   only one child process. This would imply that sockets can never be
>   re-used.
>
>2. Another data point (see also Peter's earlier post) is that
>
>     netstat -an | grep 3000
>   reports far more established connections than there are Apache
>   child processes.
>
>3. I guess this is due to the fact that the configuration record is
>   reset by Apache on each request. I observe that the comment in
>   Apache's example module (which is also in mod_lisp) says
>
>     "Note that while the per-directory and per-server configuration
>      records are available to most of the module handlers, they
>      should be treated as READ-ONLY by all except the command and
>      merge handlers.  Sometimes handlers are handed a record that
>      applies to the current location by implication or inheritance,
>      and modifying it will change the rules for other locations."
>
>   This is obviously violated by mod_lisp which modifies the record in
>   the content handler.
>
>4. My naïve attempt to fix this is to make LispSocket and
>   UnsafeLispSocket global variables. A patch which reflects my
>   changes is attached at the bottom. This currently works for
>   me. Debugging shows that sockets are in fact re-used this
>   time. Also, if I benchmark with ApacheBench (2000 request, 200
>   concurrent, 1300 byte static page, client and server on same
>   machine) then the original mod_lisp code is able to handle roughly
>   140 requests per seconds while my modified code handles more than
>   460 requests per seconds.

This was how I originally wrote the socket reuse code, (with LispSocket and
UnsafeLispSocket as global variables) but then when I tried to port it to
windows it was failing because Apache is multi-threaded on windows. So I
tried to be more clean and moved the variables to the config struct. This
still didn't worked for windows, this is why there is an #ifndef WIN32
around the reuse code. After that I just verified that it was still ok on
Linux but I didn't redo the socket reuse and speed tests.

>5. Of course, this patch will only work if the Apache children are
>   separate processes spawned by the server process as with Apache
>   1.3, i.e. another approach must be taken for the new, alternative
>   Apache 2.0 thread implementations.

Yes, Apache 1.3 has separate processes on Linux so going back to global
variables is ok for me.

>OK, that's all for now. Let me know what you think.

I think: Good catch!
I looked at your patches, they look ok for me so I commited them to the
mod_lisp repository and switched my servers to mod_lisp 2.35.

Please get the latest one (2.35) on the mod_lisp repository:
http://www.fractalconcept.com:8000/public/open-source/

Cheers,

Marc