From ocorrain at yahoo.com Mon Nov 8 19:13:10 2004 From: ocorrain at yahoo.com (=?iso-8859-1?Q?Tiarn=E1n_=D3_Corr=E1in?=) Date: Mon, 08 Nov 2004 19:13:10 +0000 Subject: [tbnl-devel] TBNL on Darwin/MacOS X with SBCL/OpenMCL Message-ID: Hello-- does TBNL work on MacOS X with either SBCL or OpenMCL? I'm using mod_lisp 2.38 and Apache 1.3.31. However, requests to the tbnl-test pages hang randomly, and sometimes I get the rather scary: [Mon Nov 8 18:48:53 2004] [error] [TBNL] illegal instruction at #X406C4FA8 -- Tiarn?n From edi at agharta.de Mon Nov 8 19:43:59 2004 From: edi at agharta.de (Edi Weitz) Date: Mon, 08 Nov 2004 20:43:59 +0100 Subject: [tbnl-devel] TBNL on Darwin/MacOS X with SBCL/OpenMCL In-Reply-To: =?iso-8859-1?q?=28Tiarn=E1n_=D3=2E_Corr=E1in's?= message of "Mon, 08 Nov 2004 19:13:10 +0000") References: Message-ID: On Mon, 08 Nov 2004 19:13:10 +0000, ocorrain at yahoo.com (Tiarn?n ? Corr?in) wrote: > does TBNL work on MacOS X with either SBCL or OpenMCL? I'm using > mod_lisp 2.38 and Apache 1.3.31. Frankly - I don't know. I'd suggest that you first try mod_lisp with Apache alone, i.e. without TBNL. You should be able to modify the example code from the mod_lisp website or ask on the OpenMCL or SBCL mailing lists. If that works then it's probably due to KMRCL because I'm pretty sure that the rest of TBNL doesn't contain implementation-specific code. > However, requests to the tbnl-test pages hang randomly, and > sometimes I get the rather scary: > > [Mon Nov 8 18:48:53 2004] [error] [TBNL] illegal instruction at > #X406C4FA8 With OpenMCL or with SBCL? Which version? Also, do you know that you can set tbnl:*show-lisp-errors-p* and tbnl:*show-lisp-backtraces-p* both to T to see more specific error messages? I'll try to help if you can provide more details. Unfortunately I don't have a Mac to test against. Cheers, Edi. From edi at agharta.de Tue Nov 9 08:49:33 2004 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Nov 2004 09:49:33 +0100 Subject: [tbnl-devel] New version 0.3.0 (Was: multipart/form-data forms, RFC2388, File Uploads) In-Reply-To: <20041027210417.GA2668@roadkill.foldr.org> (Michael Weber's message of "Wed, 27 Oct 2004 23:04:17 +0200") References: <20041027210417.GA2668@roadkill.foldr.org> Message-ID: On Wed, 27 Oct 2004 23:04:17 +0200, Michael Weber wrote: > Attached is a patch with my stab at file upload capabilities for > TBNL. :) Thanks again and sorry for the delay. I've uploaded a new version which implements file upload capabilities based on your suggestions and code. I've changed it such that it doesn't load the uploaded files into RAM but rather streams them to temporary files. The little test suite now also has a file upload page. Here's the Changelog entry Version 0.3.0 2004-11-09 Initial support for multipart/form-data (thanks to Michael Weber and Janis Dzerins) Fixed bug in CREATE-STATIC-FILE-DISPATCHER-AND-HANDLER (caught by Bill Clementson) Note that the new download link is (it's .tar.gz instead of .tgz). Some of you might also be interested in Bill Clementson's blog entry about using TBNL on Windows: I agree that it probably wouldn't be a good idea to use this as a production environment but it might be handy for development and/or demoing. The new code has been tested with CMUCL 19a on Linux, Allegro 7.0 pro on Linux, Allegro 6.2 trial on Windows, and LispWorks 4.3.7 pro on Windows. Cheers, Edi. From edi at agharta.de Tue Nov 9 21:16:11 2004 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Nov 2004 22:16:11 +0100 Subject: [tbnl-devel] New version 0.3.1 Message-ID: Changelog entry: Version 0.3.1 2004-11-09 Slight changes to support Chris Hanson's mod_lisp2 Changed GET-BACKTRACE for newer SBCL versions (thanks to Nikodemus Siivola) Download from See for Apache 2 support. Cheers, Edi. From edi at agharta.de Tue Nov 9 21:30:34 2004 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Nov 2004 22:30:34 +0100 Subject: [tbnl-devel] Apache 2 support Message-ID: Hi! Several TBNL users have reported success with Chris Hanson's mod_lisp2[1] which seems to be a full reimplementation of Marc's mod_lisp for Apache 2, so I decided to try it out myself because a customer of mine wants to use Apache 2. I turned out that it was completely painless to use mod_lisp2 with Apache 2.0.52 on Linux as well as on Windows - the module worked out of the box and some tests seem to show that it doesn't suffer from the "leaked children" and "leaked sockets" problems we had with some versions of mod_lisp. I've added some lines of code to send headers that TBNL likes to have but other than that it's all Chris Hanson's code at which I think I can recommend for TBNL. I hope some other people will try and test it as well. Marc - feel free to add this to your subversion repository if you want. I've also uploaded a pre-compiled module for the Win32 version of Apache 2 which worked for me with Apache 2.0.52, TBNL 0.3.1 (released some minutes ago), and either AllegroCL 6.2 trial or LispWorks 4.3.7 pro. It's here: Cheers, Edi. [1] From edi at agharta.de Wed Nov 10 13:43:29 2004 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Nov 2004 14:43:29 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <042a01c4c728$1969f450$0a02a8c0@marcxp> (Marc Battyani's message of "Wed, 10 Nov 2004 14:20:38 +0100") References: <042a01c4c728$1969f450$0a02a8c0@marcxp> Message-ID: On Wed, 10 Nov 2004 14:20:38 +0100, "Marc Battyani" wrote: > I always wanted to test and finalize those versions but never found > the time to do it. The major points are the treading model and the > socket reuse. Have you also tested these points ? (The problem is > to have thread specific variables for the socket variables (no > special vars in C :( )) Yes, I tried to test these things, too. The code seems to keep the socket info in the configuration records. We know that this didn't work for 1.3.x[1] but it seems to work for 2.x. I inserted logging statements to check whether the sockets are actually re-used and they obviously /are/ re-used. I also tried to test whether there are issues with multiple mod_lisp instances[2] but I couldn't break the code. What I did was something like this: Start two different mod_lisp instances from different Lisp images in different locations. Then run ApacheBench to request a page from the first image several thousand times. Finally request a page from the second image manually one or more times. This was always enough to break mod_lisp prior to 2.37 but doesn't generate errors with mod_lisp2. Maybe my test methodology is flawed but at the moment I'm pretty sure that everything works fine. Perhaps Chris wants to comment on these issues. What I saw was that mod_lisp2 with Apache 2 seems to be a bit slower than mod_lisp with Apache 1.3 - something like 280 requests per second versus 340 requests per second (on VMWare). I'm not sure if this is due to Apache 2 or due to mod_lisp2, though. From a quick glance at the code it seems that mod_lisp2 does a lot of error checking which I think is a good thing. I don't mind if this results in a speed decrease that's barely noticable unless you've hundreds of simultaneously connected clients. Also, please note that I only ran these tests on Linux. On Windows (which supposedly has another default threading model) I only checked whether the standard TBNL tests succeeded. Cheers, Edi. [1] [2] From edi at agharta.de Wed Nov 10 17:11:16 2004 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Nov 2004 18:11:16 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: (Edi Weitz's message of "Wed, 10 Nov 2004 14:43:29 +0100") References: <042a01c4c728$1969f450$0a02a8c0@marcxp> Message-ID: Hmm, looks like I was a bit too quick with my recommendation. Prompted by Marc's questions I conducted some more tests and currently it seems that mod_lisp2 is leaving some sockets open if the Lisp used is AllegroCL 7.0 - yesterday I only tested with CMUCL. This doesn't happen with AllegroCL 7.0 and Apache 1.3. I'm also just building an Apache with the "worker" threading model and will play with this combo as well. More info to come soon, hopefully. Nevertheless, if anybody is willing to give the current code a try - just do it and report here... :) Cheers, Edi. From edi at agharta.de Wed Nov 10 18:19:46 2004 From: edi at agharta.de (Edi Weitz) Date: Wed, 10 Nov 2004 19:19:46 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <058701c4c747$a985cc20$0a02a8c0@marcxp> (Marc Battyani's message of "Wed, 10 Nov 2004 18:06:46 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> Message-ID: On Wed, 10 Nov 2004 18:06:46 +0100, "Marc Battyani" wrote: > I'm making it thread friendly now. > It should be available for testing this evening. Oh, it's good you're saying this - I was about to do the same thing... :) So I'll be doing something else, I'm sure you're faster. I was going to use the APR library, something like apr_threadkey_private_set/get. Is that your plan, too? Cheers, Edi. From ocorrain at yahoo.com Thu Nov 11 10:13:13 2004 From: ocorrain at yahoo.com (Tiarnan O'Corrain) Date: Thu, 11 Nov 2004 02:13:13 -0800 (PST) Subject: [tbnl-devel] TBNL on Darwin/MacOS X with SBCL/OpenMCL Message-ID: <20041111101313.28986.qmail@web54605.mail.yahoo.com> Edi Weitz writes: > [I]t's probably due to KMRCL because > I'm pretty sure that the rest of TBNL doesn't contain > implementation-specific code. You're right, it was processes.lisp in KMRCL. I hacked in support for OpenMCL's native threads on Darwin and everything seems to work now. I also see that file uploads and multipart/form-data encoding work now as well. Wonderful! regards Tiarnan ===== Tiarnan O Corrain (on-site at Vodafone NL mobile: +353 86 831 0249 mail: ocorrain at yahoo.com) From edi at agharta.de Thu Nov 11 10:30:53 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 11:30:53 +0100 Subject: [tbnl-devel] TBNL on Darwin/MacOS X with SBCL/OpenMCL In-Reply-To: <20041111101313.28986.qmail@web54605.mail.yahoo.com> (Tiarnan O'Corrain's message of "Thu, 11 Nov 2004 02:13:13 -0800 (PST)") References: <20041111101313.28986.qmail@web54605.mail.yahoo.com> Message-ID: On Thu, 11 Nov 2004 02:13:13 -0800 (PST), Tiarnan O'Corrain wrote: > You're right, it was processes.lisp in KMRCL. I hacked in support > for OpenMCL's native threads on Darwin and everything seems to work > now. Cool! Please send your patches to Kevin Rosenberg so he can incorporate them into his code. > I also see that file uploads and multipart/form-data encoding work > now as well. Wonderful! Yep - please torture test it... :) From edi at agharta.de Thu Nov 11 10:33:44 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 11:33:44 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <060901c4c754$3538ed40$0a02a8c0@marcxp> (Marc Battyani's message of "Wed, 10 Nov 2004 19:36:35 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> Message-ID: On Wed, 10 Nov 2004 19:36:35 +0100, "Marc Battyani" wrote: > Don't bet ;-) I'm always horrified by the time it takes when I write > some C code like that. allocation/deallocation/pointers/tortuous > poorly documented APIs (I had to read the Apache sources to > understand what they mean with their threadkey stuff...) > etc. bleah... Same here... :( I'm going to spend some time with this now. If you've got something better to do (like fighting deadlines or playing with your kids) please do it and wait until I give up or deliver something. Maybe tonight. Cheers, Edi. From marc.battyani at fractalconcept.com Thu Nov 11 11:33:08 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 11 Nov 2004 12:33:08 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> Message-ID: <083501c4c7e2$387255d0$0a02a8c0@marcxp> "Edi Weitz" wrote: > On Wed, 10 Nov 2004 19:36:35 +0100, "Marc Battyani" wrote: > > > Don't bet ;-) I'm always horrified by the time it takes when I write > > some C code like that. allocation/deallocation/pointers/tortuous > > poorly documented APIs (I had to read the Apache sources to > > understand what they mean with their threadkey stuff...) > > etc. bleah... > > Same here... :( > > I'm going to spend some time with this now. If you've got something > better to do (like fighting deadlines or playing with your kids) > please do it and wait until I give up or deliver something. Maybe > tonight. We should get a better sync. ;-) I finished the modifications and put mod_lisp2.c in the mod_lisp repository. Now what is needed is to test it (and fix it ;-) If you could test it on Linux while I debug it on Windows it would be cool. Cheers, Marc From edi at agharta.de Thu Nov 11 11:42:51 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 12:42:51 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <083501c4c7e2$387255d0$0a02a8c0@marcxp> (Marc Battyani's message of "Thu, 11 Nov 2004 12:33:08 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> Message-ID: On Thu, 11 Nov 2004 12:33:08 +0100, "Marc Battyani" wrote: > We should get a better sync. ;-) Yes... :) > I finished the modifications and put mod_lisp2.c in the mod_lisp > repository. > > Now what is needed is to test it (and fix it ;-) If you could test > it on Linux while I debug it on Windows it would be cool. Just for fun here's what I have now - see attachment. I'm gonna look at your code. Cheers, Edi. -------------- next part -------------- A non-text attachment was scrubbed... Name: mod_lisp2.c Type: application/octet-stream Size: 30999 bytes Desc: not available URL: From edi at agharta.de Thu Nov 11 11:51:21 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 12:51:21 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <083501c4c7e2$387255d0$0a02a8c0@marcxp> (Marc Battyani's message of "Thu, 11 Nov 2004 12:33:08 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> Message-ID: On Thu, 11 Nov 2004 12:33:08 +0100, "Marc Battyani" wrote: > Now what is needed is to test it (and fix it ;-) If you could test > it on Linux while I debug it on Windows it would be cool. Your version fails if multiple Lisp images are running - it re-uses the socket although it shouldn't. That's why I have the lisp_private_t structure in my code. From marc.battyani at fractalconcept.com Thu Nov 11 11:54:43 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 11 Nov 2004 12:54:43 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> Message-ID: <085b01c4c7e5$3c000910$0a02a8c0@marcxp> "Edi Weitz" wrote: > On Thu, 11 Nov 2004 12:33:08 +0100, "Marc Battyani" wrote: > > > We should get a better sync. ;-) > > Yes... :) hehe... > > I finished the modifications and put mod_lisp2.c in the mod_lisp > > repository. > > > > Now what is needed is to test it (and fix it ;-) If you could test > > it on Linux while I debug it on Windows it would be cool. > > Just for fun here's what I have now - see attachment. > > I'm gonna look at your code. Looks like I'm more lazy. I just added a per thread configuration but did not touch anything else. Cheers, Marc From marc.battyani at fractalconcept.com Thu Nov 11 11:58:22 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 11 Nov 2004 12:58:22 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> Message-ID: <086801c4c7e5$bed8e460$0a02a8c0@marcxp> "Edi Weitz" wrote: > On Thu, 11 Nov 2004 12:33:08 +0100, "Marc Battyani" wrote: > > > Now what is needed is to test it (and fix it ;-) If you could test > > it on Linux while I debug it on Windows it would be cool. > > Your version fails if multiple Lisp images are running - it re-uses > the socket although it shouldn't. That's why I have the lisp_private_t > structure in my code. No. The check_cfg_for_reuse will mark the socket as unsafe if it's not the same lisp image. It's just that I forgot to call it. :( It should be called like this: local_lisp_cfg (lisp_cfg_t *cfg) { lisp_cfg_t *local_cfg = NULL; apr_threadkey_private_get((void**)&local_cfg, cfg_key); if (local_cfg == NULL) { local_cfg = copy_lisp_cfg (socket_pool, cfg); apr_threadkey_private_set((void*)local_cfg, cfg_key); return local_cfg; } check_cfg_for_reuse(local_cfg, cfg); return local_cfg; } Cheers, Marc From edi at agharta.de Thu Nov 11 12:41:42 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 13:41:42 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <086801c4c7e5$bed8e460$0a02a8c0@marcxp> (Marc Battyani's message of "Thu, 11 Nov 2004 12:58:22 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> <086801c4c7e5$bed8e460$0a02a8c0@marcxp> Message-ID: On Thu, 11 Nov 2004 12:58:22 +0100, "Marc Battyani" wrote: > No. The check_cfg_for_reuse will mark the socket as unsafe if it's > not the same lisp image. It's just that I forgot to call it. :( Yeah, I hadn't seen it when I took a quick glance at your code but now I do. I checked the modified code with and without threads on Linux and it seems to work fine now. I'll update the TBNL docs so people know where to get mod_lisp2. Thanks, Edi. From marc.battyani at fractalconcept.com Thu Nov 11 12:48:02 2004 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Thu, 11 Nov 2004 13:48:02 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> <086801c4c7e5$bed8e460$0a02a8c0@marcxp> Message-ID: <08ac01c4c7ec$aebda550$0a02a8c0@marcxp> "Edi Weitz" wrote: > On Thu, 11 Nov 2004 12:58:22 +0100, "Marc Battyani" wrote: > > > No. The check_cfg_for_reuse will mark the socket as unsafe if it's > > not the same lisp image. It's just that I forgot to call it. :( > > Yeah, I hadn't seen it when I took a quick glance at your code but now > I do. I checked the modified code with and without threads on Linux > and it seems to work fine now. > > I'll update the TBNL docs so people know where to get mod_lisp2. Maybe you should wait a little bit for some more testing... Have you also tested the multi-threaded modes ? Cheers, Marc From edi at agharta.de Thu Nov 11 13:18:09 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 11 Nov 2004 14:18:09 +0100 Subject: [tbnl-devel] Re: [mod-lisp-devel] Apache 2 support In-Reply-To: <08ac01c4c7ec$aebda550$0a02a8c0@marcxp> (Marc Battyani's message of "Thu, 11 Nov 2004 13:48:02 +0100") References: <058701c4c747$a985cc20$0a02a8c0@marcxp> <060901c4c754$3538ed40$0a02a8c0@marcxp> <083501c4c7e2$387255d0$0a02a8c0@marcxp> <086801c4c7e5$bed8e460$0a02a8c0@marcxp> <08ac01c4c7ec$aebda550$0a02a8c0@marcxp> Message-ID: On Thu, 11 Nov 2004 13:48:02 +0100, "Marc Battyani" wrote: > Maybe you should wait a little bit for some more testing... I wouldn't mind if other mod_lisp/TBNL would also do some testing... :) > Have you also tested the multi-threaded modes ? Yes, I've tested with the 'prefork' and with the 'worker' model. (After I had done the same thing with my code. I think that's enough testing for today... :) From ocorrain at yahoo.com Sat Nov 27 02:26:10 2004 From: ocorrain at yahoo.com (=?iso-8859-1?Q?Tiarn=E1n_=D3_Corr=E1in?=) Date: Sat, 27 Nov 2004 02:26:10 +0000 Subject: [tbnl-devel] backtraces in openMCL Message-ID: Hello Edi-- this piece implements backtraces in OpenMCL, though there may be a more beautiful way. Setting "detailed-p" to t results in a cranial overflow in my case :-) -------------- next part -------------- A non-text attachment was scrubbed... Name: util.patch Type: text/x-patch Size: 1190 bytes Desc: not available URL: -------------- next part -------------- -- Tiarn?n From edi at agharta.de Mon Nov 29 09:11:05 2004 From: edi at agharta.de (Edi Weitz) Date: Mon, 29 Nov 2004 10:11:05 +0100 Subject: [tbnl-devel] New release 0.3.4 (Was: backtraces in openMCL) In-Reply-To: =?iso-8859-1?q?=28Tiarn=E1n_=D3=2E_Corr=E1in's?= message of "Sat, 27 Nov 2004 02:26:10 +0000") References: Message-ID: On Sat, 27 Nov 2004 02:26:10 +0000, ocorrain at yahoo.com (Tiarn?n ? Corr?in) wrote: > this piece implements backtraces in OpenMCL, though there may be a > more beautiful way. Setting "detailed-p" to t results in a cranial > overflow in my case :-) Thanks, I've made a new release with this code added. Changelog: Version 0.3.4 2004-11-29 Added backtrace code for OpenMCL (provided by Tiarn?n ? Corr?in) Cheers, Edi.