From edi at agharta.de Tue Aug 2 09:59:22 2005 From: edi at agharta.de (Edi Weitz) Date: Tue, 02 Aug 2005 11:59:22 +0200 Subject: [tbnl-devel] Upload of text files In-Reply-To: (Edi Weitz's message of "Tue, 05 Jul 2005 13:07:45 +0200") References: Message-ID: On Tue, 05 Jul 2005 13:07:45 +0200, Edi Weitz wrote: > I sniffed around a bit and it looks like the problem is in the RFC > 2388 library TBNL uses. As a workaround [...] FWIW, Janis Dzerins has just released a new version of his RFC 2388 library which fixes this so you can now use the "official" version again. Cheers, Edi. From stesch at no-spoon.de Mon Aug 8 21:36:18 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Mon, 8 Aug 2005 23:36:18 +0200 Subject: [tbnl-devel] SCGI support? Message-ID: <20050808213618.GJ26978@parsec.no-spoon.de> Hi! Has anyone thought about implementing a SCGI interface for TBNL? That way it could work together with Apache + mod_scgi, or even better: lighttpd + mod_scgi lighttpd: http://www.lighttpd.net/ SCGI specification: http://python.ca/nas/scgi/protocol.txt Regards, Stefan From steve.huffman at gmail.com Tue Aug 9 04:01:33 2005 From: steve.huffman at gmail.com (Steve Huffman) Date: Tue, 9 Aug 2005 00:01:33 -0400 Subject: [tbnl-devel] tbnl freeze Message-ID: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> Hi all, I'm working on reddit.com, which uses TBNL as its web framework. For the most part, all is well, but I've been running into an error lately that is causing me to worry. Basically, the symptom of the problem is that our website appears to pause, and all requests just hang (eventually they'll timeout on the browser side). I work against our server using remote SLIME. The SLIME session does not hang, but in order to bring the website back to life I log into our web server, use attachtty to view the running lisp process, and hit ctrl + c a few times, after which a debugger will pop open in Emacs that I can abort. I'm running CMUCL 19a on a FreeBSD box. I use apache and mod_lisp in front of TBNL. I haven't been able to find out what causes this lockup, nor do I understand why I must open the process on the web server (when I'm already cruising in SLIME) to wake it up. Has anyone seen any issues like this before? Or perhaps know if this is even a TBNL issue? I've pasted the output I see from the lisp process (via attachtty) and in my emacs session (after I send an interrupt). Any help would be greatly appreciated, Steve Huffman In attachtty I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY ] Destroy the process Debug (type H for help) (UNIX::SIGINT-HANDLER # # #.(SYSTEM:INT-SAP #x47FD789C)) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:code/signal.lisp. 0] and in emacs I'll see something like this: Interrupted at #x2811F14E. [Condition of type SIMPLE-CONDITION] Restarts: 0: [CONTINUE] Return from BREAK. 1: [DESTROY] Destroy the process Backtrace: 0: (UNIX::SIGINT-HANDLER #<#1=unused-arg> #<#1#> #.(SYSTEM:INT-SAP #x47FD789C)) 1: (UNIX::SIGINT-HANDLER 3 #<#1=unused-arg> #<#1#> #.(SYSTEM:INT- SAP #x47FD789C))[:EXTERNAL] 2: ("Foreign function call land") 3: ("Foreign function call land") 4: ("Foreign function call land") 5: ("Foreign function call land") 6: ("Foreign function call land") 7: (LISP::FD-STREAM-READ-N-BYTES # "" 0 14 ...) 8: (LISP::FD-STREAM-READ-N-BYTES 5 # "" 0 ...)[:EXTERNAL] 9: (LISP::READ-INTO-SIMPLE-STRING "" # 0 14) 10: (READ-SEQUENCE "" # :START 0 ...) 11: ((METHOD INITIALIZE-INSTANCE (:AFTER) (TBNL::REQUEST)) (#(1 2 0 3 5 ...) . #()) #<#1=unused-arg> # #<#1#>) 12: ((MAKE-INSTANCE TBNL::REQUEST :HEADERS-IN PCL::|.P0.|) (("content- stream" . #) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip- addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 13: (TBNL::PROCESS-REQUEST (("content-stream" . #) ("content-length" . "14") ("content-type" . "application/x-www-form-urlencoded") ("remote-ip-addr" . "172.133.113.57") ("remote-ip-port" . "3332") ...)) 14: (TBNL::LISTEN-FOR-REQUEST # TBNL::PROCESS-REQUEST) From edi at agharta.de Tue Aug 9 07:06:31 2005 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Aug 2005 09:06:31 +0200 Subject: [tbnl-devel] tbnl freeze In-Reply-To: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> (Steve Huffman's message of "Tue, 9 Aug 2005 00:01:33 -0400") References: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> Message-ID: On Tue, 9 Aug 2005 00:01:33 -0400, Steve Huffman wrote: > I'm working on reddit.com, which uses TBNL as its web framework. For > the most part, all is well, but I've been running into an error > lately that is causing me to worry. Basically, the symptom of the > problem is that our website appears to pause, and all requests just > hang (eventually they'll timeout on the browser side). > > I work against our server using remote SLIME. The SLIME session does > not hang, but in order to bring the website back to life I log into > our web server, use attachtty to view the running lisp process, and > hit ctrl + c a few times, after which a debugger will pop open in > Emacs that I can abort. > > I'm running CMUCL 19a on a FreeBSD box. I use apache and mod_lisp in > front of TBNL. > > I haven't been able to find out what causes this lockup, nor do I > understand why I must open the process on the web server (when I'm > already cruising in SLIME) to wake it up. > > Has anyone seen any issues like this before? Or perhaps know if this > is even a TBNL issue? > > I've pasted the output I see from the lisp process (via attachtty) > and in my emacs session (after I send an interrupt). Hi! Unfortunately, I currently have no idea what the problem is. Just a vague idea: Could you try if the problem is still there without SLIME being in the picture? I remember I've seen a completely unrelated threading problem with a LispWorks app which went away after we stopped having SWANK loaded. Cheers, Edi. PS: From you headers it looks like you're using the newest mod_lisp version so that should be fine. From ignas.mikalajunas at gmail.com Tue Aug 9 10:09:56 2005 From: ignas.mikalajunas at gmail.com (Ignas Mikalajunas) Date: Tue, 9 Aug 2005 13:09:56 +0300 Subject: [tbnl-devel] tbnl freeze In-Reply-To: References: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> Message-ID: > > I'm working on reddit.com, which uses TBNL as its web framework. For > > the most part, all is well, but I've been running into an error > > lately that is causing me to worry. Basically, the symptom of the > > problem is that our website appears to pause, and all requests just > > hang (eventually they'll timeout on the browser side). I am not sure whether it is related yet - I was having simmilar problems with tbnl/cmucl while making mod_lisp interoperate with mod_xslt. While server was getting enough traffic it was pretty responsive yet after long pauses I was getting timeouts. Though tbnl on sbcl was working fine with such configuration. The workaround i have found was - send a ping request once in a few seconds not letting cmucl "sleep". As if one of cmucl threads was hogging all the resources and not giving any to threads that were not doing anything for a while ... Ignas P.S. it could have been related with mod_lisp + mod_xslt combination though ... From edi at agharta.de Tue Aug 9 14:33:00 2005 From: edi at agharta.de (Edi Weitz) Date: Tue, 09 Aug 2005 16:33:00 +0200 Subject: [tbnl-devel] tbnl freeze In-Reply-To: (Ignas Mikalajunas's message of "Tue, 9 Aug 2005 13:09:56 +0300") References: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> Message-ID: On Tue, 9 Aug 2005 13:09:56 +0300, Ignas Mikalajunas wrote: > I am not sure whether it is related yet - I was having simmilar > problems with tbnl/cmucl while making mod_lisp interoperate with > mod_xslt. While server was getting enough traffic it was pretty > responsive yet after long pauses I was getting timeouts. Though tbnl > on sbcl was working fine with such configuration. > The workaround i have found was - send a ping request once in a few > seconds not letting cmucl "sleep". As if one of cmucl threads was > hogging all the resources and not giving any to threads that were not > doing anything for a while ... Ah, wait! You guys use (mp::startup-idle-and-top-level-loops) on startup, don't you? If you don't, that /could/ be the reason for the symptons you describe. Cheers, Edi. From stesch at no-spoon.de Tue Aug 9 14:41:19 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Tue, 9 Aug 2005 16:41:19 +0200 Subject: [tbnl-devel] tbnl freeze In-Reply-To: References: <7CB63559-7190-47EC-B242-7A3F9ABD13DB@gmail.com> Message-ID: <20050809144119.GQ26978@parsec.no-spoon.de> On 2005-08-09 16:33:00, Edi Weitz wrote: > Ah, wait! You guys use > > (mp::startup-idle-and-top-level-loops) > > on startup, don't you? If you don't, that /could/ be the reason for > the symptons you describe. Wasn't there a conflict with SLIME when you use this? I think I remember I've heard something regarding (mp::startup-idle-and-top-level-loops) From stesch at no-spoon.de Wed Aug 10 14:33:11 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 10 Aug 2005 16:33:11 +0200 Subject: [tbnl-devel] SCGI support? In-Reply-To: <20050808213618.GJ26978@parsec.no-spoon.de> References: <20050808213618.GJ26978@parsec.no-spoon.de> Message-ID: <20050810143311.GX26978@parsec.no-spoon.de> On 2005-08-08 23:36:18, Stefan Scholl wrote: > Has anyone thought about implementing a SCGI interface for TBNL? If anyone wants to do it: In GET-REQUEST-DATA (modlisp.lisp) is a test for mod_lisp connection or other. First a single line is read with READ-LINE to have some data for the test. Here is a good place to test for SCGI. But you can't use READ-LINE for it. SCGI sends binary data: ASCII + #\Null No EOL. Could be a problem for Unicode enabled CL implementations like SBCL? (The type of the socket stream can be seen in function ACCEPT-TCP-CONNECTION of KMRCL's sockets.lisp. For SBCL it's the element-type BASE-CHAR.) Regards, Stefan From stesch at no-spoon.de Fri Aug 12 06:49:29 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Fri, 12 Aug 2005 08:49:29 +0200 Subject: [tbnl-devel] SBCL: Can't cope with too many threads Message-ID: <20050812064929.GH26978@parsec.no-spoon.de> Hi! Just ran a small test with sbcl and standalone TBNL. "-c 50" (ApacheBench, 50 simultaneous requests) was too much for the poor litte server. It just froze. Don't really know who's to blame. But maybe it's possible to limit the requests on the TBNL (KMRCL?) side that are served simultaneously -- according to some magic numbers some SBCL wizards could supply. How do the Araneida guys deal with it? Don't use threads, just serve event? Regards, Stefan From stesch at no-spoon.de Fri Aug 12 07:29:31 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Fri, 12 Aug 2005 09:29:31 +0200 Subject: [tbnl-devel] SBCL: Can't cope with too many threads In-Reply-To: <20050812064929.GH26978@parsec.no-spoon.de> References: <20050812064929.GH26978@parsec.no-spoon.de> Message-ID: <20050812072931.GI26978@parsec.no-spoon.de> On 2005-08-12 08:49:29, Stefan Scholl wrote: > Just ran a small test with sbcl and standalone TBNL. "-c 50" > (ApacheBench, 50 simultaneous requests) was too much for the poor > litte server. It just froze. BTW: SBCL 0.9.3.39 From hutch at recursive.ca Sat Aug 13 13:25:25 2005 From: hutch at recursive.ca (Bob Hutchison) Date: Sat, 13 Aug 2005 09:25:25 -0400 Subject: [tbnl-devel] SBCL: Can't cope with too many threads In-Reply-To: <20050812064929.GH26978@parsec.no-spoon.de> References: <20050812064929.GH26978@parsec.no-spoon.de> Message-ID: <069f668a84b232905b6819932673662c@recursive.ca> Hi, On Aug 12, 2005, at 2:49 AM, Stefan Scholl wrote: > Hi! > > Just ran a small test with sbcl and standalone TBNL. "-c 50" > (ApacheBench, 50 simultaneous requests) was too much for the poor > litte server. It just froze. I've done tests on a system I'm working on (Sienna) 5000 hits with 50 concurrent requests and it has run very very well, even when Sienna is running in the development environment. I've tried this apache/mod_lisp -> TBNL, Araneida -> TBNL, apache/Araneida -> TBNL, and TBNL stand-alone. I'm using LispWorks on a mac. I am quite sure I tried all of these combinations (completely sure of Araneida->TBNL and TBNL standalone) on linux as well. I let it run overnight once with 50 concurrent hits, no problem at all. I've only tried higher concurrency experiments on OS X, but never experienced a complete failure, it degraded well. I have never tried to find a breaking point, 50 concurrent requests would be way more than sufficient for any conceivable use of Sienna (of course, now that I've said that I'll be proven wrong immediately :-) Assuming my results are reproducible in other webapps, I think I'd be having a look at SBCL... > > Don't really know who's to blame. But maybe it's possible to > limit the requests on the TBNL (KMRCL?) side that are served > simultaneously -- according to some magic numbers some SBCL wizards > could supply. What happens with a smaller number of concurrent users? Is there a point where performance abruptly declines or is it a slow descent into failure? > > > How do the Araneida guys deal with it? Don't use threads, just > serve event? > > > Regards, > Stefan > > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > > ---- Bob Hutchison -- blogs at Recursive Design Inc. -- Raconteur -- From stesch at no-spoon.de Sun Aug 14 00:39:19 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Sun, 14 Aug 2005 02:39:19 +0200 Subject: [tbnl-devel] SBCL: Can't cope with too many threads In-Reply-To: <069f668a84b232905b6819932673662c@recursive.ca> References: <20050812064929.GH26978@parsec.no-spoon.de> <069f668a84b232905b6819932673662c@recursive.ca> Message-ID: <20050814003919.GQ26978@parsec.no-spoon.de> On 2005-08-13 09:25:25, Bob Hutchison wrote: > On Aug 12, 2005, at 2:49 AM, Stefan Scholl wrote: > >Don't really know who's to blame. But maybe it's possible to > >limit the requests on the TBNL (KMRCL?) side that are served > >simultaneously -- according to some magic numbers some SBCL wizards > >could supply. > > What happens with a smaller number of concurrent users? Is there a > point where performance abruptly declines or is it a slow descent into > failure? I haven't tested it after that. I'm a CMUCL user and was just testing if a current SBCL works with CL-WIKI. Regards, Stefan From stesch at no-spoon.de Tue Aug 16 20:24:56 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Tue, 16 Aug 2005 22:24:56 +0200 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends In-Reply-To: References: Message-ID: <20050816202456.GA26978@parsec.no-spoon.de> On 2005-03-17 02:17:19, Edi Weitz wrote: > Specifically, I tried to explain in the documentation how TBNL talks > with the different back-ends. I also changed the test setup so that > it now hopefully adapts to all three back-ends and gives new users a > chance to understand how TBNL works. I was just wondering ... isn't now every proxy feature in every webserver some kind of mod_lisp? What are the pros and cons of mod_lisp vs. proxy (+ TBNL stand-alone)? From stesch at no-spoon.de Wed Aug 17 07:39:33 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 17 Aug 2005 09:39:33 +0200 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends In-Reply-To: <20050816202456.GA26978@parsec.no-spoon.de> References: <20050816202456.GA26978@parsec.no-spoon.de> Message-ID: <20050817073933.GB26978@parsec.no-spoon.de> On 2005-08-16 22:24:56, Stefan Scholl wrote: > I was just wondering ... isn't now every proxy feature in every > webserver some kind of mod_lisp? > > What are the pros and cons of mod_lisp vs. proxy (+ TBNL > stand-alone)? Forget it. mod_lisp is much, much faster. 3 to 4 times faster in a local test with ApacheBench against Apache 1.3 + mod_lisp + TBNL, lighttpd + mod_proxy + TBNL, and TBNL stand-alone. Apache 1.3 + mod_lisp + TBNL was always faster. The mod_lisp request data can be parsed without any regular expressions. Maybe that's enough to cause that difference in speed. I think I'll try to remove all calls to CL-PPCRE:SPLIT and test it again (later). From hutch at recursive.ca Wed Aug 17 11:52:57 2005 From: hutch at recursive.ca (Bob Hutchison) Date: Wed, 17 Aug 2005 07:52:57 -0400 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends In-Reply-To: <20050817073933.GB26978@parsec.no-spoon.de> References: <20050816202456.GA26978@parsec.no-spoon.de> <20050817073933.GB26978@parsec.no-spoon.de> Message-ID: <8AD93DC5-E35A-476E-970E-3F0F1E542684@recursive.ca> Hi Stefan, On Aug 17, 2005, at 3:39 AM, Stefan Scholl wrote: > On 2005-08-16 22:24:56, Stefan Scholl wrote: > >> I was just wondering ... isn't now every proxy feature in every >> webserver some kind of mod_lisp? >> >> What are the pros and cons of mod_lisp vs. proxy (+ TBNL >> stand-alone)? >> > > Forget it. mod_lisp is much, much faster. 3 to 4 times faster in > a local test with ApacheBench against Apache 1.3 + mod_lisp + > TBNL, lighttpd + mod_proxy + TBNL, and TBNL stand-alone. > > Apache 1.3 + mod_lisp + TBNL was always faster. > Really? That's very interesting. What lisp implementation are you using? I use Lispworks, and in my tests (on OS X and linux, I don't have a license for windows), Apache+mod_lisp+TBNL was second and TBNL standalone the fastest by just a bit better than a factor of *two* -- except on OS X, where there is a bug that I cannot identify that causes a lot of memory to be permanently allocated and -- depending on the concurrency count -- lispworks either slowly grids to a halt or falls over dead almost instantly. I think Araneida+TBNL was third, but it is really hard to say (and I think I can improve this, I don't quite understand why it isn't closer to the standalone TBNL). And the apache+proxy+?+TBNL was slowest. However, even though using a proxy is slowest, for me, I can get my ISP to go along with this whereas they won't consider mod_lisp (or the mod_jk? java modules either, so it isn't a lisp problem). I need my ISP to go along with me only if I need to use port 80. Any other port I use standalone TBNL on linux or Araneida+TBNL on OS X -- I've had an application up and running now on linux (standalone TBNL) for just over two months now I think. And for what its worth, these are all *way* faster than Java. I have not tried lighttpd. I wonder what's causing the timing differences? I'm testing against that application I mentioned and there are some pretty significant chunks of HTML generated (sometimes 100's of kb) and sometimes a non- trivial CPU requirement. Cheers, Bob > The mod_lisp request data can be parsed without any regular > expressions. Maybe that's enough to cause that difference in > speed. > > I think I'll try to remove all calls to CL-PPCRE:SPLIT and test > it again (later). > > > > _______________________________________________ > tbnl-devel site list > tbnl-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/tbnl-devel > ---- Bob Hutchison -- blogs at Recursive Design Inc. -- Raconteur -- From stesch at no-spoon.de Wed Aug 17 14:47:31 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 17 Aug 2005 16:47:31 +0200 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends In-Reply-To: <8AD93DC5-E35A-476E-970E-3F0F1E542684@recursive.ca> References: <20050816202456.GA26978@parsec.no-spoon.de> <20050817073933.GB26978@parsec.no-spoon.de> <8AD93DC5-E35A-476E-970E-3F0F1E542684@recursive.ca> Message-ID: <20050817144731.GE26978@parsec.no-spoon.de> Hi Bob and list! On 2005-08-17 07:52:57, Bob Hutchison wrote: > On Aug 17, 2005, at 3:39 AM, Stefan Scholl wrote: > >Forget it. mod_lisp is much, much faster. 3 to 4 times faster in > >a local test with ApacheBench against Apache 1.3 + mod_lisp + > >TBNL, lighttpd + mod_proxy + TBNL, and TBNL stand-alone. > > > >Apache 1.3 + mod_lisp + TBNL was always faster. > > Really? That's very interesting. What lisp implementation are you > using? I use Lispworks, and in my tests (on OS X and linux, I don't > have a license for windows), Apache+mod_lisp+TBNL was second and TBNL > standalone the fastest by just a bit better than a factor of *two* CMUCL 19a on a Pentium III 650 MHz (2 CPUs), 512 MiB RAM. Linux 2.6.x (Gentoo) Tested with CL-WIKI serving a little 30 KiB file. ab -n 100 -c 5 ... (localhost) Regards, Stefan From stesch at no-spoon.de Wed Aug 17 17:10:07 2005 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 17 Aug 2005 19:10:07 +0200 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends In-Reply-To: <20050817144731.GE26978@parsec.no-spoon.de> References: <20050816202456.GA26978@parsec.no-spoon.de> <20050817073933.GB26978@parsec.no-spoon.de> <8AD93DC5-E35A-476E-970E-3F0F1E542684@recursive.ca> <20050817144731.GE26978@parsec.no-spoon.de> Message-ID: <20050817171007.GF26978@parsec.no-spoon.de> On 2005-08-17 16:47:31, Stefan Scholl wrote: > ab -n 100 -c 5 ... > > (localhost) That's it. Remote test (local network) isn't that bad after. About 14.6 requests/sec with Apache 1.3 + mod_lisp + TBNL. 13.8 requests/sec with stand-alone TBNL and behind lighttpd's mod_proxy. From marc.battyani at fractalconcept.com Wed Aug 17 20:03:43 2005 From: marc.battyani at fractalconcept.com (Marc Battyani) Date: Wed, 17 Aug 2005 22:03:43 +0200 Subject: [tbnl-devel] New version 0.5.0 / Multiple back-ends References: <20050816202456.GA26978@parsec.no-spoon.de> <20050817073933.GB26978@parsec.no-spoon.de> Message-ID: <015b01c5a366$c5acb930$0a02a8c0@marcxp> ----- Original Message ----- From: "Stefan Scholl" To: Sent: Wednesday, August 17, 2005 9:39 AM Subject: Re: [tbnl-devel] New version 0.5.0 / Multiple back-ends > On 2005-08-16 22:24:56, Stefan Scholl wrote: > > I was just wondering ... isn't now every proxy feature in every > > webserver some kind of mod_lisp? > > > > What are the pros and cons of mod_lisp vs. proxy (+ TBNL > > stand-alone)? > > Forget it. mod_lisp is much, much faster. 3 to 4 times faster in > a local test with ApacheBench against Apache 1.3 + mod_lisp + > TBNL, lighttpd + mod_proxy + TBNL, and TBNL stand-alone. > > Apache 1.3 + mod_lisp + TBNL was always faster. > > The mod_lisp request data can be parsed without any regular > expressions. Maybe that's enough to cause that difference in > speed. They are several reasons why mod_lisp is faster than mod_proxy and generally than a standalone lisp server in real world applications: The most important point for the speed is that mod_lisp keeps the Apache => Lisp socket open. Most Lisp implementation don't use native threads or don't allow more than one thread to be active. So using Apache as a front-end is more efficient if there are more than one processor (or an HT one) Apache can serve the static content thus leaving more Lisp CPU time to process application logic. BTW 14.6 request/s is very slow. Are you sure you keep the Apache/Lisp socket open ? FYI I get 440 req/s max on a dual Xeon 2GHz for a fixed reply. Marc From alceste.scalas at gmx.net Sat Aug 27 15:11:41 2005 From: alceste.scalas at gmx.net (Alceste Scalas) Date: Sat, 27 Aug 2005 17:11:41 +0200 Subject: [tbnl-devel] New macro: create-groups-bind-regex-dispatcher Message-ID: <1125155502.8768.69.camel@localhost> Hello, first of all, congratulations for TBNL :-) Here's a small macro (with a big ugly name) for TBNL, that allows to bind portions of a request URI to arguments passed to dispatch functions. (see attachment) I'm using it to create RESTful [1] URIs and web APIs for a web application I'm playing with. For example, let's say I want to publish a news archive. The "usual" way for accessing the news for August 27th, 2005 would be an URI like: http://www.news.tld/archive?year=2005&month=08&day=27 The proposed add-on allows to easily (well, actually *more* easily) manage URIs like this: http://www.news.tld/archive/2005/08/27/ The task is performed by a macro called create-groups-bind-regex-dispatcher, which takes three arguments: 1. a CL-PPCRE regex (a string, an s-expression or a scanner) with one or more register groups. It will be matched against the request URI; 2. a list of variable names that will be bound to the register groups above iff the regex matches; 3. a dispatch function that accepts keyword arguments named like the variables above. A code sample for the news archive: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - (defun archive-page (&key year month day) "Archive page" (format nil "Year: ~A; Month: ~A; Day: ~A" year month day)) (setq *dispatch-table* (list (create-groups-bind-regex-dispatcher "^\\/archive\\/(\\d{4})\\/(\\d{1,2})\\/(\\d{1,2})\\/?$" (year month day) archive-page) #'default-dispatcher)) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Under the hood, the list of variables provided to create-groups-bind-regex-dispatcher is used with cl-ppcre:register-groups-bind --- so you can apply some voodoo to the matched variables: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - (defun archive-page (&key year month day) "Archive page with integer keyword parameters" (format nil "(+ year month day) => ~D" (+ year month day))) (setq *dispatch-table* (list (create-groups-bind-regex-dispatcher "^\\/archive\\/(\\d{4})\\/(\\d{1,2})\\/(\\d{1,2})\\/?$" ((#'parse-integer year month day)) archive-page) #'default-dispatcher)) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - That's all. I hope it will be useful for you as well... Comments are welcome (please don't be too rude, it's one of my first attempts with CL :-) Regards, alceste References: [1] http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm -- Alceste Scalas -------------- next part -------------- ;;; -*- Mode: LISP; Syntax: COMMON-LISP; Package: TBNL; Base: 10 -*- ;;; $Id$ ;;; Copyright (c) 2005, Alceste Scalas. All rights reserved. ;;; Redistribution and use in source and binary forms, with or without ;;; modification, are permitted provided that the following conditions ;;; are met: ;;; * Redistributions of source code must retain the above copyright ;;; notice, this list of conditions and the following disclaimer. ;;; * Redistributions in binary form must reproduce the above ;;; copyright notice, this list of conditions and the following ;;; disclaimer in the documentation and/or other materials ;;; provided with the distribution. ;;; THIS SOFTWARE IS PROVIDED BY THE AUTHOR 'AS IS' AND ANY EXPRESSED ;;; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED ;;; WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ;;; ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY ;;; DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL ;;; DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE ;;; GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS ;;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, ;;; WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING ;;; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS ;;; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. (in-package #:tbnl) (defmacro create-groups-bind-regex-dispatcher (regex var-list page-function) "Works like CREATE-REGEX-DISPATCHER, but PAGE-FUNCTION will be called with the variables in VAR-LIST used as additional keyword arguments bound to the corresponding register groups of REGEX. Note: VAR-LIST has the same format of its omonymous in CL-PPCRE:REGISTER-GROUPS-BIND, so you can apply some magic here." `(let ((scanner (cl-ppcre:create-scanner ,regex))) (lambda (request) (cl-ppcre:register-groups-bind ,var-list (scanner (script-name request)) ;; Iff the regex matches, we return a closure that calls the ;; dispatch function with the proper keyword arguments (lambda () (,page-function ,@(build-keyword-list var-list))))))) (defun build-keyword-list (var-list) "Utility function for CREATE-GROUPS-BIND-REGEX-DISPATCHER. Build a keyword list out of the var-list, ready to be used to invoke the dispatcher function" (mapcan (lambda (var) (list (intern (symbol-name var) :keyword) var)) (cleanup-var-list var-list))) (defun cleanup-var-list (var-list) "Flatten all the \(FUNCTION VAR) entries in VAR-LIST, leaving only variable names." (loop for element in var-list if (consp element) nconc (loop for var in (rest element) collect var) else collect element)) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio ? firmata URL: From edi at agharta.de Mon Aug 29 22:49:27 2005 From: edi at agharta.de (Edi Weitz) Date: Tue, 30 Aug 2005 00:49:27 +0200 Subject: [tbnl-devel] New macro: create-groups-bind-regex-dispatcher In-Reply-To: <1125155502.8768.69.camel@localhost> (Alceste Scalas's message of "Sat, 27 Aug 2005 17:11:41 +0200") References: <1125155502.8768.69.camel@localhost> Message-ID: Hi! Sorry for the delay. On Sat, 27 Aug 2005 17:11:41 +0200, Alceste Scalas wrote: > first of all, congratulations for TBNL :-) Thanks... :) > Here's a small macro (with a big ugly name) for TBNL, that allows to > bind portions of a request URI to arguments passed to dispatch > functions. (see attachment) The macro is fine and I have in fact used something similar myself in applications I wrote. I'm a bit hesitant to add it to TBNL, though, as I'm not sure if this functionality belongs there. My gut feeling is that macros like yours are one level of abstraction above TBNL and should be in a separate library. What do others think about this? Cheers, Edi.