From edi at agharta.de Sun Oct 1 11:20:41 2006 From: edi at agharta.de (Edi Weitz) Date: Sun, 01 Oct 2006 13:20:41 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL and other Lisps (Was: TBNL survey / re-org) In-Reply-To: <451DAAF6.1000307@travislists.com> (Travis Cross's message of "Fri, 29 Sep 2006 19:23:34 -0400") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: On Fri, 29 Sep 2006 19:23:34 -0400, Travis Cross wrote: > Edi Weitz wrote: > >> LispWorks/Hunchentoot doesn't use KMRCL, so that's the plan for >> SBCL and the other Lisps as well. > > I had a feeling this was on your agenda as well. OK, here's a quick hack which tries to be a version of Hunchentoot which works with SBCL (see below for other Lisps): http://weitz.de/files/hunchentoot-sbcl-beta.tar.gz It'd be nice if SBCL users could give it a try and report what works and what doesn't. To use it, replace your TBNL folder with the one in the tarball and put the Hunchentoot folder somewhere where ASDF can find it, then use ASDF to load the Hunchentoot (not the TBNL!) system. Afterwards, you can load the TBNL-TEST system for an initial test website if you want. You start the server with (TBNL:START-SERVER :PORT xxx) In addition to the usual TBNL dependencies, you'll need FLEXI-STREAMS, Chunga, and CL+SSL. You should also patch RFC2388 as described here: http://common-lisp.net/pipermail/tbnl-devel/2005-December/000524.html (No need to patch KMRCL as it is no longer used.) The version numbers and the documentation haven't been updated, but the Hunchentoot documentation should basically apply. Here are some differences: 1. SBCL ignores WRITE-TIMEOUT and uses READ-TIMEOUT for the TIMEOUT argument of SB-BSD-SOCKETS. 2. The SETGID argument must be a number and cannot be a string. 3. You cannot provide a password for the private key file. 4. The certificate and the private key cannot be in one file. 5. The setting of *GC-INTERVAL* has no effect. So, please try with your own web apps and your own browser. Do things like chunked encoding and persistent connections work for you? Does https work for you? As for the implementation, please look into the port-sbcl.lisp and unix-sbcl.lisp files. If for specific parts you know a better way to do this, please report it - I'm not an SBCL expert. See also the GET-GID-FROM-NAME function. Users of AllegroCL, CMUCL, or OpenMCL might want to provide their own versions of the port-xxx.lisp and unix-xxx.lisp files. I have plans to add support for these implementations in the future, but you can certainly speed this up if you send me some code. This would be especially helpful for OpenMCL as I don't have a Mac, and my wife doesn't want to give me her iBook... :) Cheers, Edi. From edi at agharta.de Sun Oct 1 11:44:14 2006 From: edi at agharta.de (Edi Weitz) Date: Sun, 01 Oct 2006 13:44:14 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: (Edi Weitz's message of "Sun, 01 Oct 2006 13:20:41 +0200") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: On Sun, 01 Oct 2006 13:20:41 +0200, Edi Weitz wrote: > So, please try with your own web apps and your own browser. Do > things like chunked encoding and persistent connections work for > you? Does https work for you? I forgot to mention that a Hunchentoot server can also still cater to mod_lisp, so you can try that as well if you want. (In fact, this is the intended future of Hunchentoot/TBNL and this is how I'm partly using Hunchentoot with LispWorks now.) From travis at travislists.com Sun Oct 1 17:18:13 2006 From: travis at travislists.com (Travis Cross) Date: Sun, 01 Oct 2006 13:18:13 -0400 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: <451FF855.1020308@travislists.com> Edi Weitz wrote: >> Edi Weitz wrote: >> >>> LispWorks/Hunchentoot doesn't use KMRCL, so that's the plan for >>> SBCL and the other Lisps as well. >> I had a feeling this was on your agenda as well. > > OK, here's a quick hack which tries to be a version of Hunchentoot > which works with SBCL (see below for other Lisps): > > http://weitz.de/files/hunchentoot-sbcl-beta.tar.gz > Fantastic :) Thanks Edi. I have this setup, and I'm running through some tests now. Here's one quick observation: After killing the workers, STOP-SERVER tries to PROCESS-KILL the main listener. The main listener though, as returned in the object from START-SERVER, is actually a socket, not a thread. This is the case because START-UP-SERVER in port-sbcl.lisp returns a socket when START-SERVER is expecting a thread. The attached patch remedies this, as also takes care of closing the socket when the thread dies. Cheers, -- Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: fix-start-up-server.diff URL: From edi at agharta.de Sun Oct 1 20:11:51 2006 From: edi at agharta.de (Edi Weitz) Date: Sun, 01 Oct 2006 22:11:51 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: <451FF855.1020308@travislists.com> (Travis Cross's message of "Sun, 01 Oct 2006 13:18:13 -0400") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <451FF855.1020308@travislists.com> Message-ID: On Sun, 01 Oct 2006 13:18:13 -0400, Travis Cross wrote: > After killing the workers, STOP-SERVER tries to PROCESS-KILL the > main listener. The main listener though, as returned in the object > from START-SERVER, is actually a socket, not a thread. > > This is the case because START-UP-SERVER in port-sbcl.lisp returns a > socket when START-SERVER is expecting a thread. > > The attached patch remedies this, as also takes care of closing the > socket when the thread dies. Ah, yes, thanks. I had this in mind for a second, but it somehow managed to slip through. Seems like I never stopped the damn thing when testing it... :) Is there a guarantee in SBCL that the cleanup forms of an UNWIND-PROTECT are executed when a thread is killed? From travis at travislists.com Sun Oct 1 21:57:34 2006 From: travis at travislists.com (Travis Cross) Date: Sun, 01 Oct 2006 17:57:34 -0400 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <451FF855.1020308@travislists.com> Message-ID: <452039CE.2050406@travislists.com> Edi Weitz wrote: > Is there a guarantee in SBCL that the cleanup forms of an > UNWIND-PROTECT are executed when a thread is killed? In my experience, it seems to work most of the time, but I'd have to ask someone with more sb-thread-internals-jitsu to fill in whether there is an actual guarantee in place or not. My understanding is that because of issues with shared memory between threads, killing a thread should always be considered a bit drastic, as there is no real guarantee that it won't corrupt the entire lisp image. [1] When I write listeners such as this, to provide for a normal shutdown I close over a function that flips a flag variable telling the listener to stop, then initiates a socket connection to the listener to break the socket-accept block. [2] It seems that something like this [3] might be the correct solution here as well. For a normal shutdown, one tends to prefer to let current workers finish their tasks up anyway. Providing for both the 'clean' shutdown and the 'nuke them all and take your chances' approach would probably be the most flexible interface. Cheers, -- Travis [1] See this thread (which sf.net fragmented): http://sourceforge.net/mailarchive/forum.php?thread_id=14295329&forum_id=4134 http://sourceforge.net/mailarchive/forum.php?thread_id=14649427&forum_id=4134 http://sourceforge.net/mailarchive/forum.php?thread_id=14649430&forum_id=4134 Nikodemus Siivola: "No question about it. As long as we don't have safe-points and otherwise run in an uninterruptible state, then not just killing, but generally interrupting and even partially unwinding another thread is basically a horrible, yet dangerously seductive idea. "It is simple enough to write code that checks for "kill yourself" message, and deal with it, and that is what threaded applications should do." [2] The (currently SBCL-only) code that I use for doing this is here: http://code.tcross.org/releases/cl-listener-0.1.0.tar.gz http://code.tcross.org/releases/cl-sockets-0.1.0.tar.gz ... or for the darcs-inclined: http://darcs.tcross.org/cl-listener/ http://darcs.tcross.org/cl-sockets/ [3] Global special variables, gray streams, or conditions can of course be used to achieve the same ends. From edi at agharta.de Sun Oct 1 22:46:15 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 02 Oct 2006 00:46:15 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: <452039CE.2050406@travislists.com> (Travis Cross's message of "Sun, 01 Oct 2006 17:57:34 -0400") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <451FF855.1020308@travislists.com> <452039CE.2050406@travislists.com> Message-ID: On Sun, 01 Oct 2006 17:57:34 -0400, Travis Cross wrote: > My understanding is that because of issues with shared memory > between threads, killing a thread should always be considered a bit > drastic, as there is no real guarantee that it won't corrupt the > entire lisp image. [1] Ouch! > When I write listeners such as this, to provide for a normal > shutdown I close over a function that flips a flag variable telling > the listener to stop, then initiates a socket connection to the > listener to break the socket-accept block. [2] > > It seems that something like this [3] might be the correct solution > here as well. For a normal shutdown, one tends to prefer to let > current workers finish their tasks up anyway. Providing for both > the 'clean' shutdown and the 'nuke them all and take your chances' > approach would probably be the most flexible interface. Yeah, I'll probably implement something like this in the future. At the moment I'm just mimicking the LispWorks API where killing the server process is the documented way to stop the server. http://www.lispworks.com/documentation/lw50/LWRM/html/lwref-56.htm From travis at travislists.com Mon Oct 2 02:44:59 2006 From: travis at travislists.com (Travis Cross) Date: Sun, 01 Oct 2006 22:44:59 -0400 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: <45207D2B.6090105@travislists.com> Edi Weitz wrote: >> So, please try with your own web apps and your own browser. Do >> things like chunked encoding and persistent connections work for >> you? Does https work for you? > > I forgot to mention that a Hunchentoot server can also still cater to > mod_lisp, so you can try that as well if you want. (In fact, this is > the intended future of Hunchentoot/TBNL and this is how I'm partly > using Hunchentoot with LispWorks now.) I would still likely use mod_lisp2 (or mod_proxy) for certain deployments as well, so this is certainly a useful perk. It appears that the destiny of TBNL and Hunchentoot is to merge into one greater whole. Behind mod_lisp2, I observe the following things: The "Binary data, delivered from file" test downloads the picture, but it takes a long time for the browser to close its connection to the server. Refreshing the connection, Firefox flashes a warning that the picture cannot be displayed because it contains errors, but then shows the picture anyway, then hangs for awhile. The "Binary data, delivered from RAM" test seems to work fine. The "UTF-8 test file" test seems to work... eventually. In this case, it seems to take awhile before the last part of the file is delivered to the browser. The "source code of this test" example shows the same behavior. File uploads, both text and binary, seem to upload ok. When trying to download the uploaded file, however, it takes 15+ seconds before the browser will offer to save the file. None of these issue appear when I access tbnl/hunchentoot directly. Cheers, -- Travis From travis at travislists.com Mon Oct 2 04:57:47 2006 From: travis at travislists.com (Travis Cross) Date: Mon, 02 Oct 2006 00:57:47 -0400 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps (Was: TBNL survey / re-org) In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: <45209C4B.9070700@travislists.com> Edi Weitz wrote: > As for the implementation, please look into the port-sbcl.lisp and > unix-sbcl.lisp files. If for specific parts you know a better way to > do this, please report it - I'm not an SBCL expert. See also the > GET-GID-FROM-NAME function. I believe that the *CURRENT-PROCESS* symbol macros do not need to quote their return values. This was preventing the dead worker threads from being removed from the server object in SBCL. Cheers, -- Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: fix-current-process-symbol-macro.diff URL: From edi at agharta.de Mon Oct 2 08:22:04 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 02 Oct 2006 10:22:04 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: <45209C4B.9070700@travislists.com> (Travis Cross's message of "Mon, 02 Oct 2006 00:57:47 -0400") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <45209C4B.9070700@travislists.com> Message-ID: On Mon, 02 Oct 2006 00:57:47 -0400, Travis Cross wrote: > I believe that the *CURRENT-PROCESS* symbol macros do not need to > quote their return values. Blush... Yes, of course. Thanks, Edi. From edi at agharta.de Mon Oct 2 09:00:08 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 02 Oct 2006 11:00:08 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL and other Lisps In-Reply-To: <45207D2B.6090105@travislists.com> (Travis Cross's message of "Sun, 01 Oct 2006 22:44:59 -0400") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <45207D2B.6090105@travislists.com> Message-ID: On Sun, 01 Oct 2006 22:44:59 -0400, Travis Cross wrote: > I would still likely use mod_lisp2 (or mod_proxy) for certain > deployments as well, so this is certainly a useful perk. It appears > that the destiny of TBNL and Hunchentoot is to merge into one > greater whole. Right. > Behind mod_lisp2, I observe the following things: > > The "Binary data, delivered from file" test downloads the picture, > but it takes a long time for the browser to close its connection to > the server. Refreshing the connection, Firefox flashes a warning > that the picture cannot be displayed because it contains errors, but > then shows the picture anyway, then hangs for awhile. The "Binary > data, delivered from RAM" test seems to work fine. > > The "UTF-8 test file" test seems to work... eventually. In this > case, it seems to take awhile before the last part of the file is > delivered to the browser. The "source code of this test" example > shows the same behavior. > > File uploads, both text and binary, seem to upload ok. When trying > to download the uploaded file, however, it takes 15+ seconds before > the browser will offer to save the file. > > None of these issue appear when I access tbnl/hunchentoot directly. Yeah, my bad, sorry. I forgot to mention (I think this is also in the Hunchentoot docs somewhere) that you have to start the Hunchentoot server with START-TBNL instead of START-SERVER if it's going to sit behind mod_lisp. This part of the API is going to change, though. From edi at agharta.de Mon Oct 2 11:10:56 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 02 Oct 2006 13:10:56 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL, AllegroCL, CMUCL, OpenMCL (Was: Hunchentoot for SBCL and other Lisps) In-Reply-To: (Edi Weitz's message of "Sun, 01 Oct 2006 13:20:41 +0200") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: Here now is a new beta of Hunchentoot which incorporates the bugfixes sent by Travis Cross and which also contains code for AllegroCL, CMUCL, and OpenMCL. The OpenMCL code hasn't been tested at all, the AllegroCL and CMUCL code has been tested only very mildly. http://weitz.de/files/hunchentoot-beta.tar.gz See earlier emails in this thread for instructions. For CMUCL, make sure that you have the newest (0.6.3) version of FLEXI-STREAMS. From erik.enge at gmail.com Mon Oct 2 14:12:42 2006 From: erik.enge at gmail.com (Erik Enge) Date: Mon, 2 Oct 2006 10:12:42 -0400 Subject: [tbnl-devel] Re: New version 0.11.3 (Was: Add hook before uploading a file.) In-Reply-To: References: <58f839b70609171145i490568d5kc6b2814e9cbade0e@mail.gmail.com> <58f839b70609171633q1198cf71o3ebfcaac1ab2fb27@mail.gmail.com> Message-ID: <58f839b70610020712w120b33adt9808d3485c8c58ee@mail.gmail.com> On 9/30/06, Edi Weitz wrote: > I've added that now - see doc entry for *FILE-UPLOAD-HOOK*. Please > try if it works for you. I think that's the best I can do ATM without > modifying the RFC2388 code. Great, this works as expected. Thanks! Erik. From ocorrain at yahoo.com Tue Oct 3 16:43:18 2006 From: ocorrain at yahoo.com (=?utf-8?B?VGlhcm7DoW4gw5MgQ29ycsOhaW4=?=) Date: Tue, 03 Oct 2006 17:43:18 +0100 Subject: [tbnl-devel] Flexi-streams needs a change for openmcl Message-ID: Hi-- OpenMCL runs into some problems with having +latin-1+ defined as a constant, since external-format doesn't have a load-form defined. I've added make-load-form to external-format.lisp in flexi-streams, thusly: --- external-format.lisp~ 2006-06-13 00:26:12.000000000 +0100 +++ external-format.lisp 2006-10-03 17:27:15.000000000 +0100 @@ -54,6 +54,15 @@ (:documentation "EXTERNAL-FORMAT objects are used to denote encodings for flexi streams.")) +(defmethod make-load-form ((self external-format) &optional env) + (declare (ignore env)) + `(make-instance 'external-format + :name ,(external-format-name self) + :id ,(external-format-id self) + :little-endian ,(external-format-little-endian self) + :eol-style ,(external-format-eol-style self))) + + (defun make-external-format% (name &key (little-endian *default-little-endian*) id eol-style) "Used internally by MAKE-EXTERNAL-FORMAT." -- Tiarn?n From edi at agharta.de Tue Oct 3 17:29:31 2006 From: edi at agharta.de (Edi Weitz) Date: Tue, 03 Oct 2006 19:29:31 +0200 Subject: [tbnl-devel] Flexi-streams needs a change for openmcl In-Reply-To: =?iso-8859-1?q?=28Tiarn=E1n_=D3_Corr=E1in's?= message of "Tue, 03 Oct 2006 17:43:18 +0100") References: Message-ID: On Tue, 03 Oct 2006 17:43:18 +0100, ocorrain at yahoo.com (Tiarn?n ? Corr?in) wrote: > OpenMCL runs into some problems with having +latin-1+ defined as a > constant, since external-format doesn't have a load-form defined. Seems you're not using the latest version. http://weitz.de/flexi-streams/CHANGELOG Cheers, Edi. From ocorrain at yahoo.com Tue Oct 3 22:11:56 2006 From: ocorrain at yahoo.com (=?utf-8?B?VGlhcm7DoW4gw5MgQ29ycsOhaW4=?=) Date: Tue, 03 Oct 2006 23:11:56 +0100 Subject: [tbnl-devel] Flexi-streams needs a change for openmcl In-Reply-To: (Edi Weitz's message of "Tue, 03 Oct 2006 19:29:31 +0200") References: Message-ID: >>>>> "EW" == Edi Weitz writes: EW> Seems you're not using the latest version. Ah... -- Tiarn?n From graham.fawcett at gmail.com Wed Oct 4 14:40:31 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Wed, 4 Oct 2006 10:40:31 -0400 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? Message-ID: Hi folks, Is anyone still encountering problems with binary uploads using TBNL with SBCL and the mod_lisp front-end? I've applied Travis Cross' patches to KMRCL and RFC2388. But in spite of the patches, I'm seeing errors when trying to upload a binary file to the tbnl-test demo application. For example: # (:EXTERNAL-FORMAT :UTF-8): the octet sequence (226 227 207) cannot be decoded. I'm using SBCL 0.9.15 on Linux 2.6; TBNL 0.11.3, KMRCL-1.89, and a fresh RFC2388, with the Cross patches applied. *features* and SLIME backtrace are below. I'm happy to do more testing/debugging, but perhaps someone could help point me in the right direction. Thanks, Graham *features* (:CL-WHO :TBNL :TBNL-SBCL-DEBUG-PRINT-VARIABLE-ALIST :TBNL-BIVALENT-STREAMS :URL-REWRITE :KMR-NORMAL-DSDC :KMR-NORMAL-CESD :CL-PPCRE :KMR-MOP :ASDF :SB-THREAD :ANSI-CL :COMMON-LISP :SBCL :UNIX :SB-DOC :SB-TEST :SB-LDB :SB-PACKAGE-LOCKS :SB-UNICODE :SB-SOURCE-LOCATIONS :IEEE-FLOATING-POINT :X86 :ELF :LINUX :GENCGC :STACK-GROWS-DOWNWARD-NOT-UPWARD :C-STACK-IS-CONTROL-STACK :STACK-ALLOCATABLE-CLOSURES :ALIEN-CALLBACKS :LINKAGE-TABLE :OS-PROVIDES-DLOPEN :OS-PROVIDES-DLADDR :OS-PROVIDES-PUTWC) Snipped backtrace: 7: (TBNL::MAYBE-INVOKE-DEBUGGER #) 8: (SIGNAL #) 9: (ERROR SB-INT:STREAM-DECODING-ERROR) 10: (SB-INT:STREAM-DECODING-ERROR # (226 227 207)) 11: (SB-IMPL::STREAM-DECODING-ERROR-AND-HANDLE # 3) 12: (SB-IMPL::INPUT-CHAR/UTF-8 # NIL NIL) 13: (READ-CHAR # NIL NIL #) 14: ((FLET RFC2388::RUN) #) 15: (RFC2388::READ-UNTIL-NEXT-BOUNDARY # "--------------------$ 16: ((SB-PCL::FAST-METHOD RFC2388:PARSE-MIME (STREAM T)) # # # "multipart/form-data; bou$ 18: ((SB-PCL::FAST-METHOD INITIALIZE-INSTANCE :AFTER (TBNL::REQUEST)) (#(NIL 1 2 0 3 5 8 4 6) . #()) #) 20: (TBNL::PROCESS-REQUEST (("content-stream" . #) ("server-pr$ 21: (TBNL::LISTEN-FOR-REQUEST # TBNL::PROCESS-REQUEST) 22: ((LAMBDA NIL)) From erik.enge at gmail.com Wed Oct 4 15:00:04 2006 From: erik.enge at gmail.com (Erik Enge) Date: Wed, 4 Oct 2006 11:00:04 -0400 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: References: Message-ID: <58f839b70610040800r566d080haea6703cee49d910@mail.gmail.com> On 10/4/06, Graham Fawcett wrote: > Is anyone still encountering problems with binary uploads using TBNL > with SBCL and the mod_lisp front-end? I've applied Travis Cross' > patches to KMRCL and RFC2388. But in spite of the patches, I'm seeing > errors when trying to upload a binary file to the tbnl-test demo > application. For example: I'm afraid I can't be very helpful other than to point out that it works with my setup: TBNL 0.11.3, SBCL 0.9.13, RFC2388 with Travis' patches and KMRCL 1.87. Thanks, Erik. From edi at agharta.de Wed Oct 4 15:17:47 2006 From: edi at agharta.de (Edi Weitz) Date: Wed, 04 Oct 2006 17:17:47 +0200 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: (Graham Fawcett's message of "Wed, 4 Oct 2006 10:40:31 -0400") References: Message-ID: On Wed, 4 Oct 2006 10:40:31 -0400, "Graham Fawcett" wrote: > Is anyone still encountering problems with binary uploads using TBNL > with SBCL and the mod_lisp front-end? I've applied Travis Cross' > patches to KMRCL and RFC2388. Probably not relevant in this case, but I think the latest version of RFC2388 doesn't need patches for SBCL anymore. > But in spite of the patches, I'm seeing errors when trying to upload > a binary file to the tbnl-test demo application. For example: > > # > (:EXTERNAL-FORMAT :UTF-8): > the octet sequence > (226 227 207) cannot be decoded. > > I'm using SBCL 0.9.15 on Linux 2.6; TBNL 0.11.3, KMRCL-1.89, and a > fresh RFC2388, with the Cross patches applied. *features* and SLIME > backtrace are below. I'm happy to do more testing/debugging, but > perhaps someone could help point me in the right direction. If it's not urgent, you should take a look at the Hunchentoot beta which is supposed to replace TBNL soon. I hope the issues with SBCL will vanish with this release. If not, I'd like to know. See the "Hunchentoot for SBCL..." thread here http://common-lisp.net/pipermail/tbnl-devel/2006-October/thread.html and use the second beta announced on October 2. Cheers, Edi. From ignas.mikalajunas at gmail.com Wed Oct 4 16:02:37 2006 From: ignas.mikalajunas at gmail.com (Ignas Mikalajunas) Date: Wed, 4 Oct 2006 19:02:37 +0300 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: References: Message-ID: > If it's not urgent, you should take a look at the Hunchentoot beta > which is supposed to replace TBNL soon. I hope the issues with SBCL > will vanish with this release. If not, I'd like to know. > > See the "Hunchentoot for SBCL..." thread here > > http://common-lisp.net/pipermail/tbnl-devel/2006-October/thread.html > > and use the second beta announced on October 2. > > Cheers, > Edi. Hunchentoot beta allows uploading binary files with SBCL 0.9.16. everything works nicely. Than you very much. The only part of tbnl that does not play nicely with my setup is the usage of (load-time-value *load-pathname*) as it is not working with asdf-binary-locations nicely. I really want my fasl files in the different place from my lisp files (constant upgrades to sbcl become more difficult otherwise), but tbnl-test complains about not being able to find fz.jpg (sbcl it tries to find near fasl files). For my own libraries i am using #.(or *compile-file-truename* *load-truename*) which kind of works on sbcl, though i am not 100% sure it is the right way if it would work on ACL and lispworks (i do not have these implementations) i can send you the patch. Ignas From edi at agharta.de Wed Oct 4 16:31:35 2006 From: edi at agharta.de (Edi Weitz) Date: Wed, 04 Oct 2006 18:31:35 +0200 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: (Ignas Mikalajunas's message of "Wed, 4 Oct 2006 19:02:37 +0300") References: Message-ID: On Wed, 4 Oct 2006 19:02:37 +0300, "Ignas Mikalajunas" wrote: > The only part of tbnl that does not play nicely with my setup is the > usage of (load-time-value *load-pathname*) as it is not working with > asdf-binary-locations nicely. I really want my fasl files in the > different place from my lisp files (constant upgrades to sbcl become > more difficult otherwise), but tbnl-test complains about not being > able to find fz.jpg (sbcl it tries to find near fasl files). For my > own libraries i am using #.(or *compile-file-truename* > *load-truename*) which kind of works on sbcl, though i am not 100% > sure it is the right way if it would work on ACL and lispworks (i do > not have these implementations) i can send you the patch. Thanks, I've updated my local source tree accordingly. Will be in the next release. I /think/ (load-time-value (or #.*compile-file-pathname* *load-pathname*)) should be a portable solution. From graham.fawcett at gmail.com Wed Oct 4 17:26:37 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Wed, 4 Oct 2006 13:26:37 -0400 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: References: Message-ID: On 10/4/06, Edi Weitz wrote: > If it's not urgent, you should take a look at the Hunchentoot beta > which is supposed to replace TBNL soon. I hope the issues with SBCL > will vanish with this release. If not, I'd like to know. Sure, it's not urgent. :-) I downloaded the second beta, followed the build instructions, and I get the following error when I visit the app with my browser: invalid structure layout: A test for class STREAM was passed the obsolete instance # [Condition of type SB-KERNEL:LAYOUT-INVALID] I see there is a patch to SBCL[1] that is supposed to correct this; I'll give it a try again after that. Graham [1] http://sourceforge.net/mailarchive/message.php?msg_id=36304967 Graham From graham.fawcett at gmail.com Wed Oct 4 17:42:40 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Wed, 4 Oct 2006 13:42:40 -0400 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: References: Message-ID: On 10/4/06, Graham Fawcett wrote: > Sure, it's not urgent. :-) I downloaded the second beta, followed the > build instructions, and I get the following error when I visit the app > with my browser: > > invalid structure layout: > A test for class > STREAM > was passed the obsolete instance > # > [Condition of type SB-KERNEL:LAYOUT-INVALID] > > I see there is a patch to SBCL[1] that is supposed to correct this; > I'll give it a try again after that. I upgraded SBCL from 0.9.15 to 0.9.17, and this fixed the problem. Ingas reported that it is also working on 0.9.16: so that's the minimum. Initial impressions of hunchentoot on SBCL: w00t! My binary uploads are working! :-) I'll do some performance tests, but so far this seems to be working very well. Thanks so much! Graham From graham.fawcett at gmail.com Wed Oct 4 19:50:59 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Wed, 4 Oct 2006 15:50:59 -0400 Subject: [tbnl-devel] SBCL, hunchentoot, performance Message-ID: Hi again folks, Maybe it's a bit early to be asking about hunchentoot, sbcl and performance... but heck, let me ask anyway. :-) Trying the hunchentoot beta on SBCL 0.9.17, I ran tbnl-test, and uploaded a ~0.75MB binary file. (Which worked!) Testing with 'ab', I downloaded the file -- but I wasn't able to get a download faster than 359KB/sec, or 2 seconds per download, on a fast network. I also observed >95% CPU activity during the download (about 50/50 between user and system). Profiling the :TBNL package, and downloading the file again, showed this: seconds | consed | calls | sec/call | name -------------------------------------------------------- 1.740 | 35,748,752 | 1 | 1.739901 | HANDLE-STATIC-FILE 0.004 | 8,184 | 1 | 0.003998 | TBNL::READ-HTTP-HEADERS 0.004 | 24,912 | 6 | 0.000665 | TBNL::WRITE-HEADER-LINE/HTTP and profiling :FLEX instead, then repeating the download, showed: seconds | consed | calls | sec/call | name ------------------------------------------------------------- 0.009 | 81,920 | 352 | 0.000027 | FLEXI-STREAMS::READ-BYTE* 0.000 | 106,051,824 | 1,488,124 | 0.000000 | FLEXI-STREAMS::WRITE-BYTE* The consing numbers seem pretty high. :-) I'm not sure how to interpret the 0.0 seconds for the write-byte* calls -- a hundred million conses might be fast, but not instantaneous! It's notable that the reported "profiling overhead" was ~15 seconds, perhaps it was just a profiling error. (I'm not a SLIME-profiling expert -- advice is welcome.) Is this degree of performance similar to what you see under Lispworks? I'm not throwing stones at beta code, just trying to interpret what I'm seeing here. Thanks, Graham From edi at agharta.de Wed Oct 4 21:43:08 2006 From: edi at agharta.de (Edi Weitz) Date: Wed, 04 Oct 2006 23:43:08 +0200 Subject: [tbnl-devel] tbnl, sbcl and binary uploads: still a problem? In-Reply-To: (Graham Fawcett's message of "Wed, 4 Oct 2006 13:42:40 -0400") References: Message-ID: On Wed, 4 Oct 2006 13:42:40 -0400, "Graham Fawcett" wrote: > I'll do some performance tests, but so far this seems to be working > very well. Great! I expect performance to be a bit worse than "pure" TBNL because now two Gray streams are wrapped around the socket stream, but I don't think this'll be a problem (unless you are in the Google ballpark). Both FLEXI-STREAMS and CHUNGA haven't been optimized for speed yet, so there's enough room for improvement if necessary. From edi at agharta.de Wed Oct 4 21:51:38 2006 From: edi at agharta.de (Edi Weitz) Date: Wed, 04 Oct 2006 23:51:38 +0200 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: (Graham Fawcett's message of "Wed, 4 Oct 2006 15:50:59 -0400") References: Message-ID: I replied to your other email before I saw this one. On Wed, 4 Oct 2006 15:50:59 -0400, "Graham Fawcett" wrote: > Maybe it's a bit early to be asking about hunchentoot, sbcl and > performance... but heck, let me ask anyway. :-) > > Trying the hunchentoot beta on SBCL 0.9.17, I ran tbnl-test, and > uploaded a ~0.75MB binary file. (Which worked!) Testing with 'ab', I > downloaded the file -- but I wasn't able to get a download faster > than 359KB/sec, or 2 seconds per download, on a fast network. I also > observed >95% CPU activity during the download (about 50/50 between > user and system). > > Profiling the :TBNL package, and downloading the file again, showed this: > > seconds | consed | calls | sec/call | name > -------------------------------------------------------- > 1.740 | 35,748,752 | 1 | 1.739901 | HANDLE-STATIC-FILE > 0.004 | 8,184 | 1 | 0.003998 | TBNL::READ-HTTP-HEADERS > 0.004 | 24,912 | 6 | 0.000665 | TBNL::WRITE-HEADER-LINE/HTTP > > and profiling :FLEX instead, then repeating the download, showed: > > seconds | consed | calls | sec/call | name > ------------------------------------------------------------- > 0.009 | 81,920 | 352 | 0.000027 | FLEXI-STREAMS::READ-BYTE* > 0.000 | 106,051,824 | 1,488,124 | 0.000000 | FLEXI-STREAMS::WRITE-BYTE* > > The consing numbers seem pretty high. :-) I'm not sure how to > interpret the 0.0 seconds for the write-byte* calls -- a hundred > million conses might be fast, but not instantaneous! It's notable > that the reported "profiling overhead" was ~15 seconds, perhaps it > was just a profiling error. (I'm not a SLIME-profiling expert -- > advice is welcome.) > > Is this degree of performance similar to what you see under > Lispworks? I'm not throwing stones at beta code, just trying to > interpret what I'm seeing here. I haven't really checked the performance yet - I usually try to make it work correctly first. If you want to make the whole stuff faster, I think starting with FLEXI-STREAMS is a good idea. Throw a couple of optimization declarations at SBCL and see what Python has to say about them. CHUNGA might be a worthwhile target as well. From edi at agharta.de Wed Oct 4 23:19:22 2006 From: edi at agharta.de (Edi Weitz) Date: Thu, 05 Oct 2006 01:19:22 +0200 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: (Graham Fawcett's message of "Wed, 4 Oct 2006 15:50:59 -0400") References: Message-ID: On Wed, 4 Oct 2006 15:50:59 -0400, "Graham Fawcett" wrote: > seconds | consed | calls | sec/call | name > ------------------------------------------------------------- > 0.009 | 81,920 | 352 | 0.000027 | FLEXI-STREAMS::READ-BYTE* > 0.000 | 106,051,824 | 1,488,124 | 0.000000 | FLEXI-STREAMS::WRITE-BYTE* > > The consing numbers seem pretty high. :-) It just occurred to me that I know what the reason for this excessive consing is, and I think I also know how to fix it. I'll do that tomorrow - have to go to bed now... :) From edi at agharta.de Thu Oct 5 07:05:33 2006 From: edi at agharta.de (Edi Weitz) Date: Thu, 05 Oct 2006 09:05:33 +0200 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: (Edi Weitz's message of "Thu, 05 Oct 2006 01:19:22 +0200") References: Message-ID: On Thu, 05 Oct 2006 01:19:22 +0200, Edi Weitz wrote: > It just occurred to me that I know what the reason for this > excessive consing is, and I think I also know how to fix it. I'll > do that tomorrow - have to go to bed now... :) Please try with the latest (0.6.4) version of FLEXI-STREAMS. That should make a difference. Cheers, Edi. From graham.fawcett at gmail.com Thu Oct 5 15:24:26 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Thu, 5 Oct 2006 11:24:26 -0400 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: References: Message-ID: On 10/5/06, Edi Weitz wrote: > On Thu, 05 Oct 2006 01:19:22 +0200, Edi Weitz wrote: > > > It just occurred to me that I know what the reason for this > > excessive consing is, and I think I also know how to fix it. I'll > > do that tomorrow - have to go to bed now... :) > > Please try with the latest (0.6.4) version of FLEXI-STREAMS. That > should make a difference. I updated flexi-streams (and trivial-gray-streams, since the new flexi-streams required an update of it). I see quite an improvement: seconds | consed | calls | sec/call | name ----------------------------------------------------- 1.044 | 36,832 | 1 | 1.043898 | TBNL:HANDLE-STATIC-FILE 0.000 | 0 | 1 | 0.000000 | TBNL::READ-HTTP-HEADERS 0.000 | 28,664 | 6 | 0.000000 | TBNL::WRITE-HEADER-LINE/HTTP Reduced the conses for handle-static-file by 90%. On the FLEX side: seconds | consed | calls | sec/call | name ------------------------------------------------------------ 0.000 | 71,507,496 | 1,488,124 | 0.000000 | FLEXI-STREAMS::WRITE-BYTE* 0.000 | 49,152 | 349 | 0.000000 | FLEXI-STREAMS::READ-BYTE* 71M conses down from 106M conses on the write-byte* call; 45M down from 81M on read-byte*. According to the wall-clock (never trust the wall-clock!), I'm getting downloads of my large binary at ~1800KB/sec, an apparent three-fold improvement. Thanks! Though not exactly fast yet, this is useable for development purposes. I'll play with some optimization-declarations, as you suggested, and see if I can squeeze out a bit more performance. Regards, Graham From sergio.garcia at gmail.com Thu Oct 5 20:18:40 2006 From: sergio.garcia at gmail.com (Sergio Garcia) Date: Thu, 5 Oct 2006 22:18:40 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL and other Lisps (Was: TBNL survey / re-org) In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: <34ec25750610051318n5235bc60iae6f817254cd4edf@mail.gmail.com> > It'd be nice if SBCL users could give it a try and report what works > and what doesn't. So far, I see everything working right with my little application running on hunchentoot/sbcl. Of course only a couple of friends of mine use it, so its not a real test :) : http://crazyrobot.hd.free.fr:9000/tol10/ (tbnl+ajax stuff: click on the arrows.). Thanks Edi. Sergio From edi at agharta.de Thu Oct 5 20:27:20 2006 From: edi at agharta.de (Edi Weitz) Date: Thu, 05 Oct 2006 22:27:20 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL and other Lisps In-Reply-To: <34ec25750610051318n5235bc60iae6f817254cd4edf@mail.gmail.com> (Sergio Garcia's message of "Thu, 5 Oct 2006 22:18:40 +0200") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <34ec25750610051318n5235bc60iae6f817254cd4edf@mail.gmail.com> Message-ID: On Thu, 5 Oct 2006 22:18:40 +0200, "Sergio Garcia" wrote: > http://crazyrobot.hd.free.fr:9000/tol10/ I can't connect. I hope that's not a bad sign... :) From sergio.garcia at gmail.com Thu Oct 5 20:39:37 2006 From: sergio.garcia at gmail.com (Sergio Garcia) Date: Thu, 5 Oct 2006 22:39:37 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL and other Lisps In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <34ec25750610051318n5235bc60iae6f817254cd4edf@mail.gmail.com> Message-ID: <34ec25750610051339t11a09e7bj57292465b6f6f1b1@mail.gmail.com> ooops :P no, it was my fault. I meant: http://crazyrobot.hd.free.fr:4000/tol10/ Sergio On 10/5/06, Edi Weitz wrote: > On Thu, 5 Oct 2006 22:18:40 +0200, "Sergio Garcia" > wrote: > > > http://crazyrobot.hd.free.fr:9000/tol10/ > > I can't connect. I hope that's not a bad sign... :) > From edi at agharta.de Thu Oct 5 21:00:21 2006 From: edi at agharta.de (Edi Weitz) Date: Thu, 05 Oct 2006 23:00:21 +0200 Subject: [tbnl-devel] Hunchentoot for SBCL and other Lisps In-Reply-To: <34ec25750610051339t11a09e7bj57292465b6f6f1b1@mail.gmail.com> (Sergio Garcia's message of "Thu, 5 Oct 2006 22:39:37 +0200") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> <34ec25750610051318n5235bc60iae6f817254cd4edf@mail.gmail.com> <34ec25750610051339t11a09e7bj57292465b6f6f1b1@mail.gmail.com> Message-ID: On Thu, 5 Oct 2006 22:39:37 +0200, "Sergio Garcia" wrote: > http://crazyrobot.hd.free.fr:4000/tol10/ Thanks, now it works fine. The Ajax stuff looks cool! From edi at agharta.de Fri Oct 6 07:23:24 2006 From: edi at agharta.de (Edi Weitz) Date: Fri, 06 Oct 2006 09:23:24 +0200 Subject: [tbnl-devel] Re: Hunchentoot for SBCL, AllegroCL, CMUCL, OpenMCL In-Reply-To: (Edi Weitz's message of "Mon, 02 Oct 2006 13:10:56 +0200") References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: On Mon, 02 Oct 2006 13:10:56 +0200, Edi Weitz wrote: > http://weitz.de/files/hunchentoot-beta.tar.gz There's now a newer beta at the same location. No guarantees, though - don't throw the old beta away yet if it worked for you. START-SERVER has a new keyword argument :MOD-LISP-P now. If it's NIL, the server is a stand-alone web server; if it's not NIL, the server acts as a back-end for mod_lisp. START-TBNL is gone. From edi at agharta.de Fri Oct 6 07:15:28 2006 From: edi at agharta.de (Edi Weitz) Date: Fri, 06 Oct 2006 09:15:28 +0200 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: (Graham Fawcett's message of "Thu, 5 Oct 2006 11:24:26 -0400") References: Message-ID: On Thu, 5 Oct 2006 11:24:26 -0400, "Graham Fawcett" wrote: > Though not exactly fast yet, this is useable for development > purposes. Please try the just-released latest version of FLEXI-STREAMS (0.6.5) and see if you see more improvements. From edi at agharta.de Fri Oct 6 12:07:54 2006 From: edi at agharta.de (Edi Weitz) Date: Fri, 06 Oct 2006 14:07:54 +0200 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: (Edi Weitz's message of "Fri, 06 Oct 2006 09:15:28 +0200") References: Message-ID: On Fri, 06 Oct 2006 09:15:28 +0200, Edi Weitz wrote: > Please try the just-released latest version of FLEXI-STREAMS (0.6.5) > and see if you see more improvements. Sorry, for the noise today, but a) FLEXI-STREAMS 0.6.5 was buggy, so you should use 0.6.6, and b) you should also use Chunga 0.2.0 for (hopefully) more performance improvements. If all goes as planned, we probably don't need those ugly declarations to increase the speed of Hunchentoot. Expect more betas and bugs this weekend... :) From graham.fawcett at gmail.com Fri Oct 6 15:04:25 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Fri, 6 Oct 2006 11:04:25 -0400 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: References: Message-ID: On 10/6/06, Edi Weitz wrote: > Sorry, for the noise today, but a) FLEXI-STREAMS 0.6.5 was buggy, so > you should use 0.6.6, and b) you should also use Chunga 0.2.0 for > (hopefully) more performance improvements. > > Expect more betas and bugs this weekend... :) I look forward to them! :-) Thanks very much, I'll keep testing. Graham From travis at travislists.com Fri Oct 6 15:42:54 2006 From: travis at travislists.com (Travis Cross) Date: Fri, 06 Oct 2006 11:42:54 -0400 Subject: [tbnl-devel] Re: Hunchentoot for SBCL, AllegroCL, CMUCL, OpenMCL In-Reply-To: References: <451D9B3A.60001@officetone.com> <451DAAF6.1000307@travislists.com> Message-ID: <4526797E.8000905@travislists.com> Edi Weitz wrote: > On Mon, 02 Oct 2006 13:10:56 +0200, Edi Weitz wrote: > >> http://weitz.de/files/hunchentoot-beta.tar.gz > > There's now a newer beta at the same location. No guarantees, though > - don't throw the old beta away yet if it worked for you. Initial indications on the merged library look good. All the tests pass on my setup. Cheers, -- Travis From graham.fawcett at gmail.com Fri Oct 6 20:11:32 2006 From: graham.fawcett at gmail.com (Graham Fawcett) Date: Fri, 6 Oct 2006 16:11:32 -0400 Subject: [tbnl-devel] SBCL, hunchentoot, performance In-Reply-To: References: Message-ID: On 10/6/06, Edi Weitz wrote: > Sorry, for the noise today, but a) FLEXI-STREAMS 0.6.5 was buggy, so > you should use 0.6.6, and b) you should also use Chunga 0.2.0 for > (hopefully) more performance improvements. You weren't kidding! I've just installed the updated Flexi-streams and Chunga. Now I'm seeing, oh, about a 150x improvement! No time right now for any real stats, but here's the dump of a (localhost) 'ab' test: ------ $ ab -n 250 http://localhost:3000/tbnl/test/files/imgp2926.jpg .... Server Software: Hunchentoot Server Hostname: localhost Server Port: 3000 Document Path: /tbnl/test/files/imgp2926.jpg .... Document Length: 812011 bytes Concurrency Level: 1 Time taken for tests: 2.928872 seconds Complete requests: 250 Failed requests: 0 Write errors: 0 Total transferred: 203049750 bytes HTML transferred: 203002750 bytes Requests per second: 85.36 [#/sec] (mean) Time per request: 11.715 [ms] (mean) Time per request: 11.715 [ms] (mean, across all concurrent requests) Transfer rate: 67701.83 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 0 Processing: 6 11 6.6 7 54 Waiting: 2 3 2.7 2 33 Total: 6 11 6.6 7 54 Percentage of the requests served within a certain time (ms) 50% 7 66% 15 75% 15 80% 15 90% 16 95% 17 98% 20 99% 43 100% 54 (longest request) ------ Brililant! Cheers, Graham From edi at agharta.de Sun Oct 8 23:25:22 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 09 Oct 2006 01:25:22 +0200 Subject: [tbnl-devel] Hunchentoot beta no. 4 Message-ID: http://weitz.de/files/hunchentoot-beta.tar.gz Use at your own risk, as usual. FYI, I'm currently re-factoring the code (and changing parts of the API while I'm at it). I do this with LispWorks on Windows 99% of the time, so I'm not even sure if the code still compiles on SBCL or other Lisps. I'm happy if you send me notes about problems you're encountering. And, sorry, the documentation still hasn't been updated. Cheers, Edi. From travis at travislists.com Mon Oct 9 02:37:21 2006 From: travis at travislists.com (Travis Cross) Date: Sun, 08 Oct 2006 22:37:21 -0400 Subject: [tbnl-devel] Hunchentoot beta no. 4 In-Reply-To: References: Message-ID: <4529B5E1.6060005@travislists.com> Edi Weitz wrote: > http://weitz.de/files/hunchentoot-beta.tar.gz > > Use at your own risk, as usual. > > FYI, I'm currently re-factoring the code (and changing parts of the > API while I'm at it). I do this with LispWorks on Windows 99% of the > time, so I'm not even sure if the code still compiles on SBCL or other > Lisps. I'm happy if you send me notes about problems you're > encountering. Initial indications look good on SBCL. One note about test.lisp: The output from the LATIN-1 / UTF-8 tests is being escaped twice, once by ESC in INFO-TABLE and once by ESCAPE-STRING-ALL in PARAMETER-TEST. Cheers, -- Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: test-double-escape-fix.diff URL: From travis at travislists.com Mon Oct 9 03:34:05 2006 From: travis at travislists.com (Travis Cross) Date: Sun, 08 Oct 2006 23:34:05 -0400 Subject: API addition: REDIRECT-URL (was: [tbnl-devel] Hunchentoot beta no. 4) In-Reply-To: References: Message-ID: <4529C32D.8070202@travislists.com> Edi Weitz wrote: > FYI, I'm currently re-factoring the code (and changing parts of the > API while I'm at it). Here is a small API change that's been sitting in my queue for awhile: REDIRECT currently builds a full URL out of the script name, the host, and one of two hard-coded protocols. This is useful when you are redirecting to another HTTP/S URL on your site. Often, however, I have cases where I already know the full URL, either because the user has given it too me, or because I have it stored in a database. It seems wasteful and hazardous to break the URL apart only to put it back together again. Also, there are times when I need to redirect to protocols other than http:// or https:// (such as ftp:// or even sip://). Therefore I propose that a lower level function be slid underneath REDIRECT and exported. I've tentatively called this REDIRECT-URL. Cheers, -- Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: add-redirect-url.diff URL: From edi at agharta.de Mon Oct 9 06:27:06 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 09 Oct 2006 08:27:06 +0200 Subject: [tbnl-devel] Hunchentoot beta no. 4 In-Reply-To: <4529B5E1.6060005@travislists.com> (Travis Cross's message of "Sun, 08 Oct 2006 22:37:21 -0400") References: <4529B5E1.6060005@travislists.com> Message-ID: On Sun, 08 Oct 2006 22:37:21 -0400, Travis Cross wrote: > The output from the LATIN-1 / UTF-8 tests is being escaped twice, > once by ESC in INFO-TABLE and once by ESCAPE-STRING-ALL in > PARAMETER-TEST. Thanks. Fixed in my local CVS tree. From edi at agharta.de Mon Oct 9 06:47:15 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 09 Oct 2006 08:47:15 +0200 Subject: [tbnl-devel] Re: API addition: REDIRECT-URL In-Reply-To: <4529C32D.8070202@travislists.com> (Travis Cross's message of "Sun, 08 Oct 2006 23:34:05 -0400") References: <4529C32D.8070202@travislists.com> Message-ID: On Sun, 08 Oct 2006 23:34:05 -0400, Travis Cross wrote: > Here is a small API change that's been sitting in my queue for > awhile: Yeah, makes sense. I'll add something like that to the release. From vamlists at gmail.com Mon Oct 9 06:56:14 2006 From: vamlists at gmail.com (Vamsee Kanakala) Date: Mon, 09 Oct 2006 12:26:14 +0530 Subject: [tbnl-devel] Hunchentoot beta no. 4 In-Reply-To: References: Message-ID: <4529F28E.8040609@gmail.com> Edi Weitz wrote: > And, sorry, the documentation still hasn't been updated. > > Hi Edi, I'm have been learning cl in my spare time for the past few months, and I'm trying to build a small webapp in cl just to see how difficult/easy it would be from other web frameworks I use for regular work. I recently started playing with Hunchentoot, and I have to say it works great (I use it with relatively old sbcl 0.9.8 - default on Ubuntu), and installation was easy, even for a relative newbie (after I followed the list). If installation is not too much of a hassle for you, I think a wiki for documentation would help immensely, I and others who are interested can put in some steps that other newcomers might miss (or comments, etc). If you're so inclined, would like to help you with this in anyway I can. Thanks, Vamsee. From edi at agharta.de Mon Oct 9 07:19:23 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 09 Oct 2006 09:19:23 +0200 Subject: [tbnl-devel] Hunchentoot beta no. 4 In-Reply-To: <4529F28E.8040609@gmail.com> (Vamsee Kanakala's message of "Mon, 09 Oct 2006 12:26:14 +0530") References: <4529F28E.8040609@gmail.com> Message-ID: Hi Vamsee, On Mon, 09 Oct 2006 12:26:14 +0530, Vamsee Kanakala wrote: > I'm have been learning cl in my spare time for the past few months, > and I'm trying to build a small webapp in cl just to see how > difficult/easy it would be from other web frameworks I use for > regular work. I recently started playing with Hunchentoot, and I > have to say it works great (I use it with relatively old sbcl 0.9.8 > - default on Ubuntu), and installation was easy, even for a relative > newbie (after I followed the list). Great! > If installation is not too much of a hassle for you, I think a wiki > for documentation would help immensely, I and others who are > interested can put in some steps that other newcomers might miss (or > comments, etc). If you're so inclined, would like to help you with > this in anyway I can. I think that's a good idea, but due to my workload I wouldn't want to install and maintain a Wiki on one of my machines. I think there are already enough other Wikis that you could use for this task, though. One of them would be the CL Gardeners Wiki, or maybe CLiki. Also, Google will show you a lot of interesting links if you search for "free wiki". http://wiki.alu.org/Gardeners_Projects http://www.cliki.net/index http://www.google.com/search?q=free+wiki If you set something up, let us know and I'll link to it from the Hunchentoot website. Cheers, Edi. From djneu at att.net Mon Oct 9 17:47:58 2006 From: djneu at att.net (David J. Neu) Date: Mon, 9 Oct 2006 13:47:58 -0400 Subject: [tbnl-devel] TBNL and SBCL Message-ID: <20061009174758.GA37357@bach.davidneu.local> Hi, I know that there has been a lot of discussion about how nicely TBNL and SBCL play together, and as someone new to this list I wanted to ask a few question about the current situation. The architecture, I envision has Apache sitting "in-front", passing requests back to TBNL. In view of http://common-lisp.net/pipermail/tbnl-devel/2006-October/000710.html and http://common-lisp.net/pipermail/tbnl-devel/2006-October/000711.html do I understand correctly, that the new road ahead is that TBNL is sort of being "merged" with Hunchentoot to create an enhanced TBNL, that is less dependent on "other" packages and that this new release will reduce the issues related to TBNL/SBCL compatibility? Many thanks! Cheers, David From edi at agharta.de Mon Oct 9 18:22:53 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 09 Oct 2006 20:22:53 +0200 Subject: [tbnl-devel] TBNL and SBCL In-Reply-To: <20061009174758.GA37357@bach.davidneu.local> (David J. Neu's message of "Mon, 9 Oct 2006 13:47:58 -0400") References: <20061009174758.GA37357@bach.davidneu.local> Message-ID: On Mon, 9 Oct 2006 13:47:58 -0400, "David J. Neu" wrote: > I know that there has been a lot of discussion about how nicely TBNL > and SBCL play together, and as someone new to this list I wanted to > ask a few question about the current situation. > > The architecture, I envision has Apache sitting "in-front", passing > requests back to TBNL. > > In view of > http://common-lisp.net/pipermail/tbnl-devel/2006-October/000710.html > and > http://common-lisp.net/pipermail/tbnl-devel/2006-October/000711.html > do I understand correctly, that the new road ahead is that TBNL is > sort of being "merged" with Hunchentoot to create an enhanced TBNL, > that is less dependent on "other" packages and that this new release > will reduce the issues related to TBNL/SBCL compatibility? Yes. You can try out the latest beta if you want http://weitz.de/files/hunchentoot-beta.tar.gz or you can wait for the 0.4.0 release which I expect to be available in a few days. Cheers, Edi. From edi at agharta.de Mon Oct 9 23:45:05 2006 From: edi at agharta.de (Edi Weitz) Date: Tue, 10 Oct 2006 01:45:05 +0200 Subject: [hunchentoot-devel] New Hunchentoot version 0.4.0 (aka "good-bye TBNL") Message-ID: OK, as far as I am concerned, TBNL is hereby deprecated. If someone else wants to inherit and maintain it, they should contact me. I think that the current release of Hunchentoot (modulo errors) is a worthy successor/replacement for all implementations supported by TBNL. In particular, I expect less issues with SBCL due to the use of FLEXI-STREAMS. Please download, install, play with it, and report problems to the list: http://weitz.de/files/hunchentoot.tar.gz Note that some parts of the API have been slightly changed. Consult the docs if in doubt. And if you find mistakes in the documentation, please report them as well. Have fun, Edi. From travis at travislists.com Tue Oct 10 09:19:18 2006 From: travis at travislists.com (Travis Cross) Date: Tue, 10 Oct 2006 05:19:18 -0400 Subject: [hunchentoot-devel] New Hunchentoot version 0.4.0 (aka "good-bye TBNL") In-Reply-To: References: Message-ID: <452B6596.3010808@travislists.com> Edi Weitz wrote: > Please download, install, play with it, and report problems to the > list: > > http://weitz.de/files/hunchentoot.tar.gz The keyword conversion in the easy handler appears to not work correctly because of a mistaken call to MAKE-KEYWORD with two parameters in CONVERT-PARAMETER. It looks like maybe the interface to make-keyword changed from two parameters to one parameter plus a keyword argument while refactoring, and this call just slid by. Your easy handler implementation is, by the way, a fantastic addition to the library. This obsoletes a number of less general macros I'd been using to obtain similar results in my code. While I was reading your documentation for define-easy-handler, I kept thinking to myself, "Wow, you know it'd be great if this also did X" -- and then, in the very next paragraph, you would explain exactly how it does X just as I would have envisioned it. The macro delivers on nearly all of the possibilities that its interface subtly suggests. Cheers, -- Travis -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: fix-call-to-make-keyword.diff URL: From edi at agharta.de Tue Oct 10 09:28:51 2006 From: edi at agharta.de (Edi Weitz) Date: Tue, 10 Oct 2006 11:28:51 +0200 Subject: [hunchentoot-devel] New Hunchentoot version 0.4.1 (Was: New Hunchentoot version 0.4.0 (aka "good-bye TBNL")) In-Reply-To: <452B6596.3010808@travislists.com> (Travis Cross's message of "Tue, 10 Oct 2006 05:19:18 -0400") References: <452B6596.3010808@travislists.com> Message-ID: On Tue, 10 Oct 2006 05:19:18 -0400, Travis Cross wrote: > The keyword conversion in the easy handler appears to not work > correctly because of a mistaken call to MAKE-KEYWORD with two > parameters in CONVERT-PARAMETER. It looks like maybe the interface > to make-keyword changed from two parameters to one parameter plus a > keyword argument while refactoring, and this call just slid by. Right. Thanks for catching that. I've released a new version which fixes this. > Your easy handler implementation is, by the way, a fantastic > addition to the library. This obsoletes a number of less general > macros I'd been using to obtain similar results in my code. While I > was reading your documentation for define-easy-handler, I kept > thinking to myself, "Wow, you know it'd be great if this also did X" > -- and then, in the very next paragraph, you would explain exactly > how it does X just as I would have envisioned it. The macro > delivers on nearly all of the possibilities that its interface > subtly suggests. Great. That was the plan... :) From edi at agharta.de Tue Oct 10 18:57:23 2006 From: edi at agharta.de (Edi Weitz) Date: Tue, 10 Oct 2006 20:57:23 +0200 Subject: [hunchentoot-devel] New version 0.4.2 Message-ID: ChangeLog: Version 0.4.2 2006-10-10 No timeouts for mod_lisp servers (as in Hunchentoot 0.3.x) Download: http://weitz.de/files/hunchentoot.tar.gz Cheers, Edi. From edi at agharta.de Wed Oct 11 07:06:34 2006 From: edi at agharta.de (Edi Weitz) Date: Wed, 11 Oct 2006 09:06:34 +0200 Subject: [hunchentoot-devel] New version 0.4.3 (Was: [Lispweb] Announcement: Hunchentoot) In-Reply-To: <6A88FCB8-9343-4390-A643-ED52C4227EFB@stoye.com> (Ralf Stoye's message of "Tue, 10 Oct 2006 22:24:27 +0000") References: <6A88FCB8-9343-4390-A643-ED52C4227EFB@stoye.com> Message-ID: On Tue, 10 Oct 2006 22:24:27 +0000, Ralf Stoye wrote: > I just tested Hunchentoot on osx openmcl (ppc) Might as well be that you're the first one... :) > + `(ccl::atomic-incf-decf ,place ,delta)) Ugh, another unexported symbol in the OpenMCL port. Well, we'll see what happens... Thanks for the patches - I've made a new release which incorporates them. > further i had to adjust the path to libssl in cl+ssl, in file > cl+ssl.asd, just below (defparameter *libssl-pathname*....) insert: > #+(and :openmcl :DARWIN-TARGET) > (setf *libssl-pathname* "/usr/lib/libssl.dylib") That should be sent to the cl+ssl maintainer. Cheers, Edi. From jcm at FreeBSD-uk.eu.org Sun Oct 15 02:33:51 2006 From: jcm at FreeBSD-uk.eu.org (Jonathon McKitrick) Date: Sun, 15 Oct 2006 03:33:51 +0100 Subject: [hunchentoot-devel] Comparisons with araneida? Message-ID: <20061015023351.GA9247@dogma.freebsd-uk.eu.org> Hi all, the new release of Hunchentoot for other lisps piqued my interest. I've been deploying on araneida, but I've run into some issues that have required less than ideal workarounds. So, I just thought I'd check out Hunchentoot as an alternative. So for basic webapps with ajax functionality (XMLHttpRequest), how would you say the 2 platforms stack up? Jonathon McKitrick -- My other computer is your Windows box. From marijnh at gmail.com Mon Oct 16 08:06:55 2006 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 16 Oct 2006 10:06:55 +0200 Subject: [hunchentoot-devel] Latest rfc2388 Message-ID: Hello list, There appears to be some kind of incompatible interface change between the rfc2388 that can be found at cliki.net and the latest version of the library from common-lisp.net/project/rfc2388. After some head scratching I figured out that Hunchentoot was not working on my system because it uses the older version of rfc2388. So now I am switching between these versions of the library a lot, since UCW requires the other, newest version. Are there any plans to switch Hunchentoot to the latest version? Or do you wait until a new version is 'released'? I don't really know how the release process for rfc2388 works, or when a new version is due. Regards, Marijn -------------- next part -------------- An HTML attachment was scrubbed... URL: From edi at agharta.de Mon Oct 16 09:06:54 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 16 Oct 2006 11:06:54 +0200 Subject: [hunchentoot-devel] Latest rfc2388 In-Reply-To: (Marijn Haverbeke's message of "Mon, 16 Oct 2006 10:06:55 +0200") References: Message-ID: On Mon, 16 Oct 2006 10:06:55 +0200, "Marijn Haverbeke" wrote: > There appears to be some kind of incompatible interface change > between the rfc2388 that can be found at cliki.net and the latest > version of the library from common-lisp.net/project/rfc2388. After > some head scratching I figured out that Hunchentoot was not working > on my system because it uses the older version of rfc2388. So now I > am switching between these versions of the library a lot, since UCW > requires the other, newest version. Are there any plans to switch > Hunchentoot to the latest version? Or do you wait until a new > version is 'released'? I don't really know how the release process > for rfc2388 works, or when a new version is due. I'm not aware of any incompatible changes. In fact, Hunchentoot on SBCL definitely needs the newest version of RFC2388, and according to ViewCVS on common-lisp.net the latest change was two months ago and it was made for TBNL/Hunchentoot. It would be helpful if you could tell us what exactly is not working for you, which Lisp you are using, etc. Cheers, Edi. From marijnh at gmail.com Mon Oct 16 10:40:27 2006 From: marijnh at gmail.com (Marijn Haverbeke) Date: Mon, 16 Oct 2006 12:40:27 +0200 Subject: Fwd: [hunchentoot-devel] Latest rfc2388 In-Reply-To: References: Message-ID: (Accidentally sent this just to Edi, forwarding to list) ---------- Forwarded message ---------- From: Marijn Haverbeke Date: Oct 16, 2006 12:38 PM Subject: Re: [hunchentoot-devel] Latest rfc2388 To: Edi Weitz It would be helpful if you could tell us what exactly is not working > for you, which Lisp you are using, etc. On close inspection, the problem is not with Hunchentoot or rfc2388. It turns out I was previously using an rfc2388 from a repository mainained by the UCW people, who have apparantly branched it and made a few incompatible changes. So you can safely ignore these messages. Regards, Marijn -------------- next part -------------- An HTML attachment was scrubbed... URL: From edi at agharta.de Mon Oct 16 10:43:28 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 16 Oct 2006 12:43:28 +0200 Subject: [Marijn Haverbeke] Re: [hunchentoot-devel] Latest rfc2388 Message-ID: Forwarded to mailing list. -------------- next part -------------- An embedded message was scrubbed... From: "Marijn Haverbeke" Subject: Re: [hunchentoot-devel] Latest rfc2388 Date: Mon, 16 Oct 2006 12:38:27 +0200 Size: 3206 URL: From edi at agharta.de Mon Oct 16 10:44:30 2006 From: edi at agharta.de (Edi Weitz) Date: Mon, 16 Oct 2006 12:44:30 +0200 Subject: Fwd: [hunchentoot-devel] Latest rfc2388 In-Reply-To: (Marijn Haverbeke's message of "Mon, 16 Oct 2006 12:40:27 +0200") References: Message-ID: On Mon, 16 Oct 2006 12:40:27 +0200, "Marijn Haverbeke" wrote: > (Accidentally sent this just to Edi, forwarding to list) Ugh, sorry, I also just forwarded it before I saw this one... From jcm at FreeBSD-uk.eu.org Wed Oct 18 15:43:37 2006 From: jcm at FreeBSD-uk.eu.org (Jonathon McKitrick) Date: Wed, 18 Oct 2006 16:43:37 +0100 Subject: [hunchentoot-devel] Comparisons with araneida? Message-ID: <20061018154337.GA32637@dogma.freebsd-uk.eu.org> Does anyone have any comments on tbnl (now Hunchentoot) and araneida? I'd be interested in anyone that switched from araneida, features comparisons, stability, idiosyncrasies and so on. Jonathon McKitrick -- My other computer is your Windows box. From mike at sharedlogic.ca Sun Oct 22 16:48:47 2006 From: mike at sharedlogic.ca (Michael J. Forster) Date: Sun, 22 Oct 2006 11:48:47 -0500 Subject: [hunchentoot-devel] Bug Report: AUTHORIZATION Message-ID: <94CCA8C1-678F-449F-B755-7BB0F25438A4@sharedlogic.ca> Hi Edi, I found a bug in AUTHORIZATION. If I click the "Authorization" link on the Hunchentoot test page and submit an empty username and password, instead of prompting me again, the server returns `Internal Server Error'. In this case, the call to SPLIT returns nil, which does not match the DESTRUCTURING-BIND lambda list. My first reaction was simply to rewrite the DESTRUCTURING-BIND to permit a nil username as well as password: (destructuring-bind (&optional user password) (split ":" (base64:base64-string-to-string (subseq authorization start))) (values user password)) Then, we would get (VALUES NIL NIL), which would seem consistent with the documentation. However, I also realised that AUTHORIZATION will return just nil if the auth header doesn't exist. Should that be documented? Here's my environment: * Hunchentoot 0.4.4 (no Apache, no mod_lisp) * latest versions (as of today) of dependencies * LispWorks 4.4.6 * OS X 10.4.8 * Safari 2.0.4 * Firefox 1.5.0.7 Regards, Mike P.S. Started migrating our apps from Portable AllegroServe to Hunchentoot this past week. Very pleased with the API and performance. Thanks! -- Michael J. Forster From alemmens at xs4all.nl Sun Oct 22 20:53:42 2006 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Sun, 22 Oct 2006 22:53:42 +0200 Subject: [hunchentoot-devel] Bug Report: AUTHORIZATION In-Reply-To: <94CCA8C1-678F-449F-B755-7BB0F25438A4@sharedlogic.ca> References: <94CCA8C1-678F-449F-B755-7BB0F25438A4@sharedlogic.ca> Message-ID: Michael Forster wrote: > Hi Edi Edi is on vacation this week and probably won't be reading his email until October 29. Arthur Lemmens From mike at sharedlogic.ca Mon Oct 23 12:57:10 2006 From: mike at sharedlogic.ca (Michael J. Forster) Date: Mon, 23 Oct 2006 07:57:10 -0500 Subject: [hunchentoot-devel] Bug Report: AUTHORIZATION In-Reply-To: References: <94CCA8C1-678F-449F-B755-7BB0F25438A4@sharedlogic.ca> Message-ID: <1DCD9F42-0A0A-4693-9E25-E39A0C4FA675@sharedlogic.ca> On 2006-10-22, at 15:53, Arthur Lemmens wrote: > Michael Forster wrote: > >> Hi Edi > > Edi is on vacation this week and probably won't be reading his > email until October 29. > > Arthur Lemmens > Thanks, Arthur. -- Michael J. Forster From vamlists at gmail.com Mon Oct 23 12:57:33 2006 From: vamlists at gmail.com (Vamsee Kanakala) Date: Mon, 23 Oct 2006 18:27:33 +0530 Subject: [hunchentoot-devel] How to use/generate/access javascript/css? Message-ID: <453CBC3D.706@gmail.com> Hi all, Sorry for the newbie questions, I'm curious as to how people working with tbnl/hunchentoot create/maintain their javascript and css files. If you use js/css generators, what are those (I know of only parenscript). If I'm using cl-who to generate my html, how can I include a .css or .js file in this code (or, where do I need to place these files to be accessible by hunchentoot? TIA, Vamsee. From tobia.conforto at linux.it Mon Oct 23 14:50:31 2006 From: tobia.conforto at linux.it (Toby) Date: Mon, 23 Oct 2006 16:50:31 +0200 Subject: [hunchentoot-devel] How to use/generate/access javascript/css? In-Reply-To: <453CBC3D.706@gmail.com> References: <453CBC3D.706@gmail.com> Message-ID: <20061023145031.GB3038@localhost.localdomain> Vamsee Kanakala wrote: > Sorry for the newbie questions, I'm curious as to how people working > with tbnl/hunchentoot create/maintain their javascript and css files. I mostly use static css and js files. I put them in the root directory of my website and then create a handler for those specific extensions*. If I have more than a few I simply put them into css/ and js/ folders. To include them, just generate the relevant HTML code: (:html (:head (:link :href "myfile.css" :rel "stylesheet" :type "text/css") (:script :src "myfile.js" :type "text/javascript" nil) ... The last nil creates which works better than