From stesch at no-spoon.de Wed Sep 1 15:33:20 2004 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 1 Sep 2004 17:33:20 +0200 Subject: [tbnl-devel] Problems with SBCL 0.8.14 In-Reply-To: <20040831145830.GO28832@parsec.no-spoon.de> References: <20040831132145.GL28832@parsec.no-spoon.de> <87u0uj77f5.fsf@bird.agharta.de> <20040831134805.GM28832@parsec.no-spoon.de> <20040831144747.GN28832@parsec.no-spoon.de> <20040831145830.GO28832@parsec.no-spoon.de> Message-ID: <20040901153320.GL28832@parsec.no-spoon.de> On 2004-08-31 16:58:30, Stefan Scholl wrote: > On 2004-08-31 16:47:47, Stefan Scholl wrote: > > It's in function encode-session-string (session.lisp). > > (md5:md5sum-sequence "foo") > > --> segmentation violation at #X926BC80 OK, that's a bug in the current version 1.3.8 of md5. Until it's fixed TBNL can't be used together with SBCL 0.8.14. http://article.gmane.org/gmane.lisp.steel-bank.devel/3582 From xach at xach.com Wed Sep 1 15:35:21 2004 From: xach at xach.com (Zach Beane) Date: Wed, 1 Sep 2004 11:35:21 -0400 Subject: [tbnl-devel] Problems with SBCL 0.8.14 In-Reply-To: <20040901153320.GL28832@parsec.no-spoon.de> References: <20040831132145.GL28832@parsec.no-spoon.de> <87u0uj77f5.fsf@bird.agharta.de> <20040831134805.GM28832@parsec.no-spoon.de> <20040831144747.GN28832@parsec.no-spoon.de> <20040831145830.GO28832@parsec.no-spoon.de> <20040901153320.GL28832@parsec.no-spoon.de> Message-ID: <20040901153521.GU12869@xach.com> On Wed, Sep 01, 2004 at 05:33:20PM +0200, Stefan Scholl wrote: > On 2004-08-31 16:58:30, Stefan Scholl wrote: > > On 2004-08-31 16:47:47, Stefan Scholl wrote: > > > It's in function encode-session-string (session.lisp). > > > > (md5:md5sum-sequence "foo") > > > > --> segmentation violation at #X926BC80 > > OK, that's a bug in the current version 1.3.8 of md5. Until it's > fixed TBNL can't be used together with SBCL 0.8.14. > > http://article.gmane.org/gmane.lisp.steel-bank.devel/3582 Well, it's trivial to, for example, make the minor changes to tbnl so it uses the bundled SB-MD5, which does not have the bug. Zach From maxi at paoloastori.com Wed Sep 1 16:26:47 2004 From: maxi at paoloastori.com (Massimiliano Campagnoli) Date: Wed, 01 Sep 2004 18:26:47 +0200 Subject: [tbnl-devel] (post-parameters) strange behaviour ? Message-ID: <4135F847.20701@paoloastori.com> Try to handle this html form: after submitting this form in TBNL *debug-mode* you will get: > (post-parameters) > (("%3Amyfield" . "foo")) instead of >(("myfield" . "foo" )) Any explanation for this ? -- ING. MASSIMILIANO CAMPAGNOLI PAOLO ASTORI SRL VIA MIRABELLA, 9 28013 - GATTICO ITALY From edi at agharta.de Wed Sep 1 18:50:24 2004 From: edi at agharta.de (Edi Weitz) Date: Wed, 01 Sep 2004 20:50:24 +0200 Subject: [tbnl-devel] (post-parameters) strange behaviour ? In-Reply-To: <4135F847.20701@paoloastori.com> (Massimiliano Campagnoli's message of "Wed, 01 Sep 2004 18:26:47 +0200") References: <4135F847.20701@paoloastori.com> Message-ID: <87vfexvn67.fsf@bird.agharta.de> On Wed, 01 Sep 2004 18:26:47 +0200, Massimiliano Campagnoli wrote: > Try to handle this html form: > > > > > > > > after submitting this form in TBNL *debug-mode* you will get: > > > (post-parameters) > > (("%3Amyfield" . "foo")) > > instead of > > >(("myfield" . "foo" )) That would be strange. I hope you expected ((":myfield" . "foo" )) > Any explanation for this ? Yes, that's how the browser sends the field's name. When accepting POST or GET parameters the values are URL-decoded (see request.lisp and the function FORM-URL-ENCODED-LIST-TO-ALIST in util.lisp), the names aren't. Do you think it would make sense to decode the names as well? It'd be a trivial code change but I'm not sure about the implications. What do the RFCs say? Are there any restrictions on the names of the parameters? Cheers, Edi. From stesch at no-spoon.de Wed Sep 1 19:48:21 2004 From: stesch at no-spoon.de (Stefan Scholl) Date: Wed, 1 Sep 2004 21:48:21 +0200 Subject: [tbnl-devel] (post-parameters) strange behaviour ? In-Reply-To: <87vfexvn67.fsf@bird.agharta.de> References: <4135F847.20701@paoloastori.com> <87vfexvn67.fsf@bird.agharta.de> Message-ID: <20040901194821.GN28832@parsec.no-spoon.de> On 2004-09-01 20:50:24, Edi Weitz wrote: > Yes, that's how the browser sends the field's name. When accepting > POST or GET parameters the values are URL-decoded (see request.lisp > and the function FORM-URL-ENCODED-LIST-TO-ALIST in util.lisp), the > names aren't. Do you think it would make sense to decode the names as > well? It'd be a trivial code change but I'm not sure about the > implications. What do the RFCs say? Are there any restrictions on the > names of the parameters? Haven't read the RFC, but cgi.rb (CGI library of Ruby) decodes names and values. key, value = pairs.split('=',2).collect{|v| CGI::unescape(v) } CGI.pm (CGI library of Perl), too. $param = unescape($param); $value = unescape($value); Regards, Stefan From travis at crosswirecorp.com Wed Sep 1 20:52:08 2004 From: travis at crosswirecorp.com (Travis Cross) Date: Wed, 01 Sep 2004 15:52:08 -0500 Subject: [tbnl-devel] File Upload Design Issues In-Reply-To: <20040831105222.82165.qmail@web50607.mail.yahoo.com> References: <20040831105222.82165.qmail@web50607.mail.yahoo.com> Message-ID: <41363678.10105@crosswirecorp.com> Jeff Caldwell wrote: > 1) What are the relevant RFC's I need to consider? RFC2388: http://www.faqs.org/rfcs/rfc2388.html - Seems to be the real starting point for this. RFC1867: http://www.faqs.org/rfcs/rfc1867.html - This spec provides more examples and additional clarity as compared to the first one. RFC2045: http://www.faqs.org/rfcs/rfc2045.html RFC2046: http://www.faqs.org/rfcs/rfc2046.html RFC2183: http://www.faqs.org/rfcs/rfc2183.html > 2) What references/URLs/sample code is available for me to learn > about the encoding/stream issues? RFC2047: http://www.faqs.org/rfcs/rfc2047.html RFC2231: http://www.faqs.org/rfcs/rfc2231.html For sample code, it might depend on what language you want to read. PHP has been doing file upload handling successfully for quite some time, if your brain is inclined to read C code :) Their implementation is described here: http://www.php.net/manual/en/features.file-upload.php It seems that Webware in Python also supports file uploads, though I have not looked closely at their implementation. (http://www.webwareforpython.org/) > 3) General comments on the design. In particular, uploading a file > may only the first step. What is to be done with the file? Most implementations that I've seen do something like this: they take the uploaded file, store it in a temporary folder, and pass the information (the real location of the file, the form name it was posted under, the provided mime type, the size of the file, the client-side name for the file, and any errors that occurred) to the scripting language interface. The temporary file is then immediately deleted when the request is complete. A significant security consideration is assuring that no user can upload enough data to cause the server to crash. Since Lisp is far more powerful than most of the languages that I've observed (and since a continual process is assured to be running), there may be a more effective way possible to do this. (Brainstorming here) Maybe the file could be stored in a temporary file and the interface would return a stream to that file. Maybe the file could just be kept in memory and returned as a variable, which would be especially useful when one just wants to drop the data into a relational database. > 3a) There are OS dependencies, as not all OS's support the same file > names. True. In the temporary file model, I think the solution is just using some simple and meaningless but unique string. As long as the file name is passed to the application, it really doesn't matter what the file is physically named. > 3b) The file upload must generate a portable but unique internal name > while at the same time retaining the OS-specific name specified by > the sender (original name). Correct. > 3c) This implies a kind of catalog of files, mapping the original > name to the internal name and vice versa. This technique also is > necessary to prevent duplicate file names, for example from multiple > users. Other information needs to be mapped as well, including the form field information so that files can be distinguished from each other if a form supports multiple files in a single post. > 3d) The user should be able to associate a short and/or long > description, and possibly a category or two, with the file. TBNL > might not dictate this precisely but the catalog should be flexible > in what it holds. I'm not certain what you mean by this, or what the need is. It would seem that the primary index on which to find a file would be the 'name' or the form field as specified in the POST. > 3e) The catalog needs to be able to associate a file with a user and > vice versa. Users aren't a TBNL concept at the moment and I'm not > sure adding them just for this is worthwhile. On the other hand, > uploading and storing files without the possibility of a sense of > ownership doesn't seem robust. Is this really a part of the TBNL > core? How can this be handled cleanly? It would seem that this could be left to the user. ie, if the uploaded file is only stored temporarily, and then the application is expected to do something with it before the request is complete (move it somewhere else, for example), then the responsibility for file storage would be held by the user of the library, probably where it should be. If the uploaded files are held temporarily in memory, then it seems that it would become even more of a non-issue. > 3f) Where and how should the files be stored? ACS and OpenACS > (openacs.org), the last time I looked, store uploaded files in a big > directory tree with, IIRC, subdirectories named with single letters. > The destination subdirectory was chosen by the first 3 or 4 chars of > the internally-generated file name. The idea is that a filesystem can > have, or can efficiently handle, only so many files per subdirectory. > Given an idea of the total number of uploaded files we want to > support, we could calculate the depth and number of subdirectories > needed. Is there really a need for any subdirectories? Does the OpenACS approach add any real value to the temporary file or memory model? > 3g) TBNL-level security is enough and it's OK to mix uploaded files > from several users in the same catalog and subdirectories, so long as > TBNL keeps things sorted out and, depending upon the application, the > programmer is responsible for showing/making available the right file > to the right person. It would seem that TBNL should only present the information for the file(s) uploaded as part of the request that the particular dispatched thread is handling. > I bet I both left something out and put too much in :) I'd appreciate > comments on all of the above. In summary, how should the files be > stored and how should the catalog be structured? Once I have an idea > of how we think this should work, I'm happy to glue it all together. I think the idea of a catalog might be a layer of functionality above what is needed here. It seems to me that it would be better to implement simple file upload support, and then if there is a need or added value in a catalog based approach to implement that separately and on top of the basic file uploading support. > Thanks for your help. Anytime. -- Travis From edi at agharta.de Thu Sep 2 04:48:24 2004 From: edi at agharta.de (Edi Weitz) Date: Thu, 02 Sep 2004 06:48:24 +0200 Subject: [tbnl-devel] New version 0.2.11 (Was: (post-parameters) strange behaviour ?) In-Reply-To: <20040901194821.GN28832@parsec.no-spoon.de> (Stefan Scholl's message of "Wed, 1 Sep 2004 21:48:21 +0200") References: <4135F847.20701@paoloastori.com> <87vfexvn67.fsf@bird.agharta.de> <20040901194821.GN28832@parsec.no-spoon.de> Message-ID: <87u0uhp97r.fsf_-_@bird.agharta.de> On Wed, 1 Sep 2004 21:48:21 +0200, Stefan Scholl wrote: > Haven't read the RFC, but cgi.rb (CGI library of Ruby) decodes names > and values. > > key, value = pairs.split('=',2).collect{|v| CGI::unescape(v) } > > > CGI.pm (CGI library of Perl), too. > > $param = unescape($param); > $value = unescape($value); OK, thanks for the info. So I think we should follow their lead. I've released a new version: Version 0.2.11 2004-09-02 FORM-URL-ENCODED-LIST-TO-ALIST now decodes names and values Note that this change affects the names of GET and POST parameters as well as the names of cookies. Cheers, Edi. From edi at agharta.de Sun Sep 5 07:02:20 2004 From: edi at agharta.de (Edi Weitz) Date: Sun, 05 Sep 2004 09:02:20 +0200 Subject: [tbnl-devel] mod_lisp2-2.c (Was: Help needed in handler function with (sleep seconds)) In-Reply-To: <412A2466.5050006@paoloastori.com> (Massimiliano Campagnoli's message of "Mon, 23 Aug 2004 19:07:50 +0200") References: <412A2466.5050006@paoloastori.com> Message-ID: <87hdqdmc5f.fsf@bird.agharta.de> Chris Hansen asked me to forward the following message to this list. Note that he's not subscribed so you have to Cc him if you reply. Cheers, Edi. From: Chris Hanson Subject: [tbnl-devel] Help needed in handler function with (sleep seconds) To: maxi at paoloastori.com CC: tbnl-devel at common-lisp.net, marcoxa at cs.nyu.edu Date: Sun, 05 Sep 2004 00:44:58 -0400 User-Agent: IMAIL/1.21; Edwin/3.116; MIT-Scheme/7.7.91.pre Date: Mon, 23 Aug 2004 19:07:50 +0200 From: Massimiliano Campagnoli where mod_lisp2-2.c (which is based on a different implementation) works has expected if the handler contains (sleep n). But since mod_lisp2-2.c occasionally crashes Apache and has no support for the latest Edi's improvements (since is based on mod_lisp 2.33) I would say that the option to move back to apache 1.3 with the latest mod_lisp.c 2.38 is still a good move, until someone (Marco Antoniotti & Co.) with enough experience has time to port and test mod_lisp.c 2.38 to Apache2. I'm interested to know if there is a reliable way to crash apache using "mod_lisp2-2.c". If so, I'll fix it. I am also willing to update the implementation with the 2.38 improvements, provided that it doesn't take more than a day or so. The main problem is that I haven't been using the module. (I'm still using Apache 1.x.) I wrote the module for a colleague, so that he could use my server-side software, and I haven't heard about any problems from him. Are there any other known problems with this code? From stesch at no-spoon.de Mon Sep 6 21:23:28 2004 From: stesch at no-spoon.de (Stefan Scholl) Date: Mon, 6 Sep 2004 23:23:28 +0200 Subject: [tbnl-devel] Problems with SBCL 0.8.14 In-Reply-To: <20040901153320.GL28832@parsec.no-spoon.de> References: <20040831132145.GL28832@parsec.no-spoon.de> <87u0uj77f5.fsf@bird.agharta.de> <20040831134805.GM28832@parsec.no-spoon.de> <20040831144747.GN28832@parsec.no-spoon.de> <20040831145830.GO28832@parsec.no-spoon.de> <20040901153320.GL28832@parsec.no-spoon.de> Message-ID: <20040906212328.GR28832@parsec.no-spoon.de> On 2004-09-01 17:33:20, Stefan Scholl wrote: > On 2004-08-31 16:58:30, Stefan Scholl wrote: > > On 2004-08-31 16:47:47, Stefan Scholl wrote: > > > It's in function encode-session-string (session.lisp). > > > > (md5:md5sum-sequence "foo") > > > > --> segmentation violation at #X926BC80 > > OK, that's a bug in the current version 1.3.8 of md5. Until it's > fixed TBNL can't be used together with SBCL 0.8.14. > > http://article.gmane.org/gmane.lisp.steel-bank.devel/3582 OK, it's fixed. Current version 1.8.4 http://www.cliki.net/md5 Everyone with SBCL >= 0.8.14 should update md5. It gets called when sessions are used. From maxi at paoloastori.com Tue Sep 7 12:07:22 2004 From: maxi at paoloastori.com (Massimiliano Campagnoli) Date: Tue, 07 Sep 2004 14:07:22 +0200 Subject: [tbnl-devel] mod_lisp2-2.c Message-ID: <413DA47A.7080409@paoloastori.com> I am not able to reproduce a crash on Apache 2 using mod_lisp2-2.c and TBNL. As soon as I found a way I'll post to the list. Actually I would be very interested in using TBNL on mod_lisp2-2.c if it is kept at the same latest level of mod_lisp.c 2.38 with latest Edi's fixes. -- ING. MASSIMILIANO CAMPAGNOLI PAOLO ASTORI SRL VIA MIRABELLA, 9 28013 - GATTICO ITALY