From lam at tuxfamily.org Fri Jun 1 10:27:02 2007 From: lam at tuxfamily.org (Nicolas Lamirault) Date: Fri, 01 Jun 2007 12:27:02 +0200 Subject: [drakma-devel] POST request and content-type Message-ID: <87k5uo9dhl.fsf@no-log.org> hello, i make some tests with Drakma. i would like to change the content-type of a POST request, but i've got always "application/x-www-form-urlencoded" : CL-USER> (drakma:http-request "http://lisp.org/" :method :post :content-type "application/atom+xml") POST / HTTP/1.1 Host: lisp.org User-Agent: Drakma/0.7.0 (SBCL 1.0; Linux; 2.6.18-4-486; http://weitz.de/drakma/) Accept: */* Connection: close Content-Type: application/x-www-form-urlencoded Content-Length: 0 HTTP/1.1 307 Temporary Redirect Date: Fri, 01 Jun 2007 10:23:23 GMT Connection: Close Server: AllegroServe/1.2.43 Transfer-Encoding: chunked LOCATION: /index.html #() 307 ((:DATE . "Fri, 01 Jun 2007 10:23:23 GMT") (:CONNECTION . "Close") (:SERVER . "AllegroServe/1.2.43") (:TRANSFER-ENCODING . "chunked") (:LOCATION . "/index.html")) # # T someone knows if i make a mistake ? it is a bug ? thanks for anyhelp ... PS: very good soft ! -- Nicolas Lamirault From edi at agharta.de Fri Jun 1 11:21:22 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 01 Jun 2007 13:21:22 +0200 Subject: [drakma-devel] POST request and content-type In-Reply-To: <87k5uo9dhl.fsf@no-log.org> (Nicolas Lamirault's message of "Fri, 01 Jun 2007 12:27:02 +0200") References: <87k5uo9dhl.fsf@no-log.org> Message-ID: On Fri, 01 Jun 2007 12:27:02 +0200, Nicolas Lamirault wrote: > i would like to change the content-type of a POST request, but i've > got always "application/x-www-form-urlencoded" : > > [...] > > someone knows if i make a mistake ? it is a bug ? No, it's documented behaviour: http://weitz.de/drakma/#content-type > PS: very good soft ! Thanks... :) From lam at tuxfamily.org Fri Jun 1 13:52:16 2007 From: lam at tuxfamily.org (Nicolas Lamirault) Date: Fri, 01 Jun 2007 15:52:16 +0200 Subject: [drakma-devel] POST request and content-type In-Reply-To: (Edi Weitz's message of "Fri\, 01 Jun 2007 13\:21\:22 +0200") References: <87k5uo9dhl.fsf@no-log.org> Message-ID: <87bqfzaijz.fsf@no-log.org> thanks a lot Edi Weitz writes: > On Fri, 01 Jun 2007 12:27:02 +0200, Nicolas Lamirault wrote: > >> i would like to change the content-type of a POST request, but i've >> got always "application/x-www-form-urlencoded" : >> >> [...] >> >> someone knows if i make a mistake ? it is a bug ? > > No, it's documented behaviour: > > http://weitz.de/drakma/#content-type > >> PS: very good soft ! > > Thanks... :) > _______________________________________________ > drakma-devel mailing list > drakma-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/drakma-devel > -- Nicolas Lamirault From mathias.dahl at gmail.com Sat Jun 2 09:44:04 2007 From: mathias.dahl at gmail.com (Mathias Dahl) Date: Sat, 2 Jun 2007 11:44:04 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding Message-ID: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> Hi! I have a problem with drakma and character encoding. My goal is to make a small web utility which first GETs some content from a certain page on my wiki, then optionally adds prepends stuff to this content, and then POST the new content back. It works as long as only ASCII characters are involved but fails when I use characters from the higher part of Latin-1, in my case the Swedish character "?". Below I have simplified the code so that it just GETs the old content and POST it back. What one sees is that the Swedish character is correctly handled in the GET, I see the "?" in it's full glory, but after having posted it back, the content gets corrupt and the next time I run the function I get an error because of the strange character. Ok, enough blabbing, here is the code: ;;;;; code starts here (defvar *boundary* "-------------------------1852275791466338532535335716") (defconstant +crlf+ #.(format nil "~C~C" #\Return #\Linefeed)) (defun format-field (name value) (format nil "--~a~aContent-Disposition: form-data; name=\"~a\"~a~a~a~a" *boundary* +crlf+ name +crlf+ +crlf+ value +crlf+)) (defun foo () (let* ((old-content (drakma:http-request "http://klibb.com/cgi-bin/wiki.pl?action=browse;id=2007-05-31;raw=1")) (cookie-jar (make-instance 'drakma:cookie-jar)) (new-content (concatenate 'string (format-field "title" "2007-05-31") (format-field "text" old-content) (format-field "recent_edit" "on") (format-field "username" "MathiasDahl") "--" *boundary* "--" +crlf+))) (format t "Old content: ~a" old-content) (setf (drakma:cookie-jar-cookies cookie-jar) (list (make-instance 'drakma:cookie :name "pwd" :value "editeramera" :expires (+ (get-universal-time) 36000) :domain "klibb.com"))) (format t "New content: ~a" new-content) (drakma:http-request "http://klibb.com/cgi-bin/wiki.pl" :method :post :cookie-jar cookie-jar :content-type (format nil "multipart/form-data; boundary=~a" *boundary*) :content new-content))) ;;;;; code ends here Again, the code is simplified, some parts are hardcoded etc, but the above is enough to recreate the problem. Note that after running the code one time, you cannot test it again, because the content on the page is now changed. Here is what I get after running the function the first time: ==== * (foo) Old content: bl? New content: ---------------------------1852275791466338532535335716 Content-Disposition: form-data; name="title" 2007-05-31 ---------------------------1852275791466338532535335716 Content-Disposition: form-data; name="text" bl? ---------------------------1852275791466338532535335716 Content-Disposition: form-data; name="recent_edit" on ---------------------------1852275791466338532535335716 Content-Disposition: form-data; name="username" MathiasDahl ---------------------------1852275791466338532535335716-- NIL 302 ((:DATE . "Sat, 02 Jun 2007 09:30:53 GMT") (:SERVER . "Apache/2.2.3 (Mandriva Linux/PREFORK-1mdv2007.0)") (:SET-COOKIE . "MuuWiki=username%1EMathiasDahl; path=/; expires=Mon, 01-Jun-2009 09:30:53 GMT") (:LOCATION . "http://klibb.com/cgi-bin/wiki.pl/2007-05-31") (:CONTENT-LENGTH . "0") (:CONNECTION . "close") (:CONTENT-TYPE . "application/x-perl")) # # T ==== As you can see, all looks well; the old content ("bl?") looks like it should, and the new content looks the same (it's the data in the form field "text"). However, when I now run the function again, I get this: ==== * (foo) debugger invoked on a FLEXI-STREAMS:FLEXI-STREAM-ENCODING-ERROR in thread #: Unexpected value #xA in UTF-8 sequence. Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL. restarts (invokable by number or by possibly-abbreviated name): 0: [USE-VALUE] Specify a character to be used instead. 1: [ABORT ] Exit debugger, returning to top level. (FLEXI-STREAMS::SIGNAL-ENCODING-ERROR # "Unexpected value #x~X in UTF-8 sequence." 10) ==== It fails because that "?" is now something else. When I do the same thing from a browser, i.e. POST the page again and again, I don't see any problems. I have done some network sniffing with Wireshar and what I can see is that when the browser POSTs the content, the "?" is correctly encoded in UTF-8 as xC3 xA4. In the POST done by drakma, the character is encoded xE4 (which IS the unicode code point, but not encoded as UTF-8 if I understand things correctly). At first I tried to include the encoding in Content-Type, but when I saw that it did not do any difference and also saw that Firefox does not include this, I removed it. Oh, and I should show this as well: * (sb-impl::default-external-format) :UTF-8 Just so that we are clear that I DO see the content correctly and UTF-8 is used. I also tried with a version where I even hardcoded the content to be sent to be "bl?", and that gives the same problem. Maybe I should have shortened the code above to that, but what I wanted to show was that the same content I can GET nicely enough cannot be POSTed without problems. Any ideas on how I can continue debugging this? I feel kinda lost. It feels frustrating to get stuck on a problem like this when I have got the other logic to work, GETing and POSTing and stuff... I am running this in SBCL 1.0 under Mandriva GNU/Linux. Thanks! /Mathias From vodonosov at mail.ru Mon Jun 4 06:18:54 2007 From: vodonosov at mail.ru (Vodonosov Anton) Date: Mon, 04 Jun 2007 10:18:54 +0400 Subject: =?koi8-r?Q?Re=3A_[drakma-devel]_Problem_with_Drakma_and_character_encoding?= In-Reply-To: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> Message-ID: Hi Mathias, try to use (http-request ... :external-format-out :UTF-8 ...) See http://weitz.de/drakma/#external-format-out Best regards, -Anton From edi at agharta.de Mon Jun 4 06:49:33 2007 From: edi at agharta.de (Edi Weitz) Date: Mon, 04 Jun 2007 08:49:33 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding In-Reply-To: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> (Mathias Dahl's message of "Sat, 2 Jun 2007 11:44:04 +0200") References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> Message-ID: On Sat, 2 Jun 2007 11:44:04 +0200, "Mathias Dahl" wrote: > (defvar *boundary* "-------------------------1852275791466338532535335716") > > (defconstant +crlf+ #.(format nil "~C~C" #\Return #\Linefeed)) > > (defun format-field (name value) > (format nil "--~a~aContent-Disposition: form-data; name=\"~a\"~a~a~a~a" > *boundary* +crlf+ name +crlf+ +crlf+ value +crlf+)) Any reason you're doing all this instead of just using :FORM-DATA? http://weitz.de/drakma/#form-data From mathias.dahl at gmail.com Mon Jun 4 09:19:08 2007 From: mathias.dahl at gmail.com (Mathias Dahl) Date: Mon, 4 Jun 2007 11:19:08 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding In-Reply-To: References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> Message-ID: <7dbe73ed0706040219t4b77c70fl73991a2355d7fc5c@mail.gmail.com> > Hi Mathias, try to use (http-request ... :external-format-out :UTF-8 ...) Ah, what a relief! :) It works! Many thanks! /Mathias From mathias.dahl at gmail.com Mon Jun 4 09:22:37 2007 From: mathias.dahl at gmail.com (Mathias Dahl) Date: Mon, 4 Jun 2007 11:22:37 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding In-Reply-To: References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> Message-ID: <7dbe73ed0706040222p6377c04bt4afc585793fd0e97@mail.gmail.com> > > (defun format-field (name value) > > (format nil "--~a~aContent-Disposition: form-data; name=\"~a\"~a~a~a~a" > > *boundary* +crlf+ name +crlf+ +crlf+ value +crlf+)) > > Any reason you're doing all this instead of just using :FORM-DATA? > > http://weitz.de/drakma/#form-data Yes, but not a very good one; it is a combination of old code, I previously used S-HTTP-CLIENT but got problems so I switched to DRAKMA and I tried to leave the code as intact as possible, not looking into all nice features that DRAKMA might have. I will try that and see how that works for me. Thanks for a great package! It feels nice to be able to do stuff in CL, although I progress quite slowly... :) /Mathias From mathias.dahl at gmail.com Mon Jun 4 20:46:33 2007 From: mathias.dahl at gmail.com (Mathias Dahl) Date: Mon, 4 Jun 2007 22:46:33 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding In-Reply-To: <7dbe73ed0706040222p6377c04bt4afc585793fd0e97@mail.gmail.com> References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> <7dbe73ed0706040222p6377c04bt4afc585793fd0e97@mail.gmail.com> Message-ID: <7dbe73ed0706041346t6b6971b4k7aad4578b977e9ba@mail.gmail.com> > > (defun format-field (name value) > > (format nil "--~a~aContent-Disposition: form-data; name=\"~a\"~a~a~a~a" > > *boundary* +crlf+ name +crlf+ +crlf+ value +crlf+)) > > Any reason you're doing all this instead of just using :FORM-DATA? > > http://weitz.de/drakma/#form-data I have now tested this and it works like a charm, so I don't need my hacks anymore. Yay! :) I first used the :PARAMETERS keyword without specifying the :FORM-DATA keyword, and that works as well as providing it. Are there any drawbacks of NOT using :FORM-DATA when just sending form fields (i.e. no files)? Note: one of the form fields can be quite large (maybe up to a thousand characters) sometimes. Thanks! /Mathias From edi at agharta.de Mon Jun 4 20:55:20 2007 From: edi at agharta.de (Edi Weitz) Date: Mon, 04 Jun 2007 22:55:20 +0200 Subject: [drakma-devel] Problem with Drakma and character encoding In-Reply-To: <7dbe73ed0706041346t6b6971b4k7aad4578b977e9ba@mail.gmail.com> (Mathias Dahl's message of "Mon, 4 Jun 2007 22:46:33 +0200") References: <7dbe73ed0706020244i410a476bh94ae0da63d78fcb@mail.gmail.com> <7dbe73ed0706040222p6377c04bt4afc585793fd0e97@mail.gmail.com> <7dbe73ed0706041346t6b6971b4k7aad4578b977e9ba@mail.gmail.com> Message-ID: On Mon, 4 Jun 2007 22:46:33 +0200, "Mathias Dahl" wrote: > I have now tested this and it works like a charm, so I don't need my > hacks anymore. Yay! :) Good... :) > I first used the :PARAMETERS keyword without specifying the > :FORM-DATA keyword, and that works as well as providing it. Are > there any drawbacks of NOT using :FORM-DATA when just sending form > fields (i.e. no files)? Note: one of the form fields can be quite > large (maybe up to a thousand characters) sometimes. AFAIK there aren't if you're /not/ sending files. The size of the request body shouldn't make a difference as it's a POST request anyway. The only other difference that comes to mind right now is that with multipart/form-data you can potentially use different encodings for different fields. I never did that, though, and I wouldn't know why one should do it (and if the receiving end can cope with it). Oh, one other difference that just occurs to me is that for large fields Drakma might be a tad faster if you're using :FORM-DATA. But you should only care about this if you really have performance problems. From lispercat at gmail.com Fri Jun 8 15:19:43 2007 From: lispercat at gmail.com (Andrei Stebakov) Date: Fri, 8 Jun 2007 11:19:43 -0400 Subject: [drakma-devel] File uploading: adding streams and functions to the list of parameters Message-ID: Recently I've been thinking how to make http-request upload a file, but not from the hard drive but from a function which can produce (or write to a) stream. Let's say I want to create an image with cl-gd but instead of writing the image to the file and then creating the http-request with a pathname of the file I can pass a (lambda (stream) (cl-gd-create-image stream your param1 param2)) in place of the pathname parameter. Or a stream which already has the file data. I was trying to do it using :content :continuation but found it too difficult, maybe I missed other ways. Anyway, I created a patch with very minimal changes to the make-form-data-function function which allows to do so. Also I found that I set *header-stream* to not nil, I was missing the debug output produced by the character (character only) stream from make-form-data-function which basically shows all the parameters we are sending. I added the primitive support for this based on make-broadcast-stream. Here I attach the patch for your kind consideration. Thank you, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: file-as-function.patch Type: application/octet-stream Size: 3705 bytes Desc: not available URL: From edi at agharta.de Fri Jun 8 20:13:40 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 08 Jun 2007 22:13:40 +0200 Subject: [drakma-devel] File uploading: adding streams and functions to the list of parameters In-Reply-To: (Andrei Stebakov's message of "Fri, 8 Jun 2007 11:19:43 -0400") References: Message-ID: On Fri, 8 Jun 2007 11:19:43 -0400, "Andrei Stebakov" wrote: > Recently I've been thinking how to make http-request upload a file, > but not from the hard drive but from a function which can produce > (or write to a) stream. > Let's say I want to create an image with cl-gd but instead of > writing the image to the file and then creating the http-request > with a pathname of the file I can pass a > (lambda (stream) > (cl-gd-create-image stream your param1 param2)) > in place of the pathname parameter. Or a stream which already has > the file data. > I was trying to do it using :content :continuation but found it too > difficult, maybe I missed other ways. You could have passed a function designator to :CONTENT. That's a bit easier than to use :CONTINUATION, but you'd still have to create the form-data wrappers yourself. > Anyway, I created a patch with very minimal changes to the > make-form-data-function function which allows to do so. Also I found > that I set *header-stream* to not nil, I was missing the debug > output produced by the character (character only) stream from > make-form-data-function which basically shows all the parameters we > are sending. I added the primitive support for this based on > make-broadcast-stream. Here I attach the patch for your kind > consideration. The general idea is fine with me. As for the patch I have a few things I don't like: 1. Now that the value can be more than a pathname, the variable shouldn't be called PATHNAME anymore. 2. The docstring for HTTP-REQUEST and the HTML documentation need to be updated as well. See also http://weitz.de/patches.html 3. The patch contains TAB characters. 4. Where you have *STANDARD-OUTPUT*, you probably want to have *HEADER-STREAM*. 5. But I generally don't really like this part anyway. The stream is called *HEADER-STREAM* because the /headers/ are supposed to be sent there. Sending /parts/ of the form-data there as well might be useful in a few cases but it seems like a fortuitous choice to me. Thanks, Edi. From lispercat at gmail.com Mon Jun 11 17:09:59 2007 From: lispercat at gmail.com (Andrei Stebakov) Date: Mon, 11 Jun 2007 13:09:59 -0400 Subject: [drakma-devel] File uploading: adding streams and functions to the list of parameters #2 Message-ID: I changed the pathname to file-source (after giving it some thinking and asking for help from the lisp NG, sorry if it's not the best), removed the *header-stream* related modifications, removed the tabs, updated the doc-string for http-request and the html file. (for some reason Edi's response to my previous patch didn't make to my email, which is why my new message may not be part of the original thread) Thank you, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: file-as-function2.patch Type: application/octet-stream Size: 3644 bytes Desc: not available URL: From lam at tuxfamily.org Fri Jun 15 19:32:41 2007 From: lam at tuxfamily.org (Nicolas Lamirault) Date: Fri, 15 Jun 2007 21:32:41 +0200 Subject: [drakma-devel] puri error during HTTP request Message-ID: <874pl9ko86.fsf@no-log.org> hi, i make some test with Drakma and the Audioscrobbler API. for the identification, the protocol is : http://post.audioscrobbler.com/?hs=true&p=1.2& c=&v=&u=&t=&a= with auth : The algorithm for generating this token is as follows: token := md5(md5(password) + timestamp) Parse error:URI "http://post.audioscrobbler.com/?hs=true&p=1.2&c=tst&v=1.0&u=lam_&t=2007-06-15 21:29:54&a= ,???*?h|I3???" contains illegal character #\ at position 77. [Condition of type PURI:URI-PARSE-ERROR] Restarts: 0: [ABORT] Return to SLIME's top level. 1: [ABORT] Exit debugger, returning to top level. Backtrace: 0: (PURI::.PARSE-ERROR # #) 1: (PURI::.PARSE-ERROR #) Locals: SB-DEBUG::ARG-0 = : SB-DEBUG::ARG-1 = : 2: ((FLET PURI::READ-TOKEN) # #) 3: (PURI::PARSE-URI-STRING #) 4: (PURI:PARSE-URI # :CLASS #) 5: (DRAKMA:HTTP-REQUEST "http://post.audioscrobbler.com/?hs=true&p=1.2&c=tst&v=1.0&u=lam_&t=2007-06-15 21:29:54&a= ,???*?h|I3???") someone have any idea to resolve this problem ? thanks for any help -- Nicolas Lamirault From edi at agharta.de Fri Jun 15 20:31:53 2007 From: edi at agharta.de (Edi Weitz) Date: Fri, 15 Jun 2007 22:31:53 +0200 Subject: [drakma-devel] puri error during HTTP request In-Reply-To: <874pl9ko86.fsf@no-log.org> (Nicolas Lamirault's message of "Fri, 15 Jun 2007 21:32:41 +0200") References: <874pl9ko86.fsf@no-log.org> Message-ID: On Fri, 15 Jun 2007 21:32:41 +0200, Nicolas Lamirault wrote: > i make some test with Drakma and the Audioscrobbler API. for the > identification, the protocol is : > > http://post.audioscrobbler.com/?hs=true&p=1.2&c=&v=&u=&t=&a= > > with auth : The algorithm for generating this token is as follows: > token := md5(md5(password) + timestamp) > > Parse error:URI "http://post.audioscrobbler.com/?hs=true&p=1.2&c=tst&v=1.0&u=lam_&t=2007-06-15 21:29:54&a= ,???*?h|I3???" contains illegal character #\ at position 77. > [Condition of type PURI:URI-PARSE-ERROR] That's clearly an illegal URI, so PURI is right to complain. I have never heard of Audioscrobbler, but they probably mean the /hexadecimal/ MD5 representation or something like that. From edi at agharta.de Sun Jun 17 20:49:17 2007 From: edi at agharta.de (Edi Weitz) Date: Sun, 17 Jun 2007 16:49:17 -0400 Subject: [drakma-devel] New release 0.7.1 (Was: File uploading: adding streams and functions to the list of parameters #2) In-Reply-To: (Andrei Stebakov's message of "Mon, 11 Jun 2007 13:09:59 -0400") References: Message-ID: On Mon, 11 Jun 2007 13:09:59 -0400, "Andrei Stebakov" wrote: > I changed the pathname to file-source (after giving it some thinking > and asking for help from the lisp NG, sorry if it's not the best), > removed the *header-stream* related modifications, removed the tabs, No, there were still several TAB characters in there. > updated the doc-string for http-request But only parts of it. Other parts still referred to the old behaviour. > and the html file. The patch didn't contain changes to the HTML documentation. I've now released a new version with these changes as I think that the idea is generally not bad, but I still think that the patches as you sent them aren't acceptable. Thanks, Edi. From didier at lrde.epita.fr Tue Jun 19 20:56:26 2007 From: didier at lrde.epita.fr (Didier Verna) Date: Tue, 19 Jun 2007 22:56:26 +0200 Subject: [drakma-devel] Two problems with cookies expiry dates format Message-ID: Hello, I've run into 2 formats that parse-cookie-date can't currently handle: - the first one looks like this: "Sun, 18 Jun 2006 04:01:09 GMT-07:00" I presume it means GMT minus 7 hours. - the second one is almost one of the recognized formats, except that it says PDT (Pacific Daylight Time AFAIK) instead of GMT. Note that I have no idea if these formats are standard or not. I just ran into them... -- Didier Verna, didier at lrde.epita.fr, http://www.lrde.epita.fr/~didier EPITA / LRDE, 14-16 rue Voltaire Tel.+33 (1) 44 08 01 85 94276 Le Kremlin-Bic?tre, France Fax.+33 (1) 53 14 59 22 didier at xemacs.org From edi at agharta.de Wed Jun 20 02:57:10 2007 From: edi at agharta.de (Edi Weitz) Date: Tue, 19 Jun 2007 22:57:10 -0400 Subject: [drakma-devel] Two problems with cookies expiry dates format In-Reply-To: (Didier Verna's message of "Tue, 19 Jun 2007 22:56:26 +0200") References: Message-ID: On Tue, 19 Jun 2007 22:56:26 +0200, Didier Verna wrote: > I've run into 2 formats that parse-cookie-date can't currently > handle: > > - the first one looks like this: "Sun, 18 Jun 2006 04:01:09 > GMT-07:00" I presume it means GMT minus 7 hours. > > - the second one is almost one of the recognized formats, except > that it says PDT (Pacific Daylight Time AFAIK) instead of GMT. Thanks for the report. I'll take care of this once I'm back home from Boston. > Note that I have no idea if these formats are standard or not. I > just ran into them... I think that, sadly, there's no real standard for this as I've seen so many different formats used already. My impression is that you have to wait until you catch some new format in the wild and then try to integrate it. From edi at agharta.de Mon Jun 25 10:29:54 2007 From: edi at agharta.de (Edi Weitz) Date: Mon, 25 Jun 2007 12:29:54 +0200 Subject: [drakma-devel] New version 0.8.0 (Was: Two problems with cookies expiry dates format) In-Reply-To: (Didier Verna's message of "Tue, 19 Jun 2007 22:56:26 +0200") References: Message-ID: On Tue, 19 Jun 2007 22:56:26 +0200, Didier Verna wrote: > I've run into 2 formats that parse-cookie-date can't currently handle: > > - the first one looks like this: "Sun, 18 Jun 2006 04:01:09 > GMT-07:00" I presume it means GMT minus 7 hours. > > - the second one is almost one of the recognized formats, except > that it says PDT (Pacific Daylight Time AFAIK) instead of GMT. > > Note that I have no idea if these formats are standard or not. I > just ran into them... The latest release 0.8.0 should hopefully be able to parse these dates correctly. Note that I've also added a special variable so you can allow Drakma to fail when parsing cookie dates. Still, I'd prefer if people would report any date format Drakma can't parse to this mailing list. Thanks for the report, Edi. From h.duerer at gmail.com Wed Jun 27 11:27:18 2007 From: h.duerer at gmail.com (H Duerer) Date: Wed, 27 Jun 2007 12:27:18 +0100 Subject: [drakma-devel] Status text from a requst Message-ID: Hi, we have a web server giving Json responses. For proper failures it returns a non-200 status code and a failure description in the status message (e.g. the status line reads "HTTP/1.0 400 Empty method name". AFAICS this status message is not passed through by Drakma's http-request. I locally made the attached changes (doing so, I found that read-status-line added the leading separating whitespace to its third value; I fixed that as well). Any chance of having this or some other means to get to the message in the official code? Holger -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: drakma.diff Type: text/x-diff Size: 4037 bytes Desc: not available URL: From h.duerer at gmail.com Wed Jun 27 20:17:23 2007 From: h.duerer at gmail.com (H Duerer) Date: Wed, 27 Jun 2007 21:17:23 +0100 Subject: [drakma-devel] Status text from request (re-post) Message-ID: Hi, sorry about that first email -- I am still learning gmail and hadn't realised I was rudely sending HTML. Trying again in plain text: We have a web server giving Json responses. For proper failures it returns a non-200 status code and a failure description in the status message (e.g. the status line reads "HTTP/1.0 400 Empty method name". AFAICS this status message is not passed through by Drakma's http-request. I locally made the attached changes (doing so, I found that read-status-line added the leading separating whitespace to its third value; I fixed that as well). Any chance of having this or some other means to get to the message in the official code? Holger diff -rN -u old-drakma/read.lisp new-drakma/read.lisp --- old-drakma/read.lisp 2007-06-27 12:23:17.000000000 +0100 +++ new-drakma/read.lisp 2007-06-27 12:23:17.000000000 +0100 @@ -55,7 +55,7 @@ :start (1+ first-space-pos) :end second-space-pos)) (error "Status code in ~S is not an integer." line)) - (and second-space-pos (subseq line second-space-pos))))) + (and second-space-pos (subseq line (1+ second-space-pos)))))) (defun get-content-type (headers) "Reads and parses a `Content-Type' header and returns it as @@ -124,4 +124,4 @@ (trim-whitespace (subseq string (1+ old-position) position))) while old-position when (plusp (length substring)) - collect substring)) \ No newline at end of file + collect substring)) diff -rN -u old-drakma/request.lisp new-drakma/request.lisp --- old-drakma/request.lisp 2007-06-27 12:23:17.000000000 +0100 +++ new-drakma/request.lisp 2007-06-27 12:23:17.000000000 +0100 @@ -172,15 +172,15 @@ URI is where the request is sent to, and it is either a string denoting a uniform resource identifier or a PURI:URI object. The scheme of URI must be `http' or `https'. The function returns -SIX values - the body of the reply \(but see below), the status +SEVEN values - the body of the reply \(but see below), the status code as an integer, an alist of the headers sent by the server where for each element the car \(the name of the header) is a keyword and the cdr \(the value of the header) is a string, the URI the reply comes from \(which might be different from the URI the request was sent to in case of redirects), the stream the -reply was read from, and a generalized boolean which denotes +reply was read from, a generalized boolean which denotes whether the stream should be closed \(and which you can usually -ignore). +ignore), and the message text from the response line. PROTOCOL is the HTTP protocol which is going to be used in the request line, it must be one of the keywords :HTTP/1.0 or @@ -512,15 +512,15 @@ (setf (chunked-stream-output-chunking-p (flexi-stream-stream http-stream)) nil) (finish-output http-stream) - (multiple-value-bind (server-protocol status-code) + (multiple-value-bind (server-protocol status-code status-text) ;; loop until status is NOT 100 - (loop for (server-protocol status-code nil) + (loop for (server-protocol status-code status-text) = (read-status-line http-stream *header-stream*) when (= status-code 100) ;; ignore headers sent until non-100 status is seen do (read-http-headers http-stream *header-stream*) until (/= status-code 100) - finally (return (values server-protocol status-code))) + finally (return (values server-protocol status-code status-text))) (let ((headers (read-http-headers http-stream *header-stream*)) body external-format-body) (let ((connections (header-value :connection headers))) @@ -631,7 +631,8 @@ headers uri http-stream - must-close))))) + must-close + status-text))))) (when (eq content :continuation) (return-from http-request #'finish-request)) (finish-request content)))) From edi at agharta.de Fri Jun 29 23:22:29 2007 From: edi at agharta.de (Edi Weitz) Date: Sat, 30 Jun 2007 01:22:29 +0200 Subject: [drakma-devel] New release 0.9.0 (Was: Status text from request (re-post)) In-Reply-To: (H. Duerer's message of "Wed, 27 Jun 2007 21:17:23 +0100") References: Message-ID: On Wed, 27 Jun 2007 21:17:23 +0100, "H Duerer" wrote: > sorry about that first email -- I am still learning gmail and hadn't > realised I was rudely sending HTML. The first email was multipart/alternative which is fine, at least for Gnus users like me. The problem with the second email was that Gmail inserted gratuitous line breaks which rendered the (inlined) diff unusable. Anyway, thanks for the patch, it's in 0.9.0. The next time you send a patch, please make sure to update the HTML documentation as well. http://weitz.de/patches.html Cheers, Edi.