From reddaly at gmail.com  Sun Oct  4 20:39:49 2009
From: reddaly at gmail.com (Red Daly)
Date: Sun, 4 Oct 2009 13:39:49 -0700
Subject: [drakma-devel] URL-encode and rfc3986
Message-ID: <f6700f0c0910041339s366b43e2x8d0ac7c899c6bbe7@mail.gmail.com>

Drakma encodes spaces in URLs as pluses instead of "%20"s.  It also leaves
the characters "$-_.!*'()" unencoded instead of the RFC3986 unencoded set:
"-_.~".

This is okay for tolerant servers but for other applications it is a bug.
For example, in OAuth it is necessary to conform to RFC3986 or the signature
will be incorrect for the request.

To make Drakma more RFC3986-compliant, replace these four lines in
util.lisp:

                        (find char "$-_.!*'()," :test #'char=))
                     (write-char char out))
                   ((char= char #\Space)
                     (write-char #\+ out))

with these two:
                        (find char "-_.~" :test #'char=))
                     (write-char char out))

Best regards,
Red
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/drakma-devel/attachments/20091004/99500c54/attachment.html>

From edi at agharta.de  Mon Oct  5 06:09:23 2009
From: edi at agharta.de (Edi Weitz)
Date: Mon, 5 Oct 2009 08:09:23 +0200
Subject: [drakma-devel] URL-encode and rfc3986
In-Reply-To: <f6700f0c0910041339s366b43e2x8d0ac7c899c6bbe7@mail.gmail.com>
References: <f6700f0c0910041339s366b43e2x8d0ac7c899c6bbe7@mail.gmail.com>
Message-ID: <b56421a00910042309o495f03f9vaaaed53a4d6e4058@mail.gmail.com>

On Sun, Oct 4, 2009 at 10:39 PM, Red Daly <reddaly at gmail.com> wrote:
> Drakma encodes spaces in URLs as pluses instead of "%20"s.? It also leaves
> the characters "$-_.!*'()" unencoded instead of the RFC3986 unencoded set:
> "-_.~".
>
> This is okay for tolerant servers but for other applications it is a bug.
> For example, in OAuth it is necessary to conform to RFC3986 or the signature
> will be incorrect for the request.
>
> To make Drakma more RFC3986-compliant, replace these four lines in
> util.lisp:
>
> ??????????????????????? (find char "$-_.!*'()," :test #'char=))
> ???????????????????? (write-char char out))
> ?????????????????? ((char= char #\Space)
> ???????????????????? (write-char #\+ out))
>
> with these two:
> ??????????????????????? (find char "-_.~" :test #'char=))
> ???????????????????? (write-char char out))

Thanks for the info.  If you can send a proper patch I'll make a new
release immediately.

  http://weitz.de/patches.html

Edi.



From lispercat at gmail.com  Wed Oct  7 17:00:24 2009
From: lispercat at gmail.com (Andrei Stebakov)
Date: Wed, 7 Oct 2009 13:00:24 -0400
Subject: [drakma-devel] drakma:get-cookies fails to parse cookies with
	commas in values
In-Reply-To: <b56421a00909301458tc070ba8qa1b11051f3923611@mail.gmail.com>
References: <affe56e00909161359r37f51a8eva4bf026926bad9af@mail.gmail.com>
	<b56421a00909300209u22afc77evd5fc6570d9abbbd@mail.gmail.com>
	<b56421a00909300741v58fda8e4s244d81618b4d3a6b@mail.gmail.com>
	<affe56e00909300835q45e8df2bt9b0f6f1b93d0e278@mail.gmail.com>
	<b56421a00909301458tc070ba8qa1b11051f3923611@mail.gmail.com>
Message-ID: <affe56e00910071000l26bf1ca0mf0b0bf7a496c70ae@mail.gmail.com>

I think those are two different issues we are taking about.
In your case, yes, most likely bar=baz makes a new cookie (according to the
RFC).
In my case "session=6,Direct,placeholder,test.com;" is an obvious
attribute-value pair followed by a ";". (as per RFC: "av-pairs = av-pair
*(";" av-pair)"  )
What I am trying to say that drakma shouldn't stumble upon the comma after
"6", since the next construct is not "name=value", but only a token.
I agree, that they break a rule set by the RFC 2068, which defines a token
as
    token          = 1*<any CHAR except CTLs or tspecials>
where
    tspecials      = "(" | ")" | "<" | ">" | "@"
                         | "," | ";" | ":" | "\" | <">
                         | "/" | "[" | "]" | "?" | "="
                         | "{" | "}" | SP | HT

So, having a comma in a token "6,Direct,placeholder,test.com" is against the
rule but in this case it's still easily identifiable as value for the
"session=6,Direct,placeholder,test.com;" av-pair so it needs to be fixed or
some api provided so this issue could be overcome.

Thank you,
Andrei


On Wed, Sep 30, 2009 at 5:58 PM, Edi Weitz <edi at agharta.de> wrote:

> On Wed, Sep 30, 2009 at 5:35 PM, Andrei Stebakov <lispercat at gmail.com>
> wrote:
> > Looks like according to RFC 2109, "=" takes priority over "," so probably
> > when we encounter something like session=foo,bar=baz, the parser should
> > analyze sequences on both sides of an "=" character, so in this case
> comma
> > becomes a separator of two different pairs.
>
> Ah, that's something I've been missing so far.  Can you point to where
> exactly this can be found in the RFC?  That should make the cookie
> parsing code clearer and I should be able to get rid of the comma
> workaround which is already in there.
>
> Thanks,
> Edi.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/drakma-devel/attachments/20091007/cea3b18a/attachment.html>

From sthalik at test123.ltd.pl  Sat Oct 10 20:41:21 2009
From: sthalik at test123.ltd.pl (Stanislaw Halik)
Date: Sat, 10 Oct 2009 22:41:21 +0200
Subject: [drakma-devel] [patch] timeouts, take two
References: <h750pf$pfn$1@ger.gmane.org>
Message-ID: <87d44vdlse.fsf@test123.ltd.pl>

Stanislaw Halik <sthalik at test123.ltd.pl> writes:

> I made a new patch that [...]

Meow?

Edi, could you take a look at it?

-- 
polscy Lispnicy, ??czmy si? -- #lisp-pl na freenode
Lisp stuff: <http://tehran.lain.pl>




From ckonstanski at pippiandcarlos.com  Tue Oct 13 22:04:51 2009
From: ckonstanski at pippiandcarlos.com (Carlos Konstanski)
Date: Tue, 13 Oct 2009 16:04:51 -0600 (MDT)
Subject: [drakma-devel] NTLM auth headers?
Message-ID: <alpine.LNX.2.00.0910131559040.32032@localhost.localdomain>

Is there any way to get drakma to write an NTLM auth header? I am
unfortunately trying to communicate with a .NET web service in an
Active Directory environment.

If not, I suppose I could use ?`additional-headers' to write one
myself.

Carlos Konstanski

From edi at agharta.de  Wed Oct 14 10:57:22 2009
From: edi at agharta.de (Edi Weitz)
Date: Wed, 14 Oct 2009 12:57:22 +0200
Subject: [drakma-devel] NTLM auth headers?
In-Reply-To: <alpine.LNX.2.00.0910131559040.32032@localhost.localdomain>
References: <alpine.LNX.2.00.0910131559040.32032@localhost.localdomain>
Message-ID: <b56421a00910140357k4b45f761pd14ae69a4c521165@mail.gmail.com>

On Wed, Oct 14, 2009 at 12:04 AM, Carlos Konstanski
<ckonstanski at pippiandcarlos.com> wrote:
> Is there any way to get drakma to write an NTLM auth header? I am
> unfortunately trying to communicate with a .NET web service in an
> Active Directory environment.
>
> If not, I suppose I could use ?`additional-headers' to write one
> myself.

Yes, as long as we don't receive a patch for this functionality,
you'll have to send the headers manually.

Edi.



From eugoss at gmail.com  Sat Oct 17 06:12:14 2009
From: eugoss at gmail.com (Eugene Ossintsev)
Date: Sat, 17 Oct 2009 02:12:14 -0400
Subject: [drakma-devel] GET :PARAMETERS added again during redirects
Message-ID: <6ba04de90910162312s179b6f48j2c00716e340a9177@mail.gmail.com>

I use DRAKMA 1.0, Clozure CL 1.4-dev (Ubuntu Linux x86). According to
the DRAKMA documentation, parameters is an alist of name/value pairs
(the car and the cdr each being a string) which denotes the parameters
which are added to the query part of the URI or (in the case of a POST
request) comprise the request body.

In my example QUOTES1 sends a GET request passing the request
parameters in the URI, and QUOTES2 uses the :PARAMETERS instead. The
results, unfortunately, are not the same.


(defun quotes1 ()
? (multiple-value-bind (body code)
????? (drakma:http-request
"http://finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1")
??? (when (= code 200)
????? (split-sequence:split-sequence #\, body))))

("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\" ^M
")                 <=== It's fine :-)


(defun quotes2 ()
? (multiple-value-bind (body code)
????? (drakma:http-request "http://finance.yahoo.com/d/quotes.csv"
?????????????????????????? :method :get
?????????????????????????? :parameters '(("s" . "XIC.TO")
???????????????????????????????????????? ("f" . "sxl1d1t1")))
??? (when (= code 200)
????? (split-sequence:split-sequence #\, body))))

("\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\""
"\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03"
"\"10/9/2009\"" "\"3:59pm\" ^M
\"XIC.TO\"" "\"Toronto\"" "18.03" "\"10/9/2009\"" "\"3:59pm\""
"\"XIC.TO\"" "\"XIC.TO\"" "\"XIC.TO\"" "\"Toronto\"" "18.03"
"\"10/9/2009\"" "\"3:59pm\" ^M
")                 <=== It's a mess :-(


TRACE DRAKMA:HTTP-REQUEST shows that in case of redirects of GET
requests the parameters from :PARAMETERS are added again to the URI.
Thus, the same parameters may be added again and again, giving
unpredictable results.

0> Calling (DRAKMA:HTTP-REQUEST
"http://finance.yahoo.com/d/quotes.csv" :PARAMETERS (("s" . "XIC.TO")
("f" . "sxl1d1t1")))
?1> Calling (DRAKMA:HTTP-REQUEST #<URI
http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1>
:REDIRECT 4 :STREAM NIL :ADDITIONAL-HEADERS NIL :PARAMETERS (("s" .
"XIC.TO") ("f" . "sxl1d1t1")))
?<1 DRAKMA:HTTP-REQUEST returned 7 values :
?<1????? #(34 88 73 67 46 84 79 34 44 34 84 111 114 111 110 116 111 34
44 49 55 46 57 56 44 34 49 48 ..........
?<1????? 200
?<1????? ((:DATE . "Wed, 14 Oct 2009 05:00:51 GMT") (:P3P .
"policyref=\"http://p3p.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR CUR
ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi
PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE
GOV\"") (:CACHE-CONTROL . "private") (:CONNECTION . "close")
(:TRANSFER-ENCODING . "chunked") (:CONTENT-TYPE .
"application/octet-stream"))
?<1????? #<URI http://download.finance.yahoo.com/d/quotes.csv?s=XIC.TO&f=sxl1d1t1&s=XIC.TO&f=sxl1d1t1>
?<1????? #<FLEXI-STREAMS:FLEXI-IO-STREAM #x15616B76>
?<1????? T
?<1????? "OK"


Is it what may be expected from such usage of :PARAMETERS in GET
requests? If not, is it worth fixing it?


- Eugene



From edi at agharta.de  Mon Oct 19 09:34:12 2009
From: edi at agharta.de (Edi Weitz)
Date: Mon, 19 Oct 2009 11:34:12 +0200
Subject: [drakma-devel] [patch] timeouts, take two
In-Reply-To: <87d44vdlse.fsf@test123.ltd.pl>
References: <h750pf$pfn$1@ger.gmane.org> <87d44vdlse.fsf@test123.ltd.pl>
Message-ID: <b56421a00910190234j4b2072bqa456abd476b48010@mail.gmail.com>

On Sat, Oct 10, 2009 at 10:41 PM, Stanislaw Halik
<sthalik at test123.ltd.pl> wrote:

> Meow?

Sorry for the delay... :)

> Edi, could you take a look at it?

Thanks, that looks OK to me.  The question is whether any of the
non-LispWorks users has problems with this.  Anyone?

For the patches to be accepted, please have a look at

  http://weitz.de/patches.html

Note that you'll need docstrings, the HTML documentation has to be
updated, and there are some style inconsistencies (for example #+: vs.
#+) amongst other things.

Thanks,
Edi.