<div dir="ltr">Jingtao,<div><br></div><div style>please refer to <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7">http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7</a>, it clearly describes that a media type consists of exactly one type/subtype indicator followed by optional attribute=value pairs. The content type that you have presented is not valid according to these rules. Neither a lax parser like the one in CL-HTTP nor the fact that a large site sends these bogus headers makes them valid. I do not want to include code in Hunchentoot that tries to interpret such bogus data.</div>
<div style><br></div><div style>However, if you cannot get your trading partner to fix their client, I can offer this solution:</div><div style><br></div><div style><div>(defclass request-with-bad-content-type (hunchentoot:request)</div>
<div> ())</div><div><br></div><div>(defmethod hunchentoot:header-in :around ((name (eql :content-type)) (request request-with-bad-content-type))</div><div> (alexandria:when-let (content-type (call-next-method))</div><div>
(ppcre:regex-replace-all "^([^/]+/[^/]+); *[^/]+/[^/;]+" content-type "\\1")))</div><div><br></div><div style>You'll then have to use the :request-class argument to your acceptor instantiation to make it use the request-with-bad-content-type class. You also want to review the regular expression carefully and maybe profile your application to see whether you need to cache or otherwise improve performance.</div>
<div style><br></div><div style>-Hans</div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, May 26, 2013 at 5:07 AM, Jingtao Xu <span dir="ltr"><<a href="mailto:jingtaozf@gmail.com" target="_blank">jingtaozf@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Hans,<br>
<br>
I don't agree with you to say that this content type header is just bogus.<br>
As the content-type is sent by the largest B2B/B2C site in china, it<br>
must have a reason.<br>
<br>
And if you try cl-http, you can find that cl-http will parse such<br>
content type correctly.<br>
<br>
-----------------------------------------------------------------------------<br>
(parse-mime-content-type-header "application/x-www-form-urlencoded;<br>
text/html; charset=UTF-8")<br>
==> (:APPLICATION :X-WWW-FORM-URLENCODED :CHARSET :UTF-8)<br>
-----------------------------------------------------------------------------<br>
<br>
You can find the definition in cl-http/server/headers.lisp<br>
-----------------------------------------------------------------------------<br>
(define-header-type :content-type-header (:header)<br>
:parse-function parse-mime-content-type-header<br>
:print-function print-mime-content-type-header)<br>
-----------------------------------------------------------------------------<br>
<br>
Even this content-type header is bogus(actually I don't think so),<br>
hunchentoot/drakma should parse<br>
the header without raising an error if one special variable like *<br>
accept-bogus-content-type* is true.<br>
<br>
<br>
With Best Regards,<br>
jingtao.<br>
<div class="HOEnZb"><div class="h5"><br>
On Sat, May 25, 2013 at 8:11 PM, Hans Hübner <<a href="mailto:hans.huebner@gmail.com">hans.huebner@gmail.com</a>> wrote:<br>
> Jingtao,<br>
><br>
> the content-type header "application/x-www-form-urlencoded; text/html;<br>
> charset=UTF-8" is just bogus. I do not want to include code that makes<br>
> Hunchentoot work with clearly broken clients. Better error reporting would<br>
> be acceptable, though.<br>
><br>
> -Hans<br>
><br>
><br>
> On Sat, May 25, 2013 at 12:38 PM, Jingtao Xu <<a href="mailto:jingtaozf@gmail.com">jingtaozf@gmail.com</a>> wrote:<br>
>><br>
>> Hi all,<br>
>><br>
>> I found the content type header which raise the bug in my message.log<br>
>> generated by hunchentoot.<br>
>> It happened when hunchentoot get following content type header:<br>
>><br>
>><br>
>> -----------------------------------------------------------------------------------------<br>
>> application/x-www-form-urlencoded; text/html; charset=UTF-8<br>
>><br>
>> -----------------------------------------------------------------------------------------<br>
>><br>
>> I noticed that in package drakma's file read.lisp,function<br>
>> 'get-content-type'<br>
>> also assumed "/" as a token separator.<br>
>><br>
>> I hope package chunga/drakma/hunchentoot could accept such content type<br>
>> header<br>
>> without raising an exception,As Edl said,a new special variable<br>
>> similar to *accept-bogus-eols* or<br>
>> *treat-semicolon-as-continuation* which only assume " ,;" as token<br>
>> separator may be a good idea and will fix my question.<br>
>><br>
>> Any way, RFC standard is not well fit with the read world.<br>
>><br>
>> Thanks very much.<br>
>><br>
>> WIth Best Regards,<br>
>> jingtao.<br>
>><br>
>><br>
>> On Thu, May 23, 2013 at 2:01 PM, Edi Weitz <<a href="mailto:edi@agharta.de">edi@agharta.de</a>> wrote:<br>
>> > I'm not the maintainer anymore, but my take is that if some Ruby or<br>
>> > Java client misinterprets the RFC I wouldn't change Hunchentoot's (or<br>
>> > rather Chunga's) default behavior because of that. I'd rather<br>
>> > introduce a new special variable similar to *accept-bogus-eols* or<br>
>> > *treat-semicolon-as-continuation*.<br>
>> ><br>
>> > Just my .02 Euros,<br>
>> > Edi.<br>
>> ><br>
>> ><br>
>> ><br>
>> > On Thu, May 23, 2013 at 2:52 AM, Jingtao Xu <<a href="mailto:jingtaozf@gmail.com">jingtaozf@gmail.com</a>> wrote:<br>
>> >> Hi All,<br>
>> >><br>
>> >> 1. The function `read-name-value-pair' is called by `<br>
>> >> parse-content-type' in hunchentoo/util.lisp,not by my codes.<br>
>> >> 2. the slash is a token constituent in java/ruby implementation,and I<br>
>> >> think some web client/server treat it as a token constituent too,<br>
>> >> but I am waiting for the hunchentoot log to give us a live example.<br>
>> >><br>
>> >> With Best Regards,<br>
>> >> jingtao<br>
>> >><br>
>> >><br>
>> >> On Wed, May 22, 2013 at 11:40 PM, Edi Weitz <<a href="mailto:edi@agharta.de">edi@agharta.de</a>> wrote:<br>
>> >>> If I'm not mistaken, the slash is a "separator" and thus not a token<br>
>> >>> constituent according to RFC 2616 which means "path=/foo" is not legal<br>
>> >>> input for READ-NAME-VALUE-PAIR.<br>
>> >>><br>
>> >>> On Wed, May 22, 2013 at 5:27 PM, Ron Garret <<a href="mailto:ron@flownet.com">ron@flownet.com</a>> wrote:<br>
>> >>>> Very likely Jingtao's code is calling READ-NAME-VALUE-PAIR without<br>
>> >>>> being wrapped in this macro<br>
>> >>>><br>
>> >>>> But there's still a bug in READ-NAME-VALUE-PAIR:<br>
>> >>>><br>
>> >>>> ? (WITH-INPUT-FROM-VECTOR (S (MAP '(VECTOR (UNSIGNED-BYTE 8))<br>
>> >>>> 'CHAR-CODE "path=/foo"))<br>
>> >>>> (chunga:with-character-stream-semantics<br>
>> >>>> (CHUNGA:READ-NAME-VALUE-PAIR S)))<br>
>> >>>> ("path" . "")<br>
>> >>>><br>
>> >>>> On May 22, 2013, at 8:19 AM, Edi Weitz wrote:<br>
>> >>>><br>
>> >>>>> On Wed, May 22, 2013 at 4:18 PM, Ron Garret <<a href="mailto:ron@flownet.com">ron@flownet.com</a>> wrote:<br>
>> >>>>>> I found a bug in CHUNGA:READ-NAME-VALUE-PAIR.<br>
>> >>>>><br>
>> >>>>> It's not quite clear to me yet what the bug is supposed to be.<br>
>> >>>>><br>
>> >>>>> The documentation clearly says that calls to READ-NAME-VALUE-PAIR<br>
>> >>>>> and<br>
>> >>>>> friends must be wrapped with this macro:<br>
>> >>>>><br>
>> >>>>> <a href="http://weitz.de/chunga/#with-character-stream-semantics" target="_blank">http://weitz.de/chunga/#with-character-stream-semantics</a><br>
>> >>>>><br>
>> >>>>> (You might argue that this isn't very user-friendly, but Chunga<br>
>> >>>>> wasn't<br>
>> >>>>> really intended to be used that way.)<br>
>> >>>><br>
>> >><br>
><br>
><br>
</div></div></blockquote></div><br></div>