[hunchentoot-devel] Hard to catch errors below process-request

[Hans Hübner]

Andrew,

I have been trying to reproduce the problem using CCL 1.6, but it does
not occur for me.  Can you please prepare a .lisp file that is
self-contained and that I can load to reproduce the problem that you
see?

Thanks,
Hans

On Thu, Jan 20, 2011 at 11:20 PM, AVS <lispercat at gmail.com> wrote:
> The simplest I can think of is:
> (defun handle-page-request ()
>  (let ((str (make-array 0 :element-type 'character :fill-pointer 0
> :adjustable t)))
>    (dolist (char '(#\RIGHT_DOUBLE_QUOTATION_MARK #\t #\e #\s #\t
> #\RIGHT_DOUBLE_QUOTATION_MARK))
>      (vector-push-extend char str))
>    str))
>
> Put this function in a dispatch table using something like
> (hunchentoot:create-regex-dispatcher "test.html" 'handle-page-request)
> When you type in "test.html" in the browser you'll see an empty page.
>
> If you fix the function like this:
> (defun handle-page-request ()
>  (setf (hunchentoot:reply-external-format*)
> (flex:make-external-format :utf-8 :eol-style :lf))
>  (setf (hunchentoot:content-type*) "text/html; charset=utf-8")
>
>  (let ((str (make-array 0 :element-type 'character :fill-pointer 0
> :adjustable t)))
>    (dolist (char '(#\RIGHT_DOUBLE_QUOTATION_MARK #\t #\e #\s #\t
> #\RIGHT_DOUBLE_QUOTATION_MARK))
>      (vector-push-extend char str))
>    str))
>
> You'll see ”test” in the browser.
>
> Andrew
>
>
> On Thu, Jan 20, 2011 at 4:58 PM, Hans Hübner <hans.huebner at gmail.com> wrote:
>> Andrei,
>>
>> thank you for your error description.  I now understand what the
>> problem is and I agree that this is a bug that deserves fixing.  If
>> you don't have a proposal how it can be fixed, it would be immensely
>> helpful if you could provide me with a .lisp file that exposes the
>> problem (maybe using drakma to send a request that makes the handler
>> fail in the manner that you describe), as that would make it easier to
>> start working on the solution.
>>
>> Thanks,
>> Hans
>>
>> On Thu, Jan 20, 2011 at 10:45 PM, AVS <lispercat at gmail.com> wrote:
>>> Very simple. I get a string from server A. I expect the string to be
>>> of latin characters. By (their) mistake there are a couple of unicode
>>> characters in the string.
>>> Now I want to use the string as part of my own server request handler
>>> which I do via:
>>> (defun handle-page-request ()
>>>    (no-cache)
>>>    *string-with-unicode*)
>>>
>>> This request will produce an empty page. The developer says: "Why?! No
>>> debugger was triggered, it went completely silent!".
>>> The reason for that was when start-output called (string-to-octets
>>> content :external-format (reply-external-format*)), the default
>>> hunchentoot external-format is latin-1.
>>> Since there are unicode characters in content variable,
>>> string-to-octets will throw an exception which will not show anywhere.
>>> Go figure it was a problem in your content string and that
>>> string-to-octets blew up on it!
>>> I understand that in most cases the developer writes some kind cl-who
>>> code in the handler, or uses some content known to them.
>>> But there are cases when you are dealing with 3-d party content which
>>> you use to form your own.
>>>
>>> To fix their problem I now create a handler like:
>>> (defun handle-page-request ()
>>>    (no-cache)
>>>    (setf (hunchentoot:reply-external-format*)
>>> (flex:make-external-format :utf-8 :eol-style :lf))
>>>    (setf (hunchentoot:content-type*) "text/html; charset=utf-8")
>>>    *string-with-unicode*)
>>>
>>> Now, the problem I was talking about was that these types of errors
>>> are hard to find.
>>> If any exception thrown from within start-output triggered the
>>> debugger I'd find the root cause of the problem in no time.
>>>
>>> Thank you,
>>> Andrei
>>>
>>> On Thu, Jan 20, 2011 at 4:27 PM, Hans Hübner <hans.huebner at gmail.com> wrote:
>>>> On Thu, Jan 20, 2011 at 10:20 PM, AVS <lispercat at gmail.com> wrote:
>>>>> The issue is this: If any of the functions fail when called from
>>>>> start-output, you'll receive an empty response and the debugger will
>>>>> not catch which function caused the error.
>>>>> I described in the previous message that in my case (string-to-octets
>>>>> ..) failed. The failure may depend on your input.
>>>>> I just said that to reproduce that scenario (silent error without
>>>>> debugger being invoked) just put any failing function inside
>>>>> start-output and issue a page request from the browser.
>>>>> You'll receive an empty page and no debugger will be signaled.
>>>>> Maybe it's a normal scenario, but it's hard to find the failing
>>>>> function in this case.
>>>>
>>>> I'm just trying to find out how pressing the problem is - Is it
>>>> something that a user or an application developer may run into, or is
>>>> it something that just happens when you poke around in the source code
>>>> and break it in unexpected ways?  From quickly glancing at
>>>> start-output, I see no calls back into user code, but you have run
>>>> into it nevertheless.  How, why?
>>>>
>>>> -Hans
>>>>
>>>> _______________________________________________
>>>> tbnl-devel site list
>>>> tbnl-devel at common-lisp.net
>>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>>
>>>
>>> _______________________________________________
>>> tbnl-devel site list
>>> tbnl-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>>
>>
>> _______________________________________________
>> tbnl-devel site list
>> tbnl-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/tbnl-devel
>>
>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>