[tbnl-devel] Strange encoding problems with SBCL and TBNL

[Edi Weitz]

On 26 Jun 2006 14:30:18 +0900, Dan Beauchesne <dbeauchesne at gmail.com> wrote:

> I'm running SBCL 0.9.13.52 (same error with 0.9.13 release) and TBNL
> 0.9.10.
>
> I've got a simple page set up that I need to simply print the return
> value of an external function. That function returns a list
> containing UTF-8 characters. I've set *show-lisp-errors-p* to T,
> here is the error:
>
> encoding error on stream #<SB-SYS:FD-STREAM for "standard output" {A6877B9}>
> (:EXTERNAL-FORMAT :ASCII):
>   the character with code 20170 cannot be encoded.
>
> I had done some digging on google and found someone in a similar
> situation with TBNL, and he tried changing his locale LC_ALL to a
> utf-8 one. The above (:EXTERNAL-FORMAT :ASCII part was :ASCII only
> after I set my locale to UTF-8, beforehand it was :LATIN-1, which I
> do not understand.

Maybe you used a locale which wasn't recognized and therefore the
system defaulted to ASCII?

> Here is the code, if it helps:
>
> (in-package :tbnl-foo)
>
> (setq *dispatch-table*
>       (list (create-prefix-dispatcher "/tbnl/foo" 'foo)
> 	    'default-dispatcher))
>
> (defun foo ()
>   (let ((foo (jalat::search-hash "hello")))
>     (with-html-output (*standard-output*)
>       (:h2 (str foo)))))

Note that even if everything else works the "Content-Length" header
will probably be wrong.

> (setq *dispatch-table*
>       (nconc
>        (mapcar (lambda (args)
> 		 (apply #'create-prefix-dispatcher args))
> 	       '(("/tbnl/test/" foo)))
>        (list #'default-dispatcher)))
>
> The external function simply returns a list of plists containing
> UTF-8 characters. I also tried setting
> *tbnl-default-external-format* to no avail.

The main problem seems to be to find out how one has to set up the
stream to be able to send arbitrary Unicode characters.  This is not
really related to TBNL.  If you don't get help here, I'd propose to
ask on the SBCL mailing list.

Cheers,
Edi.