[cffi-devel] utf-16 encoded strings in ClozureCL on a Win32 Host

[Luís Oliveira]

On Mon, Aug 31, 2009 at 7:47 PM, John Miller <millejoh at mac.com> wrote:
> CFFI> *default-foreign-encoding*
> :UTF-8
> CFFI> (foreign-funcall "_wcsdup" (:string :encoding :utf-16) "foo" :string)
> "f" ;; That looks like the manual
> CFFI> (let ((*default-foreign-encoding* :utf-16))
>        (foreign-funcall "_wcsdup" (:string :encoding :utf-16) "foo" :string))
> "   " ;; Oops, foo in Sumerian cuneiform, maybe?

The problem is that the :UTF-16 encoding assumes big-endianness when a
BOM is not present, and that clearly is not what we want to happen in
this case. Perhaps, in the context of CFFI, we should assume native
endianness? In any case, it should be possible to specify the desired
endianness: native, big, or little.

This is an essential feature I've been meaning to add to Babel.
However, I'll only be able to get to it later next week.

If you need a quick workaround, you can edit the DEFINE-DECODER
:UTF-16 definition in babel/src/enc-unicode.lisp by changing the
default CASE clause (t #+little-endian t) to (t #+little-endian NIL).

HTH.

-- 
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/