[cffi-devel] utf-16

[Helmut Eller]

* Luís Oliveira [2010-05-31 20:57+0200] writes:

> On Mon, May 31, 2010 at 1:32 PM, Helmut Eller <heller at common-lisp.net> wrote:
>> Is this assertion
>>
>> (assert
>>  (equal
>>  (cffi:with-foreign-string (data "foobar" :encoding :utf-16)
>>    (cffi:foreign-string-to-lisp data :count 6 :encoding :utf-16))
>>  "foobar"))
>>
>> expected to fail?
>
> FOREIGN-STRING-ALLOC currently doesn't insert BOM markers and the
> :UTF-16 encoder uses native endianness. The :UTF-16 decoder defaults
> to big endian when a BOM marker is not present, as recommended by the
> Unicode specification. That's why the assertion fails.
>
> One solution is to use :UTF-16BE or :UTF-16LE explicitly, which
> unfortunately aren't implemented in Babel yet.

Thanks. I will write my on decoder then. 

Helmut