[cffi-devel] a thought on string encodings

[James Bielman]

On Mon, 2006-01-02 at 14:18 +0100, Hoehle, Joerg-Cyril wrote:

> Summary:
> ptr -> Lisp string:
> either ext:convert + memory-as uint8
> or (let ((old *foreign-encoding*))
>      (unwind-protect (progn (setq *foreign-encoding* utf-8)
>        (memory-as pointer (parse-c-type `(c-array character ,known-length)))
>        )(setq *foreign-encoding* old))) ; looks scary :-(
> Do you need unknown-length as well?
> Lisp string -> ptr:
> ext:convert + memory-as uint8 as mentioned above.

Thanks for the detailed explanation.  I did need unknown length, here's
what I'm using now to convert to/from:

#+clisp
(defmethod translate-to-foreign ((s string) (name (eql 'utf8-string)))
  (let* ((bytes (ext:convert-string-to-bytes s charset:utf-8))
         (length (length bytes))
         (buf (foreign-alloc :unsigned-char :count (1+ length))))
    (setf (ffi:memory-as ptr (ffi:parse-c-type
                              `(ffi:c-array ffi:uint8 ,length))) bytes)
    (setf (mem-aref buf :unsigned-char length) 0)
    buf))

#+clisp
(defcfun "strlen" :unsigned-int
  (s :pointer))

#+clisp
(defmethod translate-from-foreign (ptr (name (eql 'utf8-string)))
  (let* ((length (strlen ptr))
         (bytes (ffi:memory-as ptr (ffi:parse-c-type
                                    `(ffi:c-array ffi:uint8 ,length)))))
    (ext:convert-string-from-bytes bytes charset:utf-8)))

Now that this is working, I will start pulling the
implementation-specific code into an encoding-aware 
%FOREIGN-STRING-TO-LISP function in CFFI-SYS.

James