[cffi-devel] Converting foreign structures with CFFI generic functions

[Luís Oliveira]

On Sun, Sep 4, 2011 at 3:33 AM, Liam Healy <lnp at healy.washington.dc.us> wrote:
> I've written a macro define-structure-conversion to flesh out what I see as
> the way to go.  Eventually, I think this would be best as a couple of key
> arguments in the defcstruct, similar to FSBV's current :constructor and
> :deconstructor arguments.

I'm not convinced we need a macro like that. Certainly not one
specific to structures. Defining translate-* methods is indeed a bit
more verbose, but it's also plain CLOS, which makes it clear and
flexible. (E.g., you easily can use inheritance and multiple dispatch
when defining CFFI types if you already know CLOS.)


> I am stuck however on recursive conversion of slots.  What I have works for
> the complex example, but if I built on that with a  "real-and-complex"
> structure and try to convert to foreign, I get an error because the complex
> slot is already converted by the time it gets in the setf.  Even with an
> existing pointer, as would be obtained from foreign-slot-pointer, the only
> function I have at my disposal for translating the slot is
> translate-to-foreign which creates a new foreign struct, it does not write
> to an existing struct.  It seems I need something like a setf function.

    (defcstruct (real-and-complex :class real-and-complex)
     (x :double)
     (c complex-double-c))

    (define-structure-conversion value real-and-complex list (x c)
      ;; Make foreign
      ;;(setf x (first value) c (convert-to-foreign (second value)
'complex-double-c))
      (setf x (first value) c (second value))
      ;; Make CL
      (list x c))

I see a couple of issues raised by your "real-and-complex" example:
  (1) recursive translations.
  (2) differentiating between value/reference semantics for the
translation of aggregate types.
  (3) reusing translation code for both semantics.

Regarding (1), at first sight, it seems to me that if we implement
some sort of default translation from structures to CLOS objects, we
can handle recursiveness there. User-provided translations are on
their own, i.e. they should call convert-foreign-objects themselves.
(We can export a way to iterate through the slots of a structure if we
don't already.) BTW, it occurs to me that a default cstruct to CL
conversion could use opaque objects whose accessors handle the
translations (and possibly cache them).

Regarding (2), I agree that we need a translate-into-foreign-memory
function that takes 3 arguments: source value, type, and a target
pointer. (IIUC, this would be the setf function you were referring
to.) This will also be useful to properly integrate the array type
with structures, BTW.

Having solved (2) that way, (3) should be straightforward. At some
point, there'll always be a pointer to somewhere, so
translate-to-foreign can call translate-into-foreign-memory, if
applicable.

So, under this proposed scheme, your COMPLEX example would look something like:

(defcstruct (complex :class complex-type)
 (real :double)
 (imag :double))

(defmethod translate-into-foreign-memory ((value complex) (type complex-type) p)
  (with-foreign-slots ((real imag) p 'complex)
    (setf real (realpart value)
          imag (imagpart value))))

(defmethod translate-from-foreign (p (type complex-type))
  (with-foreign-slots ((real imag) p 'complex)
    (complex real imag)))

This would be adequate to implement translation both when embedding
the structure and when passing as a value. (We could also implement a
TRANSLATE-TO-FOREIGN if we wanted to do automatic translation when
passing by reference as well. I guess we probably would want want.)

I like that this transfers the responsibility of allocating a
(temporary?) structure to libffi-specific code.

The REAL-AND-COMPLEX example would in turn look like this:

(defcstruct (real-and-complex :class real-and-complex-type)
  (x :double)
  (c complex))

(defmethod translate-into-foreign-memory (value (type real-and-complex-type) p)
  (setf (foreign-slot-value p 'real-and-complex 'x) (first value))
  (convert-into-foreign-memory (second value)
                               'complex
                               (foreign-slot-pointer p 'real-and-complex 'c)))

(defmethod translate-from-foreign (p (type real-and-complex-type))
  (with-foreign-slots ((x c) p 'real-and-complex)
    (list x c)))

Does this look good to you plumbing-wise? Regarding the external API,
do you still think we need some sort of convenience macro around this?
(I guess the real-and-complex example would benefit from a
WITH-FOREIGN-SLOT-PLACES along the lines of WITH-FOREIGN-PLACE
described in <https://bugs.launchpad.net/cffi/+bug/688532>.)

We'll want matching converto-into-foreign-memory and
expand-into-foreign-memory functions for this new
translate-into-foreign-memory.

Cheers,

-- 
Luís Oliveira
http://r42.eu/~luis/