[cffi-devel] CFFI-FSBV: libffi-type-pointer for foreign structures created when needed

[Liam Healy]

For those interested, some details and progress report on the merging
of FSBV into CFFI.

On Saturday, I was about to commit the latest changes to the fsbv
branch of CFFI, and I decided at the last minute to swap the load
order of cstruct and cif in cffi-fsbv.asd.  Previously, I had cif.lisp
loading first, and then cstruct.lisp, but it made more sense to me to
put it after, because it has to do with functions, and everything
else, including cstruct.lisp, did not.  Much to my surprise, I got an
error, and I quickly realized why: cif defines a structure that libffi
needs, and when it is defined prior to cstruct, it made an ordinary
foreign structure, without the *cstruct-hooks* I had made that define
a foreign structure with the libffi-type pointers.  When it's defined
after, it got an error because some of the slots have types, like
"unsigned" that are undefined in the FSBV sense (i.e., there's no
libffi-type for it).

The immediate bug fix would be to just define the types.  Eventually
I'll do that.  But the real issue is that in that design, every type
has to have libffi-type information, even if there's no intent to use
that structure in a call by value.  In the standalone FSBV, that
wasn't an issue, because there were two macros, cffi:defcstruct and
fsbv:defcstruct, and presumably no one used the latter unless they
were going to pass or receive the foreign structure by value.  Now all
structures once defined would have the libffi plumbing defined,
whether they needed it or not.

So I changed my approach, and in the process simplified the code a
lot.  Instead of making the necessary libffi-type in the defcstruct
expansion, it will be made on demand only, when a foreign struct is
used by, e.g. defcfun.  This happens recursively, so if one structure
uses another, both will be made.  Once they structure is made, it is
attached to the plist of the symbol for the type, so it need not be
recomputed.

I'm now dealing with built-in types and coming up against a problem
I've struggled with before (in GSLL and Antik): how to map C types
(:short, :int, etc.) into established byte sizes (:uint8, :uint16).
The problem is that libffi defines the types by their size, and it's
up to the user to figure out how to map the common C types to those
sizes; this is a topic in C that has always confused me.  I notice
that CFFI has a mapping in the code at the end of src/types.lisp, but
it's not easily accessible to other code, so I think if I break that
out as some kind of table it will be useful to generate the
appropriate pointers for libffi.  Unless someone has a better way.

Liam