[fetter-devel] Re: CFFI progress

[James Bielman]

[Hi Kenny, I guess we should have talked about FFIs at ILC too, eh?]

I'm just entering this discussion, so I'm not quite sure what you guys
are planning to do with Hello-C / CFFI / etc.

(Stop me if you know all this already, just making sure since I've
never really officially announced this library or anything...)

As I understand it, the goal of Hello-C is to add callbacks and
support for additional Lisp implementations to a fork of UFFI.

CFFI is a pretty different animal, in that it tries to define an
low-level interface for frobbing memory and foreign functions, then
implement the remainder of the foreign type system in portable code.
Check out the CFFI-SYS draft specification available at
http://common-lisp.net/project/cffi for details on the system
interface.

CFFI can also be more efficient than UFFI on CMUCL and SBCL since it
uses system area pointers directly instead of consing up alien objects
that carry around redundant type information at run-time (and require
lots of type/inline declarations to optimize away).

However, it's also less mature than UFFI.  In fact, it's only 3 weeks
old, and doesn't support as many Lisp implementations, because CFFI is
asks a bit more of the Lisp implementor (providing the low-level
operations to implement the CFFI-SYS interface).

Anyway, I'll elaborate a little on this list of open issues:

Kenny Tilton <ktilton at nyc.rr.com> writes:

> Luis Oliveira wrote:
>>
>> [ideas for things to work on in cffi]
>>
>>   -> Foreign globals. 
>
> Sounds reasonable. I just never encountered that. This means, however,
> an autogenerated C DLL to create a C interface?

UFFI does this already with DEF-FOREIGN-VAR.  I think all Lisps
supported by UFFI can do this without generating any sort of C stubs.

>>   -> FOREIGN-SLOT-VALUE: let it take multiple "indices" for directly
>>      accessing structs inside structs, arrays inside structs,
>
> Hmm, not appearing clearly in my mind's eye. But I get the drift/value
> if feasible.

The idea here is to make FOREIGN-SLOT-VALUE sort of like AREF:

(defcstruct timeval
  (tv-secs :long)
  (tv-usecs :long))

(defcstruct employee
  (name :char 255)
  (hire-date timeval))

(with-foreign-object (emp employee)
  ;;                        obj  type     slot-1     slot-2...
  (setf (foreign-slot-value emp 'employee 'hire-date 'tv-secs) 10204242)
  ;; This is like 'emp.hire_date.tv_secs' in C...
  ...)

>>   -> Type-checking pointer interface. 
>
> OK, interesting. I went the other way: one type-trusting interface,
> one autoconverting interface.

Indeed, I don't really care too much about the type-checking, but
others might.  One of reasons I like defining the bulk of the foreign
type system in terms of low-level memory operations is that we can
experiment with different foreign type systems in a portable way.

This is certainly not a high priority for me though.

> "automatic access" has practically zero value. We will not diverge
> from UFFI gratuitously, so if it was a good interface it will
> survive largely unchanged even with the new underpinnings. If it was
> a sucky interface, we /want/ to kill it. If the latter obtains,
> automatic access will not fly. If, however, the UFFI API survives
> largely intact, a set of UFFI bindings can be transformed in a few
> hours. This equals "practically zero value" for backward
> compatibility.

I'm not really sure what you mean here.  In CFFI, I've pretty much
thrown out the UFFI interface and written my own.  In particular,
everything that can conceivably be a function is a function, in
contrast to UFFI which consists almost entirely of macros (many of
which can be confusing WRT what gets evaluated on which Lisp).

CFFI-UFFI-COMPAT maps the thrown-out UFFI interface back to my new
interface (minus some lossage currently with arrays, CFFI does arrays
quite differently from UFFI).

CFFI-UFFI-COMPAT has quite a bit of value to me because I can build
existing libraries like CLSQL, get a performance boost on SBCL, and
not have to change the code (and indeed I've successfully built the
CLSQL-SQLite3 backend using CFFI-UFFI-COMPAT).

James