[Ecls-list] CFFI and DFFI

[Luis Oliveira]

Juan Jose Garcia-Ripoll
<juanjose.garciaripoll at googlemail.com> writes:

>> Can you be more specific? What CFFI types are problematic?
>
> As I said, passing structures and varargs is not supported by UFFI.

CFFI doesn't support passing structures by value either, I'm
afraid. Liam Healy has implemented such support using CFFI bindings to
libffi, though.

  <http://common-lisp.net/pipermail/cffi-devel/2009-April/003090.html>

Varargs is indeed not very well supported by most backends, but seems to
work well enough on x86 (and PPC, IIRC) for stuff that really needs it,
like cl-objc.  However, as you mention, the cffi-ecl backend is in a
perfect position to support it properly.


>> Perhaps using libffi (for DFFI) would be a good, solid approach?
>> <http://sourceware.org/libffi/>
>
> I do not know about libffi. Is it just a child of libffcall? Because
> that one is no less a hack than our code is, but that is not the
> point. Those libraries, just like DFFI, represent an unnecessary
> overhead for something that could be done efficiently in ECL.

There doesn't seem to be a relation between libffi and libffcall but I
wouldn't be surprised if the implementations are similar. It seems that
the advantage in using libffi -- as opposed to rolling your own -- is
that it is widely used by other language implementations such as
CPython, Squeak and Java Native Access.


> CFFI could allow both development models, focusing on the static
> one for platforms, such as GCL or ECL, on which this might be
> advantageous.

Sure.


>> BTW, regarding the OP's problem, some UFFI libraries such as CLSQL pass
>> non-constant arguments to UFFI:LOAD-FOREIGN-LIBRARY as well. I assume
>> that would be a problem for ECL as well.
>
> That is not the only possibility. When the library is not constant we
> can resort to the non-constant version, which will be much slower, but
> UFFI is simple enough that we can intercept the name of the library
> and pass it to the linker when the file using UFFI is compiled.
> This could also be done with CFFI, but its backend only provides
> pointers, not really names.

The cffi-ecl backend discards the name for some reason I don't
understand.


> UFFI does not vararg functions it because it is not really portable.
> Even on the same platform different compilers may choose different
> conventions. But in any case, CFFI does not make use of the vararg
> information either. The code that generates the function call in the
> CFFI-ECL port has to compile _all_ function calls as varargs and hope
> that works. Please have a look at it. The reason is that the CFFI
> backend has not been passed that information to the code that
> generates the function call. I explain it below.

Indeed not trying to support varargs is much easier than trying to. :-)
But we can fix this for cffi-ecl very easily by passing the varargs
information to cffi-sys:%foreign-funcall since the mechanism for passing
other similar options is already there.


> In this case, as I said before, if instead of using the prototypes
> based on dlopen + dlsym (I mean loading library and searching for
> symbols), if the current backend was allowed to use the C names of the
> functions and produce a C code that is compiled once and used, then I
> believe it should be possible to create better prototypes and even
> keep compiler macros that expand later calls, making them more
> efficient.

I have no idea why the cffi-ecl backend doesn't use the C names. It
certainly has access to them.


> Ideally, CFFI should also allow specifying C headers that provide the
> same information and let the backend either use the header or the type
> information which is provided.

That sounds like a good recipe to have CFFI code only work properly on
ECL but not on the other Lisps.


> Is it really such a big problem in the C world? Don't we have -l flags
> that take care of resolving library names? The backend could be taught
> to strip the library names, passing the search path to the linker
> together with a list of -lA -lB ... etc

Good point. I'll look at ECL's FFI:LOAD-FOREIGN-LIBRARY to see how it
works.

It seems to me that your objections to CFFI pertain to the current
implementation of the cffi-ecl backend and not to the architecture of
CFFI per se. I suppose that's good news since the shortcomings you
described sound straightforward to fix.

Thanks for your comments,

-- 
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/