[cffi-devel] New foreign library interface

[James Bielman]

"Hoehle, Joerg-Cyril" <Joerg-Cyril.Hoehle at t-systems.com> writes:

> I feel it's a waste to reimplement in CFFI what UFFI already had to
> fuss around with directories, file extensions, features etc.,
> reimplementing itself what's possibly in various Lisp
> implementations.
>
> I nuked all of that crap in my own UFFI wrappers.

This is emphatically *not* a reimplementation of UFFI's library code.

For system libraries, there should not ever be a need to specify a
directory in DEFINE-FOREIGN-LIBRARY.  We simply pass the library name
straight to 'dlopen' or the equivalent, so it can perform the normal
search, based on ld.so.conf on Linux, searching PATH on Windows, etc.

The reason we also support specifying a pathname is for applications
that want to bundle shared libraries with their distribution; they may
not want to perform the usual search.

*FOREIGN-LIBRARY-DIRECTORIES* exists as an escape hatch for users that
want to extend this behavior (for example, if you need to load
libraries that aren't in ld.so.conf or PATH).

> That association is not even in package UFFI (I put it in
> USER). That way, it can be defined even before UFFI is loaded
> (e.g. .clisprc).  You don't need tricky constructions to the effect
> of being able to load UFFI first, patch the paths, then load system
> foo.  The proposals I've seen here for CFFI looked to me like they
> might get this wrong.
>
>>   (define-foreign-library opengl (:darwin (:framework "OpenGL"))
>> (:unix (:alternatives "libGL.so" "libGL.so.1"
>> #p"/myhome/mylibGL.so"))
>
> This is exactly what I criticize.  You, the programmer, do you know
> the library directories, extensions, version numbers of OpenGL on
> all platforms of the users of your library?  Do you know all the
> appropriate *features*?  Do you have means to test what you write
> yourself?  Presumably, you don't.

IMHO, if I'm going to link against symbols in a shared library, I had
better be aware of what major version I'm loading.  A better example
would probably look like:

(define-foreign-library opengl
  (:linux (:or "libGL.so.1" "libGL.so"))
  (:freebsd (:or "libGL.so.whatever" "libGL.so"))
  (:openbsd ...)
  ...)

The "libGL.so" entries are there as a fallback, and may or may not
work---if you want to be really careful you could omit them.  Note
that there's nothing in here about directories; the dynamic linker
will do that job for us.

I don't see how the user is in the position to specify in their CL
init file which version of a library a particular program is linked
against.

With respect to *FEATURES*, each CFFI backend is responsible for
working out the operating system and platform, and pushing the
appropriate symbol in the CFFI-FEATURES package to *FEATURES*.  These
are what the DEFINE-FOREIGN-LIBRARY macro checks against.

> A Debian distributor might know, but not the code maintainer of an
> individual package. Therefore, again, a central algorithm & means of
> mapping makes more sense that people's wild guesses, different for
> each package and author.

I disagree---these aren't wild guesses; the developer of a library
should know what version of a library he is linking against, and
document this information for each platform he's tested on.  Normally
gcc would encode these dependencies into the executable at
compile-time, but since we are loading at run-time, the programmer
needs to state the version explicitly.

> I very much prefer (defcfun "myfun" :module/library
> :my-cookie-designator :arguments ...)

I don't think these things are mutually exclusive.  If/when we add a
:LIBRARY argument, one of the things it would accept is a "logical"
library defined by DEFINE-FOREIGN-LIBRARY:

(defcfun "glEnd" :library opengl :void nil)

The reason I'm reluctant to add :LIBRARY immediately is that several
of the implementations supported by CFFI do not support it at all.  If
we are willing to put up with it being advisory on these
implementations, then it could be added without too much difficulty.

James