Shared/static libraries in static-program-op context

[Stelian Ionescu]

On Sun, 2020-05-24 at 21:57 +0200, Florian Margaine wrote:
> Good morning cffi-devel,
> 
> I have been recently sending a bunch of pull requests:
> 
> - https://github.com/cffi/cffi/pull/162
> - https://github.com/cffi/cffi/pull/157
> 
> Which are really the following issue:
> 
> - CFFI loads a static foreign library, either as :grovel-wrapper or
> :system
> - The image is dumped with the loaded library
> - When restored, the runtime will try to reload those "shared"
> libraries
> and fails because the path doesn't exist on the target system
> 
> Specifically for the 2nd one, CFFI provides a :c-file ASDF
> integration,
> which, when used with :static-program-op, will compile the C file
> into an
> object file (among others), load it as a foreign library, and
> statically
> link the object file with the program.
> 
> In order to make it possible to distribute this static-program-op
> output,
> we essentially need to close the foreign library (IOW unload it), or
> not
> load it to begin with, before dumping the image. That means tracking
> those.
> The 2nd PR I made, as sionescu pointed out, doesn't work because C-
> FILE
> code isn't aware of whether it's going to be used for static-program-
> op or
> not.So I can see 2 ways out of this:
> 
> - Either we make FOREIGN-LIBRARY-TYPE settable from the outside
> (remember
> that CFFI-TOOLCHAIN is not in the CFFI package), which doesn't sound
> like
> the correct way to go to me?
> - Or we add a CLOSE-ON-DUMP option, which we can use when registering
> the
> library. (Btw, that means exposing the REGISTER-FOREIGN-LIBRARY
> symbol, but
> that sounds uncontroversial to me.)

I don't think either of those are a good idea because they're not
solving the problem at its core: 1) the wrapper shared objects are
loaded by absolute path and 2) the filesystem layout differs between
host and target machine

To fix that you need to either
 * keep the .so as they are, and have static-program-op output a
directory containing the output binary and all the wrappers, then fixup
the wrapper paths so that the runtime finds the shared objects in their
new location (might need to close them and have them re-opened on
start). This is what most other dynamic language runtimes do (Perl,
Python, etc...)
 * statically link the wrapper: make the wrapper op also generate a
static archive (.a), then link to that in static-program-op

--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/cffi-devel/attachments/20200524/37cd6a50/attachment.sig>