Shared/static libraries in static-program-op context

[Florian Margaine]

On Mon, May 25, 2020 at 1:19 AM Stelian Ionescu <sionescu at cddr.org> wrote:

> On Sun, 2020-05-24 at 21:57 +0200, Florian Margaine wrote:
> > Good morning cffi-devel,
> >
> > I have been recently sending a bunch of pull requests:
> >
> > - https://github.com/cffi/cffi/pull/162
> > - https://github.com/cffi/cffi/pull/157
> >
> > Which are really the following issue:
> >
> > - CFFI loads a static foreign library, either as :grovel-wrapper or
> > :system
> > - The image is dumped with the loaded library
> > - When restored, the runtime will try to reload those "shared"
> > libraries
> > and fails because the path doesn't exist on the target system
> >
> > Specifically for the 2nd one, CFFI provides a :c-file ASDF
> > integration,
> > which, when used with :static-program-op, will compile the C file
> > into an
> > object file (among others), load it as a foreign library, and
> > statically
> > link the object file with the program.
> >
> > In order to make it possible to distribute this static-program-op
> > output,
> > we essentially need to close the foreign library (IOW unload it), or
> > not
> > load it to begin with, before dumping the image. That means tracking
> > those.
> > The 2nd PR I made, as sionescu pointed out, doesn't work because C-
> > FILE
> > code isn't aware of whether it's going to be used for static-program-
> > op or
> > not.So I can see 2 ways out of this:
> >
> > - Either we make FOREIGN-LIBRARY-TYPE settable from the outside
> > (remember
> > that CFFI-TOOLCHAIN is not in the CFFI package), which doesn't sound
> > like
> > the correct way to go to me?
> > - Or we add a CLOSE-ON-DUMP option, which we can use when registering
> > the
> > library. (Btw, that means exposing the REGISTER-FOREIGN-LIBRARY
> > symbol, but
> > that sounds uncontroversial to me.)
>
> I don't think either of those are a good idea because they're not
> solving the problem at its core: 1) the wrapper shared objects are
> loaded by absolute path and 2) the filesystem layout differs between
> host and target machine
>
> To fix that you need to either
>  * keep the .so as they are, and have static-program-op output a
> directory containing the output binary and all the wrappers, then fixup
> the wrapper paths so that the runtime finds the shared objects in their
> new location (might need to close them and have them re-opened on
> start). This is what most other dynamic language runtimes do (Perl,
> Python, etc...)
>  * statically link the wrapper: make the wrapper op also generate a
> static archive (.a), then link to that in static-program-op
>

The static link is already what static-program-op is doing. The problem is
that the foreign library is _also_ loaded as a shared library, and is
restored when the image restores itself. Which fails with the wrong paths.


>
> --
> Stelian Ionescu a.k.a. fe[nl]ix
> Quidquid latine dictum sit, altum videtur.
>


-- 
Florian Margaine
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cffi-devel/attachments/20200525/b15a6ab5/attachment.htm>