A CFFI -ASDF integration bug

[Robert Goldman]

On 4 Aug 2020, at 12:02, Ilya Perminov wrote:

> I think protobuf and CFFI structure their operations in a very similar 
> way
> - process-op is analogous to proto-to-lisp, it takes a "specification" 
> file
> and generates a lisp file and some other files. protobuf generates
> lisp(fasl) files only, so it does not need to do anything special to
> support bundle operations. CFFI's process-op generates some .o and .so
> files that a  bundle operation may need. The current implementation 
> adds .o
> and .so files to outputs of compile-op and it causes the problem I
> described.
> I do not know what methods need to be defined on process-op to make 
> bundle
> operations to pick up its output files. From my very limited 
> understanding
> of ASDF I do not think there is a way to do it. Method
> "component-depends-on ((o gather-operation) (s system))" determines
> input-files of a bundle-op. The method returns dependencies of one
> operation only (e.g. compile-op), but in case of CFFI's wrapper-file 
> we
> need output files of two operations: process-op and compile-op.

I don't claim to understand this process, but wouldn't it be possible 
for you to make your own `input-files :around` method for 
`gather-operation` that would collect the outputs from the 
`process-op`'s and add them to what you want?

Here's the existing definition of what I think is the relevant 
`input-files` method:

```
   (defmethod input-files ((o gather-operation) (c system))
     (unless (eq (bundle-type o) :no-output-file)
       (direct-dependency-files
        o c :key 'output-files
            :test (pathname-type-equal-function (bundle-pathname-type 
(gather-type o))))))
```

This invokes `map-direct-dependencies` which invokes the 
`component-depends-on` method for `gather-operation` on the system which 
... I don't really understand, but I believe it's the `compile-op`'s.

I think it would be easiest to write your own method that collects up 
all of the `process-op` outputs, drops any `.lisp` files (which will be 
superseded by the `.fasl` files), and adds them to the return value of 
`call-next-method`.

If you do that, and drop the `.o` and `.so` files from the 
`output-files` of `compile-op`, I think that would get what you want: 
you would collect the `.o` and `.so` files from the `process-op`, and 
you wouldn't get ASDF trying to regenerate the files when it's not 
necessary.

That said, I can think of a simpler, and easier method, and that would 
be to override the `operation-done-p` method for the `compile-op` so 
that it knows that the `.o` and `.so` files are generated by the 
`compile-op`, and only pays attention to the relationship between 
`bindings-file.lisp` and `bindings-file.fasl`.

One thing you didn't say was what it means that the compile-op is done 
over and over -- is it only compiling the bindings-file, or is it doing 
something to the object files? I don't believe it should change the `.o` 
and `.so` files, because those are already compiled by `process-op`, 
right?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20200804/30d3c4d9/attachment.htm>