<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8">
</head>
<body>
<div style="font-family:sans-serif"><div style="white-space:normal">
<p dir="auto">On 4 Aug 2020, at 12:02, Ilya Perminov wrote:</p>
</div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">I think protobuf and CFFI structure their operations in a very similar way<br>
- process-op is analogous to proto-to-lisp, it takes a "specification" file<br>
and generates a lisp file and some other files. protobuf generates<br>
lisp(fasl) files only, so it does not need to do anything special to<br>
support bundle operations. CFFI's process-op generates some .o and .so<br>
files that a bundle operation may need. The current implementation adds .o<br>
and .so files to outputs of compile-op and it causes the problem I<br>
described.<br>
I do not know what methods need to be defined on process-op to make bundle<br>
operations to pick up its output files. From my very limited understanding<br>
of ASDF I do not think there is a way to do it. Method<br>
"component-depends-on ((o gather-operation) (s system))" determines<br>
input-files of a bundle-op. The method returns dependencies of one<br>
operation only (e.g. compile-op), but in case of CFFI's wrapper-file we<br>
need output files of two operations: process-op and compile-op.</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">I don't claim to understand this process, but wouldn't it be possible for you to make your own <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">input-files :around</code> method for <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">gather-operation</code> that would collect the outputs from the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">process-op</code>'s and add them to what you want?</p>
<p dir="auto">Here's the existing definition of what I think is the relevant <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">input-files</code> method:</p>
<pre style="background-color:#F7F7F7; border-radius:5px 5px 5px 5px; margin-left:15px; margin-right:15px; max-width:90vw; overflow-x:auto; padding:5px" bgcolor="#F7F7F7"><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0" bgcolor="#F7F7F7"> (defmethod input-files ((o gather-operation) (c system))
(unless (eq (bundle-type o) :no-output-file)
(direct-dependency-files
o c :key 'output-files
:test (pathname-type-equal-function (bundle-pathname-type (gather-type o))))))
</code></pre>
<p dir="auto">This invokes <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">map-direct-dependencies</code> which invokes the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">component-depends-on</code> method for <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">gather-operation</code> on the system which ... I don't really understand, but I believe it's the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">compile-op</code>'s.</p>
<p dir="auto">I think it would be easiest to write your own method that collects up all of the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">process-op</code> outputs, drops any <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.lisp</code> files (which will be superseded by the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.fasl</code> files), and adds them to the return value of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">call-next-method</code>.</p>
<p dir="auto">If you do that, and drop the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.o</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.so</code> files from the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">output-files</code> of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">compile-op</code>, I think that would get what you want: you would collect the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.o</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.so</code> files from the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">process-op</code>, and you wouldn't get ASDF trying to regenerate the files when it's not necessary.</p>
<p dir="auto">That said, I can think of a simpler, and easier method, and that would be to override the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">operation-done-p</code> method for the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">compile-op</code> so that it knows that the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.o</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.so</code> files are generated by the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">compile-op</code>, and only pays attention to the relationship between <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">bindings-file.lisp</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">bindings-file.fasl</code>.</p>
<p dir="auto">One thing you didn't say was what it means that the compile-op is done over and over -- is it only compiling the bindings-file, or is it doing something to the object files? I don't believe it should change the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.o</code> and <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">.so</code> files, because those are already compiled by <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">process-op</code>, right?</p>
</div>
</div>
</body>
</html>