Problem with adding input files...

[Robert Goldman]

On 6/30/17 Jun 30 -10:35 AM, Faré wrote:
>> On 6/30/17 Jun 30 -10:11 AM, Faré wrote:
>>> First, can you confirm that the compute-action-stamp called by
>>> mark-operation-done called by the perform :after method on define-op
>>> returns the correct value?
>>
>> Hm.  I don't see any such after method.  All I see is this:
>>
>> (defmethod perform ((o define-op) (s system))
> ...
>>      (if-let ((pathname (first (input-files o s)))))
> ...
> This call to input-files is problematic, as it will cause the wrong
> value to be cached, before you recorded your input-files. Note that
> you might have to invalidate the cached input-files anyway, because it
> may already have been called by compute-action-stamp at that point.
> 
> But I meant the general perform :after method and mark-operation-done
> method from action.lisp.

OK, I see this:

  (defmethod perform :after ((o operation) (c component))
    (mark-operation-done o c))

That's going to compute a time stamp for the operation.  I think that's
OK, because that will be done after the first time the DEFINE-OP is
executed.  Ideally, when we go to do it the second time, this timestamp
will be compared with the last write time on the input files (in this
case a revision-number.lisp-expr file), and should trigger the DEFINE-OP
to be done again.

But obviously I'm missing something, since this is not happening.
Actually, there are a bunch of things I don't understand, including:

1. When I invoke LOAD-OP the second time, why is ACTION-UP-TO-DATE-P
never called?  What short-circuits this?

2. COMPONENT-OPERATION-TIME is called on DEFINE-OP of SYSTEM at what
looks like the right time (as well as being called a bunch of other
times, presumably to check that it's up-to-date with respect to
downstream components).

3. Looks like COMPUTE-ACTION-STATUS calls COMPUTE-ACTION-STAMP.  Looks
like that is where we compute whether or not the operation is up-to-date
wrt the input files.

4. This is likely to be the problem:  we compare the time stamp of the
input files against the earliest time stamp of the output files.  But
there are no output files from the define-op, so we do
UP-TO-DATE-P = (STAMP<= <number> NIL)
and STAMP<= is defined to always be true if the second argument is null.

So there is no point in the algorithm -- or at least no point that I can
see -- where we compare the input-file time stamps against the
component-operation-time....  I could butcher that in, but I'm a little
worried -- the logic in COMPUTE-ACTION-STAMP is quite subtle.

Anyway, that's how I'm proceeding.


> 
>> At any
>> rate, I'm having a hard time tracking down exactly where we determine
>> whether or not a define-op needs redoing.
> Ultimately, it boils down to (compute-action-stamp plan operation
> component [nil]) as called by compute-action-status in plan.lisp. And
> *that* will call input-files.
> 
>> The caching code makes things
>> much more efficient, but it also effectively camouflages the algorithm....
>>
> Not just "more efficient", but in some cases keeps O(n) what might be
> a much higher polynomial (or exponential?) when a module's input-files
> recursively depend on its contents (e.g. for warning checking). But
> yes, sometimes you need to invalidate the cache, and this looks like
> one of those times to me.
> 
> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
> Too many people are thinking of security instead of opportunity.  They seem
> more afraid of life than death.
>                 — James F. Byrnes
>