[asdf-devel] load order of dependencies

[Robert Goldman]

On 11/24/11 Nov 24 -3:14 AM, Attila Lendvai wrote:
> dear list,
> 
> my use-case: i want to make sure that when a system is loaded
> (compiled) due to its test suite being invoked, then a full-blown
> optional dependency be loaded first, and only then the system itself
> (which checks for #+(cl:find-package :foo) at compile time).
> 
> to be more specific, it's about whether to load or not a full logging
> library with heavy dependencies. when the test suite is invoked, then
> debugging will most probably follow, so load logging first that
> affects the compilation of the system itself.
> 
> and to be even more specific, we have a custom develop-op, that does
> all kind of things, including the seeking and loading of all the
> optional swank integration systems we have, etc. (note: i was doing
> the experiments below using plain 'asdf:load-system invoked on
> :hu.dwim.reiterate+hu.dwim.logger, and AFAICS there are no
> customizations that should interfere, but hit me on the head with the
> ASDF manual if you believe otherwise, and i should go back and try to
> synthesize a stripped down test case.)
> 
> this is how i wanted to achieve it:
> 
> CL-USER> (asdf:asdf-version)
> "2.017"
> 
> 
> try #1 (in any order, reiterate is compiled before the logger):
> 
> :in-order-to ((load-op (load-op :hu.dwim.logger :hu.dwim.reiterate))
>                 (compile-op (compile-op :hu.dwim.logger :hu.dwim.reiterate)))
> 
> without any :depends-on entry, which as far as i understand compiles
> into :in-order-to entries.

It would, I believe, compile into dependencies from LOAD-OP and
COMPILE-OP onto LOAD-OP, so would be a little different from what you
have here.  I am having difficulty thinking about cases where you would
really want a COMPILE-OP dependency onto COMPILE-OP instead of LOAD-OP,
although I am sure that such cases exist.

> 
> 
> try #2 (the before methods seem to get invoked too late when logger
> has fasls and reiterate doesn't?! but seems to work when neither have
> fasls?!):
> 
> (defsystem :hu.dwim.reiterate+hu.dwim.logger
>   :class hu.dwim.system
>   :description "Loads hu.dwim.logger first, so that proper logging is
> available when developing, but not making the extra dependency
> mandatory."
>   :depends-on (:hu.dwim.reiterate
>                :hu.dwim.logger))
> 
> (defmethod perform :before ((op operation) (system (eql (find-system
> :hu.dwim.reiterate+hu.dwim.logger))))
>   (load-system :hu.dwim.logger))
> 
> (defmethod perform :before ((op operation) (system (eql (find-system
> :hu.dwim.reiterate))))
>   (load-system :hu.dwim.logger))
> 
> (defmethod perform :before ((op operation) (system (eql (find-system
> :hu.dwim.reiterate.test))))
>   (load-system :hu.dwim.logger))

The above probably won't do what you want.  The problem is that, if you
have a system X containing serial file components A, B, and C, this is
the order of PERFORM calls:

(PERFORM LOAD-OP A)
(PERFORM LOAD-OP B)
(PERFORM LOAD-OP C)
(PERFORM LOAD-OP X)

So that the PERFORM on the SYSTEM actually acts as a kind of AFTER
method on the operations on the components, rather than a container, as
you would intuitively expect.  This is one of the least well-understood
attributes of ASDF, but unfortunately would take a fair amount of work
to fix.

This *appears* to work in the case you mentioned (no compilations
necessary), I believe because you see this:

(PERFORM COMPILE-OP X)
(PERFORM LOAD-OP A)
(PERFORM LOAD-OP B)
(PERFORM LOAD-OP C)
(PERFORM LOAD-OP X)

(if there are fasls that need compiling, the COMPILE-OP calls on A, B,
and C will come before the COMPILE-OP call on X, and your problem returns).

As a generalization, if you are writing a :BEFORE or :AROUND method on
PERFORM for a SYSTEM, you are probably trying to do something that won't
work.

I think, as Faré suggests, what you really want here are
DEFSYSTEM-DEPENDS-ON entries for reiterate+logger, reiterate and
reiterate.test.
> 
> 
> but to the point: what's the contract regarding the load/compile order
> of dependencies? and if there's any, then how can i tell ASDF to load
> hu.dwim.logger first? (i couldn't find anything in the manual, but i
> welcome a "try again in chapter 6", because asdf sources are not too
> easy on my parser...)
> 
> and if the order is not specified in the ASDF contract, then can we
> apply a randomizer to the otherwise equal dependencies? i've seen it
> many times before that unrecorded dependencies showed up as filesystem
> order changed (or something else, it's just an assumption, but the
> missing dependency often materialized when deploying the system on a
> different machine).

It might be an interesting test discipline (typically of interest only
in combination with :FORCE t) to allow people to run ASDF operations
with random tie-breaking.  This seems like a poor debugging technique,
though, since the number of such orderings will explode in systems with
weak RECORDED dependencies, so stumbling on a missing dependency might
well not happen quickly....

Best,
r