[asdf-devel] Unexpected behavior when loading different versions of a system

[Faré]

For what it's worth, I agree with Robert Goldman that well-defined behavior
is preferrable to a collection of patches to address quirks.

We can't completely tighten the semantics of ASDF,
because some of its fundamental assumptions regarding incremental compilation
depend on programmers following unenforceable ill-specified restrictions
(e.g. that any visible side-effect from compiling a file
is included in loading the file and not negatively interacting
with compilation of other files that may happen before said loading,
that dependencies are properly declared, etc.).
Still I think that modification of the .asd file, as detected by either
timestamp or pathname change, should trigger the invalidation of the system.

If you want to avoid unnecessary recompilation of objects,
you should be using a system that more closely tracks dependencies
between objects, such as XCVB. BTW, it is possible to load XCVB systems
from ASDF and vice-versa. I really should re-package the whole thing.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org


On Fri, Sep 2, 2011 at 14:03, Robert Goldman <rpgoldman at sift.info> wrote:
> On 9/2/11 Sep 2 -12:25 PM, dherring at tentpost.com wrote:
>>> On 9/1/11 Sep 1 -10:40 PM, Robert Goldman wrote:
>>> Here's the new problem:  when we reload Xach's foo system, we get a new
>>> definition with files "a" and "b."  Unfortunately, when ASDF reads this
>>> new definition, it first says "Oh, file a in system foo --- I have seen
>>> that before, and it's already loaded."  So when I tell ASDF to load foo
>>> (again), it says "a is already loaded, I need only load b."
>>> Unfortunately, that is not true --- the OLD "a" component is already
>>> loaded, but the NEW one has not been loaded.
>> ...
>>> [Indeed, Faré and I had in the past discussed the possibility that when
>>> a new system definition for an existing system is loaded, we should
>>> simply regard the entire system as "dirty" and always rebuild everything
>>> in it.]
>>>
>>> This is a REALLY big change to the behavior of ASDF, so I would like to
>>> hear some comments before proposing a patch.
>>
>> Makefiles handle this by checking timestamps between source and object
>> files.  Other build systems do this by checking hashes.  However, both
>> approaches rely on dependency tracking and assume that each output is
>> compiled in isolation and everything is relinked at the end.
>>
>> In a serial CL process (all files loaded/compiled by the same image), the
>> presence or absence of a file may greatly affect all files visited after
>> it.  Thus given this interactive image model, I don't think any simple
>> option will work in all cases.  The current behavior is reasonable (modulo
>> where the files are found).
>
> I fundamentally disagree about this.  For two reasons:
>
> 1.  Correctness over convenience:  It is more important to restore
> correct functioning after changes than to avoid excessive recompilation.
>  CPU cycles are cheaper than programmer cycles.  Introducing a cryptic,
> transient, buggy behavior and baffling the programmer is worse than
> recompiling a few extra files.
>
> 2.  The model is fundamentally busted.  If you reload a system file, the
> data structures in ASDF are in a deeply wrong state.  The SYSTEM objects
> (and other components) will contain an ill-characterized moosh of data,
> some from one version of the system definition and some from the later
> version.  Xach has found one bad behavior that comes from this.  But
> there could be a bottomless well of other bad behaviors from this, since
> ASDF's internal state is corrupted.
>>
>> Thus, I wouldn't force a rebuild when the system definition changes, but a
>> rebuild might be deserved when the source location changes.
>
> The bad behaviors are not limited to bad behaviors from location
> changes.  If you change the membership of the system, but the system is
> in the same location, that can introduce corruption, as well.
>
> I don't see a way to keep correctness here and maintain the current
> behavior.
>
> This is not a step I take lightly.
>
> Best,
> Robert