[asdf-devel] Unexpected behavior when loading different versions of a system

[Robert Goldman]

On 9/2/11 Sep 2 -12:25 PM, dherring at tentpost.com wrote:
>> On 9/1/11 Sep 1 -10:40 PM, Robert Goldman wrote:
>> Here's the new problem:  when we reload Xach's foo system, we get a new
>> definition with files "a" and "b."  Unfortunately, when ASDF reads this
>> new definition, it first says "Oh, file a in system foo --- I have seen
>> that before, and it's already loaded."  So when I tell ASDF to load foo
>> (again), it says "a is already loaded, I need only load b."
>> Unfortunately, that is not true --- the OLD "a" component is already
>> loaded, but the NEW one has not been loaded.
> ...
>> [Indeed, Faré and I had in the past discussed the possibility that when
>> a new system definition for an existing system is loaded, we should
>> simply regard the entire system as "dirty" and always rebuild everything
>> in it.]
>>
>> This is a REALLY big change to the behavior of ASDF, so I would like to
>> hear some comments before proposing a patch.
> 
> Makefiles handle this by checking timestamps between source and object
> files.  Other build systems do this by checking hashes.  However, both
> approaches rely on dependency tracking and assume that each output is
> compiled in isolation and everything is relinked at the end.
> 
> In a serial CL process (all files loaded/compiled by the same image), the
> presence or absence of a file may greatly affect all files visited after
> it.  Thus given this interactive image model, I don't think any simple
> option will work in all cases.  The current behavior is reasonable (modulo
> where the files are found).

I fundamentally disagree about this.  For two reasons:

1.  Correctness over convenience:  It is more important to restore
correct functioning after changes than to avoid excessive recompilation.
 CPU cycles are cheaper than programmer cycles.  Introducing a cryptic,
transient, buggy behavior and baffling the programmer is worse than
recompiling a few extra files.

2.  The model is fundamentally busted.  If you reload a system file, the
data structures in ASDF are in a deeply wrong state.  The SYSTEM objects
(and other components) will contain an ill-characterized moosh of data,
some from one version of the system definition and some from the later
version.  Xach has found one bad behavior that comes from this.  But
there could be a bottomless well of other bad behaviors from this, since
ASDF's internal state is corrupted.
> 
> Thus, I wouldn't force a rebuild when the system definition changes, but a
> rebuild might be deserved when the source location changes.

The bad behaviors are not limited to bad behaviors from location
changes.  If you change the membership of the system, but the system is
in the same location, that can introduce corruption, as well.

I don't see a way to keep correctness here and maintain the current
behavior.

This is not a step I take lightly.

Best,
Robert