[asdf-devel] Change in the way ASDF performs a plan? Or :force semantics?

[Faré]

On Wed, Feb 6, 2013 at 3:48 PM, Robert Goldman <rpgoldman at sift.info> wrote:
>> ASDF 3 will always try to upgrade itself before any operation,
>> precisely to avoid squashing your methods in the middle of the operation,
>> which would be far worse.
>
> I'm afraid that this behavior needs refinement.  Consider what we are
> doing:
> * we load my system definition
> * we load/upgrade ASDF3
> * ASDF3 destroys part of my system definition and *quietly* goes ahead
> and does the wrong thing, loading a damaged system definition.
>
> I can't imagine a case where the programmer wants this behavior.
>
Is there any reason not to load your definitions *after* upgrading ASDF?

ASDF *must* be upgraded first thing during your build.
Upgrading it at any other time is bound to cause trouble sooner or later.
There's just no reliable way we can update the basic ASDF methods
without squashing the generic functions,
which in turn will necessarily squash any extensions.

Squashing entire functions and reloading extensions is also
the only way to survive  unless we require each and every extension
to also follow the hot-upgrade discipline of ASDF.
And frankly, considering how hard that discipline is,
it's not something I wish on my enemies, much less my friends.

> I could be convinced that ASDF should upgrade itself, but I believe my
> example shows that any time ASDF upgrades itself it must reload all
> system definitions, because they have potentially been damaged.
>
That's an option I have considered:
automatically re-loading all previously defined systems when you upgrade ASDF.
But not all systems can be upgraded at all,
so this can potentially cause more damage as it can prevent.

> That seems like a tall order, so a more modest alternative would be:
>
> ASADF should raise an error and refuse to continue after reloading
> itself, forcing the programmer to re-do any operations that triggered
> the reload.  In that case, it should mark all loaded system definitions
> as invalid, since they may contain method definitions that have been
> removed.
>
Usually, the initial upgrade just works, and the restarting of operations
is automatic when that upgrade happens as part of an operate.
It only fails if your definitions
are not findable via the normal find-system mechanism, and/or
have undeclared defsystem-depends-on dependencies.
If you do things cleanly, it should work.

>> But if it was already upgraded, this should be a No-Op
>> (unless there's a bug in ASDF).
>> Moreover, :force t should not apply to ASDF,
>> because it has the :builtin-system-p t flag.
>> I just tried at home, and it worked.
>
> I just retried and got the same behavior.  No git operations done this
> morning.  This is on ACL 8.2, Mac OS X, 64-bit Enterprise:
>
> CL-USER> (push "home:lisp;asdf;" asdf:*central-registry*)
> ("home:lisp;asdf;" "home:lisp;franz-clx;" "~/lisp/asdf-systems/"
>  "~/lisp/asdf-install-systems/systems/" "~/lisp/asdf-systems/"
>  "~/lisp/asdf-install-systems/systems/")
> CL-USER> (asdf:load-system "asdf")
> ; Loading /Users/rpg/lisp/asdf/asdf.asd
> ; Fast loading
> ;
> /Users/rpg/.cache/common-lisp/acl-8.2-macosx-x64/Users/rpg/lisp/asdf/build/asdf.fasl
> ; Upgrading ASDF from version 2.23.7 to version 2.28
> T
> CL-USER> (circa)
> Configuring for CIRCA systems...
> CIRCA ASDFs now in load path.
> NIL
> CL-USER> (asdf:load-system "circa-csm" :force t)
> ; Loading /Users/rpg/circa/code/csm/circa-csm.asd
> ;   Loading /Users/rpg/circa/code/csm/logical-pathname.lisp
> ;   Loading /Users/rpg/circa/code/csm/optimization-boilerplate.lisp
> ;   Fast loading /usr/local/acl/acl82.64/code/CLX.001
> ;;; Installing clx patch, version 1.
> ;   Autoloading for package "CLTL1":
> ;     Fast loading from bundle code/cltl1.fasl.
> ;     Foreign loading clx:excldep.dylib.
> ; Loading /Users/rpg/lisp/asdf/asdf.asd
> ; Loading /Users/rpg/lisp/asdf/asdf-driver.asd
> ; Fast loading
> ;
> /Users/rpg/.cache/common-lisp/acl-8.2-macosx-x64/Users/rpg/lisp/asdf/build/asdf.fasl
> .....
>
> I verify that this behavior does *NOT* occur in SBCL, loading the same
> system definition.
>
> The only suspicious thing I see in the transcript from ACL is the
> loading of CLX.  I wonder if there's any chance that this triggers some
> interaction with ASDF?  That would be bad.
>
Looks like it's an issue with logical pathnames,
whereby on Allegro, it thinks that ASDF has changed
because it resolves to a "different" file,
what with ASDF 3 being more careful at not squashing logical-pathnames
than was ASDF 2.
Maybe (describe (find-system :asdf)) at various stages can help
diagnose the exact issue.

>> Considering that you seem to already have had ASDF 3 installed
>> (since it did the automatic upgrade attempt),
>> the issue seems to be that you git pull'ed
>> after loading your ASDF overrides, and they thus got squashed
>> during the implicit ASDF upgrade.
>
> No, the above experiment shows that is not the explanation.  Something
> stranger is going on.
>

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
"Computer programming is omnipotence without omniscience."
        — Prospero, as cited by Eliezer Yudkowsky