[asdf-devel] Subtle bug when upgrading asdf

[Faré]

I had warned before that one limitation of upgrading asdf was that it
had to be done before to load anything that depends on the new asdf.
Well, I've been bitten by it, badly, when trying to build xcvb on a
new machine. What is surprising is that I haven't been bit by it
earlier.

Symptom: on that machine, when you loaded xcvb, it failed the first
time to compile ironclad (recently uncommented as a dependency)
because the :around method on perform that changes the readtable
wasn't executed. On the second attempt at the REPL, it worked, though
(from a script, no second attempt possible). Introspection of the
perform gf showed the method was present during the first (failed)
attempt. Why oh why?

Cause: because xcvb depends on a recent asdf, it tries to make sure
that asdf is up to date, which it was trying to do from xcvb.asd (we
are not building xcvb with xcvb right now, pending some serious
refactoring and extensions). But that is too late, since that means
that the old asdf is running to compile and load xcvb and its
dependencies, not the new one. Therefore, the old asdf loads the new
asdf. The new asdf shadows problematic functions including perform
(i.e. uninterns the symbols). Ironclad defines a method on the new gf.
But it's still the old asdf trying to load ironclad, and its gf hasn't
been extended. Oops.

Why it didn't happen on my main machine: because I had been developing
cl-launch at the same time as asdf, and upping the minimal version
required for cl-launch faster than implementations had been updated;
therefore, cl-launch happened to be always be upgrading asdf from the
implementation-provided to the latest in the source-registry; by the
time xcvb was loaded, asdf was already the latest, and there was no
asdf upgrade while loading xcvb. On the other machine, with an older
cl-launch and recent sbcl, cl-launch was happy with the asdf provided
by sbcl, and therefore not upgrading it to the latest in the
source-registry, causing the above bug to surface.

Solution: make triple damn sure that asdf has been upgraded *before*
it tries to load xcvb. 1- modify cl-launch so it always upgrades asdf
whenever asdf is used. 2- have the makefile always load asdf before it
tries to load xcvb, even when running on an old cl-launch, 3- in
xcvb.asd, error out if asdf was not already the latest (and of a
sufficient version) when xcvb.asd was loaded.

What it means for you: if your code, or any code it depends on, itself
depends on a recent asdf, you cannot rely on asdf to upgrade itself
during the asdf-driven build of your code; you must upgrade it
specially *before*.

Potential cure to make it idiot-proof? In asdf:operate, always start
by reloading asdf then funcall'ing (find-symbol* :operate :asdf) if
its version changed? What do you think of such a change? Would it
cause more grief than it relieves, as it quickly reveals bad
configurations with older asdf's in their source-registry than
otherwise loaded?

Many thanks to Peter Keller, who assisted me while I was totally
baffled by this bug.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Who released the most slaves? The one who spent his wealth buying them back?
Or the capitalist who found a way to power mills with water?