Including uiop and not asdf in a built application

[Faré]

>>: Dave Cooper
>: Robert Goldman <rpgoldman at sift.info>

Summarizing:
* gendl (and gdl with it) uses monolithic-compile-bundle-op,
depends-on uiop, not asdf.
* gendl tries to use uiop from source to include it without asdf, but
asdf 3.3 broke that.

My understanding of the reason for this breakage is as follows:
* to ensure safe upgrade, asdf specially loads itself as the start of
any call to operate
 (otherwise, a dependency on asdf could cause an upgrade in the
middle, which would
 totally break everything.) To prevent catastrophic downgrade, asdf
specially ignores
 asdf.asd unless its version is at least as recent as the current one.
* asdf has a special relationship to uiop, because it kind of depends
on uiop, but
 at least since 3.3.0, asdf.asd couldn't depend on uiop.asd because
the above rule
 could then cause circular dependencies. asdf.asd instead has a magic
asdf/driver
 system that magically mimic what is in uiop.asd.
* To prevent potentially conflicting redefinition within the same
build, asdf 3.3 therefore
 specially ignores uiop.asd unless its version is *strictly* superior
to what is built into asdf.

Therefore, if you want to force asdf 3.3.x to load uiop, I recommend
patching uiop/version.lisp
to append appending .0.1 to the *uiop-version* variable value.

Sorry for the complication, but that's the simplest I could come up
with. If you can think of a better arrangement, I'm sure the next ASDF
powers that be will be listening to you.

I recommend against changing the logic of ASDF.

> Question: Is there a supported way to do what I'm trying to do, without
> modifying the ASDF source code? Intuitively, it seems to me that we should
> be able to include uiop in build products using
> monolithic-compile-bundle-op, without including asdf.
>
I would say that gendl is large enough that the 6000 lines of asdf
minus uiop shouldn't be a big burden; but if you insist on shipping
with uiop and not asdf, I recommend bumping oh-so-slightly the version
of uiop.

>:Robert
> In that case, it seems to me that
> check-not-old-asdf-system may be simply inappropriate as a check in some
> (all?) bundle operations. But I would be hard pressed to say when it is and
> is not appropriate. E.g., presumably it is appropriate in image building,
> since any image one builds would include the current running ASDF. But that
> argument does not seem to hold for bundles full of fasls or source code,
> does it?
>
Well, since at least on most implementations, building the fasls
involve loading the code in the current image, there isn't much leeway
by which asdf could afford not to use the same
check-not-old-asdf-system function. A future version of ASDF that
seriously supports cross-compilation might, but we're not quite there
yet (see the TODO for hints, if you're interested in making it
happen).

> In one sense at least, Faré is right that I should be replaced as
> maintainer. I have never used any of the bundle operations, and feel
> ambivalent about their inclusion into ASDF. They crept in as a side-effect
> of supporting lisps that use the C compiler, but they are a substantial
> increase in the breadth and maintenance difficulty of ASDF. I'm afraid I
> have neither the time nor the interest to carry that burden. I would be
> happy to yield it to others.
>
There certainly was a mission creep in ASDF 3 (see my 2014 article for
some explanations). I don't regret making it happen, even though
bugfixing lasted until 2017. The code should be pretty stable now —
though, who knows for sure?

Whatever I or other people wish, whoever it is who does the grunt work
that others won't do shan't be blamed by those who are not willing to
do (or fund) said grunt work.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
If normative judgments cannot be rational, then science is actually useless.
                — François Guillaumat