[asdf-devel] Please test ASDF 2.26.24

[Faré]

Considering that I ran out of energy with respect to ASDF hacking
following the marathons of refactoring last week,
I'm declaring 2.26.24 a candidate for release of 2.27.

Please test thoroughly, since this is quite a MAJOR release,
with a lot of significant changes.

The only reason I'm not bumping the version to ASDF 3 is that
this would needlessly break compatibility
with who depends-on (:version "asdf" "2.XXX")
when ASDF 2.27 is probably compatible with most every system out there.
There are a few incompatibilities, though.
Here are the changes since 2.26:

* ASDF-BUNDLE was merged into ASDF.
 This notably means fewer headaches for users of ECL;
 it also means a new load-fasl-op feature for all users of a modern CL,
 making it easier to deliver software as a single fasl.
 All remnants of its predecessor asdf-ecl were removed.

* Two related significant *decades-old* bugs were fixed (see note 1):
 (a) changes in dependencies now trigger a rebuild across systems, and
 https://bugs.launchpad.net/asdf/+bug/479522
 (b) timestamps are now correctly propagated
 https://bugs.launchpad.net/asdf/+bug/1087609
 In olden days, some have argued for not rebuilding systems that :depend-on
 a modified one as a "feature". It really isn't. On the other hand,
 if they want to explicitly prevent a rebuild, the actual feature :force-not
 is there to help them (released as part of 2.21 in April 2012).

* In the process, an essential bug in the ASDF object model was resolved,
 by introducing a new operation PREPARE-OP corresponding to
 "loading the dependencies of a component and its parents" and
 required before to load or compile the component,
 instead of depending on the implicit behavior of the TRAVERSE algorithm,
 which was breaking down in the above bugs.

* The TRAVERSE algorithm was gutted out. The above conceptual bug was fixed,
 and instead it has been factored into reusable higher-order functions.
 It no longer assumes all operations propagate their dependencies downwards,
 which is not the case with PREPARE-OP where they propagate upwards.
 Instead, operations can easily propagate whichever way they want
 using a one-liner defmethod to COMPONENT-DEPENDS-ON.

* COMPONENT-DEPENDS-ON is now more powerful: it can express dependencies
 on arbitrary operation objects (i.e. operations of the same class
 can have different options) acting on arbitrary component objects
 (i.e. not necessarily siblings of the current component).
 The ASDF object model can now express arbitrary build graphs, with
 special magic no more for children components vs other dependencies.

* Many internals have been refactored, and some sorry features were excised.
 The semantics of OPERATION-DONE-P is now well-specified.
 The dubious DOFIRST mechanism is gone.
 FIND-COMPONENT will pass component objects through, and
 a corresponding FIND-OPERATION replaces MAKE-SUB-OPERATION.
 Some internal accessors were renamed.

* The dubious :feature and :if-component-dep-fails :try-next features
 were removed. They were not very expressive, clunky to use, and relied
 on baking conceptually buggy behavior in the old TRAVERSE algorithm.
 Unhappily, this breaks backwards compatibility for the few who used it.
 Happily, a much cleaner :if-feature component option was added.
 However, if you use it you then depend on a recent ASDF 2.27 or later.

* POIU was updated. It is still an extension not distributed with ASDF.
 It is now much simplified and made more correct by construction by
 reusing the new TRAVERSE infrastructure of ASDF and now being able to
 assume the ASDF object model is a complete description of dependencies.
 On SBCL it will handle deferred warnings from background compilations.
 On platforms that cannot fork it will gracefully fall back
 to serial compilation.

* The documentation was updated a little. The test suite was updated.

* There were some portability updates for ABCL, CMUCL, MKCL.

Happy hacking!


Notes

1- ASDF inherited those long-standing dependency bug from MK-DEFSYSTEM,
which probably inherited it from its predecessors, and so these related bugs
have probably lived for over 30 years, for as long as defsystem existed!
I haven't seen the proprietary defsystems used by e.g. Allegro or LispWorks,
but the fact that e.g. LispWorks has caused-by and requires dependencies
(akin to in-order-to and do-first in asdf) makes it very suspect in my eyes.
And yet, it all hasn't seem to affect people too much —
certainly not so much that they would go a-hunting and a-fixing them!
Indeed, if you're working a single system at a time, until it works,
without interrupting the build (instead fixing the code and
using a restart after errors), without leaving and restarting your image,
then the bug won't hit you. If you occasionally abort a build, but
no file depends on a modified macro or other compile-time state
that wasn't itself modified to accommodate the change, then you won't hit.
Therefore, it might never have been seen by a normal Lisp Machine developer.
But when you develop plenty of Lisp libraries at a time,
each its own system, while often restarting your Lisp to build from clean,
or if you have a test-foo system separate from your foo system,
then these bugs will become quite noticeable.
And so people now notice and care.
That's interesting in what that says about how Lisp hacking has evolved.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
The truth of a proposition has nothing to do with its credibility.
And vice versa.    — Robert Heinlein, "Time Enough For Love"