What's the right way to extend ASDF with new symbols?

[Faré]

On Wed, Nov 18, 2015 at 8:59 AM, Robert Goldman <rpgoldman at sift.net> wrote:
> Commonly, one wants to extend ASDF with new operation and component classes.
>
> But the support in ASDF for referencing such classes all involves
> automagically interpreting keyword symbols (and some unqualified
> symbols?) in the ASDF package.
>
> If I understand this correctly, this leaves the ASDF extender with the
> alternatives of either jamming their new symbols into the ASDF package
> late, and exporting them, or having them live in a new package, which is
> cleaner, but relegates these new components to second-class status.
>
> Consider the following
>
> (defpackage my-system-def
>   (:use common-lisp asdf))
>
> (in-package :my-system-def)
>
> (defsystem my-system
>   :defsystem-depends-on ("my-asdf-extension")
>   :components ((:my-new-file "file1")))
>
> I don't seem to be able to define MY-NEW-FILE in my-asdf-extension and
> export it into my-system-def, because my-system-def is already defined
> by the time my-asdf-extension is loaded.  That means I pretty much have
> to either muck around in the ASDF package (which doesn't scale to
> multiple different people extending ASDF) or use side effects:
>
> (asdf:load-system "my-asdf-extension")
>
> (defpackage my-system-def
>   (:use common-lisp asdf my-asdf-extension))
>
> (in-package :my-system-def)
>
> (defsystem my-system
>  :components ((:my-new-file "file1")))
>
> Am I right about this?  If I am, is there some way to better handle
> this?  Or do we need to rethink how :defsystem-depends-on works?
>
The problem is that the defsystem form is read before the
:defsystem-depends-on clause can be processed, and hence any symbol
used in the defsystem form MUST be in a package that is already
defined by the time the defsystem form is read. If you stick to a
purely declarative style of not having definitions in your .asd file,
that pretty much means only the asdf and keyword packages are allowed.

defsystem-depends-on is also broken in that ASDF assumes that it
doesn't mark such extensions as needing to be reloaded and the .asd
files that use them reprocessed should these extensions or their
dependencies be modified.

On the one hand, ASDF is very wrong. On the other hand, side effects
everywhere mean that even if we ever fix that wrong (which will be a
lot of work), there's still a lot of wrong in trusting extensions to
not have persistent side-effects.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Be careful what you set your heart on — for it will surely be yours.
        — James Baldwin, "Nobody Knows My Name"