A modest proposition: DEFSYSTEM-DEPENDS-ON should die [was Re: What's the right way to extend ASDF with new symbols?]

[Faré]

I'm OK with declaring DEFSYSTEM-DEPENDS-ON a failure, and load-system
(or load-systems) the official way to go. But

1- This of course requires heads up, updating all users before
retiring the feature, etc. From my experience, if you start seriously
deprecating today and sending patches to all authors who use it in
quicklisp, you can expect to be removing that part of the code in two
years or so.

2- To make these dependencies work properly still requires modifying
ASDF to add explicit plan nodes for loading ASD files, that will
contain the systems loaded by load-system. The same trick will also
automatically make the :defsystem-depends-on work, since it itself
calls load-system.

3- Yes, defining things in the ASDF package is ugly, but extensions
are few enough, and using a prefix is a good enough namespace
management strategy. Not the most horrible thing that working with CL
does to you.

4- Yes, at some point, a better Lisp than CL will be needed. And
Haskell isn't quite it. Neither is Scheme or Clojure, I fear. Maybe
some future version or variant of Racket.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Who released the most slaves? The one who spent his wealth buying them back?
Or the capitalist who found a way to power mills with water? — Paul Claudel


On Fri, Feb 12, 2016 at 3:11 PM, Robert Goldman <rpgoldman at sift.net> wrote:
> On 2/11/16 Feb 11 -8:12 PM, Faré wrote:
>> On Wed, Nov 18, 2015 at 8:59 AM, Robert Goldman <rpgoldman at sift.net> wrote:
>>> Commonly, one wants to extend ASDF with new operation and component classes.
>>>
>>> But the support in ASDF for referencing such classes all involves
>>> automagically interpreting keyword symbols (and some unqualified
>>> symbols?) in the ASDF package.
>>>
>>> If I understand this correctly, this leaves the ASDF extender with the
>>> alternatives of either jamming their new symbols into the ASDF package
>>> late, and exporting them, or having them live in a new package, which is
>>> cleaner, but relegates these new components to second-class status.
>>>
>>> Consider the following
> [...snip....]
>> The problem is that the defsystem form is read before the
>> :defsystem-depends-on clause can be processed, and hence any symbol
>> used in the defsystem form MUST be in a package that is already
>> defined by the time the defsystem form is read. If you stick to a
>> purely declarative style of not having definitions in your .asd file,
>> that pretty much means only the asdf and keyword packages are allowed.
>>
>> defsystem-depends-on is also broken in that ASDF assumes that it
>> doesn't mark such extensions as needing to be reloaded and the .asd
>> files that use them reprocessed should these extensions or their
>> dependencies be modified.
>>
>> On the one hand, ASDF is very wrong. On the other hand, side effects
>> everywhere mean that even if we ever fix that wrong (which will be a
>> lot of work), there's still a lot of wrong in trusting extensions to
>> not have persistent side-effects.
>
> This is very frustrating to me.
>
> In the old days, we would just put
>
> (asdf:load-system "extension")
>
> at the top of the file.  Everybody said "oh, noes! side effects!"
>
> So we added DEFSYSTEM-DEPENDS-ON.
>
> This adds a truckload of complex code to ASDF that we must maintain, but
> basically doesn't fix anything:
>
> * The extensions don't work like native-to-ASDF constructs, unless you
> put them in the ASDF package, so we are encouraging programmers to
> perpetrate name collisions.
>
> Note that the "bad" side-effecting construct is *BETTER* than this,
> because the load form can  be put upstream of a defpackage in the asd
> file, so that the symbols will exist before the DEFSYSTEM is read.  This
> means that ASDF extenders are *not* encouraged to mess up the ASDF package.
>
> * The dependencies, as you point out, are broken.  Indeed, they may be
> *more* broken than when we put ASDF:LOAD-SYSTEM forms in the file.
>
> The only thing that DEFSYSTEM-DEPENDS-ON fixes, then, is introspection.
>  But a *declaration*, instead of a semantic form would solve this
> problem, in a much simpler way.
>
> I suggest that DEFSYSTEM-DEPENDS-ON should turn into a declaration.
> I.e., it should provide introspection semantics for, e.g., quicklisp,
> but that's it.  To keep stupid things from happening, it should error
> out if the depended-on system is not loaded at the time the follow-on
> system is loaded.  That's it.  End of story.
>
> Simplification ensues, as opposed to us having to maintain fussy
> propagation code.
>
> Also, we stop encouraging people to bork the ASDF package, which is A
> Bad Thing to do.
>
> Indeed, it's tempting to package lock ASDF on lisps that support that
> operation....
>