A modest proposition: DEFSYSTEM-DEPENDS-ON should die [was Re: What's the right way to extend ASDF with new symbols?]

[Robert Goldman]

On 2/16/16 Feb 16 -3:05 PM, Eric Timmons wrote:
> On Tue, Feb 16, 2016 at 3:14 PM, Robert Goldman <rpgoldman at sift.net> wrote:
>> On 2/16/16 Feb 16 -10:12 AM, Eric Timmons wrote:
>>> On Mon, Feb 15, 2016 at 3:34 PM, Robert Goldman <rpgoldman at sift.net> wrote:
>>>>
>>>> Sorry, I'm not trying to be difficult, but that solution is unacceptable
>>>> to me.  From my POV as maintainer, it seems like the worst of all
>>>> worlds.  We would be introducing yet more moving parts -- a new package,
>>>> ASDF-EXTENSIONS, that we would have to keep track of -- without  solving
>>>> the problem of name collisions.  This seems strictly worse than the
>>>> status quo.
>>>>
>>>> If someone wants to *fix* the double-parsing solution, that's fine.  But
>>>> I'm not going to keep around the broken double-parsing solution.
>>>
>>> Would there be anything wrong with saying everyone should use strings to name
>>> symbols from ASDF extensions? Component types and :class already seem to support
>>> this. A quick glance through the code makes me think :in-order-to and inline
>>> methods would be fairly easy to modify to support it as well. I'd be happy to
>>> send in a patch for it.
>>>
>>> It seems this would solve the issue with namespace clashes and get rid of the
>>> need for load-systems before the defsystem. There's still a lot of work if
>>> someone wants a completely declarative version of defsystem, but at least it's
>>> a step in that direction.
>>
>> I don't see that strings make an obvious improvement over the way we use
>> keywords.  Here's the situation:
>>
>> Keywords are interpreted as follows:
>>   i. names in current package (this is how we get in extensions, if we
>> have imported the extension package into the current package)
>>   ii. names in ASDF (this is how we get in extensions if we have done
>> the Bad Thing and imported our extension names there).
> 
> Sorry, I think I may not have expressed what I meant clearly. When I
> said strings,
> I really meant "strings containing a fully qualified symbol name". To
> use CFFI as
> an example, this:
> 
> ```
>  (cl:eval-when (:load-toplevel :execute)
>     (asdf:operate 'asdf:load-op 'cffi-grovel))
> 
>   (asdf:defsystem example-software
>     :depends-on (cffi)
>     :serial t
>     :components
>     ((:file "package")
>      (cffi-grovel:grovel-file "example-grovelling")
>      (:file "example")))
> ```
> 
> Would become this:
> 
> ```
>   (asdf:defsystem example-software
>     :depends-on (cffi)
>     :defsystem-depends-on (cffi-grovel)
>     :serial t
>     :components
>     ((:file "package")
>      ("cffi-grovel:grovel-file" "example-grovelling")
>      (:file "example")))
> ```
> 
> This is already supported for component types and the :class argument (using
> coerce-class). I can't find it in the documentation (but I could swear
> I saw it in
> there at some point).
> 
> The logic in coerce-class does exactly what you're describing for symbols and
> keywords. For strings it uses uiop:safe-read-from-string. Since the reader never
> sees the symbol from the extension package before it's loaded, there's no need
> to load-system it beforehand. But if someone doesn't want to always prefix with
> a package name, they can continue loading the system beforehand and using the
> symbol.
> 
> It looks like minor tweaks to the code to add the same support to operations in
> the defsystem as well. It's the same logic as coerce-class, just possibly
> without a find-class in there.
> 
> Strings do have a certain ugliness to them that keywords don't, so
> maybe this isn't
> the best direction for ASDF. But I think making support for this
> uniform, locking the
> ASDF package, and encouraging people to use it would solve the namespacing
> problem.


I see your point, and you're right: strings have the obvious advantage
of postponing interaction with the package system, so that the first
pass of parsing defsystem can slurp the whole form, without crashing on
non-existent package errors.

TBH, this seems really unpleasant to me, as a programmer, versus the
solution I have proposed.  Also, it requires substantial rewriting of
existing system definitions.  The proposal I have made simply requires
introduction of (ASDF:LOAD-SYSTEM <ASDF extension>) in a small number of
locations.  Even if this wasn't the case, I want to make ASDF simpler,
not more complicated.  Your proposal will make things more complicated
(since arbitrary parts of the defsystem form would have to be
re-engineered to accept strings in place of symbols and Do The Right Thing).

I think the "ultimate right" answer is to have a new system, YASDF,
which has system definition forms that are not processed by READ.
Instead, they would be read as files of strings by YASDF, and then
parsed by a parser, instead of by READ.  Then we wouldn't have these
worries about the package system.

Ideally, YASDF would also have an explicit design rationale.

However, although I feel that YASDF would be more of The Right Thing,
writing YASDF is not something I want to spend any hours of my life on.
 It's just not my thing.

Thanks!
r