[Asdf-devel] ASDF does not signal deferred non-style warnings as an error

[Faré]

>>>: rpg
>>: fare
>: rpg

>>> How do you distinguish between the build failing and succeeding but
>>> being followed by an error signal?
>>>
>> Nothing in ASDF should cause an error signal at the end of the W-C-U.
>
> I think I'm misunderstanding the following from your previous message:
>
>  "if deferred warnings are disabled (the default), then the
> compilation succeeds, and I get an undefined variable warning at the
> end of the build but that doesn't cause a build failure, because
> operate's with-compilation-unit doesn't catch these warnings (and
> can't replay those from previous sessions, anyway)."
>
> OK, so there's no error signal because the warning is not caught in the
> W-C-U.  TBH, I don't understand this.  When you say "caught" here, do
> you mean "handled"?  And why should OPERATE need to catch these to avoid
> a spurious compilation success? Shouldn't this warning cause
> COMPILE-FILE's to return a non-nil warningsp?
>
The warning is caught/handled/captured/call-it-what-you-may by W-C-U,
and deferred to the end of W-C-U,
***as is the one and only purpose of W-C-U*** (so far as I can tell).
The COMPILE-FILE is inside W-C-U, therefore doesn't even see the warnings,
therefore cannot and will not compute its warnings-p and failures-p
return values based on them, or otherwise display them or do anything
whatsoever based on these warnings. It thus cannot and will not notify
ASDF that cannot and will not upgrade them to full warnings or errors.

I don't know that OPERATE "needs" to do anything.
The W-C-U has *always* been in ASDF (since danb's 1.34 in 2002),
there is no more "spurious compilation success" than has always been.

When I discussed removing the W-C-U in ASDF 3 when not using
the more robust deferred-warnings mechanism, this caused a backlash:
ASDF would signal an error on SBCL when compiling buggy systems
that used misspelled, undeclared or forward-referenced special variables.
And yes, the systems were buggy, and I sent plenty of bug reports
to Quicklisp system authors based on the bugs thus detected.
It's a bug and a portability issue to fail to use a variable without
suitable declaration. If users didn't complain, it's because they never
used the functions with a misspelled variable, or happened to use an
implementation that did the right thing anyway with undeclared special
variables. The deferred-warnings mechanism catches and hushes
forward-references to functions — I don't remember what it does with
undeclared forward-references to special variables, but I would be
slightly disappointed if SBCL didn't consider that a full WARNING
worthy of a build failure.

>> You're discussing the behavior of a file that isn't checked in;
>
> No, this file is checked in, and you should see these results if you run
> the test suite after pulling master from cl.net.
>
OK, I might have been looking at the wrong place.
I see that test now, and tweaked it to make it more portable.
This test (only) checks cases where deferred-warnings are enabled.

>> does it have interesting handler-bind's?
>
> No, it's just Ilya's example.
>
It's not: you're enabling deferred-warnings. Ilya wasn't.
This explains the different in behavior.

>>> This suggested some conjectures:
>>>
>>> 1.  AFAICT, using ASDF's deferred-warnings would have required first
>>> breaking the logic that ACL already provides, and then recreating it in
>>> ASDF.
>>>
>> No. It doesn't require anything but suitable defaults.
>> It might "just" be that check-deferred-warnings needs be made more clever,
>> and/or might need to have its own separate variants of
>> *compile-file-warnings-behaviour* and *compile-file-failure-behaviour*,
>> and maybe also *compile-check*.
>
> This still means "take code that works, break it, and then add more code
> to fix the breakage."
>
No. Any code that the deferred-warnings declares broken is indeed broken code,
though the bugs that go unchecked without deferred-warnings might happen
to not be seen by users because it's in code not reached at runtime,
or executed correctly on the implementation used by the user, though unportable.

>>> 2.  If ACL can handle this sensibly w/o requiring any outside assistance
>>> from ASDF, then I don't believe that this should be ASDF's job.
>>>
>> By definition, it's ASDF's job to put the correct W-C-U, and to emulate
>> a system-wide W-C-U when warnings were deferred in a previous session
>> (i.e. file "a" was previously compiled with deferred warnings, and least
>> we forgo all intra-system incrementality, we don't want to recompile it).
>> The implementation just can't make that stuff up out of thin air.
>> Hopefully, we don't have to write clever handler-bind's
>> as part of ASDF itself, but there's a possibility we might,
>> and indeed someone should look at the undefined-variable behavior on
>> each and every of the 16 implementations.
>
> I'd be willing to meet the implementations halfway on this.  If we can
> specify the information we need from the implementors to do this right,
> then we can ask them to provide an API that will provide that
> information. If they don't, then ASDF won't perform as well on that
> implementation.
>
> I'm open to a volunteer to look at the undefined-variable behavior on
> each and every of the 16 implementations, but as Mr. Zimmerman once said
> "it ain't me you're lookin' for, babe."
>
> I'm interested in shifting a lot of this "grovel over implementation
> internals" work to the place where it can best be done -- and more
> importantly, where it can best be maintained: to the implementors.
>
Actually, it looks like you only need to check implementations with
deferred-warnings support: allegro, clozure, cmucl, sbcl, scl.
cmucl (and I presume scl) emit a full warning for undefined-variables,
but there is no provision for upgrading that to a full error;
users who want to catch them shall define their handler-bind
— or just switch to sbcl.
allegro, clozure, sbcl, seem to correctly catch the undefined variables.

Therefore, I believe the current asdf support for deferred-warnings
already works as well as we can hope it to do. It's a "simple" matter
of chasing the bugs in Quicklisp and warning all users before to
make it the default.


>>> 4.  It seems like the implementation needs to think carefully about how
>>> to defer warnings in WITH-COMPILATION-UNIT anyway.
>>>
>> No. We only need worry if some implementations convert warnings to errors
>> at the end of W-C-U, in which case it becomes much more important to
>> implement and deploy deferred-warnings.
>
> Why would anyone do this conversion? Is it because there's a warning
> because locally the compiler can't tell that code is messed up, but once
> the W-C-U has exited, the compiler assumes some sort of closure?  So if
> I don't see some form of declaration locally, I signal a warning, but it
> is held until the compilation-unit ends, and if it can't be handled
> then, we treat it as an error?  In that case why signal a warning and
> turn it into an error, instead of just signaling an error?  Is there a
> concrete example of this behavior?
>
I don't believe any implementation is doing that.
Your report made me worried, but
you just had the ASDF deferred-warnings enabled,
and it was ASDF doing this upgrading.

The *compile-file-warnings-behaviour* and *compile-file-failure-behaviour*
features have existed since ASDF 1 and are semi-documented in the manual
(with a bogus name for the latter).

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Live as if you were living already for the second time and
as if you had acted the first time as wrongly as you are about to act now!
    — Viktor Frankl, "Man's Search for Meaning"