Enforcing assumptions

Fri Sep 23 23:24:19 UTC 2016

On Fri, 2016-09-23 at 13:55 -0500, Robert Goldman wrote:
[...]
> > You surprise me. Historically, you've always been a strong voice
> > for
> > backward compatibility and being extra cautious about not breaking
> > other
> > people's code, especially not so without extra warnings and heads
> > up,
> > even more so if we're reversing something the manual used to
> > advertise.
> 
> I'm not saying that I don't want to avoid backward compatiblity
> issues.
> I just think it's too much to say "you can't change anything unless
> you
> check some large N of open source libraries by some small M of lisp
> implementations by some smaller P of operating systems."  That's
> clearly
> impossible.

You should change what you want, while being aware that if you break
people's stuff they will complain and in the worse case reject the
change altogether, so when when considering multiple paths for
implementing a new feature and deprecating old ones, it's useful to
know how much code each of the alternatives will break.

This is were testing Quicklisp (or a small but important subset
thereof) comes useful; and if one breaks the olden ways, briefly
documenting how to convert to the new ways is polite (in the old
meaning, having the same root as "politics").

> One thing about having pre-released ASDF versions is that we can run
> things up the flagpole and see how much we break.
> 
> But back at you -- you say you'd like to have MAKE-OPERATION be the
> pinch point, so that we can check initargs, exploit the fact that
> operations are unique and EQ-checkable, etc.  But then you tell me I
> can't enforce the use of MAKE-OPERATION.
> 
> I don't get it.  I don't see how you can have it both ways.  The
> alternative seems to be "we're going to let you write and run code
> that
> we KNOW will break, because we don't want to signal errors."  Having
> the
> *appearance* of backwards-compatibility is worse than obviously
> breaking
> backwards-compatibility.
> 
> So, please: either let's get rid of MAKE-OPERATION, kill its
> memoizing,
> and go back to letting people use MAKE-INSTANCE, or let me get on
> with
> enforcing its use.
> 
> I'm happy to see MAKE-OPERATION die (or be left around for backwards
> compatibility as just a shell around MAKE-INSTANCE), and reduce the
> code
> size and complexity of ASDF.  Or I'm happy to enforce its use.  But
> the
> midpoint is untenable.
> 
> I guess the final point is: please don't use the word "must," unless
> you
> mean "must."  I feel like you're stabbing me in the back when I've
> tried
> to take your stricture seriously, and suddenly it turns out "must
> means
> may."
> > 
> > I wouldn't consult Quicklisp when implementing a big fix or making
> > a
> > backward compatible change. But when proposing a backward
> > incompatible
> > change, I make sure to warn all users that I can find.
> > 
> > > > 2- No, there was never a requirement that defsystem should only
> > > > be used
> > > > within a .asd. Actually, the test system relies heavily on the
> > > > opposite.
> > > > The requirement is that .asd files be loaded in the correct
> > > > context, by
> > > > load-asd -- notably, the correct *package* must be bound, the
> > > > correct
> > > > readtable, etc.
> > > 
> > > Unfortunately, DEFSYSTEM is the only entry point we can check. 
> > >  So if
> > > LOAD-ASD is important, that's the only place I can check it. What
> > > else
> > > would you have me do?  Check *load-truename* for "asd"?
> > > 
> > > Look, if you want to push this, then you can't object to my
> > > enforcing
> > > it.  If you don't want to push this, then we should make sure
> > > DEFSYSTEM
> > > works outside the context.  But I don't want to field bug reports
> > > where
> > > someone says "I tried to define this system and it didn't work,"
> > > and
> > > 
> > > In the olden days, we relied on programmers to make sure that the
> > > context for DEFSYSTEM reading was appropriate.  That was a pain
> > > for them
> > > sometimes, but it was clear, and it kept ASDF simple.
> > > 
> > > At some point, ASDF decided to take on the burden of establishing
> > > the
> > > context for DEFSYSTEM reading.  OK, not my choice, but a
> > > reasonable
> > > decision.  But I flatly refuse to maintain *both* the DWIMing in
> > > LOAD-ASD *and* DEFSYSTEM execution in arbitrary contexts.  And as
> > > a
> > > programmer, I don't want ASDF to let me evaluate a DEFSYSTEM form
> > > only
> > > to beat me up because some invisible context, only apparent
> > > through
> > > reading the code, means that it doesn't work.
> > > 
> > > Pick one, DWIMing, or freestanding execution, but you get only
> > > one.
> > > 
> > There too I don't understand your reasoning. Who are you helping,
> > and
> > who are you harming?
> > 
> > There always was a constraint on loading a .asd, and never was one
> > on
> > using defsystem. We can assume that all software that works fits
> > those
> > constraints, but might not fit tighter ones. Why make it harder to
> > use
> > defsystem? What would you check exactly?
> 
> I don't get it.  You are saying on the one hand "you have to have the
> right context to evaluate the contents of an asd file," and on the
> other
> hand "don't stop me from evaluating ... the contents of the asd
> file."
> What is it about an ASD file reading that requires this context, if
> it's
> not the
> 
> Loading an ASD is not a lisp thing, though.  Loading an ASD file
> involves evaluating the forms inside the file.
> > 
> > To me, setting the syntax context is an essential service to
> > provide to
> > whoever writes a .asd file. The .asd maintainer cannot assume
> > anything
> > about the syntax context used by the user, who may not control the
> > syntax context from the end-user. If you can't even trust the
> > package or
> > readtable, or character encoding, you can't even write code that
> > has any
> > guaranteed meaning. Note that it's called .asd rather than .lisp
> > for a
> > good reason: it's Lisp code, but supposed to run in a specific
> > context.
> 
> But that's true of EVERY Lisp file!  If you just write arbitrary
> code,
> that relies on symbols from specific packages, and syntax from
> specific
> readtables, and you don't put an IN-PACKAGE and readtable
> specification,
> it will break.  But I don't try to control what you do when you are
> writing arbitrary lisp code.  So clearly it's not "essential." Why do
> we
> do it for ASD files?  What's so hard about leaving it to the
> programmer
> to get this right?  Why did we have to make that our job instead of
> the
> programmer's job?

Because most Lispers don't know ASDF enough to get it right. If it can
be fixed in a single point, it should.

> Worse, we have broken that invaluable debugging tool
> of interactive execution of code, by encouraging programmers to rely
> on
> our use of LOAD-ASD.

LOAD-ASD is still interactive execution of code, just in a saf(er)
implicit environment. It's not a distinct parser.

> 
> Actually, when I read your reasoning, I think I see that this *is*
> generally true (especially your remarks about the readtable). This
> suggests to me that what you are really doing here is chewing off a
> tiny
> corner of CL, and in that tiny corner, you are trying to fix a thing
> that is broken about the language -- but only in that tiny corner.
> 
> I concede that this ship has sailed, and there's no tearing down
> LOAD-ASD, although I dearly wish I could.
> 
> But again, I feel very frustrated, because I feel like you are saying
> to
> me "there's SOMETHING in every .asd file that should only be
> evaluated
> in the context of LOAD-ASD, but I'm telling you it's not DEFSYSTEM,
> and
> I'm not telling you what it is."  Or even, "asd files should never be
> loaded, or evaluated, except through LOAD-ASD, but any form in an asd
> file can be."
> 
> So, ok, what IS it about an ASD file, that is not DEFSYSTEM, that
> must
> be evaluated inside LOAD-ASD? If it's just "you could wreck the
> syntax
> behind my back," yeah, that's true, but you could wreck the syntax
> behind my back and wreck the library I'm loading, too, so why is it
> ASDF's job to fix that here?  If LOAD is broken, why is it our job to
> make LOAD-ASD and fix a tiny corner of the problem?

Because unless you have the resources to fix the language in a major
way, you can only do small incremental improvements. Very pragmatic.
Not ideal.

-- 
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20160924/e42b909e/attachment.sig>