[Asdf-devel] syntax-control branch

[Robert P. Goldman]

Faré wrote:
> On Sat, Jun 7, 2014 at 8:03 PM, Robert P. Goldman <rpgoldman at sift.info> wrote:
>> I'm still groping after a clear statement of exactly what class of bugs
>> it is that we are proposing to fix.  The existing design document is
>> still too vague, proposing to fix
>>
>> "uncontrolled, unintentional leaking of readtable side effects to
>> systems that depend on such effects not happening,"
>>
>> I think we need a clear example of a bug (ideally from a real system,
>> although it's fine if that bug has since been fixed) that would be fixed
>> by the change.
>>
> I probably should create test cases.
> 
> Scenario A: dealing with *readtable* being bound to a new value
> 1- setup the test environment
> 2- save current packages
> 3- load fare-quasiquote (or anything that has its own readtable)
> 4- while in-readtable fare-quasiquote, load-system :force t
>    something that uses normal quote,
>    and see it corrupt its fasl with fare-quasiquote annotations.
> 5- delete all new packages since 2
> 6- repeat 3, 4 — and see whether it breaks [used to]

OK, so the bug is not that systems leak the readtable *out*, it's that
readtables leak *into* systems. I had not understood that.

So unlike *PACKAGE*, the current (REPL) value of *READTABLE* leaks into
every system that you load.  It seems to me that this is a deficiency in
CL: if CL had an IN-READTABLE equivalent to IN-PACKAGE, we would not be
having this discussion.

> 
> Scenario B: dealing with the *readtable* object being modified
> 1- setup the test environment
> 2- load and compile x1, that define #^
> 3- load and compile x2, that uses #^ as per x1
> 4- load and compile x3, that defines a different #^
> 3- load and compile x4, that uses #^ as per x3
> 4- force recompilation of x2 — it will use the definition from x4 — oops
> 5- force recompilation of x1 and x4 — it will use the definition from
> x2 — oops again

This is the one I had in mind, and this is the one I don't think is
ASDF's problem.  If x1 exported *X1-READTABLE* and ADD-X1-SYNTAX (this
function adds X1 reader macros to an existing readtable passed as
argument), then the Right Thing would simply be for X1 not to alter the
global readtable, and for X2 and X4 to explicitly employ the readtables
they want.

This problem does not seem worthy of fixing in ASDF.

Scenario A seems like a stronger argument, but I'm inclined to say
"instead of limping along with a complex fix in ASDF, let's just push
NAMED-READTABLES use and leave ASDF alone."

NAMED-READTABLES seems like a much better fix than complicating ASDF
further, because it gets at the root of the problem, instead of
band-aiding around it.

The counter-argument, I suppose, is that Scenario A can only be fixed if
*EVERYONE* fixes their code to put IN-READTABLE at the top of their
source files, just the way we have to put IN-PACKAGE at the head of
every source file.
> 
>> Also, we need an argument for why this must be fixed in ASDF.
>>
> Because ASDF must be safe when called from a different readtable.
> And using ASDF safely in presence of
> 
>> Here's the devil's advocate argument:
>>
>> "OK, you have a crummy library that leaks state unintentionally, and
>> happens to do that in the readtable.  Why is it ASDF's job to make that
>> library suddenly work *without modification*?  Why shouldn't we just
>> tell the maintainer of that library to stop leaking state unintentionally?"
>>
> It's not *that library*. It's all the innocent libraries
> that know nothing about readtables, and that will be compiled
> into calls to functions from systems they don't depend on,
> which will pollute the build.

My question here came from thinking only about your Scenario B, and
overlooking Scenario A.

I'm starting to reluctantly come around, but I still need to figure out
how my system went up the spout when using syntax-control ASDF. If I
can't answer on my own behalf, I can hardly claim to understand the
patch well enough to apply it....

Best
r