Testing for ASDF 3.3.2 and beyond?

[Anton Vodonosov]

02.03.2018, 05:33, "Faré" <fahree at gmail.com>:
>>>  Often failures in cl-test-grid are "just" the result of using too
>>>  little memory,
>>
>>  not little memory, but SBCL's conservative (i.e. imprecise) garbage
>>  collection - it doesn't always recognize that an object is a garbage.
>>
>>  I saw cases where CCL compiles a project under a 100MB heap,
>>  while SBCL reproducibly runs out of 1GB heap.
>
> Maybe it's an artefact of SBCL using too much memory *while compiling*
> and would go away if you used e.g. POIU to compile inside forks.

How?

>
>>>  or batching system loads,
>>
>>  There is no batch loading, each test is run in a fresh CL process.
>
> Interesting. The memory load might still be sensitive to compilation
> order, if you reuse fasls shared with previous builds versus rebuild
> new ones.

The test order is alphabetical (by projects, and then inside a project
its ASDF systems are sorted alphabetically too).

Fasls compiled by previous tests are reused by next tests.


>
>>>  or some other reason, and
>>>  have to be retried.
>>
>>  I would put it differently: some failures may be ignored because
>>  on the previous version they were successful by a chance,
>>  and could faile the same way too.
>
> OK.
>
> PS: The newer test results look good. I'd like to see the results with
> the syntax-control branch, though. There also are a few minor bug
> fixes in the pipe that mean we'll have to re-test before release,
> anyway.
>

Yes, I plan to start syntax-control testing soon. 

You might be interested to know that ACL binds *readtable* on startup to an unmodifieble readtable
(which is a violation of CLHS http://clhs.lisp.se/Body/02_aac.htm : 
"The initial readtable is distinct from the standard readtable. It is permissible for a conforming program to modify the initial readtable.")

The most visible result of this is puri and more than 150 other systems in Quicklisp depending on it, failing to load.

Another hint: when you review test results, it might be helpful
to see how particular library is built in other implementations and in the previous Quicklisp.
Here you can find such build results for every library:
https://common-lisp.net/project/cl-test-grid/library/

Best regards,
- Anton