[asdf-devel] Testing ASDF with cl-test-grid

[Faré]

Dear Anton,

> When we tested with cl-test-grid,
> ABCL, CMUCL, CCL, CLISP, ECL, SBCL were tested on 2.31.8
> http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-21.html
>
> The latest cl-test-grid cycle was for ASDF 2.32.35, it was tested
> on CCL, CLISP, ECL, SBCL on Linux.
> http://common-lisp.net/project/cl-test-grid/asdf/asdf-diff-23.html
>
> So, as you see we ended up testing last fixes only on some lisps,
> not all lisps. Probably that's because it was stabilizing period and
> the fixes were for concrete bugs.
>
> Still, we didn't make a firm "rollback" point - an ASDF version where we
> ensured we have no regressions on all lisps. For now I suggest to consider
> 2.32.35 as more or less reliable previous comparison point, where
> all regressions were proved to be not ASDF bugs, but bugs in libraries,
> and were reported to the library maintainers.
>
> I hope in this ASDF release, we will test the released version
> on maximum number of lisps, so that in future we will have reliable
> comparison point.
>
Yes. This time, let's use 2.32.35 as our baseline, and if you can,
test both ASDF 3.0.3 as the current stable, and
3.1.0.32 (current HEAD) as our release candidate for 3.1.1.

I don't actually expect any change in cl-test-grid results between all
these versions, except maybe that my own software (e.g.
lisp-interface-library) might possibly fail on older versions of ASDF.

>> But then, you'd have to check with both old and new Quicklisp.
>
> I am going to run tests of both previous stable ASDF version and the
> current HEAD on the same Quicklisp - 2013-12-13.
>
Thanks a lot!

>> I suppose that you should compare with whatever comes with the implementation.
>
> OK, we can do this too - we will compare result for the latest ASDF
> with the results for the "main", unpatched quicklisp, which uses ASDF
> coming with the implementation.
>
> It also would be nice to compare with some concrete previous stable
> ASDF version, as described above, to be sure we cover full ASDF evolution
> on all lisps.
>
Don't overdo it. I would do it with
maybe 2.26 (default from Quicklisp), 2.32.35 (previous tested),
3.0.3 (latest stable) and 3.1.0.32 (release candidate),
and that's already a lot.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
It may be bad manners to talk with your mouth full, but it isn't too
good either if you speak when your head is empty.