Inclusion into cl-test-grid

[Raymond Toy]

>>>>> "Ian" == Ian Tegebo <ian.tegebo at gmail.com> writes:

    Ian> On Fri, Apr 29, 2016 at 8:39 AM, Raymond Toy <toy.raymond at gmail.com> wrote:

    Ray> some of the failures are really hard to fix, so they don't
    Ray> get fixed.  Some of the tests, while correct, I refuse to do
    Ray> anything about


    Ian> Ah! I think that'd be useful to have captured, formatted, and
    Ian> distributed, e.g. for people concerned with portability. 
    Ian> There doesn't need to be, but I'm not arguing against, some
    Ian> moral judgement wrt to passing/failing tests.  You can trade
    Ian> it for greater visibility in how implementations, and perhaps
    Ian> versions of the same implementation, vary over time.

I think ansi-tests does have some support for this.  There's something
like array-nil-is-string that you can set. (Or something. I don't have
the code handgy).  This disables the tests in that case.  There are
others such flags available.

    Ian> With that info in hand, someone could build the CL equivalent
    Ian> of http://caniuse.com/ (or add it into cl-test-grid's
    Ian> reporting).

    Ray> When I do run ansi-tests on the rare occassion, it's mostly
    Ray> now as a regression


    Ian> That's a great point.  I imagine one might run ansi-test and
    Ian> then have some comfortable means to annotate discrepancies. 
    Ian> Those annotations could then be used as a baseline for
    Ian> subsequent versions.  Including the deltas in release notes
    Ian> would be nice.

Yes, this is one reason I created my own fork of ansi-tests long
ago. I wanted more annotation on tests and such.  I never actually did
anything, though. :-(

--
Ray