[asdf-devel] About regression tests and libraries

[Tobias C. Rittweiler]

Nikodemus Siivola writes:

> 2009/8/5 Robert Goldman <rpgoldman at sift.info>:
>
> > Part of the problem with test-op is that the desired behavior has not
> > been specified by the ASDF community.  Because of the nature of ASDF, it
> > is impossible for
> >
> > (asdf:test-system <system>)
> >
> > to return a value indicating whether or not <system> has passed its tests.
>
> I would propose that as a matter of good style implementations of
> TEST-OP should signal an error is tests do not pass. It is the
> simplest way to make sure the result is not misinterpreted...

You wrote "tests do not pass"; just to make sure, the plural was
intentional, right? You did /not/ mean it should signal an error for
each failing test, did you?

If I'm running the TEST-OP on some system (with the user, not developer,
hat on my head), I'd like to get a nice summary about all failing tests
to give a sense about the state of the software. I'm not interested in
the minutae of the tests themselves.

Signalling an ASDF:TEST-OP-FAILED condition, perhaps even making that
hidden in some function, so people can just write (ASDF:TEST-OP-FAIL)
within their PERFORM method, is sure an idea if that better fits the
architecture of ASDF.

  -T.