[asdf-devel] About regression tests and libraries
rpgoldman at sift.info
Wed Aug 5 19:46:28 UTC 2009
Nikodemus Siivola wrote:
> 2009/8/5 Robert Goldman <rpgoldman at sift.info>:
>> Part of the problem with test-op is that the desired behavior has not
>> been specified by the ASDF community. Because of the nature of ASDF, it
>> is impossible for
>> (asdf:test-system <system>)
>> to return a value indicating whether or not <system> has passed its tests.
> I would propose that as a matter of good style implementations of
> TEST-OP should signal an error is tests do not pass. It is the
> simplest way to make sure the result is not misinterpreted...
I suppose, but typically a regression test suite should attempt to run
ALL of its tests, collecting a set of failing tests, and then returning
that information. And demanding interactive attention ("I'm going to
throw you into the debugger if I fail") may also be undesirable in
practice. I suppose handler-case would be enough for that, but I'm
still uncomfortable with the idea.
More information about the asdf-devel