[asdf-devel] About regression tests and libraries

Robert Goldman rpgoldman at sift.info
Wed Aug 5 19:46:28 UTC 2009

Nikodemus Siivola wrote:
> 2009/8/5 Robert Goldman <rpgoldman at sift.info>:
>> Part of the problem with test-op is that the desired behavior has not
>> been specified by the ASDF community.  Because of the nature of ASDF, it
>> is impossible for
>> (asdf:test-system <system>)
>> to return a value indicating whether or not <system> has passed its tests.
> I would propose that as a matter of good style implementations of
> TEST-OP should signal an error is tests do not pass. It is the
> simplest way to make sure the result is not misinterpreted...

I suppose, but typically a regression test suite should attempt to run
ALL of its tests, collecting a set of failing tests, and then returning
that information.  And demanding interactive attention ("I'm going to
throw you into the debugger if I fail") may also be undesirable in
practice.  I suppose handler-case would be enough for that, but I'm
still uncomfortable with the idea.


More information about the asdf-devel mailing list