Introspecting test suite passing/failing

[Mark Evenson]

We use ASDF to encapsulate the building and testing many systems under an
automated test runner.

For a given system, ASDF:TEST-SYSTEM always returns boolean truth as long as
the invocation of the underlying test suite succeeds.

This means there is no programatic way to provide a boolean as to whether all
the tests passed or not on the invocation of a test suite to the invoker of
ASDF:TEST-SYSTEM.

It seems that this issue has been raised before, as the ASDF manual documents
TEST-OP <https://www.common-lisp.net/project/asdf/asdf/Predefined-operations-of-ASDF.html#Predefined-operations-of-ASDF> with:

> The results of this operation are not defined by ASDF. It has proven difficult
> to define how the test operation should signal its results to the user in a way
> that is compatible with all of the various test libraries and test techniques
> in use in the community, and given the fact that ASDF operations do not return
> a value indicating success or failure. For those willing to go to the effort,
> we suggest defining conditions to signal when a test-op fails, and storing in
> those conditions information that describes which tests fail.

Is this still the best current practice to introspect the situation of failing
tests?  Can someone point me to an example implementation of this technique?


-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."