[PATCH] Add a TEST-OP-TEST-FAILURE condition for test libraries to sub-class

[Vladimir Sedach]

Anton Vodonosov <avodonosov at yandex.ru> writes:

> - success should also be signaled, so we can distinguish a version where
>   this new protocol is not implemented from the version where tests pass

That is a good idea.

> - For a caller of asdf:test-op it would be more convenient to have a single
>   signal. Ideally, it should be just a return value of the asdf:operate function,
>   as I understand we only consider the possibility of test result being signaled
>   multiple times during test-op because we hope to make it work for everyone
>   without library authors explicitly modify their code, but adding this new
>   functionality to test frameworks. A good goal, although I can imaging
>   some corner cases. Still, even if we expect test results being signalled
>   multiple times during a test-op, it would be good to provide a wrapper
>   which aggregates them into a single return value.
>
>         (common-test-results:collect (asdf:test-system "my-system"))

That is a good idea. I think it goes together well with the fully
qualified test names recommendation.

> - as others mention, to me it also occurred this new functionality
>   should not necessarily be declared inside of ASDF, it could be
>   some separate library, say common-test-result. I'm not 100% sure
>   about this, but currently, lean more towards separate lib, at least
>   for the beginning. ASDF test-op docs could just referer to it.

Raising a signal is a work-around for the inability of TEST-OP to
return a result. I would like to avoid making an entire library out
of a work-around that is specific to ASDF.

--
Vladimir Sedach
Software engineering services in Los Angeles https://oneofus.la