[asdf-devel] About regression tests and libraries

[Robert Goldman]

Tobias C. Rittweiler wrote:
> Juan Jose Garcia-Ripoll writes:
> 
>> What I would like to ask CL developers is to begin shipping regression
>> tests with their libraries, and perhaps agree on an standard so that
>> one can programatically check whether a library builds or not. This
>> feature could be even included in ASDF, asdf-install, etc, just like
>> "make check" is now a de-facto standad in Autoconf.
> 
> There's (ASDF:OOS 'ASDF:TEST-OP <system>), and on very recent revisions
> even (ASDF:TEST-SYSTEM <system>) as an abbeviation.
> 
> I don't think this has been properly documented in the manual of ASDF
> yet. I hence CC'd the asdf-devel mailing list so this will hopefully get
> promoted with the due it deserves.
> 

Part of the problem with test-op is that the desired behavior has not
been specified by the ASDF community.  Because of the nature of ASDF, it
is impossible for

(asdf:test-system <system>)

to return a value indicating whether or not <system> has passed its tests.

Some time ago, I proposed that ASDF provide a stream argument to the
test-op, providing a place into which the testing tool could dump its
test report for human inspection.  I can't say that this suggestion
received universal approbation.  Or disapprobation.  It merely received
near-universal lack of interest! ;-)

Related to the question of what the test-op should provide to its
invoker is the question of how dependencies should be propagated.

Cheers,
r