[asdf-devel] TEST-OP made useful by declarative options

[Robert Goldman]

On 4/17/10 Apr 17 -2:18 PM, Juan Jose Garcia-Ripoll wrote:
> The attached patch adds a new option to DEFSYSTEM. This option is
> 
> :TESTS test-list
> 
> test-list = (:system system-name) | function-name
> system-name = keyword | string
> function-name = symbol | string
> 
> The function name is a string designator that denotes a function with no
> arguments to be called. This string will be read using the standard I/O
> syntax only when the test is executed. This ensures that the mechanism
> works once the appropriate dependencies are loaded.
> The system name denotes a system that will be loaded when the TEST-OP
> operation is invoked.
> 
> The patch can be improved with some tests done when executing DEFSYTEM
> instead of what it does now, which is to signal an error once the test
> is performed.

I applaud the idea of having a declarative spec of tests to be run, but
is the test function really the only way to do this?  Seems like there
are test frameworks where what you want to do is to specify (somehow) a
set of test objects to be run, where the framework "knows" how to run them.

suggestion:  if we are to propose an ASDF extension for testing, we
should first survey the CL test frameworks (off-hand: FiveAM, NST,
cl-unit, LIFT, RT, and ? ...) and see what they would most "like" in the
way of a test specification.  Then we can decide how to extend ASDF to
suit.  Alternatively, possibly different test frameworks could propose
different extensions.

(defsystem foo
  :lift-tests ...
)

etc.

Best,
r