[asdf-devel] TEST-OP made useful by declarative options

[Juan Jose Garcia-Ripoll]

On Sat, Apr 17, 2010 at 10:29 PM, Robert Goldman <rpgoldman at sift.info>wrote:

> I applaud the idea of having a declarative spec of tests to be run, but
> is the test function really the only way to do this?  Seems like there
> are test frameworks where what you want to do is to specify (somehow) a
> set of test objects to be run, where the framework "knows" how to run them.
>

There are several reasons why I used the function name and did not create
framework specific options

* People are using their own custom functions for running tests, and never
in an uniform way.
* It was argued here that ASDF should never be bound to any particular test
framework. I have become convinced of the arguments in that respect.
* We can not impose test frameworks how to behave, but we can recommend
users what they should output their tests (please have a look at the thread
I started long ago).


> suggestion:  if we are to propose an ASDF extension for testing, we
> should first survey the CL test frameworks (off-hand: FiveAM, NST,
> cl-unit, LIFT, RT, and ? ...) and see what they would most "like" in the
> way of a test specification.
>

Given that the :tests keyword work, I have a mean to do what you want
without really binding ASDF to the test framework.

The idea would be to provide plug-ins for those frameworks with system
classes that take those new options. We would then extend the syntax of
:tests to include closures (would be great!) and let the system-test class
set the appropriate value of :tests.

But note that is completely independent of the current patch and does not
preclude or make it less useful, since users may also want to write their
own tests.

But what this does bring into the light is the need of means to describe
that a given DEFSYSTEM form depends on a given plug in or other system. This
would be what I was calling the :ASDF-DEPENDENCIES or :ASDF-SUPPORT option
in previous emails. Something like

(defsystem :my-test-system
   :asdf-dependencies (:rt-system)
   :rt-tests ("TEST1" "TEST2" ...)
   ...)

Let's see if I can produce something like that tonight -- stranded in Madrid
because of the volcano cloud :-/

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100417/9c818fd7/attachment.html>