[asdf-devel] ASDF test-op question

[Tobias C. Rittweiler]

Robert Goldman writes:

> Faré wrote:
>
> > Maybe ASDF is the wrong place to try to standardize testing infrastructure?
>
> This is the conclusion I have reached, as well.  I was hoping that some
> very weak standard could be arrived at that would make the test-op more
> generally useful to people installing systems, so that they could simply
> run the test-op and easily tell whether or not the test operation was
> successful.
>
> However, it may be that it's just a combination of features of our test
> framework and the way we have built our systems that makes it difficult
> for us (the original problem --- we find that we get compilation and
> loading output mixed together with the test output), so this is not a
> place where the ASDF community can readily get consensus.
>
> So please consider the suggest dropped.

Please excuse me that I cannot grow at peace with this conclusion.

The missed utility is far too great to let this rest:

Juanjo's automatic regression testing work to check for regressions in
ECL is dead on, and I'm sure other implementators would be interested in
such infrastructure as well.

The same is true for D Herring and his LibCL project which looks very
promising, but is also a candidate for automatic regression testing. I
think that Daniel is the right guy for the job and I hope he'll have the
necessary perseverance for what he has started.
  

I have to excuse myself again because I'm not familiar with ASDF in any
great detail; on the other hand, ASDF -- unlike most other Lisp project
-- consists of over half a dozen of people who do know it pretty well,
so I cannot believe that we can not come up with some solution.

In particular because it's my impression that the problems have been
over-stated. 

What's wrong with something on the line of defining a class
OPERATION-RESULT which consists of a single flag SUCCESSP, making each
OPERATION store such an object in a RESULT slot. Then define an :AROUND
method on PERFORM which does

   (setf (operation-result op) 
         (multiple-value-call #'make-operation-result op 
                              (call-next-method)))