[asdf-devel] ASDF test-op question

[Juan Jose Garcia-Ripoll]

On Mon, Oct 19, 2009 at 12:00 AM, Robert Goldman <rpgoldman at sift.info> wrote:
>> In particular because it's my impression that the problems have been
>> over-stated.

I think so. And this is indicated by the following paragraphs

> 1.  Not all regression test frameworks are functional, returning values.
>  Some write reports instead.

That does not mean anything. ASDF may impose one behavior w.r.t to the
output and the package maintainer may decide to output T saying "hey,
there might be errors, but I do not know." Later on other software and
test suites will catch up. With RT, for instance it is trivial to grab
the number of failed tests.

> 2.  One needs to come up with a means of combining operation results
> that takes into account the structure of the plan that traverse produces
> and that operate then executes.  If in order to test-op X, I must
> test-op A and B, how do I combine together the test results from A, B,
> and X into a top-level operation result.  Perhaps OPERATION-ANCESTOR can
> be pressed into service.

You are just imposing too much complexity. If I want to test package
Cl-UNICODE, I do nont want to test FLEXI-STREAMS or U-SOCKETS. Tests
should be atomic and not generate a tree of actions like ASDF does not
for everything.

> 3.  Is there some way to do this such that the regression-testing
> framework need not be made aware of ASDF?

No, there is none. Regression tests are libraries which are used by
people using ASDF. Eventually they may provide their own operations,
extending what ASDF provides, but currently it is better to have some
bare minimum from ASDF and let each package maintainer decide how they
compute the success of their test suite.

Blocking this development just because there are 5 test suites and you
do not know how to combine them with ASDF is really absurd. ASDF's
specifications can not depend on what your company or other people's
companies are setting up for their workflow.

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://juanjose.garciaripoll.googlepages.com