[imp-hackers] Procedure for adding tests to the ansi test suite

[Tobias C. Rittweiler]

Can anyone think of a good procedure for getting new tests into the test
suite?

I think there are a few requirements:

  * the new tests should be visibly separate from what's currently
    there. Perhaps this should be reflected on the file system layout of
    the source tree.

  * it should be possible to exclude the new tests when running the test
    suite.

  * a proposal explaining the justification for the test.

  * there should be at least N people who agree that the test checks for
    something that matches with their understanding of the
    specification.

More to the last point:

Preferably the N people represent different implementations.

What's a good N? >= 2 ?

Should a disapproval by as few as one person who can not be convinced
differently in reasonable time be enough to deny a proposal? Perhaps
this should be tightened up to a person representing an implementation.

Where should discussion take place? I'm sending this mail both to the
implementation-hackers and ansi-test mailing list mostly to reach as
many people who may be concerned.


  -T.