[cffi-devel] how to treat expected failures in tests

[Jeff Cunningham]

On 01/10/2012 04:28 AM, Luís Oliveira wrote:
>
>> I am interested in both short answer - as a library author, how do you think CFFI test suite
>> should be marked if only expected failures present - OK or FAIL?
> Depends on the use case. In terms of notifications, I would rather be
> warned about new failures. In terms of a summary, I'd like to see the
> results broken down into OK, FAIL, KNOWNFAIL.
>
>
How about OK, FAIL, UNEXPECTEDOK, and EXPECTEDFAIL? You have to consider 
the the cases where one expects a failure but it passes too. If those 
are too long, use the Bayes approach: FP and FN (false-positive and 
false-negative)

Jeff