[cffi-devel] how to treat expected failures in tests

[Anton Vodonosov]

Hello.

For my common lisp testing project I aggregate results of a library test suite into single value - ok/fail.

I just tested ECL and have the following output from CFFI test suite:

4 out of 228 total tests failed: DEFCFUN.NOOP, CALLBACKS.BFF.1, STRING.ENCODING.UTF-16.BASIC, 
   STRING.ENCODINGS.ALL.BASIC.
No unexpected failures.

What meaning do you put into the term "expected failure"? Does it mean the library is buggy, but
these bugs are known? Or it means that some non-required features are absent, but the 
library in general OK?

I am interested in both short answer - as a library author, how do you think CFFI test suite 
should be marked if only expected failures present - OK or FAIL?

And also I am curious in this concrete example, what these 4 failures mean for CFFI on ECL?

Best regards,
- Anton