[fiveam-devel] Unit Test Framework

[Marco Baringer]

Tapiwa Gutu <tapiwa.gutu at gmail.com> writes:

> Faithful hackers,

> I decided to take up the challenge laid down here
> http://fare.livejournal.com/169346.html and try to consolidate the
> Common Lisp unit testing frameworks. I have written a framework that
> aims to consolidate all the major features of all your frameworks
> mentioned in this blog
> http://aperiodic.net/phil/archives/Geekery/notes-on-lisp-testing-frameworks.html.

> You can find it on Github https://github.com/tgutu/clunit. I also
> wrote a blog on the development of the framework and reasons for it
> here http://ml.sun.ac.za/2012/11/09/
> developing-a-unit-test-framework-part-1/ if you are interested.

i mean no disrespect to your effort, but i think it'd be better to
improve one of the existing test frameworks (and i'm partial to fiveam)
than to try and start from scratch. now, if fiveam really doesn't (or
can't) do what you want/need it to do, then i'd love to talk about what
that is. if you end up writing your own test framework, well, i can
totally understand that too.

i also think that this problem isn't nearly as bad as it seems, fiveam
already does many of the things the other test suites do, but the
documentation is pretty horrible (so nobody knows this). did you know
about 5am:debug! ? (i didn't. until a few days ago). 

so, here's an attempt at improving fiveam's documentation (this is 60 or
70% complete):

http://bese.it/fiveam/manual.html

i'd love someone to have a quick look at it and tell me if it makes
sense (i know fiveam too well to be a good judge) or if it needs to be
fundamentally changed somehow.

it all lives in this repo:

https://github.com/segv/fiveam

which, other than the docs contains a few minor cleanups and one new
feature (no pull request yet).

ps - you know what would be really cool? source location of the failing test
(if i have 20 is checks in a test and one of them fails it'd be nice to
know the filename and line number of those checks which failed. i'd
settle form something that was #+(or sbcl ccl)...)

pps - to get stefil's debugger when a test fails in fiveam, just do
(setf 5am:*debug-on-failure* t), if you want low over head just ignore
suites (the default suite, nil, always exists) and use (5am:run!)  to
run all the currently defined tests, if you want ever lower overhead
then set 5am:*run-test-when-defined* and you can skip the call to run!.

--
-marco