[fiveam-devel] Unit Test Framework

[Tapiwa Gutu]

Hey Chris,

Thanks for the quick response. I took a look at the link you posted. The
syntax I used was most heavily influenced by that in lisp-unit because
what any given function or macro did was very clear from the name without
having to read the documentation first. Names like ASSERT-TRUE
and ASSERT-EQUAL are pretty self explanatory! Also the syntax is very
simple and easy to read and use.

Because the syntax is so close to that of lisp-unit I will make a quick
translation with a few changes and test it. The changes would be as shown
below,
the rest is pretty much the same.

(in-package :cs325-user)
;;; Define a suite for all our tests since CLUnit only groups
according to test suites instead of the implicit package grouping in
lisp-unit.
(defsuite cs325-user ())
(define-test testname . body) -> (deftest testname (cs325-user) . body)

I had left out ASSERT-PRINTS simply because I had not considered its
uses well enough. But after you pointed it out, I realised having a
built in form

to test the messages you print in your library would be very handy.

The definition of ASSERT-TRUE that I used is actually a predicate
tester according to its common use in Lisp conditional
forms such as IF and WHEN. I figured if you want to test for the
specific value T you would use (assert-eq t form).

(assert-equality 'set-equal '(car) (functions-referenced '(car l))) ->
(assert-true (set-equal '(car) (functions-referenced '(car l)))


There is an idea I have been toying with in my head. It seems the
library generally works now so I think its now
safe to write unit tests for the library itself. Now, I obviously need
test the lower level functions and make

sure that they are doing the correct thing. So I have a situation were
I want to test A and B, but since B depends
of A, if A fails its test there is no point testing B. I would need to
first fix A before I can trust what B will do.

Basically, I want to be able to short circuit tests. I want to add
this without changing the current behaviour.
Since execution of tests in a test suite is unordered, you would place
the dependent tests in a child test suite.

So something like this:

1. (defsuite A ())
2. (defsuite B (A))
3. Define low level function tests in A.
4. Define tests in B that depend on functions tested in A.

5. (run-suite 'A :stop-on-fail t)

The RUN-SUITE keyword argument :use-debugger, is for interactive
tests, this option would work for short circuiting batch tests.
What do you think?



On 10 November 2012 21:28, Christopher K Riesbeck <
c-riesbeck at northwestern.edu> wrote:

>  On a quick skim, looks like nice work. If you want to see if it handles
> a lot of the cases (better or as well) that lisp-unit was designed for, see
>
>
> http://www.cs.northwestern.edu/academics/courses/325/programs/exercise-tests.lisp
>
>  This is what tests my students' solutions to exercises from Graham and
> elsewhere. I'd prefer to use a standard test framework in my class, but I
> would want that particular file to continue to be simple for novices to
> read and understand, since they're the target audience.
>
>   I'll try making the translation myself sometime but it won't be for the
> next few months at least, due to time constraints.
>
>  I notice that you didn't include an "assert-prints" though obviously
> it's not hard to use some wrapper macro like
>
>     (assert-equal "...." (collect-output (print-dots 4)))
>
>  I would recommend adding something comparable to lisp-unit's
> assert-equality that lets you handle special equality testers, like
> set-equal, epsilon-equal, etc. (I used to call it assert-predicate but that
> implied a unary rather than binary predicate.)
>
>  Nice job.
>
>
>
>  On Nov 10, 2012, at 11:41 AM, Tapiwa Gutu wrote:
>
> Faithful hackers,
>
> I decided to take up the challenge laid down here
> http://fare.livejournal.com/169346.html and try to consolidate the Common
> Lisp unit testing frameworks. I have written a framework that aims to
> consolidate all the major features of all your frameworks mentioned in this
> blog
> http://aperiodic.net/phil/archives/Geekery/notes-on-lisp-testing-frameworks.html
> .
>
> You can find it on Github https://github.com/tgutu/clunit. I also wrote a
> blog on the development of the framework and reasons for it here
> http://ml.sun.ac.za/2012/11/09/developing-a-unit-test-framework-part-1/if you are interested.
>
> I would very much appreciate it, if you could join me in this effort and
> we all work together torwards making this project a success.
>
> Regards,
> Tapiwa
>
>
> Christopher Riesbeck
> Home page: http://www.cs.northwestern.edu/~riesbeck
> Calendar: http://calendar.yahoo.com/criesbeck
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/fiveam-devel/attachments/20121111/6bdbf2be/attachment.html>