Pointing out ABCL "extensions" to ANSI-TEST
Raymond Toy
toy.raymond at gmail.com
Tue Sep 15 15:34:50 UTC 2015
>>>>> "Mark" == Mark Evenson <evenson at panix.com> writes:
[snip]
Mark> 1) ASDF
Mark> By the nature of how ANSI-TEST loads and runs itself, encapsulation as a
Mark> "true" ASDF system (i.e. one where all the source is declared) seems a
Mark> little farfetched. And maybe it is useful to be able to run the suite
Mark> in places where ASDF is not available, such as when bootstrapping a new
Mark> implementation. Nevertheless, we put some work into encapsulating the
Mark> ANSI-SUITE into ASDF to the point that it can be loaded and the tests
Mark> run that might be of interest.
This is interesting. It sounds nicer than running a makefile where I
always have to look to see what the magic variables are to run what I
want.
Mark> 2) ABCL.TEST.ANSI
Mark> In order to support the necessary shenanigans to make the ANSI-COMPILED
Mark> and ANSI-INTERPRETED definitions work, we [created a small set of
Mark> utility routines][2]. CLEAN-TESTS performs the removal of intermediate
Mark> files that the old Makefile used to do. DO-TESTS-MATCHING will run only
Mark> those tests whose symbols match a regular expression, which is useful
Mark> for repeatedly running a subset of tests.
This also sounds interesting because I often do want to run just a
subset.
Mark> 3) ANSI-TEST error "database"
Mark> We [created a simple s-expr "database"][4] for recording test failures
Mark> and then [reporting on the results][3]. ABCL is a unique CL
Mark> implementation because it is hosted on a JVM for which there are
Mark> multiple implementations and whose behavior often varies by operating
Mark> system so running the ANSI-TEST suite has proven to be fairly sensitive
Mark> to hosting JVM implementation version and operating system. Thus, such
Mark> a database of regressions has proven to be of special importance to
Mark> ABCL. That being said, my implementation of this idea is pretty
Mark> amateurish in hindsight, so I think what is important here is the idea
Mark> rather than the implementation.
Again, this sounds pretty cool too. Cmucl has a few extensions that
cause the test suite to fail. It would be nice to annotate these test
as expected failures. Idealy, a reason for the failure would be part
of the annotation. I don't run the test suite that often anymore, but
when I do, I always forget exactly which tests are known expected
failures and which are new. Having some kind of annotation (other
than comments on the test itself) would be awesome!
--
Ray
More information about the armedbear-devel
mailing list