<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8">
</head>
<body>
<div style="font-family:sans-serif"><div style="white-space:normal">
<p dir="auto">I have pushed the system <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">fiveam-asdf</code>, which supports integration between ASDF and the FIVEAM test library, to the contribs directory in the ASDF repo. Please have a look.</p>
<p dir="auto">Be warned! It is old, and not being broke, hasn't been fixed. It inappropriately is housed in the <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">ASDF</code> package and inappropriately exports extensions from that package.</p>
<p dir="auto">But I believe it's still useful as an example of how to raise conditions when the test operation goes wrong. Catching those exceptions can be used to cause a build to fail in a CI system, typically by running lisp in batch mode and having it exit with a nonzero error code if the test operation fails.</p>
<p dir="auto">Best,<br>
r</p>
<p dir="auto">On 5 Jun 2018, at 4:09, Mark Evenson wrote:</p>
<blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px">
<p dir="auto">We use ASDF to encapsulate the building and testing many systems under an<br>
automated test runner.</p>
<p dir="auto">For a given system, ASDF:TEST-SYSTEM always returns boolean truth as long as<br>
the invocation of the underlying test suite succeeds.</p>
<p dir="auto">This means there is no programatic way to provide a boolean as to whether all<br>
the tests passed or not on the invocation of a test suite to the invoker of<br>
ASDF:TEST-SYSTEM.</p>
<p dir="auto">It seems that this issue has been raised before, as the ASDF manual documents<br>
TEST-OP <a href="https://www.common-lisp.net/project/asdf/asdf/Predefined-operations-of-ASDF.html#Predefined-operations-of-ASDF" style="color:#777">https://www.common-lisp.net/project/asdf/asdf/Predefined-operations-of-ASDF.html#Predefined-operations-of-ASDF</a> with:</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999">
<p dir="auto">The results of this operation are not defined by ASDF. It has proven difficult<br>
to define how the test operation should signal its results to the user in a way<br>
that is compatible with all of the various test libraries and test techniques<br>
in use in the community, and given the fact that ASDF operations do not return<br>
a value indicating success or failure. For those willing to go to the effort,<br>
we suggest defining conditions to signal when a test-op fails, and storing in<br>
those conditions information that describes which tests fail.</p>
</blockquote>
<p dir="auto">Is this still the best current practice to introspect the situation of failing<br>
tests? Can someone point me to an example implementation of this technique?</p>
<p dir="auto">-- <br>
"A screaming comes across the sky. It has happened before but there is nothing<br>
to compare to it now."</p>
</blockquote>
</div>
</div>
</body>
</html>