Two test errors in latest slime from github

[Paul Bowyer]

On 02/21/2014 09:04 AM, João Távora wrote:
> Luís Oliveira<luismbo at gmail.com>  writes:
>
>> On Fri, Feb 21, 2014 at 3:34 PM, Paul Bowyer<pbowyer at olynet.com>  wrote:
>>> 2 unexpected results:
>>>     FAILED  autodoc-tests-23
>>>     FAILED  autodoc-tests-24
>>>
>>>
>>> I ran the test suite from the command prompt to check if it was my script
>>> causing the problem, but I got the same two errors.
>> I believe this is a known issue, registered in this very spartan
>> issue:<https://github.com/slime/slime/issues/106>.
> Paul,
>
> The autodoc tests, were, until fairly recently, hidden from the
> automated test run by a mistake of mine. They were "unhidden"
> recently and hence that failure shows.
>
> Alternatively, it might be that you have updated your SBCL, since those
> two tests don't fail in the 32bit version, which is the version that
> Travis CI runs. They also don't fail on some cmucl version.
>
> The problem is that autodoc tests are fairly brittle and when the
> swank-side reports a little more or less info, it thinks it's a failure.
>
> Part of Luís's work is determining when that is true and when it isn't.
>
> Maybe you can help. A good place to start would be to register in the
> issue, for each problematic test case, exactly:
>
>     what is happening across different versions;
>     what the tests believe should be happening;
>     what you think should be happening.
>
> João
>
>
>
>
Hello João,

I've been able to get both the 32-bit and 64-bit versions of CCL running 
on my system and they load into slime OK, but I've not yet tried to work 
with any lisp code with them.

I tried the following with the 64-bit version of CCL from a command prompt:
cd ~/my-slime-sandbox/slime
make LISP=ccl clean
make LISP=ccl compile

make LISP=ccl clean check
make LISP=ccl check-fancy

But it gets to a point in one of the tests and produces no further 
output. It may be just very slow, but I waited for a very long time for 
more output. The point where it stops producing output:

<clip>
Fontifying *slime-macroexpansion*... (regexps.............)
Fontifying *slime-macroexpansion*... (regexps..............)
Fontifying *slime-macroexpansion*... (regexps...............)
waiting for condition: top-level [21:08:22.373847]
Undo!
passed 58/184 macroexpand-1
Saving file /tmp/slime-recipe-11858GDD.el...
Wrote /tmp/slime-recipe-11858GDD.el
passed 59/184 readme-recipe
Saving file /tmp/slime-recipe-11858DnF.el...
Wrote /tmp/slime-recipe-11858DnF.el
passed 60/184 readme-recipe-autoload-on-lisp-visit
=> 3 (2 bits, #x3, #o3, #b11)
passed 61/184 report-condition-with-circular-list-1
waiting for condition: top-level [21:15:46.975566]
waiting for condition: top-level [21:15:47.075842]
=> 3 (2 bits, #x3, #o3, #b11)
passed 62/184 report-condition-with-circular-list-2

When I run the tests using SBCL, it is very fast and I have backtrace 
info and and the cause of the error. Using the following:
cd ~/my-slime-sandbox/slime

make clean
make compile

make clean check
make check-fancy

I got the this partial output.

<snip>
Test autodoc-tests-23 backtrace:
signal(ert-test-failed (((should (equal "(error 'simple-condition &r
ert-fail(((should (equal "(error 'simple-condition &rest arguments &
#[nil "\306\307!\210\310 \210\311\312!▒q\210\313\216\314\315 \210
byte-code("\306\307\310\311\31D\313FE\314\211\315\211\316H
byte-code("\301\302!▒q\210\303\216\304\213\210+\305 \207" [temp-bu
ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
byte-code("\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216
ert-run-test([cl-struct-ert-test autodoc-tests-23 "Check autodoc wor
ert-run-or-rerun-test([cl-struct-ert--stats (tag contrib) [[cl-struc
ert-run-tests((tag contrib) #[(event-type &rest event-args) \306=\
ert-run-tests-batch((tag contrib))
slime-batch-test((tag contrib))
eval((slime-batch-test (quote (tag contrib))))
command-line-1(("-L" "." "-L" ".." "-L" "test" "--eval" "(require (q
command-line()
normal-top-level()
Test autodoc-tests-23 condition:
(ert-test-failed
((should
(equal "(error 'simple-condition &rest arguments &key format-arguments 
format-control)"
(slime-autodoc-to-string)))
:form
(equal "(error 'simple-condition &rest arguments &key format-arguments 
format-control)" "(error 'simple-condition &rest arguments &key 
(format-arguments 'nil) (format-control 'nil))")
:value nil :explanation
(arrays-of-different-length 78 92 "(error 'simple-condition &rest 
arguments &key format-arguments format-control)" "(error 
'simple-condition &rest arguments &key (format-arguments 'nil) 
(format-control 'nil))" first-mismatch-at 46)))
FAILED 16/112 autodoc-tests-23
Test autodoc-tests-24 backtrace:
signal(ert-test-failed (((should (equal "(cerror \"Foo\" 'simple-con
ert-fail(((should (equal "(cerror \"Foo\" 'simple-condition &rest ar
#[nil "\306\307!\210\310 \210\311\312!▒q\210\313\216\314\315 \210
byte-code("\306\307\310\311\31D\313FE\314\211\315\211\316H
byte-code("\301\302!▒q\210\303\216\304\213\210+\305 \207" [temp-bu
ert--run-test-internal([cl-struct-ert--test-execution-info [cl-struc
byte-code("\306\307!\211▒r\310\311!q\210\312 d\313\223)L\210\314\216
ert-run-test([cl-struct-ert-test autodoc-tests-24 "Check autodoc wor
ert-run-or-rerun-test([cl-struct-ert--stats (tag contrib) [[cl-struc
ert-run-tests((tag contrib) #[(event-type &rest event-args) \306=\
ert-run-tests-batch((tag contrib))
slime-batch-test((tag contrib))
eval((slime-batch-test (quote (tag contrib))))
command-line-1(("-L" "." "-L" ".." "-L" "test" "--eval" "(require (q
command-line()
normal-top-level()
Test autodoc-tests-24 condition:
(ert-test-failed
((should
(equal "(cerror \"Foo\" 'simple-condition &rest arguments &key 
format-arguments format-control)"
(slime-autodoc-to-string)))
:form
(equal "(cerror \"Foo\" 'simple-condition &rest arguments &key 
format-arguments format-control)" "(cerror \"Foo\" 'simple-condition 
&rest arguments &key (format-arguments 'nil) (format-control 'nil))")
:value nil :explanation
(arrays-of-different-length 85 99 "(cerror \"Foo\" 'simple-condition 
&rest arguments &key format-arguments format-control)" "(cerror \"Foo\" 
'simple-condition &rest arguments &key (format-arguments 'nil) 
(format-control 'nil))" first-mismatch-at 53)))
<snip>

I tried again with CCL to run the the tests, but it stopped at the same 
place and never returned. I finally had to close the konsole window to 
stop the test.

Paul Bowyer