Need to call asdf:load-system (or equivalent) from asdf:test-system

[Robert Goldman]

On 28 Dec 2021, at 21:51, Albus Matt Piroglu wrote:

> Note I have opened an issue on gitlab, but I know it's an overkill to 
> open
> issues for questions. I couldn't find an appropriate channel until I 
> found
> this mailing list. I'll close the issue, and here is my question for 
> your
> consideration:
>
>
> Hi, I'm trying to test a functionality that depends on loading 
> systems.
> Thusly I ended up writing some tests that need to load a "test-system"
> during the tests, and I clear the system afterwards.
>
> A simplified code for the test in question is:
>
> (test check-the-system-is-loaded-as-expected ()  (asdf:load-system
> "my-test-system-under-test-folder")  (is --check that the system
> loaded as expected--)  (asdf:clear-system
> "my-test-system-under-test-folder")  (delete-package
> "PACKAGE-FROM-TEST-SYSTEM"))
>
> and, to clarify, another test relies on the same system not being 
> loaded,
> so I don't think a dependency would solve the issue, it is rather a 
> dynamic
> decision to load or not.
>
> Now, I know that this feature is deprecated and may not work. What I'm
> wondering if there's a safer way to achieve this. Wouldn't some tests 
> for
> asdf itself need loading some test systems for example? (hmm it 
> actually
> might be a good idea to go check them now). OK, for example there's a
> hello-world-example.asd in test, which looks similar to my
> my-test-system-under-test-folder.
>
> So, instead of asdf:load-system in the above test code, should I be 
> using a
> lower-level operation call?

I'm not entirely sure what you mean by "test" in the above. If you look 
at the tests in ASDF (in the test/ subdirectory), you will see that 
these are typically run from the shell, and so don't have any lasting 
effects.  Lisp starts, the test is run, lisp exits. So typically we 
don't have to worry about clearing systems.

If you want to understand better how ASDF does its testing, a good 
resource is to read the file test/script-support.lisp which has the 
testing utilities. (If you would like to submit a merge request with 
additional comments and/or docstrings for this file, that would be 
welcome!)

You will see that this file contains a def-test-system macro 
(unfortunately, without a docstring) that can be used in tests to create 
a system without needing an asd file.  See, for example, 
test/test-sysdef-asdf.script for uses of def-test-system

This file also has an example of the use of CLEAR-SYSTEM, which is 
exported by ASDF and is defined as follows:

>     "Clear the entry for a SYSTEM in the database of systems 
> previously defined. However if the system was registered as PRELOADED 
> (which it is if it is IMMUTABLE), then a new system with the same name 
> will be defined and registered in its place from which build details 
> will have been cleared.
> Note that this does NOT in any way cause any of the code of the system 
> to be unloaded.
> Returns T if system was or is now undefined, NIL if a new preloaded 
> system was redefined."

Note from the above that you cannot unload a system.  Loading a system 
has arbitrary effects on the running lisp image, and those effects 
cannot, in general, be undone.  This is part of the reason that each 
ASDF test is run in a separate lisp process.

As for testing whether or not a system has been loaded, typically what 
is done is to define a package and a variable in that package in the 
test system, and then check for that variable having the expected value 
in order to see if the system has been loaded.