Upgrading/Installation Instructions Clarification

[Eric Timmons]

"Robert Goldman" <rpgoldman at sift.info> writes:

> This is true, but it causes just the kind of problems I have alluded to 
> -- your ASDF configuration is now smeared all over your system, and 
> debugging it becomes essentially impossible.

You're right that understanding what the source registry is doing is
extremely difficult right now. (And I also find the twisty turny
passages of the code difficult to follow). I started
https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/156 to try and
make it better.

The source registry computation works in two phases: the DSL is
flattened into a list of directories to search, and then the directories
are actually searched. There's a ton more info on the first commit to
!156, but now provenance info is recorded in the flattening
step. Recording it in the second step should also be trivial once we
decide where to record it (*source-registry* or another variable).

> Also not a fan of invisible config files.  Yeah -- it's ok to have dot 
> files in your home dir so that you don't get overwhelmed, but why is it 
> a good idea to hide the file in your repo that configures the CI so that 
> `ls` won't show it to you?

I personally lean toward hiding the files that are kind of background
infrastructure and won't be edited all that frequently. But I also now
not everyone does (obviously!). Perhaps one thing we can do is change
ASDF to use cl-source-registry.cache if it exists, falling back to
.cl-source-registry.cache if it doesn't? (Also mentioned this on !156)

> Note also that if ASDF isn't configured in your lisp init file, but by a 
> config file or environment variable, *it will be configured as it 
> loads*.  This means that all of your debugging tools are taken away from 
> you.  There will be no tracing, because by the time you can set up a 
> trace, the damage will be done.

I don't think this is true. I thought ASDF configured the source
registry on the first invocation of find-system? So you can set up
traces before that happens. But that may not be useful since the first
thing most people do when starting Lisp is load swank which ends up
initializing ASDF (on SBCL at least) through a REQUIRE.

If you wanted to trace or otherwise debug a catastrophically broken
source registry configuration using SLIME, I think adding a
(asdf:initialize-source-registry '(:source-registry :ignore-inherited-configuration))
to your init file would let you get SLIME up and running. Then you can
trace the config like normal by evaluating
(asdf:initialize-source-registry nil)

And now for my turn on the soap box:

While configuring only through an init file may help localize the
configuration from the perspective of a single developer, I think the
system level fallbacks are useful to localize the configuration from the
perspective of an integrator for a loosely coupled team. I manage
multiple development Docker images for my group and have routinely used
the system level source registry config to ensure that some of our
development tools are available. This is nice because I don't have to
badger N people to update their init files if I need to tweak something
and they can mount their init files directly from their Docker host into
multiple containers (with varying tools prebundled in each) and know
that things will Just Work.

On the other end of the spectrum, I've found a use for
CL_SOURCE_REGISTRY that I use to hyper-localize my configuration. Unless
you jump through hoops, your Lisp init file is generally user wide
config. I've been developing CLPM <https://www.clpm.dev> that configures
ASDF on a per-project basis. One of the features it provides is a Ruby
Bundler-like exec feature where you can run a command (say in CI or your
terminal) and know that any instance of ASDF in that environment will
see only the systems you've specified for that
project[1]. CL_SOURCE_REGISTRY is the only way I can think of doing this
that works seamlessly across all Lisps no matter how they're invoked.

-Eric

[1] plus the systems that are forced upon you by WRAPPING-SOURCE-REGISTRY.