[Asdf-devel] startup times and initialize-source-registry

[Faré]

On Mon, Aug 25, 2014 at 12:28 PM, Robert P. Goldman <rpgoldman at sift.info> wrote:
>> Once again, where under a hierarchy the .asd files are is ultimately
>> the knowledge and responsibility of the curators of the respective
>> source trees, not of the end-user. Therefore, the absence of recursion
>> should be a matter of said source trees including a file
>> cl-source-registry.conf or .cl-source-registry.conf (visible having
>> priority over hidden), that specifies how (not) to recurse in that
>> tree. A simple script could be provided for system writers to create
>> or update said file. A year after all relevant packages have migrated,
>> we can make it the default and people can drop said configuration file
>> when they follow the default behavior of not having meaningful .asd
>> files except directly under the top directory. Furthermore, with a
>> :file directive, these configuration files can directly list the .asd
>> files without any further filesystem access.
>
> I don't think that this new behavior should ever be the default.
> Scripting is the edge-case for CL, not building of large, complex
> systems.  Further, for some CL implementations (like ABCL, as Mark
> points out), scripting use is *never* a good option.
>
Well, we'll see. But faster startup times benefit everyone, not just
people using Lisp for scripting. Shaving a second or so from startup
time (somewhat less or much more depending on the implementation and
the size of your source trees)  is useful whether you're (re)starting
SLIME, running tests in a batch job, building in a slave or
cross-compiling as well as if you're scripting.

> >From a cost tradeoff PoV, there are few scripting configurations now,
> and fixing them all is easy and cheap, since they are for
> early-adopters. Fixing all systems that contain nested system
> definitions is neither easy nor cheap.
>
That's why I proposed a two year migration plan, of the same kind we
did for unicode support.

I believe it may be worth it.

On the other hand, if we support this search for
cl-source-registry.conf while recursing, then whichever trees are
managed (by e.g. quicklisp or debian or clbuild) can have this
cl-source-registry.conf automatically optimized by the same manager,
so only unmanaged source code downloaded by the user remains slow, and
this should be small, except for power users who are big boys enough
to do the system administration step of regenerating that file when
they update their repositories.

> When one wants scripting, it should be easy to specify a "scripting
> lisp." For now, I suggest that people who want to script with CL should
> build themselves a pre-configured image for the purpose.  That image
> could have a feature or ASDF configuration variable set so to change the
> default behavior to cut off recursive ASDF search.  Configuration files
> in source trees would complement this behavior.
>
Yes, you can already cut a lot of the time by dumping an image.

> That approach would serve the purpose of making this behavior easy to
> specify at the command-line or mouse-click.  Later, if CL-based
> scripting catches on, lisp implementations could ship with versions that
> are intended for rapid start-up and scripting, avoiding the need for
> scripters to build their own images.  Or separate scripting packages
> could be provided.
>
The problem is that it's not a simple matter of a scripting lisp vs a
non-scripting lisp, since many libraries (and/or managed set of
libraries) needs to be modified to keep working in this scheme for
(faster) source-registry initialization.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
To stay young requires unceasing cultivation of the ability to
unlearn old falsehoods. — Robert Heinlein, "Time Enough For Love"