[Asdf-devel] startup times and initialize-source-registry

[Faré]

On Thu, Aug 21, 2014 at 2:38 AM, Mark Evenson <evenson at panix.com> wrote:
>
> On 21 Aug 2014, at 02:36, Faré <fahree at gmail.com> wrote:
>
> […]
>
>> The trick here is in this new stop-at-asd flag, which here defaults to
>> t and isn't configurable, but which should default to nil and be
>> configurable, for backward compatibility. Its effect is that recursing
>> into subdirectories stops if a .asd file is found in the toplevel
>> directory. This saves a lot of recursing, and would save even more if
>> a .asd file of symlink to one exists at the top of a git hierarchy.
>> But this is incompatible with a lot of existing code, and so the
>> transition will be long and painful if this is adopted.
>
> If you proposing that the stop-at-asd property would be somehow configurable in
> the DEFSYSTEM form, like:
>
>         (asdf:defsystem :foo
>            :contains-interior-asdf-defintions
>            :components …
>
> then please ensure that this is present when/if you introduce this change to
> ASDF.  But I get the feeling that in order to speed things up, you weren’t
> intending to parse the DEFSYSTEM form in your search.
>
Indeed, requiring to parse a .asd file is a bad idea — and is even
worse when there are hundreds of .asd files in the directory.

But maybe we could detect a file called source-registry.conf or
something similar, and parse that to look for subdirectories with .asd
files in them. In the absence of such a file, the default behavior
would for backward compatibility be to always recurse, or maybe for
speed in a future version years from now be to recurse only if no .asd
file was found.

> If you are proposing that the user would have to do explicitly do some sort of
> configuration “for this instance of a user using asdf with this Lisp
> implementation”, I won’t be so happy because:
>
> 1)  This sort of configuration hasn’t been necessary before, so we will
> introduce complexity in ASDF deployment for efficiency in using CL as a
> scripting language which is something I don’t currently use (Admittedly because
> my platform, ABCL, based on the JVM, is just not going to ever have reasonable
> startup times.  Although there are systems that keep a JVM “warmed up” for
> firing such one-off commands to, and for specialised JVM there are memory
> mapped solutions for faster startup).
>
> 2)  I am using a system (lsw2) not in Quicklisp that has many such “interior”
> ASDF definitions. Usually when systems get in this state it is because they are
> big enough that nobody has time to package them correctly, so they tend to stay
> that way.  If I can’t put a flag in the top-level system, I’m going to run into
> problems when users haven’t done the per-user-per-lisp configuration.
>
What about supporting a source-registry.conf or .asdf-search.conf or
similar file to control recursion of the search, and eventually
requiring it when a directory both has a .asd file yet requires
recursion?

ASDF should do the right thing and require no configuration from
users, but minimal configuration of their own directory structures by
programmers is acceptable — except that for backward compatibility, it
should default to always recurse for now.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
The problem with Unix ever becoming a widely popular desktop operating system
is referred to as the 'guru in the box' problem. To get and keep Unix running
smoothly you need a captive guru on site and there just aren't enough gurus
to put in the shipping boxes.
        — Brian Kernighan