[armedbear-devel] Autoloader progress and experiences

[Erik Huelsmann]

Hi all,

Although I wanted to work on ticket #201 (&whole broken in
DEFINE-METHOD-COMBINATION), I found a regression (ticket #227) which I
believe is strongly correlated with how our process of auto loading works
(ticket #226).

So, I've done a bit of work, some of which already committed, to extract
the symbols of toplevel defined functions/methods and macros from our
bootstrap build. I'm now working to use that list to create a more complete
"autoloads.lisp" file.

During that work I found at least one cause of instability regarding our
autoloading and the definition of generic functions: we were resolving
(autoloading) symbols with an autoloader in the function slot when defining
a generic function on them. Usually however, the reason to be defining a
generic function on a symbol with an autoloader in the function slot will
be the fact that autoloading has been triggered.
The only reason this worked in the past is that our list of autoloaded
symbols was so small we haven't run into this situation. Should that ever
have happened, that'd have been very hard to find though.

Another thing I found is that MACROEXPAND-1 (and by extension of its use in
PRECOMPILE, the precompilation) try to resolve symbols with autoloaders on
them. That's a problem as well, because we have forward-referenced
functions: e.g. we forward-reference SLOT-VALUE-USING-CLASS from
SLOT-VALUE. When clos.lisp is being loaded and SLOT-VALUE is being defined
using DEFUN, it's being "precompiled": translated to an efficient execution
form.
However, that process finds a reference to SLOT-VALUE-USING-CLASS, which it
will autoload. This recursively triggers loading of clos.lisp.

Do we still want to do precompilation? Yes, I think so, because on my
machine usual build times are ~ 6 minutes for a full build; this time goes
up to ~16 minutes without precompilation. So, the process does what it's
designed to do well.


The answer then - I think - is to stop precompiling during function
definition and delay that until the first actual invocation.

---- breaking news:

While writing the above, I came up with another option, which is to make
MACROEXPAND-1 only resolve symbols which are macro functions. That worked,
or at least, that's what it looks like. I had a successful build and will
run the compiled tests in a minute.

As things stand, we'd be able to start generating most of our
autoloads.lisp file.

[even later: seems the test suite is completing now. I'm having a PSETF
regression, I think, but no idea why yet.]
-----

So, I think I'll work out the remainder of the change and make sure it gets
committed. After that, I'll have a look at the PSETF failure - which is
hopefully fall-out from another change or at least has a workaround in a
different way: having automated autoloaders seems a valuable asset.


Bye,

Erik.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20120731/957ccbdc/attachment.html>