[armedbear-devel] Some thoughts on classloaders, inner classes and in-memory compilation

[Alessio Stalla]

On Tue, Sep 22, 2009 at 3:13 PM, Mark Evenson <evenson at panix.com> wrote:
> There is a little more to the explanation of how ABCL loads its FASLs
> that should be included for completeness, namely the use of the
> so-called init-fasl Lisp code.  For every compilation unit ("packed
> FASL") there is a section of Lisp code that is read in by the Java
> implementation of LOAD that contains forms like:
>
> (SYSTEM:INIT-FASL :VERSION 32)
> (SETQ SYSTEM:*SOURCE* #P"/Users/evenson/work/abcl/dir.lisp")
> (SYSTEM:%IN-PACKAGE "CL-USER")
> (DEFPARAMETER *TEST-PATHNAME* *LOAD-TRUENAME*)
> (SYSTEM:FSET 'RUN (SYSTEM:LOAD-COMPILED-FUNCTION "dir-1.cls") 71 'NIL NIL)
> (SYSTEM:FSET 'RUN-2 (SYSTEM:LOAD-COMPILED-FUNCTION "dir-2.cls") 485 'NIL
> NIL)
>
> Special variables–like *TEST-PATHNAME* here–are declared, the proper
> IN-PACKAGE forms are emitted, and then forms at the end issues the
> loadCompiledFunction() calls to load all of the top-level compilands
> created by compiling "dir.lisp" which are then FSET to the proper
> functions.  It is classes like "dir-1.cls" that may contain further
> static initializers
>
> Any new load mechanism consisting purely of Java code would have to do
> the equivalent Java side presumably in a static initializer of some
> kind.  The current implementation rather simplifies what sort of Java
> classes can exist, namely that they can only represent functions.  I
> suppose the static initializer might be able to feed a string containing
> these forms to EVAL.
>
> Alessio's musings here are interesting, but I think there is a simpler
> solution to the problem of removing the need for intermediate temporary
> files on the filesystem by changing the behavior of
> loadCompiledFunction() to look for some sort of special variable (like
> *LOAD-TRUENAME*?) to figure out where to look for the bytes to turn a
> into a class that can be loaded by the JVM.  By "simpler solution" I
> mean something that can be worked out without drastic impact on the
> current codebase.

I'd like to point out that I'm not advocating a pure Java solution:
the Lisp init-fasl code is perfectly fine and I don't think it should
be changed. As for impact on the codebase, I might agree with you; I
still need to closely look at the code to really understand the
implications of what I'm suggesting. The compiler is a complex piece,
maybe the most complex of abcl, and I know and understand only a tiny
part of it.
I think that, as a personal experiment, I'll start changing the
implementation of LOAD-COMPILED-FUNCTION to just do something like
(pseudocode) new
'sys::*compiler-classloader-class*'().loadClass("dir-1.cls"), move the
current loadCompiledFunction to, say,
AbclSystemFileClassLoader.findClass(), and change the compiler to stop
emitting loadCompiledFunction calls and refer to the class directly
instead. That should preserve the current behavior yet demonstrate my
solution. If it turns out to be working and actually simplifying
things, I will report back and, as a next step, go on and try to add
the ability to work in memory.

Bye,
Ale