[armedbear-devel] Some thoughts on classloaders, inner classes and in-memory compilation

[Alessio Stalla]

On Tue, Sep 22, 2009 at 9:15 PM, Ville Voutilainen
<ville.voutilainen at gmail.com> wrote:
> 2009/9/22 Mark Evenson <evenson at panix.com>:
>> Great!  I think experimentation with the codebase is a great way to
>> proceed, but hopefully you aren't deterred from posting your musings
>> here.  I, for one, really appreciate both!
>
> Me too, I'm looking forward to what this experiment leads to.

I did the experiment, more or less the way I described. I haven't
touched the init-fasl code, which still uses load-compiled-function;
however loadCompiledFunction itself has been changed to use 2 new
classloaders, one for .cls files and one for entries in jar files (it
contemplates other 2 cases, zipped .abcl files inside jars and a
fallback case which I don't know when it is used, I left them out).
The compiler has been changed only in one part, to stop using
loadCompiledFunction and directly instantiate the class.

>From preliminary testing it appears to work well. It is not optimized
at all (a different classloader is created for every function to load,
even if one could be reused), yet it is only slightly slower than
before. On my machine compiling abcl takes a few seconds less than 2
minutes, with the changes it takes a few seconds more than 2 minutes.
Code complexity imho is not particularly changed, as I basically just
moved some code from loadCompiledFunction to the classloaders. The
classloaders themselves are not complex, and they could be simplified
by moving common things in a base superclass. This design is more
modular than before, but not much, because everything is still
controlled in one central place, loadCompiledFunction.

A couple of probably uninfluential limitations:
- compiled classes are always in the org.armedbear.lisp package. In
jar files they appear to be placed inside org/armedbear/lisp, while
normal files are not, so the file classloader always removes
org.armedbear.lisp. from the class name to obtain the file name.
- compiled class files are always assumed by the classloader to end in .cls
- some name mangling is necessary on class names and file names as not
every lisp symbol is a valid Java class name. I coded the minimal
amount of mangling necessary for abcl to compile itself, but there are
probably some awkward symbols that can break it.

So, I think I can try to extend my experiment to memory compilation.

> I wonder
> why abcl does the loadCompiledFunction emission, is it just a remnant
> of an old design or is there some clever idea behind it?

Who knows? Classloaders are a bit tricky, maybe the initial design
avoided them for simplicity.

Bye,
Alessio