[armedbear-devel] Less-reflection branch: profiling and comparison with trunk

[Blake McBride]

Dear Alessio,

It sounds like you are doing some great stuff.  However, I wanted to
be vocal about an issue that is critically important to me (for
whatever that's worth).  Erik is aware of this so perhaps there is no
potential issue.

It is critically important for my use of ABCL that data _and_ code be
GC'able by _Java_.  So, for example, I need to be able to load an
infinite number of Lisp source files in a fixed amount of memory (so
long as not all the functions are being referenced at the same time).
This way I can load a Lisp source file, execute code, de-reference the
entire package making the entire code module GC'able.

I haven't completely tested this but I've been in communication with
Eric and I believe this is possible now.

I don't care about GC'ing individual functions necessarily, however,
being able to completely GC either a source module or a package is
important.

Thanks.

Blake McBride


On Mon, May 17, 2010 at 11:28 AM, Alessio Stalla
<alessiostalla at gmail.com> wrote:
> Hello,
>
> as I have already informally discussed with Erik and David Miles, I
> have been working on a branch which changes how functions are compiled
> in FASLs (and by consequence how they are loaded from FASLs). The
> purpose was to reduce the use of reflection by using a single
> classloader per FASL (as opposed to one per function) plus a master
> "loader" function that will initialize all other function classes
> using the new operator, rather than passing through reflection. Now
> such loading is done eagerly, as soon as the FASL is loaded; but with
> a bit more work, I should be able to restore autoloading semantics, so
> that a given function object is only instantiated the first time it is
> called (but the class representing it will always be loaded eagerly).
> I suspect it won't change much, though.
>
> The overall performance results (reported below) are not very
> encouraging. However, I do think that the branch has a better design
> than our current approach, and goes towards the objective of having a
> per-fasl object holding various things (e.g. a FASL constant pool). In
> the long run, not having functions rely on loadCompiledFunction in
> their static initialization code is a nice step towards serializable
> closures, if we'll ever want to have them.
>
> The results on load times and overall runtime speed have been
> insignificant. Apparently by large the most important factor in our
> load time is I/O and the branch doesn't change anything about that. It
> just saves ~1% of the loading time spent on reflection.
>
> The results on memory consumption are mixed: they do show a great
> decrease in the number of classloaders, as expected, but the numbers
> are not tremendously life-changing: 56 objects (live and allocated,
> for a total of 4032B, 0.1% of total live objects) vs 789 live objects
> of 1014 allocated, 44.184B, 1.1% of total live objects.
> There are also less reflection-related objects, but more Lisp objects,
> presumably due to eager loading. The numbers for those from a quick
> glance more or less balance out with each other.
>
> Doing these profiles showed that a lot of our memory is actually made
> of the stored byte[] containing class data for compiled functions.
> Those amount to ~4M on trunk and almost 8M on my branch (probably due
> the eager loading of compiled functions). Given that I now load all
> functions with a classloader that knows where functions have been
> loaded from, I think I should not store the byte[] in the function at
> all, but rely on the fasl classloader to re-extract them from the FASL
> on demand. However, I haven't still implemented this.
>
> For those who are interested, I'm attaching the memory profiles done
> with NetBeans. I left out the CPU profiles because they're fatter
> (~6M) and I'm on a slow connection now; but I can send them
> separately.
>
> Cheers,
> Alessio
>
> _______________________________________________
> armedbear-devel mailing list
> armedbear-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel
>
>