[armedbear-devel] Compiling large Java-classes slow?

[Erik Huelsmann]

Hi Ole,

Thank you for the additional information. Can you tell me which version you
were using before and which exact version you're using now? Did something
change in your environment?

I'm using ABCL as a JSR-223 scripting language for the Electric VLSI system.
> To access the Electric's Java-classes I have a hacked-up version of JFLI,
> conveniently allowing me to say eg
>
> (def-java-class "com.sun.electric.database.network.Netlist")
>
> to get wrappers for the Java-members of that class. The JFLI-magic
> obviously happens by introspection (JCLASS-METHODS), basically generating a
> ton of (lazy) member-wrapping functions.
>
> As I watch the embedded ABCL compile a file #p"tool.lisp" of such
> DEF-JAVA-CLASS statements, it immediately spits out hundreds of temporary
> cls-files and a abcl-file,
>
> -rw-r--r--  1 mrohne  staff    1790 Aug 24 08:04 ASDF_TMP_tool_9.cls
>
> -rw-r--r--  1 mrohne  staff    1800 Aug 24 08:04 ASDF_TMP_tool_8.cls
>
> -rw-r--r--  1 mrohne  staff    1787 Aug 24 08:04 ASDF_TMP_tool_7.cls
>
> (...)
>
> -rw-r--r--  1 mrohne  staff    1254 Aug 24 08:04 ASDF_TMP_tool_244.cls
>
> -rw-r--r--  1 mrohne  staff    1359 Aug 24 08:04 ASDF_TMP_tool_243.cls
>
> -rw-r--r--  1 mrohne  staff    1248 Aug 24 08:04 ASDF_TMP_tool_242.cls
>
> -rw-r--r--  1 mrohne  staff    1362 Aug 24 08:04 ASDF_TMP_tool_241.cls
>
> -rw-r--r--  1 mrohne  staff  108500 Aug 24 08:04 ASDF-TMP-tool.abcl
>
> then goes on chewing on those files for quite a while. Eventually it
> removes the cls-files and (I assume) renames ASDF-TMP-tool.abcl to
> tool.abcl. The chewing-time grows non-linearly with the number of cls-files,
> which in turn seems to depend on the "size" of the Java-class being wrapped,
> hence my confusing subject-line.
>

My personal experience with classes being compiled -and especially with
slowness of it- is that it can be terribly slow over network connections,
especially if the network is a busy one.

Last week I added a change which aggrevated the situation, but that
aggrevation has again been aleviated a bit this week. There was no change
which would cause the numbers to be as bad as the ones you mention.



> A few hundred cls-files finishes in minutes, while more than a thousand
> easily takes the full morning. A few months back ABCL behaved quite
> differently, compiling many tens of  JFLI-imported classes in a few minutes,
> so I'm wondering what might have changed?
>


The exponential component in the growth suggests either a very busy network
or huge numbers of files in a single directory - with a filesystem not
configured to handle it. Eg. the linux filesystems ext2 and ext3 explicitly
need "directory indexing" enabled to perform well in situations with "many
thousands" of files in a single directory.

246 files should not affect performance, however and would suggest a problem
in the network, as I indicated before.

Still: there are solutions which I want to look at which would reduce our
dependency on performance of network I/O specifically and maybe even I/O
systems in general.

Bye,

Erik.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20110824/85ed6722/attachment.html>