Monolithic fasls (Was Re: [Armedbear-devel] How to load compiled Lisp stuff from a jar)

Tue Dec 2 17:07:07 UTC 2014

Mark Evenson wrote:
> On 11/28/14 21:16, Robert P. Goldman wrote:
> […]
>> I'm a little confused: doesn't ABCL use ASDF-JAR *instead* of Fare's
>> monolithic-<foo> operations? So are you looking to harmonize the two in
>> some way?
> 
> We attempt to support both methods, and are not currently looking to
> harmonize.  ASDF-JAR originated from the JVM host for ABCL:  since the
> jar file is the unit of distribution for most Java applications, it
> seemed natural to be able to distribute ABCL applications in such units.
>  Monolithic fasls came from ASDF's support for ECL which Erik saw as
> easy to support with a small amount of additional work on ABCL.
> 
>> I don't claim to understand [monolithic fasls].  Faré put them in before I took over,
>> I believe mostly for the benefit of ECL. My attempts to use them with
>> ABCL (as you know) have always failed for me: on the Mac the jar
>> pathnames get garbled in ways I don't understand.  We had some
>> correspondence about this, but I don't think either of us were able to
>> figure it out.
> 
> Hmm, I can't find the record of our correspondence.  From what I
> remember, I eventually traced the error you encountered down to OS X's
> use of a symlink for the temporary directory root, and fixed ABCL.  A
> fair amount of wall clock time elapsed (on the order of months) before I
> managed to figure out what was wrong, so I think ASDF has dropped the
> failing test, as you had, understandably, moved on.

I just checked test-bundle, and it still fails for me on Mac OS X:

TEST ABORTED: Loadable FASL not found for
'#P"/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl"'
in
'#P"jar:file:/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl!/bundle-1--system._"'

Here's what looks like the interesting part of the backtrace:

16: (SIGNAL #<FILE-ERROR {447EAE5A}>)
17: #<JAVA-STACK-FRAME org.armedbear.lisp.Lisp.error(Lisp.java:382)
{32BD8B9C}>
18: #<JAVA-STACK-FRAME org.armedbear.lisp.Load.load(Load.java:192)
{3A18C8CA}>
19: #<JAVA-STACK-FRAME org.armedbear.lisp.Load.load(Load.java:726)
{31266392}>
20: #<JAVA-STACK-FRAME
org.armedbear.lisp.Load$_load.execute(Load.java:674) {4F531F73}>
21: #<JAVA-STACK-FRAME
org.armedbear.lisp.Symbol.execute(Symbol.java:852) {5A35DFB4}>
22: #<JAVA-STACK-FRAME
org.armedbear.lisp.LispThread.execute(LispThread.java:698) {6F1D0B1}>
23: #<JAVA-STACK-FRAME org.armedbear.lisp.load_1.execute(load.lisp:33)
{6DBC1806}>
24: #<JAVA-STACK-FRAME
org.armedbear.lisp.CompiledClosure.execute(CompiledClosure.java:98)
{57CD6A32}>
25: #<JAVA-STACK-FRAME
org.armedbear.lisp.Symbol.execute(Symbol.java:803) {6EF725A6}>
26: #<JAVA-STACK-FRAME
org.armedbear.lisp.LispThread.execute(LispThread.java:644) {23C8EE34}>
27: #<JAVA-STACK-FRAME org.armedbear.lisp.Lisp.funcall(Lisp.java:172)
{5C8EAA46}>
28: #<JAVA-STACK-FRAME
org.armedbear.lisp.Primitives$pf_apply.execute(Primitives.java:2825)
{1A4D0923}>
29: (SYSTEM::%LOAD
#P"/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl"
T NIL T :DEFAULT)
30: (LOAD
#P"/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl")
31: (APPLY LOAD
#P"/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl"
NIL)
32: (#<FUNCTION {3792E652}>)
33: (#<FUNCTION {D4F2DFF}>)
34: (JAVA:JRUN-EXCEPTION-PROTECTED #<FUNCTION {D4F2DFF}>)
35: (UIOP/UTILITY:CALL-WITH-MUFFLED-CONDITIONS #<FUNCTION {3792E652}>
("Overwriting already existing readtable ~S." #(#:FINALIZERS-OFF-WARNING
:ASDF-FINALIZERS)))
36: (UIOP/LISP-BUILD:CALL-WITH-MUFFLED-LOADER-CONDITIONS #<FUNCTION
{3792E652}>)
37: (UIOP/LISP-BUILD:LOAD*
#P"/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl")
38: (ASDF/LISP-ACTION:PERFORM-LISP-LOAD-FASL
#<ASDF/BUNDLE:LOAD-BUNDLE-OP > #<ASDF/SYSTEM:SYSTEM "test-asdf/bundle-1">)

When I look into that bundle file, I see this:
$ jar tf
/Users/rpg/lisp/asdf/build/fasls/abcl-1.2.1-fasl42-macosx-x64/asdf/test/test-asdf/bundle-1--system.abcl
file3/
file3/__loader__._
file1/
file1/__loader__._

> 
>> As far as symbol obsolescence, what's going on is that we decided the
>> original names weren't very clear, so the operations got renamed.
>> Unfortunately, there had already been a release, so instead of just
>> renaming them, we kept both names, with the old names left around and
>> deprecated.  I'll probably blow them away whenever ASDF 3.2 rolls around.
> 
> There should be some documentation for monolithic fasls, somewhere, at
> some point, which, once I figure things out, I will be happy to
> contribute.
> 
> Do monolithic FASLs work on MKCL, since ECL is effectively dead?  If
> not, then we should probably document things in the ABCL manual.  But
> then that approach is a little odd, as the majority of the code resides
> in ASDF.  But it seems a bit out of place to document something in the
> ASDF manual that only applies to one, and possibly two, Common Lisp
> implementations.

As far as I know, the bundle operations run on MKCL, at least to the
extent that it passes all the tests.

I think actually the bundle operations mostly work everywhere, it's just
that I'm not sure what they're for!

That part of this discussion should probably get moved to ASDF-devel,
where Faré can weigh in.