[armedbear-devel] Fwd: ABCL packaging ASDF system in jars (was Re: Disabling "built-in" output translation)

[Mark Evenson]

-------- Original Message --------
Subject: ABCL packaging ASDF system in jars (was Re: Disabling 
"built-in" output translation)
Date: Tue, 17 May 2011 21:47:07 +0200
From: Mark Evenson <evenson-VmQCmMdMyN0AvxtiuMwx3w at public.gmane.org>
To: asdf-devel-F1HGIaG5STRyXAeb93iumQ at public.gmane.org
CC: Armed Bear <armedbear-devel-F1HGIaG5STRyXAeb93iumQ at public.gmane.org>
Newsgroups: gmane.lisp.asdf.devel
References: <iqtmt0$7ec$1 at dough.gmane.org> 
<BANLkTin8YBewU_=NnwOB53sxsn=rkqsOxA at mail.gmail.com>

On 5/17/11 3:28 PM, Faré wrote:
[…]
> Hope this helps.

I'm gonna need a bit more time to chew over the ASDF code, but thanks
for the general direction.  I must confess that in spite of the
reasonable looking documentation and having contributed the function
translation code to ASDF, its whole output translation API has never
really gelled in my understanding as a totality.  Maybe I can contribute
some examples to the texinfo file when the fuzz in my understanding
resolves a bit.

> BTW, for cl-launch and XCVB, I indeed am looking for a way to create
> bundles from compiled stuff. How do I create a jar from ABCL and a set
> of lisp files, precompiled or not?

A reasonable packaging mechanism for ABCL plus "compiled stuff" composed
of ASDF system definitions that works well in a Java ecosystem is
precisely the itch I'm in the process of scratching for ABCL right now.
   The base unit of packaging in Java is the jar file, which is nothing
more than a ZIP archive with (possibly) some Java-the-language specific
metadata.  Currently, one can package an ASDF definition in a jar file,
push the location of the asd file into ASDF:*CENTRAL-REGISTRY* using the
[ABCL JAR-PATHNAME conventions][1] for which a subsequent
ASDF:LOAD-SYSTEM will do the right thing.

[1]:
http://trac.common-lisp.net/armedbear/browser/trunk/abcl/doc/design/pathnames/jar-pathnames.markdown

For instance, if you were to package up cl-ppcre installed via
[quicklisp][10^9monkey-wants-this] in a jar file via

    unix$ cd ~/quicklisp/dists/quicklisp/software && jar cfv
/tmp/cl-ppcre-2.0.3.jar cl-ppcre-2.0.3

[10^9monkey-wants-this]: http://www.quicklisp.org

one could subsequently enable an ASDF controlled load of this system via

    CL-USER> (push "jar:file:/tmp/cl-ppcre-2.0.3.jar!/cl-ppcre-2.0.3/"
asdf:*central-registry)

When loaded, ASDF will compile the system to the user cache.

Some problems here that I'm working through:

1)  You can't immediately load FASL out of the jar.  In my current
hackish way—i.e. without comprehending your advice yet—one has first
ASDF compile the system with output translations disabled, and then

   (defmethod asdf:perform ((o asdf:compile-op) (c asdf:cl-source-file)))
   (setf (asdf::output-translations) '((t t))))

in the target JVM to load the ABCL FASLS from the jar.

2) One has to specify the absolute path on the local filesystem (or
potentially via a URI) for the jar, which makes things a bit fragile in
the typical Java ecosystem usage which shuffles jars around like win32
DLLs (or, to be fair, Unix dynamic libraries) relying merely on the
filename to keep things straight.  My current insight into a way around
would be to define another PATHNAME extension in ABCL for CLASSPATH
entries, i.e. "classpath:cl-ppcre-2.0.3.jar!/cl-ppcre-2.0.3/" would
refer to the named jar in the JVM CLASSPATH if it exists.

3)  ABCL [has a bug][#149] that currently prevents ASDF systems located
in the top-level entry of a JAR from being accessed.

[#149]: http://trac.common-lisp.net/armedbear/ticket/149

4)  The extremely nice use of [JSS][jss] and [ABCLD's slight
modification][abcld] of ASDF to also refer to jar files to dynamically
load into the JVM probably needs to be rewritten, otherwise we run into
the situation whereby we have jars (i.e. the packaged Java code) within
the ASDF packaged jar which A) needs changes within ABCL to completely
work and B) would be rather inefficient in that the naive implementation
   each request for a new entry in the JAR within a JAR would require a
complete "reseek" through the enclosed ZIP file.

[jss]: http://code.google.com/p/lsw2/source/browse/#svn%2Ftrunk%2Fjss
[abcld]:
http://code.google.com/p/abcl-dynamic-install/source/browse?repo=abcld

5)  A fear of mine: if I enable all this, I presume that people would
start going around creating 'abcl.jar' files with different inclusions
of different ASDF packagings.  Without a real smart dynamic
introspection system that essentially solves the problems in JVM's
classpath we would end up with, at the minimum, a rather hostile
situation for the end user.  "Put 'abcl.jar' in your classpath."  "I
did, and it still didn't load Maxima."  "Well, what's the checksum of
your abcl.jar?" "c48d359a23ee"  "Oh, you need 846f78c279cb".  Ideally, I
would like to come up with a mechanism that would require that
'abcl.jar' come from "official" ABCL packaging, but would somehow be
able to introspect the JVM classpath to include ASDF definitions.

Comments solicited.

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."


_______________________________________________
asdf-devel mailing list
asdf-devel at common-lisp.net
http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel