Finding abcl-contrib packages inside a JAR: approach and suggestions

[Mark Evenson]

On 4/29/17 21:48, Frederico Munoz wrote:
> Hello,
> 
> I'm almost done with my small experiment:
> 
> * I'm loading lisp code via a Java file using Load.loadSystemFile in an
>   init method
> 
> * I'm adding the file file to *lisp-home* via gradle
> 
> * I'm using gradle tasks to create a single JAR with abcl and
>   abcl-contrib added (not as JAR files)
> 
> Note that I need to have only one jar file and with specific
> requirements around the main method; the above allows me to have it
> working fine, a single  JAR which is essentially standalone.

If you could detail the requirements you have around the main method a
little bit, that would help understand more exactly where you are
heading.  From reading between the lines based on your [comments around
your Watson IoT example][1], I *think* you are trying to make a WAR that
will deploy to Jetty, so I am going to take an educated guess that that
is what you are trying to create.

[1]:
https://developer.ibm.com/recipes/tutorials/watson-iot-with-common-lisp/

A couple pointers on making WAR with ABCL:

1) [abcl-servlet][2] provides infrastructure for making an example WAR
archive which has ABCL-CONTRIB available.  The WAR only works for Java
Servlet containers that provide an "exploded WAR" deployment strategy,
i.e. at deployment the WAR archive is unzipped onto the local filesystem.

[2]: https://bitbucket.org/easye/abcl-servlet

2) Mucking with Load.loadSystemFile() and *LISP-HOME* should not be
necessary, as these are really meant to be internal interfaces to boot
the Common Lisp environment.  Of course, you are welcome to experiment,
but it would be best not to rely on these interfaces if at all possible.
The values for the logical pathname SYS *should* be the "right way"
forward to identify what *LISP-HOME* points to.  Logical pathnames are
better as they easily allow us to "update" the directory with the
runtime value without having to explicitly re-merge the whole slew of
PATHNAME objects derived from the build-time value.

3) Gradle recipes are cool.  We would like to eventually include Gradle
support in the ABCL build mechanism as it seems the most progressive of
current ecosystem (especially now that Groovy has matured a bit).  We
would like to "call into" Ant for parts of the build as the current
mechanism for controlling the artifacts and contents of 'abcl.jar' and
'abcl-contrib.jar' reside there and duplication of code paths would
incur maintenance overhead.

> My only remaining doubt (apart from other possible solutions which could
> also work and be more elegant, I'm open to suggestions of course) is
> around loading abcl-contrib.jar or more specifically the included
> packages (JSS and JFLI specifically). I've spent some hours tracking the
> loading mechanism and learned a bit about the process but I'm still
> missing some pieces.

The as yet unreleased abcl-1.5.0 contains [a top-level Ant target
'abcl-aio.jar'][4] that creates a single archive containing both
abcl.jar, abcl-contrib.jar, plus whatever has been added to the
build-time <file:abcl/contrib/> directory.

[4]: http://abcl.org/trac/browser/trunk/abcl/build.xml#L521

With [abcl-aio][] we introduced a mechanism that introspects the jar
manifest for declarations of ASDF systems to load when (require
:abcl-contrib) is issued.

[abcl-aio]: http://abcl.org/trac/changeset/14908

We intend for 'abcl-aio' to serve as the mechanism for the
end-user/developer to craft custom jar files.

Separately, there has been a long-standing mechanism to [include
arbitrary code at system boot in a 'system.lisp' file][5] that could
plausibly act as a hook for system intitialization, but I have never
needed it to get WAR archives loaded in Java Servlet contexts.

[5]: http://abcl.org/trac/browser/trunk/abcl/build.xml#L281

In short, if you were to build from ABCL trunk, I think you should have
the mechanisms to do what you want.  ABCL trunk (aka 'abcl-1.5.0-dev')
is quite stable, essentially representing the current release candidate
for abcl-1.5.0.  I am still working on a couple things (Java 6 support
for UIOP/RUN-PROGRAM, using ABCL-BUILD to install Ant and/or Maven via
interactive restart, better tests for PATHNAME shennigans), but I keep
the patches for this work separate from the ABCL trunk source until it
passes tests.

[…]
> I think there is something obvious I'm missing, the pieces are all there
> (the jar handling code works fine, etc) but I can't seem to find the
> right way to put them together.

ABCL-AIO isn't currently documented (a bug), so I would not consider it
obvious.  [ABCL-JAR][] essentially uses the same mechanism (i.e. pushing
PATHNAMEs to ASDF:*CENTRAL-REGISTRY*) you are prototyping here, but of
course you are trying to get ABCL-CONTRIB to work in the first place
here so it won't be much help for the immediate task at hand.

[abcl-jar]:
http://abcl.org/trac/browser/trunk/abcl/contrib/asdf-jar/README.markdown

Good luck, and lemme know what else you need,
Mark

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."