Proposal to improve the loading of the abcl-contrib dependency.

Sat Aug 1 17:30:35 UTC 2015

Mark,

   When I refer to the application lifecycle I am specifically referring to
the "start" phase of an application where dependencies need to be loaded
and configured.  ASDF is a lisp system with no specific support for java.
Java 9 is coming and is scheduled to include new semantics for modules and
that may carry with it a need for additional configuration semantics
related to the loading of modules. I wasn't aware that ABCL already has an
extensibility system for the lifecycle, such as
SYS:*MODULE-PROVIDER-FUNCTIONS*. If that can be used to override the
default places to search for the abcl-contrib packages, than great. I
didn't notice it in the code I was looking at.  The core problem that needs
to be solved is making a reasonable deployment system for distributed
single-jar (uberjar) applications. I don't see much need to go beyond that
since quicklisp is included in the contrib library. In the current state of
ABCL the contrib package won't load in an uberjar without executing system
implementation code that carries with it no guarantees of future support.
The same technology that will enable execution from an uberjar, will also
make it possible to integrate ABCL into many java systems, with one notable
exception of OSGI based systems.  These lifecycle issues are something that
should be dealt with at the system level, otherwise every ABCL application
will need to provide it's own lifecycle code and applications will likely
not be interoperable with each-other.  When it comes time to deal with OSGI
and Java 9 integration issues, the problem of loading contrib libraries, or
other libraries, will only get worse if this isn't solved within ABCL. The
Java logging systems is a good example of how bad it can get when everyone
decides to make their own implementation of core system features. If all
libraries end up implementing their own loading lifecycle implementations,
it is extremely likely that they will conflict with each-other. ASDF is a
good system, but it will be rendered useless in a java environment if no
uniform method is provided to properly configure it. In the current state,
uberjars produced with abcl and abcl-contrib do not properly configure asdf
so the standard code for loading the contrib packages in such applications
does not work.

Best Regards,
  Ralph Ritoch

On Sun, Aug 2, 2015 at 12:39 AM, Mark Evenson <evenson at panix.com> wrote:

>
> > On 01 Aug 2015, at 10:20, Ralph Ritoch <rritoch at gmail.com> wrote:
>
> […]
>
> > jar:file:SOME.jar!/META-INF/common-lisp/source-registry-conf.d/*/*.conf
>
> Might as well just take all files recursively:
>
> jar:file:SOME.jar!/META-INF/common-lisp/source-registry-conf.d/**/*.conf
>
> which will allow both styles to be used.
>
> > My second concern is the method of configuring the contrib search. The
> default should certainly support the "legacy" search methods and fallback
> to the classpath, but to avoid imposing limitations on future systems I
> believe a better algorithm would be to allow registration of custom
> configuration search handlers such as having a
> sys:*configuration-providers* which contains and ordered list of
> configuration provider functions. In the case of web based apps, this opens
> the potential to cache remote configuration files.
>
> We’re not really defining a contrib search anymore, per se, but rather an
> additional mechanism to configure ASDF locations in jar files.  ASDF
> already
> has a robust set of APIs for configuring itself, so I’m not entirely
> convinced
> that there is a need for anything beyond defining a well-known location
> within
> JAR archives to search.  Note that ABCL already has an extension mechanism
> for
> CL:REQUIRE in the SYS:*MODULE-PROVIDER-FUNCTIONS* hook. Could you give me a
> couple more specific use cases in which one would need additional
> mechanisms to
> specify the location of ASDF source registry configuration files?
>
> > I am looking forward to the progress of this, expanding this
> functionality beyond the configuration of one specific library could
> greatly improves the application lifecycle for ABCL applications.
>
> Again, ABCL already has a rich configuration mechanism for specifying ASDF
> system locations except for the location of the systems that constitute
> ABCL-CONTRIB.  Could you explain what you mean by application lifecycle
> here or
> what domain you are using the term from?
>
> >
> > As for the maintenance of the maven build system, the pom's I submitted
> compile the java files, and builds the jar, but is calling ant targets.
> While ant can call maven it still comes down to the key issue that maven
> doesn't pick-up a jar created by an external process. Maven central
> requires a source release, a binary release, and javadoc release, which all
> must be GPG signed. The complexity of trying to make maven a slave to ant
> is comparable to the complexity of the current "manual" system and I
> believe it will lead to the same situation you have now where no one does
> releases to maven. While there may be room for improvement with the maven
> system, there's normally little to no maintenance needed, once it is
> properly configured. The only thing that must be changed are the version
> numbers, and any dependencies. If you can make maven a slave to ant such
> that it will deploy to central and local repositories, that would be fine,
> but other than creating a custom maven plugin I don't know any way it would
> be possible without allowing maven to package and sign the jars.
>
> One can make releases to Maven using the existing POMs as was done in the
> past:
> all we need is someone with the correct amount of upstream juice to
> establish
> the political connections.
>
> There is nothing about the complexity of the current build process that
> prohibits releases to Maven, just a lack of a volunteer to ensure that it
> happens.  The [instructions have been written][1]
>
> [1]: http://abcl.org/trac/browser/trunk/abcl/maven-release.txt
>
>
> > I was advised that it would be possible to do away with the ant system
> all together. What was suggested to me by java developers was to have the
> java sources built in one module, which has a post-build script to compile
> the lisp files, and a second module which uses the target (generated) files
> of the first module as it's resource path. Calling ant is much easier since
> the ant system is already built, but it is possible to completely deprecate
> ant in favor of maven.
>
> As I pointed out earlier, your POMs do not share the intermediate build
> artifact locations nor packaging instructions with the Ant system.  In
> addition, there is a fair amount of other functionality in the Ant file
> which
> is definitely not duplicated in your POMs (running tests, running debugging
> instances, generating TAGS, usw.)  Until we approach a level of competence
> with
> Maven that allows us to adapt Maven to the needs of ABCL, I think
> discussions
> about deprecating Ant in favor of Maven are premature.
>
>
> --
> "A screaming comes across the sky.  It has happened before but there is
> nothing
> to compare to it now."
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20150802/b9896184/attachment.html>