[armedbear-devel] ASDF-JAR vs ASDF3 deployment support

[Erik Huelsmann]

On Thu, Apr 4, 2013 at 10:50 AM, Mark Evenson <evenson at panix.com> wrote:

> On 4/4/13 12:44 AM, Anton Vodonosov wrote:
> […]
> > General note about deployment CL systems.
> >
> > Often the deployment package should contain not only compiled code,
> > but also resources: web libraries and frameworks often contain .css,
> > .html, .javascript; bordeaux-threads uses version.lisp-sexp files,
> > and so on.
> >
> > In addition to the #. reader macro as used in bordeaux-threads,
> > applications/libraries access their resources during run-time using
> > asdf:system-relative-pathname or similar functions. In other words,
> > such libraries assume presence during run-time of their full source
> > control checkout .
>

Well, systems should declare the resources they want packaged into their
applications in their system files. That's a requirement both ECL and ABCL
now put forward, but I think other implementations which are able to
deliver more or less stand alone programs would want the same.


> >
> > A general purpose deployment solutions should account this aspect.
> >
> > One way to deploy applications with full content of the dependencies
> > code and resources:
> >  1. install quicklisp in a custom directory,
> >  2.  (ql:quickload :my-application)
> >  3. copy the directory of my-application, the custom quicklisp
> >     directory and probably prebuild .fals files to server.
> >
> > A little bit more docs and example of this approach is here:
> > https://github.com/avodonosov/heroku-buildpack-cl2
>
>  [ ... ]
>
> But because "universal distribution" (i.e. for the majority of users of
> a given system) has more-or-less subsumed by Quicklisp, there is no real
> constraint on the developers of system definitions to ensure that all
> components are so enumerated:  loading the system from Quicklisp works
> so nothing is broken, right?
>

Surely we can add tests for that in cl-test-grid as well?


> After Xach, Anton probably has the most practical experience with
> packaging ASDF definitions for deployment due to his pioneering work on
> CL-TEST-GRID.  I find Anton comments to be a succinct summary of the
> issues involved in packaging a given ASDF system in Quicklisp.  But when
> facing deployment for an ASDF system that is not in Quicklisp, but has
> dependencies on Quicklisp systems, Anton's approach won't work so well.
>  One would have to find the .asd definition on the filesystem, then
> transverse its sub-directories to find the source files.


The above is no longer true: with MONOLITHIC-FASL-OP you can build a fasl
which holds a system and all its depedencies, including QL provided
dependencies. This means you only need a working quicklisp setup on the
development machine.


>  This has the
> problem that there is no guarantee that the components of an asd
> definition exist in subdirectories, as theoretically the PATHNAME could
> point to an ancestor.


True, but with the above, they don't need to anymore: they'll be included
in the monolithic fasl. Even better: the monolithic fasl really doesn't
require ASDF at all: you can simply load the FASL with a normal LOAD call
and it will load all the embedded FASLs in the right (ASDF derived) order.


>  And even if one discarded this possiblity, often
> directory hierarchies share multiple ASDF definitions, so one would
> package "source" components that aren't really source components.
>

I'm sorry, I don't understand this bit. Could you try to explain your
concern with different words?

So, assuming that "deployment outside of Quicklisp from an ASDF
> definition" would be a greater good for the community, we seem to be
> stuck at a bootstrap problem.  I can't think of a way to force ASDF
> authors to enumerate their static components, but until such enumeration
> is widespread, I can't demonstrate its utility by developing an
> ASDF/BUNDLE operation.
>

Well, there is no need to develop the ASDF/BUNDLE operation: Fare did. Buth
other than that, the fact that I can now succesfully package
bordeaux-threads which we couldn't with ASDF-JAR, I think this is really a
step ahead?

Any ideas?
>

[I actually started writing this mail bottom to top, so what's below may be
a reiteration...]

Well, the great thing about the solution that Fare has now rolled out is
that I was able to package and load Bordeax-threads even though it is
problematic because it uses the #. reader macro and does not include the
version.lisp-sexp in the ASDF definition...

The infrastructure I've added to ABCL has been hooked up into ASDF with his
new release. Much more doesn't seem to be required to get it all correctly
packaged. Package providers can simply use #. reader macros to inject the
list of :static-files they need to be distributed with their code and asdf
will pick it up.  We'll need to work and test ASDF + ABCL to see if
everything works as intened though.


Bye,


Erik.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20130404/f14613f4/attachment.html>