[asdf-devel] Fwd: Faster loading for deployed systems?

[Erik Pearson]

Hi Robert,

Thanks for the Sunday afternoon reply.

On Sun, Apr 22, 2012 at 2:31 PM, Robert Goldman <rpgoldman at sift.info> wrote:

> On 4/22/12 Apr 22 -3:47 PM, Faré wrote:
> > Dear Erik,
> >
>
> >> ASDF assumes that since the system def file has not changed, the system
> >> hasn't been changed either (I know there has been discussion of this
> >> assumption.) But lets assume that at least part of the reason for that
> is
> >> because it is very time consuming to check the entire set of files every
> >> time the developer wants to recompile and load just a single file.
> >>
> > I don't think that's true, but my head hurts just thinking about it.
>
> This is not an accurate understanding.  It has come up over and over.
> The right thing to do would be to check systems recursively, but there
> are a number of reasons why this does not work with the current state of
> ASDF.  You can look in the archives for explanations; it has been
> discussed many times [I am not trying to be rude; I just don't have the
> energy to write up another copy of the explanation.  Probably we should
> put it in as a FAQ in the manual...]
>

Don't worry. I've been hanging around Lisp long enough to have a high bar
on "rude", and this doesn't even come close.

Thanks for the correction, I'll go back and revisit those threads.


> >
> >> Why not assume in a deployment mode that NO system will have changed if
> it
> >> is already loaded in memory, and unless the application has asked that
> it be
> >> reloaded? This would be for performance as well as for safety -- we
> really
> >> don't want dependency checking to result in any filesystem checks, etc.
> >>
> > Because that's wrong? I don't have as many dependencies as you do when
> > I reload a system with asdf, but usually, I do want the dependencies
> > to be reloaded.
>
> The deployment mode is an edge case and can already be handled very
> efficiently by dumping an image with the (stable) system you want
> already loaded (cf. Xach's buildapp, although I know that's for SBCL,
> rather than CCL; and CCL:SAVE-APPLICATION --
> http://ccl.clozure.com/manual/chapter4.9.html#Saving-Applications).  In
> some sense you are right, ASDF is slow for this use case, but it's
> really not the use case ASDF is looking for.
>

This is what I've been doing for several years now with no problems (none
due to asdf, that is!)

I'm exploring now a method for deploying small systems which are based on a
large-ish compiled, dumped image, but with small to medium sized systems to
be loaded and run as batches. Yes, I could just load these files manually,
but would at least like to be able to do run-time inquiries of module
availability (ala require), which, it seems to me, is the purview of asdf.

Also, it is possible that I'd need to add a specific 3rd party lisp package
to a deployment without incorporating it into the master runtime image, and
integration via asdf also seems natural.

BTW the deployment mode, for me, is the purpose of my development, so it is
kinda important. But I realize we all sit in a different place in the
development process. The main object of my work is a lisp web server that
sits under mod_lisp, but I also need to run lisp processes in batch, as
well as plenty of lisp ad-hoc analytical work.

I have thought that perhaps ASDF could output a deployment structure, which
is a set of instructions for constructing the running system. Not well
worded, perhaps, but if you capture the actions of asf you should be able
to play them back and create a functionaly equivalent image (albeit not
necessarily with asdf loaded.) E.g. if you just emit the filenames that are
loaded during (perform load-op cl-source-file), you can just place them
into a list and load them in sequence using just simple common lisp
machinery. Faster than asdf, but still slow compared to loading a big image.


>
> And ASDF is already too liberal about assuming stability, in a way that
> breaks a lot of people's development practice (see my earlier remarks);
> the last thing I would want to do is make it even more likely to miss a
> change and build an inconsistent system.
>
> Best,
> r
>
> _______________________________________________
> asdf-devel mailing list
> asdf-devel at common-lisp.net
> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20120422/8c3d1695/attachment.html>