[asdf-devel] Re: Why Lisp is Now an Acceptable Scripting Language

[Faré]

On Sun, Mar 2, 2014 at 8:50 PM, Steve Haflich <shaflich at gmail.com> wrote:
> A quarter century ago I devised similar machinery for Allegro.  It's still
> in the implementation, and I use it daily for personal scripting tasks:
>
> http://franz.com/support/documentation/current/doc/startup.htm#starting-unix-script-3
>
Wonderful! Unhappily, every implementation had its own incompatible variant.
Also, I wanted to provide best practices in terms of --eval, --load, etc.

> Your work is a very useful thing to have, but let me suggest there are
> additonal issues to consider.
>
> The Allegro machinery allows a compiled file to be included as the
> application.  This eliminates the time necessary to cl:read the source and
> compile the definition on systems that automatically compile.  For many
> scripting applications none of this matters.  For most purposes interpreted
> executions, and/or the compiler, now require such inconsequential time that
> one shouldn't care how the deed gets done.  But there are situations where
> it might:
>
>   If the script does nontrivial computation, the difference between
> interpreted and compiled execution might matter.  (For implementations that
> always compile, the compilation cost is fixed, but happens upon each
> execution.)
>
>   If the script is executed with high frequency, perhaps as the
> implementation of a web page, the cost of launching a heavyweight Lisp image
> in almost _any_ implementation is likely to be objectionable.  A lightweight
> C a.out has a small footprint in addition to the various .so libraries it
> uses, which will tend to stay loaded, and requires minimal startup execution
> overhead.  Most Common Lisp applications have a larger startup cost because
> Lip environments are, shall we say, bigger and more capable.
>
> It would be nice to think further how startup overhead could be reduced with
> precompiled execution
>
Interestingly, cl-launch takes the stance that the user specifies source,
but that compilation happens systematically and implicitly in the background,
using the same output-file cache as ASDF3.
Therefore, the first time it sees a given file or system,
or after they have been updated, there may be a startup delay
while the compiler processes the files;
but subsequent invocations will be faster as the compiled code is
directly loaded.
I feel this is both faster than what scripting languages do,
and more usable and maintainable than having to
explicitly deal with compiled files.

If you want to deal with precompiled files, you can use cl:require
for your implementation's files via --require, or you can use ASDF's
single-fasl delivery mechanism with fasl-op
and the accompanying deliver-asd-op. Also, it's portable.

(Historically, it is more accurate to say that ASDF imported
the cache technology previously implemented by cl-launch,
which itself was reusing a technique popularized by
Debian's common-lisp-controller as far back as 2004,
and by the more widely portable asdf-binary-locations after it.
By defining an :around method for the output-files function,
it was possible for the user to control where ASDF
would store its compilation output,
without the authors of ASDF having had to create an explicit hook.
It is unclear who first came up with the idea,
but the fact that this technique could be developed as an extension to ASDF
without the author explicitly designing the idea into it,
and without having to modify the source code,
is an illustration of how expressive and modular CLOS can be.
Of course, for the solve the bootstrapping issue of
where the fasl for this object file should itself be stored,
this functionality had to be part of ASDF itself,
and so was merged into ASDF2, becoming ASDF's output-translations layer.)

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Toleration is not about believing that stupid people are intelligent,
it's about letting stupid people be victims of their own stupidity
rather than being victims of yours.