[asdf-devel] Subtle bug fixed in restore-image & dump-image

[Faré]

Dear Lisp hackers,

I wasted quite some time today with a weird weird bug.

I was streamlining the build of Lisp code within Google, which will
now use as little of ASDF as possible to take advantage of their
massively scalable build system, blaze:
http://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html

Of course, I wanted to reuse the infrastructure of UIOP, without the
rest of ASDF/DEFSYSTEM that my colleagues were wary to exclude (at
least by default) from the build. Now, I was dumping an image with
dump-image, restoring it with restore-image, which was hooking into
the declared *image-entry-point*, and everything was sailing from
there, receiving via the shell interface commands to execute build
actions. One of the actions, of course, was in turn to build new lisp
executables. And there: fail. Somehow the new binary was refusing to
see anything but the default values of the *image-restore-hook* or
*image-entry-point*. What the $%^@?

The problem, fixed in ASDF 3.1.0.8, was that restore-image itself was
binding the special variables for *entry-point* and
*image-restore-hook*. This meant that the global bindings were now
shadowed, yet, it is those global bindings now unreachable that SBCL
would be preserving at next dump-image. Oops. Conclusion: there are a
few cases particularly in application construction, where you really
want to be accessing the global bindings of a variable, and where
dynamic binding is a no-no. Thanks to pkhuong for confirming the
behavior of SBCL.

I learned that lesson the hard way. I hope you don't have to.

In other news, when you have massive distributed systems, you can
really trade off 10x CPU for ¼x latency on top of a previous a
tradeoff of O(n)xCPU for O(1)xlatency. Slave away, CPUs, we want low
latency on large Lisp compilations!

(The O(n)xCPU for O(1) saving in latency is parallelizing, where every
file has to be loaded once for every other file that depends on it,
rather than once — the classic XCVB more. The further 10xCPU for 4x
saving in latency is loading dependencies as source code in
unoptimized interpret mode, rather than waiting for 4x slower
compilation then loading fast from 10x faster fasl or 50x faster
cfasl.)

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
"If men are good, you don't need government; if men are evil or ambivalent,
you don't dare have one." — Robert LeFevre