Thoughts on CL-scripting

[Faré]

On Sat, Feb 27, 2016 at 11:32 PM, Robert Goldman <rpgoldman at sift.net> wrote:
> Originally, I was very concerned to minimize the overhead of
> dependencies for building and testing ASDF, and I still think this is
> important.
>
Agreed.

Well, building ASDF strictly speaking only ever necessitate Make, a
shell and cat (and, recently, Make was made superfluous, and on
Windows, only CMD.EXE and its builtin COPY command were needed). [And
under ASDF 1 and 2, there was no building: we directly edited
asdf.lisp]

Releasing ASDF and testing it, on the other hand, always relied on
some scripting infrastructure. Back in the day, it was an unhealthy
mix of shell and perl, sed, and more. Early while writing ASDF 3
(January 2013), I started replacing a lot of the release scripting  by
a Lisp script bin/asdf-builder that was so much nicer than either
shell or perl at dealing with lists of files, lists of files and
regexps, lists of (tarball and list of tarball contents), etc.; but
testing remained written in a mix of make and shell, which worked even
when asdf was badly broken.

In May 2014, I started the minimakefile branch, which replaced
run-tests.sh by a lisp script, and was eventually merged into master
in November 2015 after the 3.1.6 release. Lisp scripting allowed much
better error reporting, with a summary of failures at the end, and
accurate-by-construction messages for reproducing and debugging
failures, as well as a few bug fixes. The downsides were that:

1- It makes a hard requirement for testing that recent enough versions
of plenty of lisp libraries should be installed for testing, when they
were previously only weakly required for testing or strongly required
for releasing but not for testing. This was mostly solved when we
adopted git submodules in August 2014.

2- The setup was quite fragile, particularly on platforms that I
wasn't testing it on, or in certain failure scenarios that I hadn't
anticipated or insufficiently tested. This caused many bugs that were
only weaned by being found by Robert the hard way over these two
years. This really sucks, and I apologize to Robert again for each of
them.

3- It fails badly when asdf itself is so broken that it can't load
asdf-tools. I only very recently (January 2016) modified the lisp
scripting infrastructure to build an executable ./build/asdf-tools the
first time they are called, hopefully with a stable working asdf.
[This unfortunately won't work well if the same checkout is shared on
multiple architectures (e.g. Windows vs Linux vs OS X, x64 vs x86 vs
ARM, etc.). But I believe we somehow already decided not to support
that.] Robert still reinstantiated the old Makefile (February 2016),
though he didn't delete the lisp scripting code (thanks!) In
retrospect, I should probably have kept the two working side-by-side
in master, rather than developing it in a branch — it would probably
both have made errors easier to find, and easier to cope with when
found. I was wrong to expect an easy transition, which was the idea
behind the branch.

> But I now conclude that trying to pull in the cl-scripting
> infrastructure was wrong, because it had the side effect of making the
> system used for testing be a dependency of the system under test,
> introducing unhealthy circularities.
>
Yes, see point 3 above.

> Going forward, if you want to experiment further along these lines, I'd
> suggest an entirely separate project -- with its own test suite! -- that
> would build an executable capable of executing CL scripts.  That
> executable would have to NOT require bleeding-edge ASDF capabilities,
> only simple core capabilities.  It should be built with the ASDF that's
> delivered with implementations, not with bleeding-edge ASDF.
>
Agreed. That's what asdf-tools might mutate into.

> That executable could be built, saved, tested, and installed, and then
> could be used as the engine for ASDF building, testing, etc., if one
> liked.  It could be a required build tool, the way today "make" is a
> required build tool.
>
That was the idea behind XCVB, though it was never as portable as ASDF is.
Bazel also kind of does provide this, though it's not lightweight at all.

> To be honest, I think such an engine should first be tested with much
> less demanding problems than building and testing ASDF.  Simple
> scripting tasks, say within the ambit of the /Perl Cookbook/, would be a
> reasonable starting point.
>
> Even doing this, I think there's only a 50-50 chance the approach will
> work out.  CL's numerous incapabilities where dealing with the host OS
> and filesystem will still stand in the way.  Committing to a single
> implementation -- probably CCL or SBCL -- might solve this problem.  A
> less cumbersome means of line-by-line processing of streams should be
> developed, and some means of using regexp literals should be added.
> Variable interpolation into strings should be provided.
>
> Just some thoughts....
>
Well, inferior-shell allows for variable interpolation into strings on
top of the functionality provided by uiop:run-program. optima-ppcre
makes regexp's quite usable on top of the functionality provided by
cl-ppcre:register-group-bind, etc. uiop:run-program already provides a
variety of line-by-line processing options, with an extension
mechanism to support more.

But yes, to make it more usable by the masses, a documentation effort
would be required. My goal so far was only to make sure the basic
capability is ubiquitous, which it now is thanks to ASDF3's UIOP,
whereas Quicklisp makes it easy to install extensions (thanks, Xach).
We require more minerals.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
If being against something is a phobia, then being for is mania.
Peace and understanding through slurs of mental illness.
Homomania, islamomania, etc.