[asdf-devel] In defense of ASDF & Semantic versioning

[Faré]

>>: antonv
>> Note also, in 2009 ASDF didn't consider version "0.6" as satisfying
>> requirement for "0.55":
>
>: rpgoldman
> Yes, that's because the version scheme in ASDF is not a sequence of
> period-separated integers, but a sequence of period-separated strings.
>
I suppose you mean the opposite:
"the version scheme in ASDF is a sequence of period-separated integers,
not a sequence of period-separated strings."

Once again, it looks like the model that Dan Barlow was aiming to emulate
was the numbering of Linux dynamic libraries; which at least superficially
looks like the same as so called "semantic versioning".

The intent was to help solve DLL hell for Lisp libraries,
but was insufficiently documented, and was lost to the community
when Dan Barlow left.

At the same time, I believe the solution was a bad fit then,
and even worse now, because this versioning scheme is designed
for binary releases of software developed in a centralized way.
ASDF libraries are distributed as source,
and more and more developed in a distributed way.

Thanks to #+foo, #.(if (find-symbol "FOO" :bar) ...)
and (eval-when ...), CL can deal with source-level compatibility
in ways that C cannot.

> When Faré wrote up the design rationale for ASDF, one of the principles
> was that ASDF should ask the right person for the right information.
>
Emphatically yes. But make it design rationale for ASDF 2, not ASDF.
I claim the successes and failures of this principle and its implementation,
not Dan Barlow though he did provide a starting point.
See Robert's and my paper: "ASDF 2: More Coordination, Less Coordination"
http://common-lisp.net/project/asdf/ilc2010draft.pdf‎

> For example, the library should not dictate where files go -- that's the
> job of the library's installer (user).  The library itself knows where
> its components are *relative to the installation* -- that's information
> the library implementor will have, which the installer will not have.
>
Great example.

> Similarly, if I am the library supplier, I am the one who knows when I
> have changed the API incompatibly.  That is why major component version
> changes were intended *not* to satisfy version requirements that have
> different major components.  This is reasonable, because it provides a
> channel of information from the library supplier, that can be checked
> automatically, and if the downstream is still compatible, it should be a
> relatively easy fix (but is it?  I don't know if there's a good way to
> say "I'll accept version 0.55+ or 1.0+").
>
> The only other option would be to have the library client provide both
> upper and lower bounds on the version numbers that the client will
> accept.  But, except for cases of *known* upgrade incompatibility, this
> is information that the library client simply cannot have.
>
And here I stop you.

The client library knows which version of the supplier library
it was designed to work with, and which version it was tested with.
But it is not responsible for how the supplier will evolve his versions.

The supplier library knows which previous versions it is compatible with,
and which versions it has broken compatibility with regard to features
that he actively supports — as opposed to having made incompatible changes
where no promise of compatibility was made, or where he explicitly reserved
the right to make a change (including fixing bugs and removing long
deprecated features).

Therefore, I contend that the supplier library, not the client library,
should provide the information as to which older versions it is or isn't
compatible enough with to version-satisfies the requirement
of a given version. The explicit requirement of a newer versions,
by definition, it fails to satisfy.

By default, it could be, as in ASDF 1, the "semantic versioning"
requirement of newer version AND same major version number.

Or it could be "This version can stand in as a replacement for
any older version whatsoever", as is the current ASDF 3 behavior,
and as ASDF needs for itself.

Or it could be "This version can stand in as a replacement for
any older version as far back as this given version", where each
library supplies its own number.

In any case, it should be the supplier library's role to specify that,
not the client library's, at least in general and as a first approximation.
The added advantage is that libraries can then implement their own
version parsing schemes that need not be the same as ASDF:
just override version-satisfies and have it call your own functions
to parse and compare version strings.

Yes, sometimes a client might insist on a specific old version range
for a library, rather than the open-ended range of the versions
being actively developed. But then what that client really wants
is a fork of the library started at some old version, so
maybe the solution isn't a matter of adding support in ASDF,
but in encouraging authors to fork projects when needed
— or encouraging them to find solutions that don't rely on
explicit or implicit forking.

> The rules here are simple and easy to understand: if you change the API
> in an incompatible way, bump the major version number.
>
> Yes, this rules out the false modestly of version numbers like 0.0.145,
> which looks like an alpha, but in fact turns out to be the fifth full
> release, but that's a sacrifice we can all live with! ;-)
>
I think that makes a meaningful default, or at least option,
that we should encourage people to use.

But that's just not what most library authors have done in practice,
and not what ASDF has done itself (it has maintained backward compatibility
from ASDF 1 to ASDF 2 to ASDF 3, modulo unavoidable bugs).

> As my own devil's advocate, the counter-argument is that the state of
> the art in CL libraries is so poor that even getting a version number is
> unusual.  But this seems to be a counsel of despair: it says that since
> we have a bad state of affairs now, we are doomed to live with it forever.
>
Or maybe we should also strive for more cooperation with less coordination:
tweak ASDF's version handling so it becomes extensible,
and document the protocol including the default and one non-default
(which it seems we already have).

> I suppose I could take a little while and write up a candidate
> "versioning systems with ASDF" node for the ASDF manual, and push it for
> consideration by the community.
>
Yes, that too.

>> Also, as we speak about versioning, I have been trying to use semantic
>> versioning as described at http://semver.org/ and I don't think it is
>> a silver bullet - it doesn't solve all problems.
>
> Isn't the subtitle of /The Mythical Man-Month/ "No Silver Bullet"?
>
> Yes, this doesn't solve every problem, but it might give you a useful
> warning if your upstream has changed under you....  Pascal's case is an
> odd one, because the bump to 1.0 is, in some sense, a false positive.
>
> OTOH, it's not really so bad to have to take a look at a library that
> has seen no commits in three years.
>
I don't think we're doing badly. At least not in THIS regard.

That's what I think the problem is with Lisp libraries:
http://fare.livejournal.com/169346.html
Sometimes, cooperation DOES require coordination!

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Gun: weapon of individual vs individual. Bomb: weapon of group vs group.
That's why collectivists hate guns and love bombs.