[asdf-devel] versioning [was Re: asdf verbosity]

[Faré]

NB: Meh, shouldn't have gone into this long defense. I somehow took
dherring's remarks personally and don't have the guts to not post this
reply after writing it. Please ignore.

> In a nutshell, things like changes to inputs (enforcing contracts), outputs
> (changing what's printed) and internal behaviors (e.g. path names vs
> objects) are very sensitive in ASDF.
>
I've tried to always support previously documented accepted inputs.
Sometimes, it was not sufficient - and I realized the hard way that I
had to support and document previous practice, too.

As for what's printed - I've added a bit of output, at least in the
case of upgrade, and no one's complained. In general, I'm in the
opinion that since ASDF's default "verbose" output is not verbose
enough to help diagnose any issue, it's as good as useless, and it
should be silenced by default, and made much more verbose when in
"verbose" mode — but obviously, this doesn't fly with a lot of
existing users.

Anyway, I understand the concern, and my position is that unless it's
consensual (at least amongst concerned users on this list), a
compatibility-breaking change in output is not kosher — and so I do
make suggestions once in a while, usually with actual code committed,
usually to be rebuffed, with the change backed out before the next
stable release.

> There are many users of the system,
> and a good number were tuned to the implementation instead of any
> specification.  ASDF's design leaks all sorts of details.
>
I should know. A huge part of my job in taking ASDF from 1.369 to 2.000
and beyond has been to replace implementation-dependent behavior and
underspecified interfaces with well-defined portable semantics.

> While you are tagging "minor" releases roughly once a month, since there are
> no separate trunk/release branches, all the changes appear in each month's
> release.
>
I tried to have two branches — it amounted to a lot more work and confusion
for no actual benefit whatsoever: *all* the work ended being "backported"
to the stable branch anyway.

Indeed, there are no new features to speak of,
only bug fixes and that prompt refactorings and API tweaks:
one corner case of system+implementation+OS leads to having to refactor
the code, fix an existing interface and expose the new, fixed, one.
These changes all too often introduce bugs that cascade for a few commits
before stabilizing.

I try to select the time for a stable release for when there is no more
pending fix, cascaded or not, and before I do any sizable refactoring.
Unhappily, absent wider testing, I often miss some corner case, and
sometimes need to issue patches right after a "stable" release.

Of course, I make mistakes along the way — but that has nothing to do
with being "development" vs "stable". It has to do with any change at all,
and it's why I encourage power users to test development versions.

As for my actual ideas for new features: they all go into XCVB,
which is getting more usable and more portable every week,
thank you for asking, but not yet universally good enough
to justify casual users migrating from ASDF.


> Thus I think you either need to restrict yourself to truly minor
> changes (not good for development), only make major releases once a year or
> so, or split development and only backport critical fixes and clean API
> extensions to the release branch.
>
"Development" so far has been prompted mainly by one or the other
corner cases emerging from having to provide a portable interface
supporting 14 underlying implementations on more than 3 OSes.
Minor changes, bug fixes and exposing APIs after cleaning them up
is *ALL* I'm doing with ASDF. I'm sorry I'm not doing it without
occasional breakage, hopefully breakage that remains unreleased
(but unhappily, not always, as I do make mistakes).

Then again, if you claim you can do a better job at maintainership,
I'm most willing to pass on the tiara to the next princess.

> In short, I wasn't really aware that people were actively seeking
> enhancements, but I have seen a number of potentially breaking changes, and
> the current development cycle means people have to catch them every month.
>
What do you propose? Less frequent releases? How would that help,
given the rate of change? Less frequent changes? Considering that
"development" is mostly driven by fixes to reported bugs, with
cleanups attempted along the way, that really means, don't fix bugs so
fast.

Or maybe you're suggesting to try fewer cleanups as I fix bugs.
That's possible, but would make the code so disgusting that I
would quickly throw up and throw down the towel. — If you think
ASDF can be maintained this way, you'll have to do it yourself.

>> Do you call 2.014 vs 2.011 "major"?
>
> By number, no.  By content, ...
>
> From a quick scan, here are a few good-looking changes that might count as
> major.
>
> eb1ed61 Substituted (default-directory) for *default-pathname-defaults*
> 09b9b8a Binding of *default-pathname-defaults*
> [special variables in CL have high visibility]
>
For instance the first changes you mentionned happened for 2.011.1;
they are an improvement to the way we parse our configuration file.
There is no negative impact whatsoever to any user, since it only affects
people who previously were frustrated in desiring to write
location-independent configuration files for their directory hierarchies,
or who were using #. in configuration files to work around this or other
limitations, and will be able to trust the *default-pathname-defaults*
to be bound to an absolute directory pathname.

> 2.012.2: cleanup name of cache directory for clisp (remove trailing dash)
> [the user should be told to rename or delete the old directory]
>
CLISP was returning non-sensical results for a component of
implementation-identifier. A user complained. I fixed in 2.012.2.
This is slightly incompatible, but not in a way that
should ever matter negatively to the end user, except for
a slight waste of cache space, if he never cleans it manually.
And it's not more incompatible than a wider change I made in 2.142
(~2.009.8) to implementation-identifier to keep strings shorter,
which is necessary to make some Windows implementations happy,
and another issue reported by a user.

If you want a "major" change, consider rather how I am now
pre-caching source-registry results, and you have to re-run
(asdf:initialize-source-registry) for ASDF to see newly-installed .asd files.
Previously, the behavior was left unspecified, and you could install a
new .asd file in a previously-configured directory that already had .asd
files but not in a new sub-directory or one previously without a .asd file.
This semantic change was prompted by a request from Xach to use pre-caching
in quicklisp. Now he can do it by simply using the source-registry,
or rolling his own alternative with the same simplified interface.

> [...] end users should be able
> to trustingly accept minor and micro versions, generally without even
> reading a changelog.  Their grey matter should only need to kick in on the
> major release cycles.  This also allows other people (e.g. debian or sbcl)
> to safely make the upgrade choice for them.
>
I agree. And I believe that this is the case: none of the changes I am making
should be *negatively* impacting anyone.

Negative impacts come from:
1- bugs.
2- my failure to realize the (usually un(der)specified) API I'm touching
 is actually being relied upon by clients.

Oops. My bad. Please help me test before release.

[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
Quality means doing it right when no one is looking. — Henry Ford