[asdf-devel] Proposal for improved support for versions of systems

[Vsevolod Dyomkin]

Hi!

Approximately after the release of ASDF 2 I've got interested in creating
yet another system distribution management system on top of ASDF, and that
made me investigate ASDF internals.  Through that investigation, I've
understood 2 things:
* ASDF has support for much more, than is well known and in ordinary use by
most Common Lisp programmers :)
* still it's support for versioning (which is essential to distribution
management) is lacking due to:
  - possibility to specify only exact version
  - somewhere in the depths of code version checking is lost (for example,
there's no condition signaled, if in the process of system definitions
traversal we load ASD file with different version for some system, that was
already loaded, which actually causes errors in some situations, i.e. its a
bug)
  - the syntax is not very friendly

So, during last month I was developing ASDF in the direction of more robust
version support.  I've started with 2.000, but fortunately there's no
intersection between my work and recent developments up to version 2.106
(except for new DEFGENERIC*, which is trivial).

Attached is the proposed patch to ASDF, that, in my view, solves the
aforementioned versioning problems.  It does the following:
* Unifies internal version representation format in the form of a list of
integers: (for example, ASDF 2.106 will have an internal version of '(2 106
0), while ASDF 2.005 — '(2 5 0)).  At the same time, the users are given
quite a lot of freedom in specifying versions, for example they can do that
as before in string form (like "1.2.3"), so it's backwards compatible.  This
new format is more programming-friendly and several utility functions are
added: PROPER-VERSION (to transform to it), V> (version-greater-or-equal),
MAJOR-VERSION and MINOR-VERSION,— while VERSION-SATISFIES is much improved
in the manner, that is explained below.  Also a method for SLOT-UNBOUND is
added to VERSION slot on a SYSTEM class, to return version '(0 0 0), which
is considered (on par with NIL) to be a wildcard version spec, so to say.
* Adds VERSION-P (version-predicate) parameter to different ASDF functions,
that need to deal with versions.  This specifies in which way to match
versions with VERSION-SATISFIES and currently has the following variants:
:EXACT (nil also translates to :EXACT), :ABOVE, :MAJOR+, :MINOR+, and also a
second version spec, which means that version should be in the interval from
the given version to version-p itself.  This is the change, that I had had
in mind, when I started looking at ASDF
* Adds a continuable error of type SYSTEM-VERSION-CONFLICT, when we try to
read ASD file with a different version of some system, that is already in
memory
* Adds new format of version specification in :DEPENDS-ON clause of
DEFSYSTEM.  Besided (:version <system> <version>), which is retained and
extended with version-p, i.e. (:version <system> <version> <version-p>) —
example: (:version :hunchentoot "1.2.3" :above) ,— there's now also a
simpler (<system> <version> <version-p>) — example: (:hunchentoot "1.2.3"
:above)

A final note about versioning: I needed to use pre-reading of ASD files in
order to know their versions without arising version conflicts.  This works,
except for the fact, that READ-EVAL is turned off in the process, so such
version specifiers as:
* :version #.*hunchentoot-version*
* :version #.(with-open-file (vers (merge-pathnames "version.lisp-expr"
*load-truename*)) (read vers))
do not produce expected results and are treated as empty versions.  There
will be a need to make a note about that to library developers.

Besides, an incompatible change is introduced to FIND-SYSTEM.  The ERROR-P
optional argument is removed, so plain NIL is unconditionally returned, when
system is not found, and VERSION and VERSION-P optional arguments are added
instead.
I'd like to argue, that this is justified by two things:
* there's no substantial use of ERROR-P argument neither in ASDF itself, nor
anywhere in dependent code (at least I can tell that after a brief look at
google search results)
* more importantly, although there's another option to add
version/version-p: add VERSION and VERSION-P as keyword arguments and leave
ERROR-P also as keyword argument — I don't think, that overall the use of
such argument is a proper architectural decision, because it's not an error
actually, when system is not found — it's a common situation and should be
dealt with in the caller appropriately to the context

My last note concerning this patch is about testing.  After syncing with the
development branch of ASDF, I see the addition of TEST directory.  When I'd
started my experiments, I had somehow missed it (or it had not been
there?).  Anyway, I have supplied a number of unit tests to the functions,
that I have added.  This is done with a very simple unit testing library of
my own (a microframework so to say) MUTEST, that can be found at:
http://github.com/vseloved/mutest/.  The tests are specified right in the
code side-by-side with functions definitions, but are protected with
#+mutest guards, so effectively can be ignored.  They also have the benefit
of showing, how each of the functions can be called.  There's also a dummy
directory structure under TEST, that is used in the tests and should be
reproduced in the same directory, where ASDF-VERSION.LISP is located.

I have also done functional testing of the changes with the systems, that I
have got installed already (around 70 libraries), and have verified, that
there are no problems.  At the same time I would not claim, that there will
not be any problems with some system definitions, that use obscure ASDF
features (although, I have acquainted myself with ASDF to the point, that I
quite well understand most of the features as well as internals now).  So I
ask anyone interested in this set of changes to thoroughly test them.

At the same time I'd be glad to know more about utilizing the current ASDF
test-suite.

Finally, the patch can be applied by simply loading ASDF-VERSION.LISP on top
of the loaded ASDF 2.106.

I hope, this work proves useful and is integrated into ASDF.  Anyway I'm
willing to answer any questions and/or improve the implementation, if
needed.

Vsevolod Dyomkin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100628/c289ec86/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: asdf-version.tar.gz
Type: application/x-gzip
Size: 8799 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100628/c289ec86/attachment.bin>