Dear Faré,<br><br>thanks for the detailed reply. I wanted to clarify some points below.<br><br><div class="gmail_quote">On Mon, Jun 28, 2010 at 4:14 PM, Faré <span dir="ltr"><<a href="mailto:fahree@gmail.com">fahree@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Dear Vsevolod,<br>
<div class="im"><br>
> Approximately after the release of ASDF 2 I've got interested in creating<br>
> yet another system distribution management system on top of ASDF, and that<br>
> made me investigate ASDF internals. Through that investigation, I've<br>
> understood 2 things:<br>
> * ASDF has support for much more, than is well known and in ordinary use by<br>
> most Common Lisp programmers :)<br>
> * still it's support for versioning (which is essential to distribution<br>
</div>> management) is lacking due to: [...]<br>
><br>
Yes, ASDF is lacking in terms of support for versioning. And I admit I<br>
am not convinced that extending that support is a good thing. Unless<br>
you're going to reimplement and maintain something that's better than<br>
existing toolchains, then there's no point at all in the exercise:<br>
Lisp users are much better off using existing tools to manage versions<br>
than roll their own half-assed mock up. Dpkg or RPM or Nix are very<br>
well suited for managing the mix-and-match of many different versions<br>
of packages. Git or SVN are very well suited for managing the exact<br>
match of package combinations.<br>
<br>
I'm not saying that distribution management is useless, or that you<br>
shouldn't be coding anything, but rather that there are a lot of<br>
interesting problems in the world, and a lot of things that could help<br>
the Lisp community, and that it's not obvious that yet another<br>
distribution management system is the one most worth of the *huge*<br>
cost of doing it right. — And I speak from the experience of trying to<br>
turn a simple one-sentence idea of an improvement upon ASDF into<br>
software (XCVB).<br></blockquote><div><br>I understand that and I'm not pushing the particular issue of a distribution management tool here. I'm more in the mood of trying to build something on top of ASDF with the smallest effort. Moreover this is not directly connected to versioning issue, or let's say, proper versioning may have other application besides what I have in mind.<br>
<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
If I may suggest a simpler, cheaper and more useful strategy for<br>
helping with the version management of Lisp software, it would be to<br>
automatically produce Debian, RPM or Nix packages from ASDF<br>
specifications, and let those tools do the rest of the integration.<br>
<div class="im"><br>
> Attached is the proposed patch to ASDF, that, in my view, solves the<br>
> aforementioned versioning problems. It does the following:<br>
> * Unifies internal version representation format in the form of a list of<br>
> integers: (for example, ASDF 2.106 will have an internal version of '(2 106<br>
> 0), while ASDF 2.005 — '(2 5 0)).<br>
</div>Why only three integers? While you're at it, why not accept the whole<br>
range of Debian or RPM version comparisons?<br></blockquote><div><br>Actually, the number of integers is not limited, it's just that this is the default practice and currently available predicates only take into account the first two. So the internal representation is a list of integers. But the user can specify versions in many different formats, like "1.2.3" or even 1.2 (float number). Moreover it's possible to build adapters for more representations by defining methods for the generic function PROPER-VERSION.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im"><br>
> A final note about versioning: I needed to use pre-reading of ASD files in<br>
> order to know their versions without arising version conflicts. This works,<br>
> except for the fact, that READ-EVAL is turned off in the process, so such<br>
> version specifiers as:<br>
> * :version #.*hunchentoot-version*<br>
> * :version #.(with-open-file (vers (merge-pathnames "version.lisp-expr"<br>
> *load-truename*)) (read vers))<br>
> do not produce expected results and are treated as empty versions. There<br>
> will be a need to make a note about that to library developers.<br></div></blockquote><div><br>To clarify my point: there are cases of libraries, that use the read-eval syntax in version definitions, most notable of which are Edi Weitz's and Cyrus Harmon's. I agree here with Robert Goldman, that this is a little contrary to the declarative nature of DEFSYSTEM form. Still it is not a problem with neither the existing, nor the proposed variants. It's just that in the FIND-SYSTEM pipeline such version declarations (or rather evaluations) won't be counted (i.e. considered equal to wildcard version). But during the evaluation of DEFSYSTEM form itself the version would be properly evaluated. So this peculiarity will effect the new behavior, but not the old, that is why I don't consider it a problem: if someone would like to use the new behavior, he is free to adapt his code, otherwise he can leave it as is and no problems would arise.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
><br>
</div>If we ever go this way, it might be better to extend ASDF to accept<br>
reading versions from file, either by READing an object, or by<br>
READ-LINEing it.<br>
<div class="im"><br>
> Besides, an incompatible change is introduced to FIND-SYSTEM. The ERROR-P<br>
> optional argument is removed, so plain NIL is unconditionally returned, when<br>
> system is not found, and VERSION and VERSION-P optional arguments are added<br>
> instead.<br>
</div>This sounds very bad to me. What is the use case?</blockquote><div><br>The use case is that we can now (oos 'load 'system :version "1.2.3") or write in defsystem :depends-on (system "1.2.3" :above). As all those user-facing facilities use FIND-SYSTEM as the underlying tool. Independently from adding VERSION and VERSION-P I would still argue to remove ERROR-P argument, as it adds unnecessary complexity. There's only one place in the original ASDF code (OPERATE), where it's expected and it is easily handled in the caller.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">How does that affect<br>
the semantics of find-system? Can there now be many instances of a<br>
system in the source-registry?</blockquote><div><br>The semantics is the following: only zero or one instance of a system with a particular name can be in memory (in *defined-systems*) — just like before. The current behavior is that if some system is loaded, there's no possibility to load the new version. The new behavior is that, if FIND-SYSTEM is forced to load the different version of the system, a continuable error is signaled, that informs the user of the situation and it's possible either to abort or continue. If the user continues, the new definition is loaded instead of the old.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Does find-system now have to grovel for<br>
*all* these instances, load the .asd files to determine the version<br>
and pick the most recent one by default? That's C R A Z Y.<br></blockquote><div><br>Yeah, now SYSDEF-CENTRAL-REGISTRY-SEARCH et al. return a list of found ASD files instead of the first one. Then the most suitable is chosen by these criteria:<br>
* if no version is specified it will be either the one already in memory, or the first one in the list<br>* if version is specified it will be the one in memory, if VERSION-SATISFies, or the first from a list, for which version satisfies<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Note that the current interface of find-system seems to be copied from<br>
that of find-class.<br></blockquote><div><br>Hmm, I see. I still don't support such decision, but maybe I just not understand something...<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
The way I see things is that whoever sets up the source-registry<br>
thereby specifies which versions of each piece of software is going to<br>
be used, at which point version management can be taken completely out<br>
of ASDF and into the hands of whoever specifies said source-registry.<br></blockquote><div><br>Yeah, that's also a possible approach. It has it's advantages for sure.<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
If you want to check that what is configured is correct, you can check<br>
versions. And e.g. POIU, XCVB or ASDF-DEPENDENCY-GROVEL do have such<br>
consistency checks. But that's as much as you need.<br>
<div class="im"><br>
> I have supplied a number of unit tests to the functions,<br>
> that I have added. This is done with a very simple unit testing library of<br>
> my own (a microframework so to say) MUTEST, that can be found at:<br>
> <a href="http://github.com/vseloved/mutest/" target="_blank">http://github.com/vseloved/mutest/</a>.<br>
><br>
</div>I'd rather ASDF has only one test framework, if possible one with few<br>
dependencies if at all.</blockquote><div><br>I agree. If there's one already chosen, I'll stick to it. as I've said, when I've started this work, I didn't find any tests, so I used my own approach. By the way MUTEST is dependency-free.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">The current one kind of sucks but works so<br>
far. I'm not against using a different one, but someone should then do<br>
the conversion of existing tests.<br>
<div class="im"><br>
> I have also done functional testing of the changes with the systems, that I<br>
> have got installed already (around 70 libraries), and have verified, that<br>
> there are no problems. At the same time I would not claim, that there will<br>
> not be any problems with some system definitions, that use obscure ASDF<br>
> features (although, I have acquainted myself with ASDF to the point, that I<br>
> quite well understand most of the features as well as internals now). So I<br>
> ask anyone interested in this set of changes to thoroughly test them.<br>
><br>
</div>I think that's a good strategy, and one many people have been<br>
following lately. Maybe there's a way to automate and share these<br>
things further.<br>
<div class="im"><br>
> Finally, the patch can be applied by simply loading ASDF-VERSION.LISP on top<br>
> of the loaded ASDF 2.106.<br>
><br>
> I hope, this work proves useful and is integrated into ASDF. Anyway I'm<br>
> willing to answer any questions and/or improve the implementation, if<br>
> needed.<br>
><br>
</div>I admit this is not on top of my priority list. I'll try to review<br>
your code in detail later this week. Ideally, a new ASDF maintainer<br>
would step forward and replace me.<br>
<br>
Thanks for your efforts,<br>
<br>
[ François-René ÐVB Rideau | Reflection&Cybernethics | <a href="http://fare.tunes.org" target="_blank">http://fare.tunes.org</a> ]<br>
Most economic fallacies derive... from the tendency to assume that there<br>
is a fixed pie, that one party can gain only at the expense of another.<br>
— Milton Friedman<br>
</blockquote></div><br>Thanks,<br>Vsevolod<br>