[asdf-devel] Make the CL syntax predictable

[Pascal Costanza]

On 28 Mar 2014, at 20:37, Attila Lendvai <attila.lendvai at gmail.com> wrote:

>>> regarding the recent discussions i'm generally baffled why it is at
> 
>>> all a question whether to make a build software deterministic or
>>> not. in my view if there's anything in the global state that has an
>>> effect on the building of a software, anything, then it's a bug.
>> 
>> I think one question is whether it's worth taking a path to this goal
>> that breaks programs that currently work. I don't think all such
>> programs are bitrotted junk that can be fixed up in a day.
> 
> those programs that currently work will keep on working if ASDF is not
> upgraded.
> 
> no upgrade, no breakage.

If you can’t upgrade ASDF, you may also not be able to upgrade quicklisp. If you can’t upgrade quicklisp, you may also not be able to get updates to existing libraries. At least, it may be harder than necessary (like in the pre-quicklisp, pre-asdf-install days), and you can’t benefit from the efforts that were put into ASDF and quicklisp to the same extent as others anymore. [1]

Don’t try to abuse the popularity of ASDF and quicklisp to enforce changes to Common Lisp, the Language that may not be accepted by the community.

> to me it's a very strange argument that the right way to avoid
> breaking unmaintained old code is by hindering change that makes new
> things better in the future.

If the proposed change to enforce double-float as a default *read-default-float-format* was unambiguously an improvement, there would be no problem. But it’s not an improvement, not by a long shot.

Keep in mind that floats are always only approximations. Doubles are only better approximations, but they are approximations nonetheless. You can rarely use a particular float format without thinking about the impact it has on the results of your algorithms. Blindly assuming that whatever default defined by the language, or by some tool, is the best one for whatever purpose will not help you get good results in the general case.

Doubles are not generally better than singles. Doubles can be better than singles because they are more precise - and they are better only if this is what you care about. However, singles can be better because they use less space, and operations on singles can be executed more efficiently on CPUs. Vector units can fit more singles in their vector registers than doubles, so you will get even better performance from singles if/when Common Lisp implementations start to support vector operations better (especially AVX). 64-bit Common Lisp implementations can represent singles as immediate values, whereas doubles need to be boxed, which causes additional performance overheads. Depending on your goals, singles may therefore be a much better default than doubles.

If portability is a concern, than the answer may again not be so clearcut: For example, a language like Java enforces particular semantics for floats to the extent that, depending on the underlying CPU, you may not get the best performance out of it. Java allows you to specify strict or non-strict semantics, which allows you to choose between result portability and performance portability. Which of those is the best default is again open for debate. It’s not so clear that a focus on result portability is the best choice, considering that we’re dealing only with approximations anyway. Choosing doubles as the default *read-default-float-format* out of a concern for result portability may not help you because, if the CL implementations involved choose better performance over duplicating results exactly /across processors/, you didn’t win that much in the first place.

Common Lisp allows library authors to be very precise with regard to what they expect from floating point operations: The number syntax allows you to be very precise about the number representation (D, F, L, or S). It also allows you to specify that you want to be generic with regard to number representation (E). Choosing E is akin to stating that your library can work with any representation.

*read-default-float-format* lets you choose between different representations for such generic libraries. The scoping rules for dynamic variables in Common Lisp give you two choices: Either you tell a particular library that it chooses a particular representation for its generically chosen representations - you do this by dynamic binding. Or you tell a set of libraries (all that you are loading from now on) that they should all choose the same representation for their generically chosen representations - you do this by side effect. This is desirable when your goal is to ensure that the different libraries work well together. (This is for example how I understand Femlisp’s choice: It states that it works well only for doubles, and you better use doubles also with other libraries that you want to use in conjunction with Femlisp. I have no particular opinion about whether this is a good idea or not, since I’m not a domain expert and cannot judge this, but it’s likely that neither can you.)

I am very happy that Common Lisp gives a lot of choices in this (and many other [!]) regards. These options shouldn’t be restricted due to the lack of imagination of some developers who believe that limiting choices is always better. There are already (far too) many languages out there that go down that route. If you want Racket, you know where you can get it. Common Lisp is a language for people who know what they are doing, not for people who need to be held by their hands.

You also have the option to fork a particular Common Lisp implementation and change it in incompatible ways to demonstrate to a larger audience that such changes could be beneficial, if that is your goal. But again, don’t attempt to force /all/ Common Lisp implementations to behave a certain way. This will certainly fail, and that would be a pity, because this would invalidate many benefits that did come with ASDF and quicklisp. The fact that the proposed changes are discussed so actively is a clear sign that people care about these things.

For what it’s worth, I’m very happy that Robert is taking a conservative stance here, and that ASDF is slowing down with regard to pushing out features that potentially cause problems for existing working libraries. It already created too many unnecessary problems in the past.


Pascal

[1] I already encountered libraries that don’t load with current ASDF versions anymore, although they are defined as ASDF libraries, which is a shame! Personally, I am maintaining Closer to MOP, and because of that, I can afford not paying attention to newer ASDF/quicklisp versions only to a limited extent.

--
Pascal Costanza
The views expressed in this email are my own, and not those of my employer.