Upgrade failures for asdf-3.1.7 and later

[James M. Lawrence]

OK I now see the last bullet point in "Pitfalls...", thanks. I had
searched the manual for "central-registry", which isn't mentioned
there directly.

This part could be improved: "startup scripts should load, configure
and upgrade ASDF among the very first things they do". Let's not tempt
the user to configure before upgrading!

Another bullet says, "Now that all implementations provide ASDF 3.1 or
later...", but that's not true. LispWorks PE does not. Yes, it's LW
version 6.1.1. And CLISP does not, though I understand that situation
is somewhat forlorn.

Here's the reason all this is immensely unexpected. If upgrading on
the fly requires reconfiguration, then I wonder what the purpose of
upgrading on the fly is supposed to be. Now that I understand that
caveat, I don't know why the on-the-fly feature exists. If one already
has such control over the configuration, then one wouldn't have loaded
ASDF2 in the first place The whole purpose of on-the-fly upgrades, it
seemed to me, was to handle the case where such control has passed.
For example the situation I mentioned: the user has the bog standard
Quicklisp setup in their init file and, after running an image for
some time, needs to load an ASDF3-only library.

I suppose the root problem here is LispWorks PE. Perhaps it should be
put into the same category as CLISP. On the other hand I place high
value on having things just work, no matter where the user is coming
from. If that's not possible or too difficult or too annoying then so
be it. It seemed reasonable to try.

Best,
lmj


On Thu, May 4, 2017 at 11:30 AM, Robert Goldman <rpgoldman at sift.net> wrote:
> Actually, I find that there already WAS a discussion of just this issue
> in the ASDF manual.  See the node "Pitfalls of the upgrade to ASDF 3."
> I have added another FAQ node to try to make this information easier to
> find, based on what went wrong.  Review welcome.
>
> Cheers,
> r
>
>
> On 5/4/17 May 4 -9:59 AM, Robert Goldman wrote:
>> On 5/4/17 May 4 -8:54 AM, James M. Lawrence wrote:
>>> LispWorks PE bundles an old asdf, which is loaded with (require "asdf").
>>
>> Is this because LWPE is still LW 6 instead of 7?
>>>
>>> CLISP optionally bundles asdf -- (require "asdf") -- and I would
>>> expect some Linux distributions to have that configuration in the
>>> CLISP they supply. (require "asdf") also looks in directories
>>> (including the user home directory!) for asdf.lisp, so an old version
>>> could be unintentionally loaded. My real concern is LispWorks, though.
>>
>> We can't really handle clisp effectively, because as far as releases are
>> concerned, it's dead.  I realize that the code repo is active, but
>> releases aren't being made, which means the de facto standard is now
>> something going on 7 years old.  That's not the ASDF project's fault.
>>>
>>> Maybe this wasn't clear enough, but my communications here are on
>>> behalf of users, not me. Many -- perhaps most, perhaps nearly all --
>>> people use asdf only indirectly through Quicklisp. I am trying to help
>>> the poor end-user who has a borked system and doesn't understand what
>>> is wrong. I would like to prevent the borkedness from happening in the
>>> first place.
>>>
>>> Most people initialize Quicklisp in their startup file. After using
>>> the lisp image for a while, they may wish to load a system and
>>> discover that the system requires asdf3. So they load asdf3. And then
>>> everything is borked. It may be difficult even for an experienced user
>>> to discover what is wrong, much less a casual user, and next to
>>> impossible for a newcomer.
>>>
>>> In the manual I didn't see any of the caveats you mention about the
>>> central registry. It says that asdf can be upgraded on the fly, and
>>> that's what people will expect. They don't expect that upgrading will
>>> bork the lisp image for some reason unknown to them.
>>
>> I will see if I can put in a FAQ about this.  Look for something soon.
>>>
>>> The quick and dirty workaround I mentioned is not something that would
>>> be part of any real code, just something a user could do to get things
>>> unborked again, that is, to enable Quicklisp to load again.
>>>
>>> I don't want to use *central-registry*. I'm not advocating using
>>> *central-registry*. I don't use *centry-registry* myself, except
>>> indirectly through Quicklisp. I am not insisting on weird upgrades.
>>> All I want to do is fix problems that end-users encounter.
>>
>> I'm not familiar with the guts of QL, but I thought QL didn't use
>> central registry.  I thought it used its own extension to the loading
>> process.
>>
>> Best,
>> R
>>
>>>
>>>
>>> On Wed, May 3, 2017 at 11:16 PM, Faré <fahree at gmail.com> wrote:
>>>> On Wed, May 3, 2017 at 7:10 PM, James M. Lawrence <llmjjmll at gmail.com> wrote:
>>>>> The manual says "it is possible to upgrade from ASDF 1 to ASDF 2 or
>>>>> ASDF 3 on the fly", and "asdf:*central-registry* is not recommended
>>>>> anymore, though we will continue to support it". From the
>>>>> documentation it is not immediately clear that upgrading is
>>>>> purposefully broken.
>>>>>
>>>> Upgrading works. Central-registry works. Central-registry is not
>>>> preserved by upgrading. And doesn't need to be, because
>>>> central-registry is something you insert into a special configuration
>>>> file that needs to first load the proper asdf, anyway. Whoever writes
>>>> that configuration file by hypothesis knows where all the software is
>>>> located. It just doesn't make sense to load the wrong asdf then
>>>> configure your central-registry only then to load yet another asdf. If
>>>> you do things like that you deserve to lose.
>>>>
>>>>> I suppose a quick and dirty workaround would be
>>>>> something like (setf asdf:*central-registry*
>>>>> asdf-2.26:*central-registry*).
>>>> That doesn't make sense, and asdf cannot guess what ancient version of
>>>> asdf was moved aside. Once again, it used to try much harder to
>>>> upgrade from 2.26 on sbcl and several other implementations, but that
>>>> got too unwieldy to support, for no good reason.
>>>>
>>>>
>>>>> Quicklisp's behavior of using the asdf version bundled with the
>>>>> implementation, if it exists, seems reasonable, at least at face
>>>>> value. After all, that's the version the vendors tested, and it may
>>>>> already be part of the image (or speedily loadable).
>>>> That part is totally reasonable indeed, and works perfectly.
>>>>
>>>>> Even if Quicklisp
>>>>> includes asdf-3.1.7, it would still try to load the bundled version
>>>>> first, so things would still be broken on LispWorks PE and CLISP.
>>>>>
>>>> Does not compute. Neither LispWorks PE nor CLISP release from 2010
>>>> provides ASDF. Quicklisp will then load its own ASDF, but that entails
>>>> no upgrade. If you want a more recent ASDF on top of that provided by
>>>> Quicklisp, you are going to lose anyway — instead overwrite
>>>> Quicklisp's asdf.lisp with the recent one, or convince Xach to upgrade
>>>> Quicklisp's ASDF to 3.1.7. Or use asdf/tools/install-asdf.lisp to make
>>>> your implementation provide ASDF despite it not being provided out of
>>>> the box.
>>>>
>>>> If you insist on such a weird upgrade, many things may go wrong beside
>>>> the *central-registry*. Yet even then you shouldn't be using the
>>>> *central-registry* to begin with. Use the source-registry.
>>>>
>>>>
>>>>> Therefore the real question is whether people should load the asdf
>>>>> bundled with the implementation, either on their own or through
>>>>> Quicklisp. If upgrading wasn't broken, things would just work and we
>>>>> wouldn't have to debate that question.
>>>>>
>>>> Upgrading is not broken. Your use pattern is broken. Don't initialize
>>>> the central registry after you load the wrong asdf then load the
>>>> correct one then expect things to work.
>>>>
>>>>> How about preserving *central-registry* when upgrading? That seems
>>>>> completely natural and expected to me, even apart from the fact that
>>>>> it happens to solve the problem at hand.
>>>>>
>>>> It's completely unnatural and backwards to load a wrong asdf,
>>>> initialize it, then upgrade it. Please configure *after* you upgrade
>>>> (and yes, *if* the configuration is for ASDF to find ASDF itself, you
>>>> may have to configure that part twice; or just skip the part about
>>>> loading the wrong asdf). And try using install-asdf.lisp where
>>>> applicable.
>>>>
>>>> Finally, please don't use the central-registry for cases like these.
>>>> Use the source-registry.
>>>>
>>>> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
>>>> Only a fool tests the depth of the water with both feet. — African proverb
>>>
>>
>>
>
>