Upgrade failures for asdf-3.1.7 and later

[Robert Goldman]

Actually, I find that there already WAS a discussion of just this issue
in the ASDF manual.  See the node "Pitfalls of the upgrade to ASDF 3."
I have added another FAQ node to try to make this information easier to
find, based on what went wrong.  Review welcome.

Cheers,
r


On 5/4/17 May 4 -9:59 AM, Robert Goldman wrote:
> On 5/4/17 May 4 -8:54 AM, James M. Lawrence wrote:
>> LispWorks PE bundles an old asdf, which is loaded with (require "asdf").
> 
> Is this because LWPE is still LW 6 instead of 7?
>>
>> CLISP optionally bundles asdf -- (require "asdf") -- and I would
>> expect some Linux distributions to have that configuration in the
>> CLISP they supply. (require "asdf") also looks in directories
>> (including the user home directory!) for asdf.lisp, so an old version
>> could be unintentionally loaded. My real concern is LispWorks, though.
> 
> We can't really handle clisp effectively, because as far as releases are
> concerned, it's dead.  I realize that the code repo is active, but
> releases aren't being made, which means the de facto standard is now
> something going on 7 years old.  That's not the ASDF project's fault.
>>
>> Maybe this wasn't clear enough, but my communications here are on
>> behalf of users, not me. Many -- perhaps most, perhaps nearly all --
>> people use asdf only indirectly through Quicklisp. I am trying to help
>> the poor end-user who has a borked system and doesn't understand what
>> is wrong. I would like to prevent the borkedness from happening in the
>> first place.
>>
>> Most people initialize Quicklisp in their startup file. After using
>> the lisp image for a while, they may wish to load a system and
>> discover that the system requires asdf3. So they load asdf3. And then
>> everything is borked. It may be difficult even for an experienced user
>> to discover what is wrong, much less a casual user, and next to
>> impossible for a newcomer.
>>
>> In the manual I didn't see any of the caveats you mention about the
>> central registry. It says that asdf can be upgraded on the fly, and
>> that's what people will expect. They don't expect that upgrading will
>> bork the lisp image for some reason unknown to them.
> 
> I will see if I can put in a FAQ about this.  Look for something soon.
>>
>> The quick and dirty workaround I mentioned is not something that would
>> be part of any real code, just something a user could do to get things
>> unborked again, that is, to enable Quicklisp to load again.
>>
>> I don't want to use *central-registry*. I'm not advocating using
>> *central-registry*. I don't use *centry-registry* myself, except
>> indirectly through Quicklisp. I am not insisting on weird upgrades.
>> All I want to do is fix problems that end-users encounter.
> 
> I'm not familiar with the guts of QL, but I thought QL didn't use
> central registry.  I thought it used its own extension to the loading
> process.
> 
> Best,
> R
> 
>>
>>
>> On Wed, May 3, 2017 at 11:16 PM, Faré <fahree at gmail.com> wrote:
>>> On Wed, May 3, 2017 at 7:10 PM, James M. Lawrence <llmjjmll at gmail.com> wrote:
>>>> The manual says "it is possible to upgrade from ASDF 1 to ASDF 2 or
>>>> ASDF 3 on the fly", and "asdf:*central-registry* is not recommended
>>>> anymore, though we will continue to support it". From the
>>>> documentation it is not immediately clear that upgrading is
>>>> purposefully broken.
>>>>
>>> Upgrading works. Central-registry works. Central-registry is not
>>> preserved by upgrading. And doesn't need to be, because
>>> central-registry is something you insert into a special configuration
>>> file that needs to first load the proper asdf, anyway. Whoever writes
>>> that configuration file by hypothesis knows where all the software is
>>> located. It just doesn't make sense to load the wrong asdf then
>>> configure your central-registry only then to load yet another asdf. If
>>> you do things like that you deserve to lose.
>>>
>>>> I suppose a quick and dirty workaround would be
>>>> something like (setf asdf:*central-registry*
>>>> asdf-2.26:*central-registry*).
>>> That doesn't make sense, and asdf cannot guess what ancient version of
>>> asdf was moved aside. Once again, it used to try much harder to
>>> upgrade from 2.26 on sbcl and several other implementations, but that
>>> got too unwieldy to support, for no good reason.
>>>
>>>
>>>> Quicklisp's behavior of using the asdf version bundled with the
>>>> implementation, if it exists, seems reasonable, at least at face
>>>> value. After all, that's the version the vendors tested, and it may
>>>> already be part of the image (or speedily loadable).
>>> That part is totally reasonable indeed, and works perfectly.
>>>
>>>> Even if Quicklisp
>>>> includes asdf-3.1.7, it would still try to load the bundled version
>>>> first, so things would still be broken on LispWorks PE and CLISP.
>>>>
>>> Does not compute. Neither LispWorks PE nor CLISP release from 2010
>>> provides ASDF. Quicklisp will then load its own ASDF, but that entails
>>> no upgrade. If you want a more recent ASDF on top of that provided by
>>> Quicklisp, you are going to lose anyway — instead overwrite
>>> Quicklisp's asdf.lisp with the recent one, or convince Xach to upgrade
>>> Quicklisp's ASDF to 3.1.7. Or use asdf/tools/install-asdf.lisp to make
>>> your implementation provide ASDF despite it not being provided out of
>>> the box.
>>>
>>> If you insist on such a weird upgrade, many things may go wrong beside
>>> the *central-registry*. Yet even then you shouldn't be using the
>>> *central-registry* to begin with. Use the source-registry.
>>>
>>>
>>>> Therefore the real question is whether people should load the asdf
>>>> bundled with the implementation, either on their own or through
>>>> Quicklisp. If upgrading wasn't broken, things would just work and we
>>>> wouldn't have to debate that question.
>>>>
>>> Upgrading is not broken. Your use pattern is broken. Don't initialize
>>> the central registry after you load the wrong asdf then load the
>>> correct one then expect things to work.
>>>
>>>> How about preserving *central-registry* when upgrading? That seems
>>>> completely natural and expected to me, even apart from the fact that
>>>> it happens to solve the problem at hand.
>>>>
>>> It's completely unnatural and backwards to load a wrong asdf,
>>> initialize it, then upgrade it. Please configure *after* you upgrade
>>> (and yes, *if* the configuration is for ASDF to find ASDF itself, you
>>> may have to configure that part twice; or just skip the part about
>>> loading the wrong asdf). And try using install-asdf.lisp where
>>> applicable.
>>>
>>> Finally, please don't use the central-registry for cases like these.
>>> Use the source-registry.
>>>
>>> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
>>> Only a fool tests the depth of the water with both feet. — African proverb
>>
> 
>