[asdf-devel] [Sbcl-devel] Logical pathnames vs ASDF & SBCL

Mon Jun 13 06:46:52 UTC 2011

On 13 Jun 2011, at 06:29, Faré wrote:

> On 12 June 2011 11:49, Pascal Costanza <pc at p-cos.net> wrote:
>> Hi,
>> 
>> This is all very frustrating. With ASDF 1.x, everything worked fine and I have never encountered any issues. Since CL implementations "upgraded" to ASDF 2.x, things run into problems on a regular basis.
>> 
> I'm sorry you're running into problems. I could have done things
> better. Yet, I believe ASDF 2 is a clear improvement over ASDF 1. I
> also believe a lot of your problems come from your rash attitude in
> dealing with adversity.
> 
>> What is really annoying is that the ASDF 2.x documentation is misleading. *central-registry* is described as old-style and deprecated that is hopefully going away in the future, and logical pathnames are also described as old-style and better to be avoided. If I read the recent messages by Fare and James correctly, it turns out that if I had sticked to *central-registry* and switched to logical pathnames, everything would be fine and I wouldn't have had to waste time on a very frustrating experience.
>> 
> I still believe central-registry and logical pathnames are old style,
> since they necessitate implementation-dependent configuration (i.e.
> more buck) for results of more limited applicability (i.e. less bang).
> I am unwilling to dig into each implementation's details and document
> the 14 different ways to get things working this way.
> 
> For a less frustrating experience, you could communicate better and
> earlier about your problems.
> 
> For instance, you apparently switched to logical pathnames after you
> experienced a bug in the ASDF support for RMCL. If only you had
> reported that my previous series of fixes to RMCL support still didn't
> solve your issues, and kept pressing the issues, I would have provided
> a solution then, rather than when Binghe reported the issue again a
> few months afterwards.

Your reactions to bug reports are either null, or very discouraging. I usually only get fixes or workarounds by people other than you. Instead I repeatedly hear from you that either I am doing something wrong, or that what I'm trying to do is deprecated, or legacy, or old-style, or whatever. Or you don't react at all (like with my _bug_ report that the documentation is misleading, which is the main reason why I switched from a working setup to a setup that doesn't work). Or like here, you react by saying that I'm not reporting problems properly, or not early enough, or whatever.

I don't get such reactions from any other library/tool maintainer!

You either support a feature, or you don't. Don't do this mixed bag of "I support it, but you're a loser if you use it."

The main reason why I'm having problems with ASDF 2.x is because it's alpha-quality software at best (you're still trying to figure out some very basic concepts, it seems to me), yet you don't even call it beta quality, but instead choose to call it 2.x, push it down everybody's throats, and turn a whole community into alpha testers who didn't volunteer to be guinea pigs in the first place.

I have used ASDF 1.x and it's central registry approach for years in almost all CL implementations available, and I have never encountered any problems with it. I have also used logical pathnames to some limited extent years ago, and my experience with those weren't even remotely as bad as with ASDF 2.x. When using them recently, for a few weeks they actually made something work that ASDF 2.x claims to support, but actually didn't really, until ASDF screwed it up again.

I'm not sure how to proceed from here. Communicating with you and reporting bugs doesn't seem to buy anything. If I had the choice I would just drop ASDF 2.x and go back to ASDF 1.x, but this doesn't seem a viable option. Or is it?

Pascal

>> Please take one or two steps back and try to see the global picture: Logical pathnames and *central-registry* have their quirks and complications, that is correct. But all you're doing is to replace this with an alternative that has again their own quirks and complications. So overall you're not making things easier, you're just replacing one difficult-to-use system with another difficult-to-use system. This is not a win, not for anybody.
>> 
> I believe I made things easier: no more per-implementation
> configuration, no more attempts to badly documentation half of them.
> 
>> ASDF is not the central focus of my work. It's a tool that is supposed to make my life easier, not harder. I want to contribute to the Common Lisp community by providing my own libraries, not by fighting a build tool.
>> 
>> I will try to switch to *central-registry* again. It seems promising that this could actually work in the mid run and not cause any further problems down the line. Please make sure that it stays that way.
>> 
>> Best,
>> Pascal
>> 
>> On 12 Jun 2011, at 02:25, Faré wrote:
>> 
>>> TL;DR: ASDF currently doesn't play so well with logical pathnames.
>>> I'm seeking advice on how to make things better —
>>> or on whether it's worth the trouble.
>>> 
>>> 
>>> Dear all,
>>> 
>>> Pascal's recent plight with logical pathnames on SBCL
>>> once again rises the issue of how much ASDF (or SBCL)
>>> should support logical pathname users, and
>>> how they should behave to make logical pathname users happy.
>>> 
>>> With case-sensitivity issues, things only get worse!
>>> 
>>> (Note: personally inviting James Anderson to the discussion, since
>>> he's the only strong proponent and user of logical pathnames I know.)
>>> 
>>> 
>>> REMINDER ABOUT LOGICAL PATHNAMES (LPNs)
>>> 
>>> So, the CLHS says LPNs are case-converted to upper-case,
>>> portably restricted to such upper-case characters,
>>> and get translated to physical pathnames
>>> in an implementation-dependent way.
>>> On Unix, with its case-sensitive filesystems and lower-case conventions,
>>> the useful thing is then downcasing names.
>>> 
>>> At least SBCL and CLISP follow both the standard
>>> and the useful pathname mapping convention.
>>> SBCL goes on to length to throw an error
>>> when non-portable code tries to break
>>> the standard restrictions on logical-pathnames;
>>> CLISP will let you manually make-pathname a LPN
>>> with lower-case or mixed-case characters, but
>>> will still downcase them at the end
>>> (the latest CLISP 2.49 also seems to not want to create
>>> a logical pathname with parse-namestring).
>>> CCL tries to make users happy by leaving the logical pathname host
>>> case-insensitive (per the standard) but case-preserving,
>>> and being case-sensitive in the rest of the pathname
>>> (if you want upper-case, you know where to find it).
>>> To collect and analyze the behavior of each of 14 implementations
>>> is left as an exercise to the logical-pathname loving reader [M20].
>>> To devise, document and get implementations to adopt a common behavior
>>> that would be universally accepted by all is left as another exercise [50].
>>> 
>>> 
>>> ASDF NAMING OF SYSTEMS
>>> 
>>> Similarly, when naming an ASDF system, to allow
>>> to designate systems with symbols (CL legacy), which are case-converted,
>>> yet map them to a unixy lower-case name, and still
>>> allow systems with all kind of cases,
>>> symbols are downcased whereas strings preserved.
>>> Then the filesystem is queried with the resulting normalized string name
>>> (see function coerce-name).
>>> 
>>> Now with its good old central-registry,
>>> ASDF used to query each directory every time a system was requested.
>>> This works great with logical pathnames, as it defers
>>> all case handling to the last minute, and allows .asd files
>>> to be loaded with a LPN as the *load-pathname*.
>>> However, this was somewhat slow, and
>>> it didn't scale to recursively searching directories,
>>> and so came the habit of creating "link farms",
>>> single directories with plenty of symlinks to adequate .asd files.
>>> Of course, symlinks are both not so portable
>>> (not very well supported in Windows, if at all, though
>>> later versions of ASDF recognize Windows shortcuts as a substitute),
>>> and a maintenance nightmare (unlike Macintosh aliases,
>>> they are not automatically updated
>>> when a system is installed, moved or deleted).
>>> Therefore link farms were not a good interface to end-users.
>>> 
>>> For ASDF 2, I implemented a new way to manage system files:
>>> the source-registry. It allows you to specify trees to search recursively
>>> as well as single directories. Whether the registry is searched lazily
>>> (when a system is requested) or eagerly (one the first time around) or
>>> yet something else, was left unspecified. In my first implementation of it,
>>> I built upon the previous central-registry, and at initialization time
>>> I was looking for which subdirectories in the specified trees
>>> did contain .asd files, then saved these in a list that I consulted
>>> in the very same way as the central-registry.
>>> (Note: the central registry is still there, and consulted first.)
>>> 
>>> However, DIRECTORY returns fully resolved TRUENAMEs, and
>>> any portable way of searching inside subdirectories is doomed
>>> to squash away logical pathnames. Currently, any use of :tree
>>> in source-registry will resolve into physical pathnames (PPNs)
>>> on most implementations, and always did.
>>> 
>>> However, entries with :directory used to preserve LPNs.
>>> Not so with 2.014.7, where, prompted by a Quicklisp feature request,
>>> I implemented eager caching of ASDF systems. So I query DIRECTORY
>>> and save the results in a hash-table indexed by pathname-name.
>>> The table is case-sensitive, to reflect the case-sensitive names of ASDF.
>>> One consequence is that :directory entries do not preserve LPNs anymore.
>>> 
>>> Another more subtle consequence is that things may go wrong
>>> if your filesystem is case-insensitive but case-preserving.
>>> In the old way of doing things (still available with the central-registry),
>>> you'd typically be looking for the file with lower-case name,
>>> and you would find it even if named in different cases on disk.
>>> With the old way, you'll record the name in its case mix,
>>> and will never match it against the (lowercase) name.
>>> So some systems that depend on this behavior will disappear.
>>> 
>>> 
>>> POTENTIAL SOLUTIONS
>>> 
>>> An easy way out is to declare that ASDF from now on
>>> shall be case-insensitive, using EQUALP to compare system names
>>> and in hash-tables that cache system files.
>>> Another way out is to declare that ASDF is still case sensitive,
>>> and that the only case allowed is lower-case for physical filenames.
>>> Any system that doesn't work with this new interpretation
>>> probably didn't work portably with the old one, anyway.
>>> 
>>> A harder way out would be to assume case-sensitivity in the filesystem
>>> (since some filesystems are case-sensitive, and any asd file that depends
>>> on case-insensitivity is non-portable and deserves to lose),
>>> and make extra effort when using DIRECTORY against logical pathnames
>>> to always reconstitute LPNs from the PPNs given by directory,
>>> and drop the reconstituted results if they don't resolve to the same thing.
>>> But is it worth the pain?
>>> 
>>> Finally, I could do nothing, and assume that
>>> people who go through the pain of setting up logical pathnames
>>> before they run ASDF can just as well setup the *central-registry*
>>> and have LPNs be recognized in the good old way.
>>> If you use the source-registry, your LPNs will be squashed,
>>> and nothing will break in the code since code works fine with PPNs,
>>> just like most everyone use it, but your debug information
>>> will be stored according to the PPN, not the LPN. Meh.
>>> 
>>> Zach (if you're still reading), do any systems in the wild
>>> use anything but lower-case names for system files?
>>> 
>>> [ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
>>> Two members of the Political Police salute a giant portrait of our Great Leader
>>> - Comrade, do you think the same as I do? inquires the first one, with
>>> a sincere gaze pleeing into his colleagues' eyes.
>>> - Comrade, the second one replies and returns the same gaze: yes, I do.
>>> - In that case, comrade, you're under arrest! concludes the first one.
>>> 
>>> ------------------------------------------------------------------------------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Sbcl-devel mailing list
>>> Sbcl-devel at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/sbcl-devel
>> 
>> --
>> Pascal Costanza
>> 
>> 
>> 
>> 
> 
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Sbcl-devel mailing list
> Sbcl-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/sbcl-devel

--
Pascal Costanza