[asdf-devel] source file encoding

[Douglas Crosher]

It may be significant that a number of the quicklisp releases use non-ascii in the system definition files.  Can this be addresses
in ASDF alone?   Should an attempt be made to add an encoding argument to 'find-system, and to have quicklisp record the encoding in
its release database and use this when calling 'find-system?  If so then perhaps this could be stored as a default encoding for a
system.

Looking at non-ascii usage in quicklisp releases shows that the UTF-8 usage is not that significant.

Releases considered: 716
Releases with UTF-8 lisp source files:  86  (12%)
Releases with UTF-8 in comments only :  34
Releases using UTF-8 in their system definitions: 21
Releases for which all the UTF-8 could be recoded to ISO-8859-1:  59
Releases with other non-ascii source files:  21  (3%)
Releases with other non-ascii source files in comments only: 12

Releases using non-ascii characters from only the ISO-8859-1 set: 59 + 12? = 71? (10%)

Releases using only ASCII in source files: 716 - 86 - 21 = 609 (85%)

Some of the UTF-8 is rather gratuitous and if portability was a concert there would have been suitable ASCII substitutes.  There
does not appear to be much respect for portability in some of these releases, so even adding encoding support to ASDF system
definitions files many not help for some of these releases.

If you accept that library authors will choose their encoding, even for the system definition files, then the only solution seems to
be to add an encoding option to 'find-system and suggest this be used to load the system definition.

Regards
Douglas Crosher


On 04/12/2012 12:43 AM, Faré wrote:
>>> No. Library authors have *already* largely adopted UTF-8.
>>> See previous analysis by Orivej Desh:
>>>       "I did a ckeck of quicklisp systems.
>>>         There are 263 lisp files in 107 systems which assume non-ASCII,
>>>         and only 31 of them in 20 systems assume non-UTF-8"
>>
>> I saw those statistics.  I have no idea what "assume non-ASCII" means.
>> That there are files that have non-ascii characters in them?  And that
>> only 31 files are not in utf-8 already?
>>
> Yes, of which only 13 files were actually managed by ASDF as opposed
> to examples, one is a MCL-only file that doesn't support UTF-8 anyway,
> two have already been fixed, and the rest are only latin1 or such in
> comments. Bugs filed for all the other systems (but no response so
> far).
> 
> IOW, I believe we're mostly arguing about a non-issue.
> 
> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
> 
> _______________________________________________
> asdf-devel mailing list
> asdf-devel at common-lisp.net
> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel
>