[asdf-devel] source file encoding

[Faré]

On Thu, Mar 29, 2012 at 07:07, Douglas Crosher <dtc-asdf at scieneer.com> wrote:
> My suggestion is to just use :external-format plus some minimal translations for CLISP to help write portable code, for example
> :utf-8 and :iso-8859-1 or :latin1, which should cover most portable CL projects, and to make the default :default.
>
Well, :external-format is not quite portable, and any translation
layer means that
reusing the same name will lead to confusion.

> It is not possible to validate the external-format so the default action should be to pass it through.  With a default of passing it
> through, it would seem most appropriate to name it :external-format, and to call any processing a translation.
>
The issue is that the only qualified person to specify what the encoding is
is whoever writes (or repackages) the source code.

Pushing the responsibility back to whoever tries to use the source code
is the current situation, and just doesn't work so well.

> The external-format is implementation dependent and user extensible which making it impossible for ASDF to validate.  Requiring a
> custom ASDF hook to be installed seems an unnecessary burden with no utility.
>
But that's precisely the point: whoever packages the source
does not control which implementation is used, and
needs to specify the encoding in an implementation-independent way.
Sure, we could force packagers to use #. all over the place,
but it's more declarative and better overall
to move the translation to a hook in ASDF:
same work for whoever codes the translation library
up to trivial registering of hook,
and much less work for whoever uses it.

> CL implementations are required to recognise :default as an external-format,
> and could be expected to have a sensible default for their environment.
But the file encoding is inherent in the source code as distributed,
not in the implementation's environment.

> Any other default value is problematic as it may not be supported.
> If a project really needs UTF-8 source files
> then it would not appear to be a big burden to require this to be specified.

> A lot of portable code can be written without needing
> UTF-8 source files, even code that supports UNICODE.
>
Whenever people restrict themselves to ASCII, they are already
pretty much guaranteed to have their code work everywhere.
I'd like for a similar guarantee to be available to everyone
using some superset of ASCII; and UTF-8 is the obvious choice
for a superset of ASCII that is suitable for everyone.

Obviously, some legacy 8-bit only implementations just can't do it,
and that's where we'll use :default.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
What we Are is God's gift to us.
What we Become is our gift to God.