Handling of readme and license files

[Robert Goldman]

On 21 Oct 2022, at 15:53, Russell Sim wrote:

> Robert Goldman <rpgoldman at sift.info> writes:
>
>> I have been out of town and away from my email for a while. Sorry for
>> the late response. Checked ASDF code and I think you are right: it 
>> would
>> make sense to make `:read-file-form` more generally usable than just 
>> in
>> the `:version` property.
>>
>> Unfortunately, if we were to do this, it would cause new `defsystem`
>> forms to be backwards incompatible, which would be undesirable.
>
> I was thinking about this and I understand the need for backwards
> compatibility, I know that read time conditionals could be used to
> enable backwards compatibility `#+asdf-3.4` or something of that
> nature?

Yes, that is correct: I expect that we would add a new feature for such 
constructs.  See https://gitlab.common-lisp.net/asdf/asdf/-/issues/131
>
>> We could probably instead add a `:license-file` `defsystem` option. 
>> We
>> should think carefully about how this is to be done, though. Probably
>> this should *not* point to a file directly, but to a component
>> (generally a `:static-file`): that would be desirable because using a
>> component already provides a mechanism for specifying the location of 
>> a
>> source file.
>>
>> At this point, though, we are starting to get a structure that is so
>> fussy that it is likely to cause more problems than it fixes.
>
> I agree this sounds overly complicated for the user, introducing
> dependent fields i think is too much. Would it make sense to add a
> special :license-file component type like
> https://gitlab.com/daewok/asdf-release-ops introduces? or implicitly 
> add
> a `:static-file` when a `:license-file` is declared as a `defsystem`
> option?

The problem with implicitly adding a `:static-file` from a 
`:license-file` metadata item is that metadata doesn't have a way of 
allowing the programmer to specify the location of the file, the way 
that a component does (through `pathname` option or the use of modules).

Having a `:license-file` component type would work, but I hate to mix 
together program and meta-program that way.
>
> Supporting multiple license files is better for dual licensed 
> projects,
> so in some ways adding a new component type is the simplest.
>
>> I wonder if this is a technical problem at all. Maybe the problem is
>> just that developers are not putting the right static files into 
>> their
>> systems, and the issue is that our social practices need to be fixed,
>> not our code. If programmers should be putting license files in as
>> static files, maybe we should just be teaching them to do this.

I could have said "most of them don't even put version information in"!

>
> I think it's a combination of the 2, if the tool doesn't provide an
> easy/obvious way to do fill the `:long-description` field.  We are
> relying on the users to copy and paste good examples, or to try and
> write portable code.  I do agree it's a balance, and in a library like
> this it's important consider all options.  Some of this could be fixed
> by providing tooling and better examples.

Allowing `:read-file-form` to be used more widely would be a reasonable 
extension, although it won't be backward compatible.
>
>
> This discussion has given me some good ideas.  Like you say, it's 
> partly
> a social practices issue, which makes me think that tooling like a
> linter would help.  But read time evaluation makes that hard to
> implement.  If the need for that could be removed somehow, that would
> improve things significantly.  One thing that can be seen from the
> `:version`, `:read-file-form` option usage below is that if there is a
> better way to read a file, people will use it.
>
> Cheers,
> Russell
>
>
> --------------------
> Below are some numbers to gauge usage of existing fields and features 
> of
> ASDF.  I have gone and pulled all the systems from Quicklisp so we can
> look at the current usage.
>
> So looking at 4921 asd files found in quicklisp (NB, this total isn't
> perfect there a few minor duplicates, ~10 due to locally having 
> multiple
> versions of some packages) But the rest of the results I manually
> verified.
>
> 70 asd files currently declare a license via a `:static-file` 
> component.
> 1961 asd files currently declare a `:license` option
>
> 76 asd files that declare the readme as either a `:static-file` or a
> `:doc-file` (6 use the `:doc-file` declaration) There is some overlap
> with the following libraries, but not all, so some people are 
> correctly
> declaring a `:static-file` and using read time evaluation.  Sorry I 
> was
> too lazy to calculate the set difference.
>
> 130 systems using read-file-string
> -----------
>   :long-description #.(uiop:read-file-string
>                        (uiop:subpathname *load-pathname* 
> "README.org"))
> -----------
>
> 95 asd files using the `with-open-file` similar to this
> -----------
>   :long-description
>   #.(with-open-file (stream (merge-pathnames
>                              #p"README.markdown"
>                              (or *load-pathname* 
> *compile-file-pathname*))
>                             :if-does-not-exist nil
>                             :direction :input)
>       (when stream
>         (let ((seq (make-array (file-length stream)
>                                :element-type 'character
>                                :fill-pointer t)))
>           (setf (fill-pointer seq) (read-sequence seq stream))
>           seq)))
> -----------
>
> Regarding general use of read time evaluation, there are 7 total
> instances where users are incorrectly using `with-open-file` to fill 
> the
> `:version` defsystem option from a file instead of using
> `:read-file-form` or equivalent line option.
>
> There are ~258 correct usages of the `:version (:read-file-form`
> expression.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20221021/10068f8f/attachment.html>