[asdf-devel] Guidelines for ASDF system authors

[Felix Lange]

Hi,

I've been thinking a lot about the structure of Common Lisp projects lately
and would like to turn those thoughts into a public discussion.

I believe that some written guidelines for ASDF system authors could really
improve how the Common Lisp ecosystem functions as a whole.

The documentation on the ASDF website is comprehensive with respect to
the capabilities of its implementation, but there are few
guidelines on how developers are supposed to structure their systems.
I perceive a need for a manual chapter or document that consolidates
best practices advice and somehow documents the `intended' use of ASDFs
countless features. This information is currently scattered all over the
ASDF manual. Getting the big picture is possible but kind of difficult.

The new section `Extensions' on the ASDF project website as well as the
FAQ section in the manual are very much a step in the right direction.

Some of the questions that I think need a documented solution
which could also act as a community convention:

- How to properly conditionalize on *FEATURES*.

  This is touched briefly in manual section 6.2.1.3 `Required features'.
  There's a note explaining that excluding components through the #+
  syntax is a bad idea, and that the correct way of doing it
  i.e. (:file "foo" :depends-on (:feature :sbcl)) does not work at this time.
  In the meantime, Nikodemus has written MADEIRA-PORT for this exact purpose.

- Implementing TEST-OP

  The manual tells me that `it has proven difficult to define how the
  test operation should signal its results' and that TEST-OP does nothing
  by default.
  Current practice seems to be defining TEST-OP as a method that
  loads another system and then somehow executes all tests, failing
  the operation when some test did not pass. This is noted in the FAQ section
  and should really be the official way.

- How to specify optional dependencies in the `correct' way?

  Suppose I'm working on a library that can use either IRONCLAD or CL+SSL
  to generate random bytes using a strong crypto algorithm.
  I don't want to clutter the final application image with both crypto
  libraries but rather use whatever happens to be the application
  developer's choice.
  
  I have several options to do this.

  1) Use :WEAKLY-DEPENDS-ON to rely on the application's transient dependencies
     to load either one. This is obviously a nice solution, but
     I have to document the requirement of loading at least one of them
     in the README and figure out which one to use at runtime.
  2) Define two systems, MY-LIBRARY-IRONCLAD and MY-LIBRARY-CL+SSL.
     This would force the user to require one of them in order to use the
     library. I also need to document this, but do not have to switch
     at runtime.
  3) Solve the problem outside of ASDF

- How much `formalization' of the build process pays off in the long run?

  ASDF could obviously have more uses if people followed a common guideline.
  What's the advantage of custom component classes vs. a custom
  operation that is hooked into COMPILE-OP via :IN-ORDER-TO?
  What's used by existing systems?

- Whether or not to define a package for the system definition
  
  The documentation uses (IN-PACKAGE :ASDF) in section 5.1.
  Common practice seems to be defining a package inside the system
  definition file if the system uses custom component classes or
  features that require toplevel forms other than ASDF:DEFSYSTEM.
  :DEFSYSTEM-DEPENDS-ON is also an option.

- What are some good practices when building applications with ASDF?

  As the manual points out, `ASDF presents three faces'.
  People who want to use the ASDF for applications have very different
  needs from library developers and will often want tight control
  over their dependency graph.
  
  Are versioned dependencies usable and practical?
  How can I override decisions made by library authors in my
  application's build process?
  What's a good way to prevent package name conflicts?
  
  Some `war stories' from commercial application development
  could really help others arrive at a good solution quickly.

I see that `style guide for .asd files' is a TODO item in the manual,
so it seems some of those question are already being addressed.
I am willing to do documentation work as soon as I understand people's
intentions.

Looking forward to your point of view,

Felix