[asdf-devel] ASDF traverse changed behavior?

Wed Mar 17 21:57:13 UTC 2010

On Wed, Mar 17, 2010 at 9:46 PM, Robert Goldman <rpgoldman at sift.info> wrote:

> The inclusion of LIB-OP should open up
>  (defmethod PERFORM ((op LIB-OP) (c COMPONENT)) ...)
> to you.

> Summing up, please you all consider this: if you forbid --in the ASDF
> > specification-- to write any method that uses directly ASDF classes that
> > means we are not allowed to extend ASDF systems at all.
>
> I think the above is overboard.  It would be more helpful for you to
> work to provide a clean description of what is an acceptable API and
> what is not than to indulge in this kind of overstatement.
>

Wait Roger, seems you are angry because you get too much mail, but do not
patronize me or talk to me about indulging or whatever.

It has been explicitly written in this thread at some point that it should
be forbidden to allow writing methods over any class that ASDF defines. I
wanted to make sure that people understand the consequences of this.

Second, ECL's extensions are part of ASDF. It is not just a game we play to
be on this list or to ship ASDF with our system. It also sucks my
development time and prevents me from working on other useful things like
ECL's compiler, performance, etc, but I believe on the utility of the
project: being able to build and ship arbitrary ASDF systems as libraries,
executables and the like, and improving integration of ASDF with systems
that work better with monolithic binary files instead of thousands of small
compiled files. This is a path other implementations may follow, but no one
seems to care right now.

Third, just for the record as well, if ECL's extensions are allowed to be
part of ASDF then they should be allowed to have room for development and
there should be some consideration as how to integrate them with further
developments other maintainers do on the remaining components in ASDF. That
means perhaps advertising incompatible changes, better defining how to
integrate with or track the build process, etc, etc.

I can partially contribute to this with a test suite that tests ECL's
extensions together with ASDF. I can also provide an automated test farm if
it helps http://ecls.sourceforge.net/logs.html because I believe almost
nobody here uses ECL at all. Right now I do not know how to integrate this
with ASDF, but guidance is welcome

We cannot allow people to write arbitrary methods on arbitrary bits of
> ASDF --- the whole system would become unstable and unmaintainable.  So
> we MUST find a crisp way to characterize what extenders can and can't do.
>

I agree that a common ground should be found, but be careful with your
sentences.

This is not just 'people' and this is not just 'extenders'. Either you
accept this is part of ASDF and accept its growing user base, or not. If
asdf-ecl.lisp is part of ASDF then any maintainer should also care for what
those other components need and do, and what they are based on.

> Please focus your attention on helping us base define a clear set of
> guidelines rather than attacking the notion of limitations on the
> extenders.

What we do need is clear and self explained in the asdf-ecl.lisp extension
and I discussed this here long long ago. If this has to be done again, then
so it be.

asdf-ecl.lisp provides six new operations that have to integrate with ASDF
and that can be used instead of LOAD-OP or COMPILE-OP. These operations are
DLL-OP, LIB-OP, FASL-OP, monolithic versions (that is everything in one
file) and a PROGRAM-OP.

These operations proceed in a simple way: they use TRAVERSE with a LOAD-OP
to know what a system uses, and then they use PERFORM with a COMPILE-OP to
actually compile it and from the list that TRAVERSE creates they learn the
list of compiled files and build things with them (library, program,
whatever)

Integration with TRAVERSE is crucial and depending on how _you_ _all_ decide
that TRAVERSE should behave then we will have to adapt the code one way or
another. This integration is fragile. If at some point the algorithm is
randomly changed and you decide that systems should be listed before
components, then we are f*ck*d.

Until now our own operations listed no dependencies other than the LOAD-OP
and COMPILE-OP mentioned before. TRAVERSE was just happy with that. Right
now ASDF is adding additional dependencies such as performing the same
operations (DLL-OP, LIB-OP...) on the components itself. This was my source
of surprise when I started this thread.

asdf-ecl.lisp only defined methods on COMPONENT-DEPENDS-ON, INPUT-FILES,
OUTPUT-FILES, PERFORM and OPERATION-DONE-P. All the signatures we used
contain at least either OUR components our OUR operations, so in that sense
they should be allowed to exist and ASDF should be careful on deciding what
to do with these methods -- for instance, rewriting paths as we discussed on
another thread.

Note that the protocol for writing these new operations was never clear and
it has grown up with experience and by inspecting ASDF's codebase. At some
point the extensions were accepted in the source tree, point at which I
asked whether the integration that was done the right way or not. The lack
of answer was understood as an ok.

Right now you are claiming that this is not at all like that, that we cannot
hook at random functions in asdf because we make your system unmaintainable.
Does the protocol I explained above sound so random or is it a reasonable
path at asdf-ecl and other people willing to extend ASDF may follow?

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100317/7b162022/attachment.html>