[asdf-devel] ASDF traverse changed behavior?

[Robert Goldman]

On 3/17/10 Mar 17 -5:54 PM, Juan Jose Garcia-Ripoll wrote:
> On Wed, Mar 17, 2010 at 11:28 PM, Robert Goldman <rpgoldman at sift.info
> <mailto:rpgoldman at sift.info>> wrote:
> 
>     For example, if I'm writing the ASDF
>     definition for my FLOYD-WARSHALL transitive closure library, I can't be
>     putting into it
> 
>     (defmethod PERFORM ((op load-op) (c cl-source-file))
>      ....)
> 
>     because that means that anyone who loads my library will be loading it
>     into a radically different environment.
> 
> 
> Understood. Then the general restriction would be that at least one of
> the classes in the method should be proprietary to allow the user to
> write such methods. Does this seem reasonable?

Correct.

And I think that you point out an important corollary to this --- we
need to figure out what sorts of things the ASDF core is allowed to do.

For example, we should avoid method definitions with signatures like

(T <component-type>)

because these are likely to cause problems downstream when people try to
extend them, because of multiple argument dispatch.

I'm hoping that we can come up with some form of specification that is
more terse and general than simply specifying what we are going to do
for each generic function that we export.

Perhaps this will emerge from trying to write a manual section on
writing one's own components and operations.


> 
>     > Integration with TRAVERSE is crucial and depending on how _you_ _all_
>     > decide that TRAVERSE should behave then we will have to adapt the code
>     > one way or another. This integration is fragile. If at some point the
>     > algorithm is randomly changed and you decide that systems should be
>     > listed before components, then we are f*ck*d.
> 
>     Can you say what exactly has goofed this up for you?  Note that TRAVERSE
>     is not deterministic --- it's possible to specify multiple different
>     orderings that satisfy the dependency specification in the ASDF file.
> 
> 
> Oh, there is nothing with TRAVERSE's output _right now_. It has bitten
> us hard in the past, though and I was just mentioning for the record, to
> give a complete overview of how this kind of operations bind to the ASDF
> core: these are operations rely on traversing a system and performing
> things on them, a task that can be very useful for many other purposes,
> such as grovelling information from libraries (kind of TAGs files),
> metaprogramming, etc. All these things should be doable with ASDF if the
> API is fixed with a long term scope.
>  
> 
>     > asdf-ecl.lisp only defined methods on COMPONENT-DEPENDS-ON,
>     INPUT-FILES,
>     > OUTPUT-FILES, PERFORM and OPERATION-DONE-P. All the signatures we used
>     > contain at least either OUR components our OUR operations, so in that
>     > sense they should be allowed to exist and ASDF should be careful on
>     > deciding what to do with these methods -- for instance, rewriting
>     paths
>     > as we discussed on another thread.
> 
>     I thought you were specifically referring to core methods.  I see that I
>     was not fully understanding.
> 
> 
> Ok, so then we may consider that these are non-volatile definitions,
> which is great because as I said this is basically all one really needs
> to build things on top of ASDF!

I think so, but see my notes up above.  I don't know how to handle cases
like

someone wants to write a method for (MY-OP T) but the core wants to
write methods for (T COMPONENT-CLASS).  How do we avoid or arbitrate
these cases?  Does anyone have a simple rule to apply here?

>  
> 
>     > At some point the extensions were accepted in the source tree,
>     point at
>     > which I asked whether the integration that was done the right way or
>     > not. The lack of answer was understood as an ok.
> 
>     This exhibits two problems ASDF development.  The underlying problem is
>     that people are doing this work on very thin time slices.
> 
>     1.  Lack of answer can never effectively be treated as "ok" --- lack of
>     an answer is more likely to simply mean "I haven't had time to look at
>     this."  My changes to TRAVERSE went in a very long time ago, and only
>     after a very long discussion in the mailing list.  It's clear that you
>     didn't have time to review them then, and so an issue that we thought
>     was closed will have to be reopened.
> 
> 
> I miss a lot of emails when the subjects are not directly related. On
> reading the archive of emails I see what you were discussing -- to me it
> just looked like a minor detail in dependency binding and nothing that
> would change the logic of TRAVERSE.

Right.  And I missed a lot of your ECL discussions for the same reasons.
>  
> 
>     2.  The documentation is badly bit-rotted.  The section on extending the
>     ASDF object model is in particularly bad shape.  It was never complete,
>     as far as I can tell, and it has not been updated.  We badly need a
>     protocol specification stating exactly what has to be defined to make
>     new components and operations.  I can't say that I fully understand how
>     to do this --- my experiments have often seemed to go just fine, then
>     led to me stumbling much later on a problem that came up because some
>     critical method was missing.
> 
> 
> I must say I am basically in the same position. That is why I ask so
> often and may sound sometimes like nagging. Seems we are all a bit
> insecure on how ASDF should behave, maybe two things would help -- one
> would be writing down a description of what the main function, TRAVERSE,
> does, another one is perhaps enlarging the set of tests so that we all
> feel safer about experimentation.

I would actually like to have an abstract description of the task.  That
seems preferable to describing what TRAVERSE does, since I think the
task would leave us free to implement TRAVERSE in different ways, and
have users just know "we will invoke perform on all these nodes, under
these circumstances," where "these circumstances" are dictated by
dependencies and the state of the file system.

Also, I think the current TRAVERSE is somewhat broken with respect to
the abstract problem description.  Some early decisions, reasonable at
the time, seem to me to have trapped us in unfortunate ways.

I've actually been trying to sketch out an abstract description of ASDF
as an AI planning problem, but I keep being side-tracked by the need to
work for money!

Best,
R

> 
> Juanjo
> 
> -- 
> Instituto de Física Fundamental, CSIC
> c/ Serrano, 113b, Madrid 28006 (Spain)
> http://tream.dreamhosters.com