[asdf-devel] new asdf does not like matlisp.asd

[Robert Goldman]

On 4/11/10 Apr 11 -10:28 AM, james anderson wrote:
> good afternoon;
> 
> reading traverse is not my idea of a good time, but when i persevere,  
> i do not arrive at the same conclusion.

Here is the logic as I understand it.  I will cite line numbers.  I wish
I knew how to do this reliably --- I can only say that these numbers
work for me as of this writing (git pull done).

Jargon:
I think of this as a tree walk, so use the term "node" and "component"
interchangeably.
Plan = return value of TRAVERSE; an ordered sequences (specifically a
list) of Planops
Planop = a piece of a plan:  a dotted pair whose car is an operation and
whose cdr is a component.

1.  L1400:  make sure we don't visit the same nodes in the tree > 1.  If
we have already visited this, skip.  A pruned-op may be returned.  Check
for circular dependencies, and mark this node (component) as in progress
(visiting).

2.  loop at 1414: check all of the current nodes dependencies.  This
loop side-effects onto the local variable FORCED.  FORCED is going to
accumulate all the operations needed for this sub-plan (i.e., the part
of the TRAVERSE plan that is generated from the current component).

3.  1421:  block that is run only for the benefit of components of type
MODULE (which includes SYSTEMs).  Note that at the top we bind a number
of variables, including the FORCED special, which is the only way to
permit us to call do-one-dep here [someday, in my fantasy world,
traverse will be rewritten to be purely functional and not side-effect
onto FORCED everywhere.].  As a result of running the code in this
block, we will bind the MODULE-OPS lexical variable, which tells us what
operations need to be performed on the current module.

   3.1  1428: MUST-OPERATE is bound to tell us whether the components of
the current module must be operated on.  The check is whether there is
*FORCING* bound around us, or if the component is not a system.  This is
my attempt to see if the component IS a module simpliciter.  [In my
ideal world, ASDF would be refactored so that there are three classes
--- HAS-COMPONENTS, MODULE and SYSTEM, and MODULE isa HAS-COMPONENTS and
SYSTEM isa HAS-COMPONENTS but SYSTEM is no longer a sub-class of
MODULE...  Currently there is no really reliable way to tell that
something is "just" an internal module, since the class hierarchy is
mutable...]:
  (and (not (typep c 'system)) forced)
recall that FORCED was bound earlier to the set of planops that must be
performed on this node's dependencies.

This was done for backward compatibility, and because we don't have good
tests for a system being operated on.  Space does not permit me to
explain this here.

Note that there is no check here for OPERATION-DONE-P on the module
itself.  That test is performed downstream.

  3.2  1435:  Foreach of the module's components, bind *FORCING* to
MUST-OPERATE.  This ensures that all the modules components will be
operated on if any of the module's dependencies have changed.  Then
traverse each of the component's children, checking for component
dependency failures.

4.  1456:  If we have a module as our node, then we are done here
operating on the module's components and are now working on the module
object proper.  So you see what I have been saying about the module
object representing, at one and the same time, both the module as thing
with components AND any post-operations on the module, after the
operation has been done on its components (postorder tree traversal).

   4.1 Do we need to operate on the object?  OR:
       1456: are there upstream operations done (FORCED) or were
operations done on this components children (if any --- MODULE-OPS)?
       1457: are all operations forced (*FORCING*)?
       1458: is this operation as yet not done (OPERATION-DONE-P)?  As
you see, the OPERATION-DONE-P test on a module will be performed AFTER
working on its children in step 3.  Note that *it must be this way* for
backward compatibility.  This is a consequence of the double-meaning of
MODULE.
       1459 - 1470: A test that I do not understand and was too
frightened of to remove or rewrite.
       1471-1473: The op x component's DO-FIRSTs are processed.
       1475-7:  accumulate the return values.
       1478-9:  update the visiting logic

5.  Mark the node as visited and return the sub-plan.

I think this analysis bears out my claim.  I can't pretend to respond to
your experience with this particular system; I can't make MATLISPBUG run
on my linux box.  Currently, if I try to do load-system on it a second
time, it crashes my lisp hard, taking down SLIME.  I am updating SLIME
and SBCL now, and will try again when I have time.

Hope this analysis helps.  Fare, maybe it would be worth writing an
article about ASDF and the ASDF2-ing for ILC.  This reconstruction might
be an important part of such an article.

best,
r