Incremental defsystem dependencies

[Faré]

Dear Stelian, Dear Lispers,

in Gitlab MR !37 I try to address some fundamental bugs regarding the
proper treatment of incrementality for defsystem dependencies. See for
instance the test I just added, that still fails at this time,
test-defsystem-depends-on-change.script.
https://gitlab.common-lisp.net/asdf/asdf/merge_requests/37

Now, the good news is that it's a generally understood problem,
analogous to proper phase separation of macro evaluation in e.g.
Racket, or of proper stratification of dynamic computation graphs in
reactive programming: "just" have an operation define-op for every
primary-system, that can represent in the graph the definition of
other computations, and its dependencies. Many current open ASDF bugs
can be fixed, and ASDF will correctly handle the incremental
rebuilding of systems when a defsystem-depends-on dependency is out of
date.

The bad news is that it becomes even harder than before to support a
system that has defsystem-depends-on on another system defined in the
very same file, a feature currently relied upon by iolib.

Indeed, if every reference to a system by operate now triggers a
dependency on define-op, then loading a file already being read while
defining a system triggers the circular dependency detector. To solve
that while preserving some semblance of atomicity for the perform
seems hard. Stelian had a proposal to defer the evaluation of the
defsystem forms, which could help if done well, but itself is not a
trivial change, and might not help that much when methods are defined
on (eql (find-system ...)). We'll see.

Stelian, how sad would you be if you have to segregate
defsystem-depends-on to refer to systems in separate files?

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Malthusianism: because growth may stop within centuries, we must use
extreme violence to put an end to it now and make everyone poor today.