Define a Simple Echo-Op
Robert Goldman
rpgoldman at sift.net
Wed Apr 27 15:26:55 UTC 2022
Phoebe Goldman writes (but can't post):
For some frustrating reason I can't email the asdf-devel list, so cc'ing rpgoldman & he can forward.
> Why can't I subclass ASDF:OPERATION? This is unexpected since
> ASDF:OPERATION is the base class for all operations.
The various UPWARD- DOWNWARD- etc -OPERATION classes are subclasses of OPERATION. You can explicitly add OPERATION to your superclasses list alongside SIDEWAY-OPERATION if you want to, but it's redundant.
In ASDF 2, the direction-OPERATION classes didn't exist; you just subclassed OPERATION directly. All operations were implicitly DOWNWARD and SIDEWAY unless you wrote your own COMPONENT-DEPENDS-ON method. ASDF 3 added some built-in operations that were not SIDEWAY/DOWNWARD (PREPARE-OP is UPWARD, for example), so Faré codified all the sane dependency relationships as the direction-OPERATION classes, and deprecated the default COMPONENT-DEPENDS-ON method for OPERATION. In ASDF 3, if you subclass OPERATION directly and don't define a method on COMPONENT-DEPENDS-ON, you get a deprecation warning.
> I'm following the manual: "If the action of performing the operation on a component has
> dependencies, you must define a method on component-depends-on."
This is wrong; I don't know why the manual says that. You still can define a method on COMPONENT-DEPENDS-ON, but there's no reason to unless you have a really weird operation. Writing your own COMPONENT-DEPENDS-ON methods is error-prone and a pain in the ass, so it's much easier to subclass one of the direction-DEPENDS-ON methods and get a built-in method that does the right thing.
> But still I don't get why echo-op depends on itself. I didn't specify it
> anyway. Or it that the default behaviour for all *-operation classes?
Honestly, I have no clue why there's an (ECHO-OP . NIL) pair at the head of the COMPONENT-DEPENDS-ON list for an ECHO-OP. But MAP-DIRECT-DEPENDENCIES ignores it. Shrug.
> "A method for [INPUT-FILES] is often not needed, since ASDF has a pretty
> clever default input-files mechanism."
Again, I don't know why the manual says that. You don't need to define INPUT-FILES methods for built-in operations, but you will need to define them for custom operations that take files as input. Most user-defined operations subclass either COMPILE-OP or LOAD-OP, which already have their INPUT-FILES (and OUTPUT-FILES) methods, but I've had to define INPUT-FILES methods in the past.
> Is there a way to obtain the pathname to (:FILE "foo") from the
> CL-SOURCE-FILE object like what COMPILE-OP does?
COMPONENT-PATHNAME.
> Why do I get nothing for my input-files? I'm expecting it to print out
> the pathname to (:FILE "foo") object, very much like the input files
> into COMPILE-OP.
Taking the COMPONENT-PATHNAME as the sole INPUT-FILE can't be the default, because lots of operations do something different. If you think of PREPARE-OP, COMPILE-OP and LOAD-OP as being the "normal" operations on a LISP-SOURCE-FILE, only COMPILE-OP takes the source file as an INPUT-FILE. LOAD-OP takes as an INPUT-FILE the compiled fasl, which is the OUTPUT-FILES of COMPILE-OP. I don't think PREPARE-OP takes any INPUT-FILES at all, but I could be wrong.
> One question: How do you know whether to define PERFORM primary method
> or an :AROUND method for the custom operation class? It's not obvious to
> me which one to choose for different use cases.
Write an :AROUND method if you're subclassing an existing operation like LOAD-OP to modify its behavior, and you intend to use CALL-NEXT-METHOD to invoke the default behavior for your superclass at some point in your method. This is useful if you want to invoke the next method in a dynamic context e.g. with handlers or restarts bound, or with local bindings of special variables; or if you want to conditionally invoke the next method sometimes but not always. Actually, you can do all of these things in a primary method via CALL-NEXT-METHOD, so now that I think of it, just define a primary method. Write an :AROUND method only if you're defining a mix-in class that other people will subclass which wants to do those things.
If you're subclassing an existing operation and you want to add additional behavior, but you still want the default behavior to run unconditionally in a default dynamic context, define a :BEFORE or :AFTER method as appropriate.
Write a primary method if you're defining a new method, or if you want to completely override the behavior of an existing operation. Or if you want to do the things I said earlier with CALL-NEXT-METHOD.
When in doubt, write a primary method.
cheers,
phoebe
> On Apr 27, 2022, at 3:21 AM, zacque <technical+asdfdevel at zacque.tk> wrote:
>
>
> Hi phoebe,
>
> Thanks for your thoughtful reply!
>
>> A few things.
>>
>> 1. ECHO-OP should not be selfward. SELFWARD-OPERATION is for (OPERATION COMPONENT) pairs which depend on
>> (DIFFERENT-OPERATION COMPONENT) for the same COMPONENT. For example, LOAD-OP is selfward with respect to
>> COMPILE-OP, because in order to perform (load-op FILE), you must first perform (compile-op FILE). Your ECHO-OP has no such
>> dependency. In this case, I believe you want ECHO-OP to be downward and sideways, meaning that (ECHO-OP MODULE)
>> depends on (ECHO-OP MODULE-COMPONENT) for each of the children MODULE-COMPONENTs of the MODULE, and that
>> (ECHO-OP SOURCE-FILE) depends on (ECHO-OP EARLIER-SOURCE-FILE) for each of the EARLIER-SOURCE-FILEs in
>> SOURCE-FILE's :DEPENDS-ON list. This way, when you call (OPERATE 'ECHO-OP (FIND-SYSTEM "whatever")), ASDF will do a
>> depth-first dependency-order traversal of your system.
>
> After playing a little with downward and sideway, I can observe that
> downward-operation provides a depth-search/"postorder" traversal for the
> current system; sideway-operation provides a dependency-first traversal
> for dependent systems.
>
> * Note:
> Use :force all with OPERATE to force reloading all dependent
> systems. E.g. (asdf:operate 'echo-op:echo-op :echo-op-test :force :all)
>
> * Question:
> Why can't I subclass ASDF:OPERATION? This is unexpected since
> ASDF:OPERATION is the base class for all operations.
>
> My code:
> ----------- file: echo-op.lisp -------------------
> (in-package #:echo-op)
>
> (defclass echo-op (asdf:operation) ())
>
> (defmethod asdf:perform ((op echo-op) c)
> (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%"
> op c (asdf:input-files op c)))
> -------------------------------------------------
>
> Output:
> ----------------- REPL --------------------------
> CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t)
> WARNING:
> DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON
> :FOR-OPERATION
> #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API.
> WARNING:
> DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON
> :FOR-OPERATION
> #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API.
> WARNING:
> DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON
> :FOR-OPERATION
> #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API.
> WARNING:
> DEPRECATED-FUNCTION-WARNING: Using deprecated function (ASDF/ACTION::BACKWARD-COMPATIBLE-DEPENDS-ON
> :FOR-OPERATION
> #<ECHO-OP:ECHO-OP >) -- please update your code to use a newer API.
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> has input files:
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> has input files:
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> has input files:
> Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> has input files:
> #<ECHO-OP:ECHO-OP >
> #<ASDF/PLAN:SEQUENTIAL-PLAN {1015FB0BC3}>
> -------------------------------------------------
>
>> 2. Your COMPONENT-DEPENDS-ON method is wrong. No pair of (OPERATION COMPONENT) should ever depend on the same
>> (OPERATION COMPONENT). What you're saying is, "in order to perform (ECHO-OP FILE), you must first perform (ECHO-OP
>> FILE)."
>>
>> 3. For operations that subclass one or more of DOWNWARD- UPWARD- SIDEWAY- SELFWARD- or
>> NON-PROPOGATING-OPERATION, you don't need to define a COMPONENT-DEPENDS-ON method.
>
> Why is my COMPONENT-DEPENDS-ON method wrong? It's not obvious that one
> doesn't need to define a COMPONENT-DEPENDS-ON method for >=1
> *-OPERATION. Is it stated anywhere?
>
> I'm following the manual: "If the action of performing the operation on a component has
> dependencies, you must define a method on component-depends-on." and following the examples in cffi-grovel: https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/grovel/asdf.lisp#L66 <https://github.com/cffi/cffi/blob/3c76afe7ba03ce015e0df99ac9ddcd61320a44a4/grovel/asdf.lisp#L66>
>
>
> That said, my code works fine by removing my COMPONENT-DEPENDS-ON
> method. And I can see what you meant by (ECHO-OP FILE) depending on
> itself by printing it out:
>
> My code:
> ----------- file: echo-op.lisp -------------------
> (in-package #:echo-op)
>
> (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
>
> (defmethod asdf:perform ((op echo-op) c)
> (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%"
> op c (asdf:component-depends-on op c)))
> -------------------------------------------------
>
> Output:
> ----------------- REPL --------------------------
> CL-USER> (asdf:operate 'echo-op:echo-op :echo-op-test :force t)
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "package"> depends on
> (#<ECHO-OP >)
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "example"> depends on
> (#<ECHO-OP >)
> Operation #<ECHO-OP > on component #<CL-SOURCE-FILE "echo-op-test" "main"> depends on
> (#<ECHO-OP >)
> Operation #<ECHO-OP > on component #<SYSTEM "echo-op-test"> depends on
> (#<ECHO-OP >)
> (#<ECHO-OP > #<CL-SOURCE-FILE "echo-op-test" "package">
> #<CL-SOURCE-FILE "echo-op-test" "example">
> #<CL-SOURCE-FILE "echo-op-test" "main">)
> (DEFINE-OP echo-op-test)
> #<ECHO-OP:ECHO-OP >
> #<ASDF/PLAN:SEQUENTIAL-PLAN {10162397C3}>
> -------------------------------------------------
>
> But still I don't get why echo-op depends on itself. I didn't specify it
> anyway. Or it that the default behaviour for all *-operation classes?
>
>> 4. Most (OPERATION COMPONENT) pairs have very uninteresting sets of input files. (COMPILE-OP CL-SOURCE-FILE) has one
>> input file, the .lisp source file. (LOAD-OP CL-SOURCE-FILE) has one input file, the .fasl compiled file. (ECHO-OP CL-SOURCE-FILE)
>> will have no input files at all, unless you define a method on INPUT-FILES to list them.
>
> Why do I get nothing for my input-files? I'm expecting it to print out
> the pathname to (:FILE "foo") object, very much like the input files
> into COMPILE-OP.
>
> Again, I thought ASDF is smart enough to infer it for me? To quote the manual:
>
> "A method for this function is often not needed, since ASDF has a pretty
> clever default input-files mechanism."
>
> Is there a way to obtain the pathname to (:FILE "foo") from the
> CL-SOURCE-FILE object like what COMPILE-OP does?
>
>> I think the following definition of ECHO-OP might be enlightening to you:
>>
>> (uiop:define-package :echo-op
>> (:use :cl)
>> (:export #:echo-op))
>> (in-package :echo-op)
>>
>> (defclass echo-op (asdf:sideway-operation asdf:downward-operation) ())
>>
>> (defun print-input-files (op c)
>> (format t "~&Operation ~a on component ~a has input files:~{~% ~a~}~%"
>> op c (asdf:input-files op c)))
>>
>> (defun print-dependencies (op c)
>> (format t "~&Operation ~a on component ~a depends on ~{~% ~a~}~%"
>> op c (asdf:component-depends-on op c)))
>>
>> (defmethod asdf:perform ((op echo-op) c)
>> (flet ((do-operations (thunk)
>> (funcall thunk op c)
>> (funcall thunk (asdf:make-operation 'asdf:compile-op) c)
>> (funcall thunk (asdf:make-operation 'asdf:load-op) c)))
>> (format t "~&~%Input files for component ~a with a variety of operations:~%~%" c)
>> (do-operations #'print-input-files)
>> (format t "~&~%Dependencies for component ~a with a variety of operations:~%~%" c)
>> (do-operations #'print-dependencies)))
>>
>> Note that:
>> 1. The only method I have defined is on PERFORM, and it is a primary method, not an :AROUND method. ASDF already has all
>> the COMPONENT-DEPENDS-ON methods I need.
>> 2. I print the COMPONENT-DEPENDS-ON list in addition to the INPUT-FILES list.
>> 3. I print both the COMPONENT-DEPENDS-ON and INPUT-FILES lists for all three of ECHO-OP, COMPILE-OP and LOAD-OP.
>>
>> I recommend you load this version, try (ASDF:OPERATE 'ECHO-OP:ECHO-OP (ASDF:FIND-SYSTEM "echo-op-test") :FORCE T) and
>> see what output you get.
>
> Thanks! It's indeed very enlightening. And I've separated it out into
> few parts to play with, as shown in the code snippet I pasted above.
>
> One question: How do you know whether to define PERFORM primary method
> or an :AROUND method for the custom operation class? It's not obvious to
> me which one to choose for different use cases.
>
> Thanks!
>
> --
> Regards,
> zacque
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 265 bytes
Desc: OpenPGP digital signature
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20220427/081b1cff/attachment.sig>
More information about the asdf-devel
mailing list