[asdf-devel] SBCL port of ECL's extensions

[Juan Jose Garcia-Ripoll]

Now to the point.

ASDF right now has two modes of operation. One is "building" the other one
is "loading". User defined operations belong to one or another class.

* compile-op is clearly a "building" operation, for it takes lisp files and
compiles them
* load-op is clearly a "loading" operation: it just loads a compiled file
* load-source-op just as well.
* cffi-grovel is a "building" operation: it takes some header and produces a
lisp file
* a ficticious ffi-dlopen would be a "loading" operation, installing a
shared library in memory.

One would be tempted to classify these operations according to the value of
OUTPUT-FILES. If it is NIL, then is a "builder", otherwise it is a "loader".
However this is not always the case as ASDF has not imposed that restriction
so far.

A simpler way to differentiate "builders" and "loaders" is the one I
mentioned before: the two lisp images process, which is close to xvcb
philosophy
1 Start a clean lisp image.
2 Apply a (asdf:oos 'load-op :target-system)
3 Quit the image
4 Start another clean lisp image
5 Apply (asdf:oos 'load-op :target-system)
The second pass will give a set of operations that is just "loaders", for
the first phase created all files that the lisp needs: shared libraries,
etc, etc

We thus all agree that a standalone system consists only of "loaders": from
the second lisp image I could dump whatever is needed and produce a fresh
and ready lisp image.

The same concept can be applied to binary distributed, standalone
replacements for an ASDF system. If in step "5" I identify all the
operations that are done on a given system I can identify all "loaders" that
relate to that particular system.

The only missing ingredient would be identifying that the system needs, such
as "resources" that are neither produced nor actually related to an ASDF
operation, because currently there is no need for that. This can be left for
a second stage, an ASDF 2.1 if you wish, for 3.0 looks too far and more like
an excuse to forget things. In that ASDF 2.1 we can add another component
such as

(:resource "my-bitmap-file" :type "bmp")

and force that component have always OUTPUT-FILES the file itself. Actually
I can happily do it right now it is just two lines.

Now the question is how do we "fake" the two lisp images process, which is a
perfectly valid specification of what we need.

Right now the extensions do not implement the previous specification.
Instead, at the risk of including spurious things, I just invoke TRAVERSE
with a LOAD-OP and find out all files that are actually loaded. I know this
is a hack but it worked so far.

One way that conforms better to the previous specification would be as
follows: First force a COMPILE-OP on the system, either in the current or in
a forked image. This ensures that al build operations are performed. A
second LOAD-OP using
  (defmethod operation-done-p :around ((operation load-op) c)
     (if *force-load-p* nil (call-next-method)))
and setting *force-load-p* to T will then identify which components actually
form part of the lisp image.

This has the disadvantage that it triggers a full compilation of a system
for a simple query. It is justified if the goal is just to make a bundle
operation, but it may be too prohibiting for other extensions.

A third way would be to TRAVERSE using a system that assumes all builders
are done and only "loaders" are required. How would I do that? Explicitely
telling TRAVERSE what I assume that has been done and what not.

(defvar *gather-component-filter* nil)

(defmethod builder-operation-p ((o compile-op) c)
  t)

(defmethod builder-operation-p (o c)
  (and (output-files o c) t))

(defmethod operation-done-p :around ((o t) c)
  (if *gather-component-filter*
      (funcall *gather-component-filter* o c)
      (call-next-method)))

(defun gather-loaded-components (system &key (op-type 'compile-op)
                                 filter-system (filter-type t) include-self)
 "Finds all components that SYSTEM depends on when applying LOAD-OP.
The output is a list of pairs of operations of the type OP-TYPE and
components they act upon. The list can be filtered either by component
type (FILTER-TYPE) or by imposing that the component belong to a given
system (FILTER-SYSTEM). INCLUDE-SELF adds a final pair for the
operation and the system itself."
  (let* ((system (find-system system))
         (*gather-component-filter* #'builder-operation-p)
         (operation (make-instance op-type))
         (tree (traverse (make-instance 'LOAD-OP) system)))
    (append (loop for (op . component) in tree
               when (and (typep component filter-type)
                         (or (not filter-system)
                             (eq (component-system component)
filter-system)))
               collect (progn
                         (when (eq component system) (setf include-self
nil))
                         (cons operation component)))
            (and include-self (list (cons operation system))))))

This is an extension of what is currently being used in asdf-ext.lisp but in
a more reasonable way because it would be extensible to other things like
resources, etc. Note that the :around method is harmless: it only takes
effect when gathering components and what it does is discarding the
operations we assume done.

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28006 (Spain)
http://tream.dreamhosters.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100329/ad25d79c/attachment.html>