[asdf-devel] ASDF 2.0 requirements : simplify the output locations mechanism

james anderson james.anderson at setf.de
Tue Apr 20 23:19:46 UTC 2010 

On 2010-04-18, at 00:20 , Faré wrote:

> Dear all,
>
> I think time has come to declare ASDF 2.0 in beta-test, and focus on
> releasing it before we get more features into it.
>
> If there are any bug fixes or improvements you think should definitely
> be in 2.0, please open a bug in launchpad and tag them as such
> (milestone: version 2), or otherwise make it prominent in your message
> on this mailing-list.

before one can declare asdf 2.0 to be feature complete, one should  
have a specification of its intended function and then gauge the  
implementation in those terms. while the users' manual does explain  
aspects of the user and/or programmer interface, in addition to the  
descriptions of the interface, there should be something which  
describes the requirements which are being beta-tested - both to  
assess completeness and to inform future maintenance. the additions  
to the code base over the past few months were devoted to two areas:

- configuration
- output translations

i neglect, for the moment, the configuration component of the asdf  
evolution and concentrate on questions related to output  
translations. even though the configuration mechanism contributes  
settings which affected the latter, they are in sufficiently  
independent, that an isolated analysis is reasonable.

in lieu of a specification, several asdf-devel messages on the topic  
of output translations[1--5] discuss requirements. the asdf-binary- 
location users' guide includes a "what it does" passage[6]. the asdf- 
binary-locations-devel archives[7] include no additional discussions.  
to follow those discussions, an output translation mechanism should  
make it possible to

- place (output) files under a single root
- place (output) files in a tree specific to (runtime x version x  
project x file-type)
- include process information (eg, pid or user id) in the location
- specify elements of the mapping programmatically
- specify elements of the mapping by configuration
- combine independent specifications
- express location maps both globally and in connection with specific  
sub-systems (cl-controller, cl-launch, etc)
- support simultaneous, independent mapping for binary output and  
ancillary data files
- disable translations for a given path
- disable translations completely

in response to these requirements, "asdf binary locations" was  
developed, first as an extension to, then integrated with asdf, and  
further developed into a general output translation facility. while  
the evolution brings definite benefits, in the process, between  
version 1.362, when a-b-l was incorporated into asdf core, and the  
current +/- 1.7, the source size has increased from 56K to 132K, the  
function count has grown from less than one hundred to more than two  
hundred, and the call count has almost tripled.

the call graphs[8] depict the situation. each node in the graph  
corresponds to a function in the asdf implementation, each edge to a  
call. the green nodes were present in 1.362, the yellow nodes in  
1.502, and red nodes in 1.7. filled nodes have remained, while empty  
nodes indicate functions which are no longer present. the increase in  
code mass and complexity is evident. since, as expressed, the  
requirements could be met by logical pathname translations, the  
situation causes concern. there is nothing in the functional  
requirements, as such, which dictates an implementation of this  
complexity. there are, however, non-functional concerns which appear  
to have informed the decision: it has been held, that
- logical pathnames are not portable;
-  the facility must operate in terms of physical pathnames as real- 
world lisp applications require file names which do not conform to  
logical pathname syntax constraints.

in the course of testing the output location mapping itself, it was  
demonstrated[9] that the non-portability claims were unfounded. an  
experiment based on 1.502 demonstrated[10] that a mechanism which  
separates location translation from truename processing can provide  
the required functionality with much less code in terms of logical  
pathnames despite non-conformant physical pathnames. the logical  
pathname based location maps were then re-implemented in 1.702 and  
further experiments demonstrated that - with changes to the scripts  
to account for functions which would be removed, all regression tests  
pass, and that with appropriate initialization, a multi-runtime build  
is supported for the normal abcl/acl/ccl/clisp/cmucl/ecl/lw/sbcl  
complement. the following host definition, for example, produces a  
centralized per-runtime translation tree for output files.

(let ((root (make-pathname :directory (butlast (pathname-directory  
*build-init-pathname*) 2)
                            :name nil :type nil :defaults *build-init- 
pathname*))
       (host "SOURCES"))
   (setf (logical-pathname-translations host) '(("**;*.*"  "**;*.*")))
   (let ((lbin (pathname-type (compile-file-pathname (make- 
pathname :host host :name "TEST" :type "LISP"))))
         (bin (pathname-type (compile-file-pathname "test.lisp"))))
     (setf (logical-pathname-translations host)
         `((,(make-pathname :host host :directory '(:absolute :wild- 
inferiors) :name :wild :type lbin :version :wild)
             ,(make-pathname :directory `(,@(pathname-directory root)  
"bin" ,(asdf::implementation-identifier) :wild-inferiors)
                            :name :wild :type bin :version :wild))
	   (,(make-pathname :host host :directory '(:absolute :wild- 
inferiors) :name :wild :type :wild :version :wild)
             ,(make-pathname :directory `(,@(pathname-directory  
root) :wild-inferiors)
                             :name :wild :type :wild :version :wild)))))

   (setq asdf::*translations-root*  (make-pathname :host  
host :directory '(:absolute) :name nil :type nil)))

given which, the question occurs, whether the output locations  
implementation deserves further thought before it advances to beta  
status?

---
[1] : http://common-lisp.net/pipermail/asdf-devel/2010-March/000943.html
[2] : http://tinyurl.com/yyv2o4w
[3] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 
000691.html
[4] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 
000699.html
[5] : http://common-lisp.net/pipermail/asdf-devel/2010-February/ 
000702.html
[6] : http://common-lisp.net/project/asdf-binary-locations/user- 
guide.html
[7] : http://common-lisp.net/pipermail/asdf-binary-locations-devel/
[8] : http://reports.setf.de/asdf-versions-twopi.svg,
       http://reports.setf.de/asdf-versions-twopi.pdf,
       http://reports.setf.de/asdf-versions-dot.svg,
       http://reports.setf.de/asdf-versions-dot.pdf
[9] : http://tinyurl.com/y7jmzee : the test cases are present in dual  
form, one for a physical pathname and once for a logical pathname
[10] : http://common-lisp.net/pipermail/asdf-devel/2010-April/ 
001351.html

More information about the asdf-devel mailing list