[asdf-devel] :logical-hostname

[james anderson]

On 2010-03-31, at 20:00 , Juan Jose Garcia-Ripoll wrote:

> On Wed, Mar 31, 2010 at 4:39 PM, james anderson  
> <james.anderson at setf.de> wrote:
> i simply don't understand your requirements. on one hand both with  
> and without asdf, but then it is objected, that a mechanism which  
> requires asdf does not work without it. please explain.
>
> What I do not understand is what YOU do not understand.
>
> In an standalone system there is no ASDF. That is all.
>
> I want to be able to take an ASDF system describing my program and  
> its dependencies on other libraries and compile it and produce a  
> standalone executable. That program will not have ASDF inside, it  
> will not have the notion of a central registry it will have  
> nothing, but it still has to work.

it's going to need to take it's logical host definitions from somewhere.

either they survived, in which case, in the simplest form they are re- 
rooted, or in more complex forms distinct branches are mapped  
distinctly, or they didn't survive, and the program will either apply  
heuristics to locate the intended files, or will benefit from  
external information to accomplish that task. to these ends, it will  
use at least the cl operators for logical pathnames. in the case of  
the examples which i posted, it might in addition use the same  
operators from the cited library which were used to establish the  
host definitions in the first place.

asdf did not appear in that paragraph.

>
> Please understand that the program I am talking about is not a  
> dumped lisp image that contains everything needed to build the  
> system and even some extra stuff that was there (database of ASDF  
> systems and other trash). The programs I produce are just a set of  
> compiled files, one per ASDF system, linked together with a core  
> library that is the Common Lisp environment ECL offers.

that's ok.

>
> When the program starts, the linked files are "loaded" one after  
> another, as if LOAD was used but actually much faster. But I insist  
> again ASDF is not part of the things that is loaded simply because  
> there is no need.

that's ok.

>
> The only thing that one might need ASDF for is to locate the data  
> associated to the program, but if we provide logical pathname  
> translations (one host per library), then a small piece of code can  
> be inserted at the beginning of the program which locates the data  
> and sets up the appropriate translations.

as noted above.

>
> if the requirement is that exact same exact formulation be used  
> with and without asdf, then
> a) the additional system initialization argument is excluded
> b) the second option in my earlier message provides this path.
>
> I only find "b" acceptable because I still do not understand how  
> "a" is going to work without a central registry and a lot of  
> complicated logic that has to propagate to those distributed binaries.

asdf uses a central registry. that is a given. that is how it finds  
the system definitions. i reported that it is possible for asdf to  
use the registry list to infer the logical host which should apply to  
a given system definitions. without any additional specification in  
the system declaration. that asdf does that has no necessary bearing  
on what a release process or the released program does with logical  
pathnames. it just makes it possible for the build process to share  
information with the later processes.

>
> Having "b", which I recall was one logical host name per system,  
> one may simply prepend to the compiled set of files a tiny bit of  
> lisp that looks as follows
>
> (defvar *program-data-location*
> #+windows ;; Data and binaries live together
> (program-location)
> #+unix ;; Data is installed somewhere standard
> "/usr/share/program/"
> #+osx ;; Assume this is an application bundle
> (merge-pathnames "Content/Resources/" (program-location)))
>
> (defun set-host-translation (root extra)
>   ...)
>
> (set-host-translation "cl-ppcre" "cl-ppcre/")
> (set-host-translation "cl-unicode" cl-unicode/")

this is what the code in the examples i cited does. not in exactly  
these terms, but the pathname world is not large.
the only difference between the two approaches which i described is  
whether asdf uses the operators or they are used otherwise, in  
addition to the defsystem form.

>
> With this simple, automatically generated code I can pack the given  
> systems and let them work anywhere, even though ASDF is not present.
>
> The same or a similar procedure can be used to link many systems  
> into a single distributable binary fasl + resources, install it in  
> a standard location that can be found by certain lisps and prepend  
> a simple logic for finding out its data... Without using ASDF in  
> the file itself.
>
> There is even a third model in which installation paths are known  
> beforehand. Say that a package maintainer from Debian has the role  
> of generating binaries for 10 libraries out there and they have  
> complex inter-dependencies. Libraries can be compiled in the  
> machine of the maintainer and then installed by ASDF in their  
> standard locations. ASDF can, in the process, hard-code the  
> pathname translations when installing the files.
>
> For instance, if we produce binaries for cl-ppcre-1.0.1 and cl- 
> unicode-2.0.1 and the latter depends on the former, and we know  
> that they are going to live in
> /usr/lib/common-lisp/cl-ppcre-1.0.1
> /usr/lib/common-lisp/cl-unicode-2.0.1
> when "asdf-build" compiles the files and "asdf-installs" them we  
> may get
>
> ;; Stripped down version of a system definition
> ;; which only contains the precompiled files
> ;; and which can be searched and used by ASDF
> /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.asd
> /usr/lib/common-lisp/cl-unicode-1.0.1/cl-unicode.asd
>
> ;; The binaries themselves
> /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre-1.0.1.fas
> /usr/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode-1.0.1.fas
>
> ;; Proxy binaries that can be loaded using "require"
> ;; even if ASDF is not present. These files define the logical
> ;; pathname translations based on hardcoded locations.
> /user/lib/common-lisp/cl-ppcre-1.0.1/cl-ppcre.fas
> /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.fas
>
> ;; Data used by the library
> /user/lib/common-lisp/cl-ppcre-1.0.1/cl-unicode.dat
>
> ;; Symbolic link to the last version
> /usr/lib/common-lisp/cl-ppcre -> cl-ppcre-1.0.1
>
> The FASL without version numbers files can explicitly load the  
> required versions of the libraries because they know the hardcoded  
> paths of their files. In doing so they can also set up the  
> appropriate pathname translations that are used by the other files.
>
> I have listed just two examples with different paradigms on how to  
> build and distribute software, and which go beyond current use in  
> which the user has to set up a collection of registries, in your  
> case you even define pathname translations and have magic to make  
> ASDF understand them, sources are always present, things are  
> recompiled...

the first two clauses are true, although the characterization is  
unwarranted. the third and fourth are not.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/asdf-devel/attachments/20100331/2ffacd7b/attachment.html>