From senatorzergling at gmail.com Wed Sep 2 13:37:30 2009 From: senatorzergling at gmail.com (szergling) Date: Thu, 3 Sep 2009 01:37:30 +1200 Subject: [xcvb-devel] Installation problem In-Reply-To: <653bea160908240550t63a307dbvab942ad623b30ebb@mail.gmail.com> References: <8a4b88390908140232x5bfc95dco86e3fb1eb3428423@mail.gmail.com> <653bea160908141316w1d6a114dna5c4c603e0e807a6@mail.gmail.com> <8a4b88390908240458x3b4c5e53xf91297d30dac2ff4@mail.gmail.com> <653bea160908240550t63a307dbvab942ad623b30ebb@mail.gmail.com> Message-ID: <8a4b88390909020637y1c97e64fk1b038c9f1a94b584@mail.gmail.com> Hi again, I'm still alive :) On 8/25/09, Far? wrote: > thanks a lot for taking the time and trouble to beta-test our software. No problem. There're some interesting ideas on the docs, so I was attracted to check this out :) > After your previous email, I've tried my best to address your legitimate > concerns about being able to use an SBCL at a non-standard location, and > the latest release of XCVB (0.360) should include provisions to let you I will try the latest release soon. The stuff I'm talking about here is still 0.357 only. > use whichever SBCL is properly installed to a directory in your $PATH. > So make sure that "sbcl" works at your shell prompt (don't try to run > it uninstalled from its source directory -- that's painful). > If the autodetection fails, please report a bug again. Right, I see that actually installing will put the content of the contribs dir and the sbcl.core in the same folder in /lib/sbcl/ so SBCL_HOME works as expected. I have successfully rebuilt xcvb with the new path. It was quite muddled, and unfortunately, I cannot retrace the steps/minor changes I made to get everything to run, but I will probably need to check with 0.360 or later versions anyway. > I recommend using SBCL 1.0.30.4 or later (1.0.31.0 is due RSN), which will > have the advantage of providing your with CFASL support in addition to > being something with which we have tested our software. I have not tried the latest SBCL yet. >> However, at the moment, I still couldn't get make-makefile to run: >> >> tyc20 at binks:~/hda/code/lisp/my-first-xcvb-test$ xcvb make-makefile --build >> . >> end of file on #> /home/tyc20/hda/code/lisp/my-first-xcvb-test/obj/target-properties.lisp-expr" >> {AB591F1}> >> 0: >> unhandled PRINT-NOT-READABLE: # cannot be >> printed readably. >> >> Argh! error within --disable-debugger error handling > Ouch. Which target lisp implementation are you using? > If that still happens with the latest SBCL, can you trace how xcvb > invokes the target lisp to extract its properties, > and what makes the target lisp choke? In the end, I traced this to query-target-lisp-helper which calls asdf:run-shell-command with something like sbcl ... > some-file My trouble arose from the 'sbcl' and path issue, where the exec silently dies. This is confirmed by piping 2> to a file instead, where I find fatal error encountered in SBCL pid 7972: can't find core file at /home/tyc20/hda/sbcl/sbcl-1.0.29-x86-linux/contrib//sbcl.core I fixed this by setting PATH and SBCL_HOME appropriately. I also needed to set XCVB_PATH to my current module location to get a build to run. None of this is very "referentially transparent" (if I may abuse that terminology), since there are lots of hidden variables. So I think I'm set, and right now, I'm stuck on this error: There is no applicable method for the generic function # when called with arguments (NIL). 0: (SB-DEBUG::MAP-BACKTRACE #)[:EXTERNAL] 1: (SB-DEBUG:BACKTRACE 536870911 #) 2: (XCVB-DRIVER::PRINT-BACKTRACE #) 3: (XCVB-DRIVER::BORK #) 4: (SIGNAL #)[:EXTERNAL] 5: (ERROR "~@")[:EXTERNAL] 6: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) # # #)[:EXTERNAL] 7: (XCVB::GRAPH-FOR-LISP-MODULE # "/xcvb/driver") 8: (XCVB::CALL-WITH-GRAIN-REGISTRATION (:LISP "/xcvb/driver") #)[:EXTERNAL] 9: (XCVB::GRAPH-FOR* # ((:LISP "/xcvb/driver"))) ... etc ... I think I will try and bring up a REPL and slime debugger first. This will be interesting. Part of the reason I am looking at xcvb is because I have lots of issues with multiple versions of (often incompatible) Lisps, individual projects, slime, boxsets (& dependencies) etc. I wonder if there's a way to have multiple versions of modules all run seamlessly? In other words, how can I run two different versions of the same library simultaneously in the same Lisp image, switching between the two as I see fit? Perhaps library implementors might have to give up asdf & defpackage, and use some new constructs for system definitions and dependency management instead? >> Next time, I'm gonna try and run MAKE-MAKEFILE, REMOVE-XCVB-COMMAND >> (in main.lisp) etc on the REPL. Is that how you are all using it at >> the moment? > At the moment, I am using it at the shell prompt, though sometimes > indeed I use the repl for debugging -- particularly with > C-u M-x slime env LISP_FASL_CACHE=NIL XCVB_PATH=... xcvb repl > >> Thanks for the help, I will continue to explore when I find time. > Thank you so much for your testing. > > My apologies for the trouble you're having. No apology required. I knew what I was up against the moment I saw all the Makefiles... I'm only testing this out, so there is no urgency or any great need for xcvb (for now) on my part. > I'm definitely committed to make XCVB a usable tool for everyone and > not just something that merely works for me in my particular setting. Good to hear. Hopefully, xcvb can one day be the tide that lifts all Lisp projects upwards through a useful module system. > [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org > ] > The trouble with opportunity is that it always comes disguised as hard > work. > -- Herbert V. Prochnow > From fahree at gmail.com Wed Sep 2 16:29:16 2009 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Wed, 2 Sep 2009 12:29:16 -0400 Subject: [xcvb-devel] Installation problem In-Reply-To: <8a4b88390909020637y1c97e64fk1b038c9f1a94b584@mail.gmail.com> References: <8a4b88390908140232x5bfc95dco86e3fb1eb3428423@mail.gmail.com> <653bea160908141316w1d6a114dna5c4c603e0e807a6@mail.gmail.com> <8a4b88390908240458x3b4c5e53xf91297d30dac2ff4@mail.gmail.com> <653bea160908240550t63a307dbvab942ad623b30ebb@mail.gmail.com> <8a4b88390909020637y1c97e64fk1b038c9f1a94b584@mail.gmail.com> Message-ID: <653bea160909020929s6048f68dt6dc6de106159f75b@mail.gmail.com> Honorable Senator, > Hi again, I'm still alive :) Glad the reavers didn't get you... > I will try the latest release soon. The stuff I'm talking about here > is still 0.357 only. A lot of big and small bugs have been fixed in .366 including many issues with the packaging of the release tarball. > In the end, I traced this to query-target-lisp-helper which calls > asdf:run-shell-command with something like > > sbcl ... > some-file > > My trouble arose from the 'sbcl' and path issue, where the exec > silently dies. This is confirmed by piping 2> to a file instead, where > I find > > ? ?fatal error encountered in SBCL pid 7972: > ? ?can't find core file at > /home/tyc20/hda/sbcl/sbcl-1.0.29-x86-linux/contrib//sbcl.core > > I fixed this by setting PATH and SBCL_HOME appropriately. I also > needed to set XCVB_PATH to my current module location to get a build > to run. None of this is very "referentially transparent" (if I may > abuse that terminology), since there are lots of hidden variables. > Oops. I'm adding this paragraph to the README under `Compiling with XCVB`_: Note that you need to properly setup your `Search Path`_ with ``XCVB_PATH``, or specify it with option ``--xcvb-path``. Also, XCVB will assume that the specified implementation, ``sbcl`` or otherwise, is in your ``PATH``, though you may override with ``--lisp-binary-path``. If needed, make sure you export the proper ``SBCL_HOME`` or ``CCL_DEFAULT_DIRECTORY``, etc., or that some wrapper script to your Lisp binary does. > So I think I'm set, and right now, I'm stuck on this error: > > ? ?There is no applicable method for the generic function > ? ? # > > ? ?when called with arguments > ? ? ?(NIL). [...] > ? ?7: (XCVB::GRAPH-FOR-LISP-MODULE > ? ? ? ?# ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?((:LISP "/xcvb/driver") (:IMAGE "/_") > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (:FASL "/test-module/module1")) > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?XCVB::DEPENDENCIES-R NIL > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?XCVB::INCLUDED-DEPENDENCIES NIL > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?XCVB::LOAD-COMMANDS-R NIL)> > ? ? ? ?"/xcvb/driver") Oops. This means that the installed XCVB is not in your XCVB_PATH. I'm adding this to the README under `Search Path`_: Importantly, an installed version of XCVB itself (at least its ``build.xcvb`` and ``driver.lisp`` in the same directory) must be present under the search path since XCVB will look for the module ``/xcvb/driver`` to be included in the target Lisp image as a necessary prelude to any build. In a good installation of XCVB, this should be the case by default, and users would use ``!`` in their search path specifications to inherit this default; however, until XCVB comes packaged with your system, you shouldn't trust these defaults until you have verified them. > I think I will try and bring up a REPL and slime debugger first. Don't bother, I understand the issue. Hopefully, XCVB .366 has more informative error message, too. > This > will be interesting. Part of the reason I am looking at xcvb is > because I have lots of issues with multiple versions of (often > incompatible) Lisps, individual projects, slime, boxsets (& > dependencies) etc. I wonder if there's a way to have multiple versions > of modules all run seamlessly? In other words, how can I run two > different versions of the same library simultaneously in the same Lisp > image, switching between the two as I see fit? Perhaps library > implementors might have to give up asdf & defpackage, and use some new > constructs for system definitions and dependency management instead? > This is a problem that XCVB can't directly help you with. XCVB will help you easily have multiple versions in multiple images, by having a different XCVB_PATH. To have the SAME lisp image use multiple versions of a library, you need to be able to load them at the same time, which at the very least would require you to play tricks with RENAME-PACKAGE to rename the code from one version before you load the code from another. Then again, client code can only be compiled with one version of the server code, so you may have to duplicate and rename not just the one package, but each and every package after it -- which wouldn't give you much more than having multiple images. NOW, *if* you only want to switch between a few identified run-time things that happen to be functions and/or variables, whereas you trust all the macros to be functionally the same, then you could indeed do tricks with something that would walk the package for bound and fbound symbols and do a switcheroo between two versions of them. But that's assuming a lot, and still doing stuff that CL isn't helping you with. And XCVB currently has no support for such thing (though it could be added with no more difficulty to XCVB than to ASDF). >> I'm definitely committed to make XCVB a usable tool for everyone and >> not just something that merely works for me in my particular setting. > > Good to hear. Hopefully, xcvb can one day be the tide that lifts all > Lisp projects upwards through a useful module system. > Or maybe we should all switch to R7RS. [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Genius is one percent inspiration and ninety-nine percent perspiration. -- Thomas Alva Edison From fahree at gmail.com Fri Sep 11 19:08:20 2009 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Fri, 11 Sep 2009 15:08:20 -0400 Subject: [xcvb-devel] XCVB .370, the it's mainly working edition Message-ID: <653bea160909111208j3b836491u19621ae035f44f9@mail.gmail.com> I've compiled most of QRes with XCVB. 1307 lisp source files so far. On a quad-core machine using SBCL and CFASL, it was an order of magnitude slower than ASDF, notably because of some files with deep dependencies on up to 277 other files the CFASLs of which had to be painstakingly loaded. If your dependencies run less deep and you have enough cores, you might achieve a speed up instead of a slowdown. In any case, XCVB provides determinism, repeatability, incremental compilation and phase separation to CL builds. Otherwise, all is not lost for me, as there is plenty of progress to be made to automate the optimization of dependencies, possibly separating compile-time, load-time and cfasl-load-time dependencies. Or possibly in using fork to cache image state without dumping images. I'll keep you tuned. As usual, the latest release tarballs can be found at the URL below, and I welcome any feedback on the use of it: http://common-lisp.net/project/xcvb/ [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] Success is getting what you want. Happiness is wanting what you get. -- Dale Carnegie From fahree at gmail.com Tue Sep 29 23:30:44 2009 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Tue, 29 Sep 2009 19:30:44 -0400 Subject: [xcvb-devel] XCVB .382, now with ASDF and POIU backends Message-ID: <653bea160909291630p4c44973bs21a3ca53c2a12ec5@mail.gmail.com> Since our previous attempt at using XCVB on a very large project had bad performance issues because of very deep dependency tree, and until we optimize this tree, we now provide a fast compilation backend to XCVB using POIU to compile in parallel (POIU only supports SBCL and CCL at this point in time). This was tested by compiling XCVB itself with this fast backend: xcvb nm --build xcvb --parallel && make -f xcvb-ne.mk A variant of the backend can also be used to create a .asd file for your XCVB builds so you can integrate XCVB software into systems built with an ASDF workflow. The output is simplified, and will have less information than an .asd file written by hand; it will not make use of any ASDF extension that may provide features comparable to what XCVB does (generated files, conditional compilation, requires, data dependencies, warning control). Nonetheless it provides a usable bridge between XCVB projects and ASDF projects, and its output can serve as the input for further manual massaging into a full-featured .asd file: xcvb x2a -b /xcvb -o /tmp/xcvb.asd The latest XCVB release can be found here: http://common-lisp.net/projects/xcvb/releases/ POIU can be found on QITAB: http://common-lisp.net/projects/qitab/ [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] There are three types of people in the world; those who can count, and those who can't. From attila.lendvai at gmail.com Wed Sep 30 06:42:50 2009 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Wed, 30 Sep 2009 08:42:50 +0200 Subject: [xcvb-devel] XCVB .382, now with ASDF and POIU backends In-Reply-To: <653bea160909291630p4c44973bs21a3ca53c2a12ec5@mail.gmail.com> References: <653bea160909291630p4c44973bs21a3ca53c2a12ec5@mail.gmail.com> Message-ID: > dependencies, warning control). Nonetheless it provides a usable > bridge between XCVB projects and ASDF projects, and its output can > serve as the input for further manual massaging into a full-featured Fare, do you happen to have a pool of build.xcvb files available somewhere? (for projects available on the net without an official build.xcvb in their repos yet) asdf-system-connection is badly broken, so we switched to explicit dependencies but it's highly uncomfortable... or do we need more than just an additional build.xcvb file? -- attila From fahree at gmail.com Wed Sep 30 09:47:00 2009 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Wed, 30 Sep 2009 05:47:00 -0400 Subject: [xcvb-devel] XCVB .382, now with ASDF and POIU backends In-Reply-To: References: <653bea160909291630p4c44973bs21a3ca53c2a12ec5@mail.gmail.com> Message-ID: <653bea160909300247h7ac244a2n80aa8985c99254fd@mail.gmail.com> 2009/9/30 Attila Lendvai : > Fare, do you happen to have a pool of build.xcvb files available > somewhere? (for projects available on the net without an official > build.xcvb in their repos yet) > > or do we need more than just an additional build.xcvb file? > I have a pool of such build.xcvb's and associated patches (mostly regarding the proper use of eval-when) for a few systems that we use at ITA. I'll probably publish those patches next thing on the XCVB web site, with announcement on xcvb-devel. In addition to the list below, there's also the fare-* (fare-csv, fare-utils, fare-matcher, meta, scribble, exscribe), and the dependencies of xcvb itself (cl-launch, asdf, asdf-dependencies-grovel, closer-mop, xcvb), and poiu for which I'm actively maintaining git repos (darcs for closer-mop) that include xcvb support. alexandria asdf-additions babel bordeaux-threads cffi chunga cl-base64 cl-fad cl-ppcre cl+ssl cl-unicode cl-who drakma flexi-streams hunchentoot iterate-1.4.3 lw-compat md5 metabang-bind net-telent-date parse-number-1.0 ptester puri rfc2388 split-sequence stefil trivial-features trivial-gray-streams trivial-ldap usocket [ Fran?ois-Ren? ?VB Rideau | Reflection&Cybernethics | http://fare.tunes.org ] First, they said Java was good for app-lets; then they recoiled, and said it was good for serv-lets; actually, what it's good for is toy-lets.