From fahree at gmail.com Thu Feb 14 22:57:28 2013 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Thu, 14 Feb 2013 23:57:28 +0100 Subject: [xcvb-devel] xcvb In-Reply-To: <20765.7685.800819.551103@hermes.wukix.com> References: <20765.7685.800819.551103@hermes.wukix.com> Message-ID: Dear Wes, sorry, I am travelling right now, after spending so much time focused on ASDF; If I vaguely remember your mentioning XCVB in a previous conversation, I never registered that you were serious about advancing it! There is so much to do about it. I suppose that the most urgent would be: 1- make it take advantage of the new asdf/driver system, that replaces most (but not all) of what used to be xcvb-driver. 2- fix it so it compiles itself again, post ASDF3. 3- fix the test suite so it doesn't run out of memory on SBCL anymore. 4- add all the features that have made it to ASDF itself since I last updated XCVB over a year ago, so it reaches parity again: test around-compile, add compile-check, add encodings, deferred warnings, bundle support, translation to/from if-feature, etc. 5- complete a client/server model of compilation. 6- use the above to have a more seamless bridge to/from ASDF, that allows for system-grained parallel compilation of code with ASDF or POIU. 7- add support for a system-grained blaze backend (Google internal), and for equivalent free-software backends if there's anything like that (distributed build system), or else implementation of such using iolib. I suppose the right place to discuss such things is xcvb-devel mailing-list, so I am adding the list to this conversation. ??? ? Fran?ois-Ren? ?VB Rideau ?Reflection&Cybernethics? http://fare.tunes.org Computer Science is no more about computers than astronomy is about telescopes. ? E. W. Dijkstral From fahree at gmail.com Mon Feb 18 15:01:47 2013 From: fahree at gmail.com (=?ISO-8859-1?Q?Far=E9?=) Date: Mon, 18 Feb 2013 16:01:47 +0100 Subject: [xcvb-devel] XCVB for Mocl? In-Reply-To: References: <20761.34512.329892.561841@hermes.wukix.com> Message-ID: I am traveling and I haven't been touching my work email in some time, but your previous message prompted me to look at it. For XCVB discussions, I prefer my personal email, and the xcvb-devel mailing-list, both of which I added to this conversation. I am otherwise replying inline to your message. > Hi Far?, > > Right now Mocl only supports ASDF (not XCVB) and it seems to work for 99% of the use cases but the remaining 1% is becoming a source of frustration. I was hoping maybe XCVB could become the cure to our woes. If XCVB can work out for us, maybe Wukix can help contribute momentum around XCVB and help to deprecate ASDF. However, if this is going to happen anytime soon, I need to figure out XCVB *quickly* so I was hoping you could help me understand some things about it. > Sorry for not replying that quickly. > Mocl is fundamentally a cross-compilation tool. As such, our needs for a build system are somewhat different than an ordinary Lisp. Perhaps the biggest and most obvious concern is that the build computer is not the same as the target computer/device. > > From the reading materials for XCVB I gather that XCVB can handle this situation better. Already, that's a good sign. > Yes. XCVB is designed for cross-compilation, whereas ASDF is designed for self-compilation. > Here are some gripes about ASDF (correct me if I'm wrong, I'm not an expert in ASDF): > > ASDF does not seem to support (in any obvious way) separation of the build-time lisp code and the run-time lisp code. We are trying to produce a compact executable for mobile devices, but ASDF pollutes this executable with things that are only needed during the build step. We can try to optimize this dead weight out, but some libraries do call ASDF during execution which makes removing ASDF non-trivial. > While ASDF itself could be modified to support such a separation, the entire ASDF eco-system, with ASDF extensions, etc., cannot. It's a losing proposition to try to evolve it that way. That was one of the main reasons I started XCVB as something separate rather than try to evolve ASDF (which I ended up doing anyway, for other reasons). On the other hand, if you want to deliver small executables, you may want to have some kind of "tree shaker" or garbage collector that removes from the dumped image code not reachable from some set of roots. I'm told LispWorks has some decent API for that; maybe Allegro too? In any case, you might need something like that if you want to optimize away EVAL and COMPILE-FILE from your delivery environment. > Pollution aside, another issue is the reliance on open-ended, state-filled, compile-time execution to define the build process. In many ASD files I am seeing lisp code or at least defgeneric. Usually it doesn't cause a problem. *Usually*. But the idea that any code can be running here, to interact with the build system, can make it impossible to reason about a library's build-time behavior. > Exactly. It's not *just* ASDF itself, it's also the body of existing code and the culture of how to use it. See also: https://bugs.launchpad.net/asdf/+bug/541562 > Again, based on what I've read about XCVB, it seems separating this stuff and cleaning up the use of state was a goal of XCVB so that sounds good to me. However, here's what I don't know about XCVB: > > 1. Is the build.xcvb purely declarative? No execution of arbitrary lisp statements? > Yes, build.xcvb is purely declarative data. By design, it cannot express arbitrary lisp code. There were vague discussions on how to better extend the build language than the current hard way, but so far it hasn't happened. > 2. Does XCVB need to stick around in the final executable? > If so, can it have a small footprint? > No it doesn't. XCVB only requires a small "driver" to be present during compilation, and on linking platforms such as ECL or MKCL and I suppose soon MOCL, it would not require the driver to be linked into the final executable. However, note that since I merged most of the former xcvb-driver into the current asdf/driver, xcvb at present requires ASDF during the build itself. But this could be reduced to just asdf/driver, and only during the build for linking platforms. > 3. Can XCVB deal with non-lisp data files easily? For instance, LOCAL-TIME looks for time zone data files in its source code directory, but since we aren't copying source code to the target device, we need a way to specify needed data files declaratively in build.xcvb, and somehow bundle them with the executable. The program, too, would then need a way to ask XCVB where its data files are at runtime. > Yes, there is some way in XCVB to specify dependencies on data files. > 4. XCVB seems to have a lot of features. I'm sure this is great for your large, distributed builds, but it looks overwhelming for our "1.0". We just want something simple that works (for now). How feasible would it be to fork a "tiny xcvb" or write one from scratch? For now I would be happy with just parsing very simple build.xcvb files and LOADing each file serially in the proper order. To me it looks possible and maybe a good strategy, but what are your thoughts? > Actually, considering how difficult it is to extend XCVB at present, I was considering rewriting it from scratch, or mostly so, in a way that would be more readily extensible, possibly using a prototype-based object system and/or interface-passing style. On the other hand, it would be *much* easier for you to just restrict yourself to a working subset of XCVB, without having to care much about all the other features. If you're not willing to evolve your reimplementation into a full product that you thereafter maintain, I strongly recommend against doing it; it's much more effort than you hope, and whatever little gains would be dwarfed by the cost. > Looking forward to your reply! > Sorry for the long delay. > Thanks, > > Wes > Wukix, Inc. ??? ? Fran?ois-Ren? ?VB Rideau ?Reflection&Cybernethics? http://fare.tunes.org We reject: kings, presidents and voting. We believe in: rough consensus and running code. ? David Clark for the IETF