[xcvb-devel] XCVB for Mocl?

[Faré]

I am traveling and I haven't been touching my work email in some time,
but your previous message prompted me to look at it.

For XCVB discussions, I prefer my personal email,
and the xcvb-devel mailing-list, both of which I added to this conversation.
I am otherwise replying inline to your message.

> Hi Faré,
>
> Right now Mocl only supports ASDF (not XCVB) and it seems to work for 99% of the use cases but the remaining 1% is becoming a source of frustration. I was hoping maybe XCVB could become the cure to our woes. If XCVB can work out for us, maybe Wukix can help contribute momentum around XCVB and help to deprecate ASDF. However, if this is going to happen anytime soon, I need to figure out XCVB *quickly* so I was hoping you could help me understand some things about it.
>
Sorry for not replying that quickly.

> Mocl is fundamentally a cross-compilation tool. As such, our needs for a build system are somewhat different than an ordinary Lisp. Perhaps the biggest and most obvious concern is that the build computer is not the same as the target computer/device.
>
> From the reading materials for XCVB I gather that XCVB can handle this situation better. Already, that's a good sign.
>
Yes. XCVB is designed for cross-compilation,
whereas ASDF is designed for self-compilation.

> Here are some gripes about ASDF (correct me if I'm wrong, I'm not an expert in ASDF):
>
> ASDF does not seem to support (in any obvious way) separation of the build-time lisp code and the run-time lisp code. We are trying to produce a compact executable for mobile devices, but ASDF pollutes this executable with things that are only needed during the build step. We can try to optimize this dead weight out, but some libraries do call ASDF during execution which makes removing ASDF non-trivial.
>
While ASDF itself could be modified to support such a separation,
the entire ASDF eco-system, with ASDF extensions, etc., cannot.
It's a losing proposition to try to evolve it that way.
That was one of the main reasons I started XCVB as something separate
rather than try to evolve ASDF
(which I ended up doing anyway, for other reasons).

On the other hand, if you want to deliver small executables,
you may want to have some kind of "tree shaker" or garbage collector
that removes from the dumped image code not reachable from some set of roots.
I'm told LispWorks has some decent API for that; maybe Allegro too?
In any case, you might need something like that
if you want to optimize away EVAL and COMPILE-FILE
from your delivery environment.

> Pollution aside, another issue is the reliance on open-ended, state-filled, compile-time execution to define the build process. In many ASD files I am seeing lisp code or at least defgeneric. Usually it doesn't cause a problem. *Usually*. But the idea that any code can be running here, to interact with the build system, can make it impossible to reason about a library's build-time behavior.
>
Exactly. It's not *just* ASDF itself, it's also the body of existing code
and the culture of how to use it. See also:
https://bugs.launchpad.net/asdf/+bug/541562

> Again, based on what I've read about XCVB, it seems separating this stuff and cleaning up the use of state was a goal of XCVB so that sounds good to me. However, here's what I don't know about XCVB:
>
> 1. Is the build.xcvb purely declarative? No execution of arbitrary lisp statements?
>
Yes, build.xcvb is purely declarative data.
By design, it cannot express arbitrary lisp code.

There were vague discussions on how to better extend the build language
than the current hard way, but so far it hasn't happened.

> 2. Does XCVB need to stick around in the final executable?
> If so, can it have a small footprint?
>
No it doesn't.
XCVB only requires a small "driver" to be present during compilation,
and on linking platforms such as ECL or MKCL and I suppose soon MOCL,
it would not require the driver to be linked into the final executable.

However, note that since I merged most of the former xcvb-driver
into the current asdf/driver,
xcvb at present requires ASDF during the build itself.
But this could be reduced to just asdf/driver,
and only during the build for linking platforms.

> 3. Can XCVB deal with non-lisp data files easily? For instance, LOCAL-TIME looks for time zone data files in its source code directory, but since we aren't copying source code to the target device, we need a way to specify needed data files declaratively in build.xcvb, and somehow bundle them with the executable. The program, too, would then need a way to ask XCVB where its data files are at runtime.
>
Yes, there is some way in XCVB to specify dependencies on data files.

> 4. XCVB seems to have a lot of features. I'm sure this is great for your large, distributed builds, but it looks overwhelming for our "1.0". We just want something simple that works (for now). How feasible would it be to fork a "tiny xcvb" or write one from scratch? For now I would be happy with just parsing very simple build.xcvb files and LOADing each file serially in the proper order. To me it looks possible and maybe a good strategy, but what are your thoughts?
>
Actually, considering how difficult it is to extend XCVB at present,
I was considering rewriting it from scratch, or mostly so,
in a way that would be more readily extensible,
possibly using a prototype-based object system and/or interface-passing style.

On the other hand, it would be *much* easier for you
to just restrict yourself to a working subset of XCVB,
without having to care much about all the other features.
If you're not willing to evolve your reimplementation
into a full product that you thereafter maintain,
I strongly recommend against doing it;
it's much more effort than you hope, and whatever little gains
would be dwarfed by the cost.

> Looking forward to your reply!
>
Sorry for the long delay.

> Thanks,
>
> Wes
> Wukix, Inc.

—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
We reject: kings, presidents and voting. We believe in:
rough consensus and running code. — David Clark for the IETF