Advice sought on structuring a system

[Daniel Herring]

Hi Don,

System configuration, and configuration of dependencies in particular, is 
in some sense an AI-complete problem.  There is no general-purpose 
solution.  Every solution has tradeoffs.  At the end of the day, some end 
user may need to tweak something.

My impression is that that Quicklisp takes the 80% or 90% approach. 
Quicklisp tries to provide what "the average" user wants.  Anyone outside 
that profile should download and configure packages independently.

>From what you describe, sqlite adds minimal functionality to Roan, and 
requiring sqlite adds significant risk of a failed Roan install.  Thus 
sqlite should not be required in the Quicklisp distribution of Roan. 
Power users can read the documentation, download Roan, and do any manual 
tweaks that are required.

In terms of functionality and soundness of intent, the most advanced 
configuration tool I know of is GNU Autoconf.  It provides a standard 
approach for running experiments on a system, providing default config 
selections, and providing user-selectable options.  Unfortunately, it 
targets C/C++ and is written in "portable shell script" that is 
partially auto-generated by M4 and Perl.  In other words, it is fairly 
hard to grok.  Unfortunately, CL has nothing that even comes close in 
terms of functionality.

I haven't looked recently, but in the past there were a number of ASDF 
files with reader conditionals (or comments) in order to support some user 
customization.  I believe Fare got many of these cleaned up.  You should 
ask the ASDF list for the current recommended practice.

- Daniel


On Wed, 6 Apr 2016, Don Morrison wrote:

> I'm tidying up a library of code, Roan (for building things that play with change ringing, you probably don't want to know the details :-), for eventual inclusion in Quicklisp. In thinking about its dependencies I've run into an
> issue I'm not sure how best to address. I asked Zach Beane for advice, but he, too, isn't sure how best to deal with it, and suggested I consult folks on this list.
> 
> My library has dependencies upon several others, all readily available from Quicklisp. One them, though, cl-sqlite (or perhaps it's better referred to as sqlite, it's rather conflicted about what it wants to call itself) in turn has
> a dependency on a binary library, sqlite3. For example, if the sqlite3 binary library is not install, doing
> 
> (ql:quickload :sqlite)
> 
> signals an error when it's setting up FFI things.
> 
> However Roan's use of SQLite is not essential to Roan's usefulness (well, it's probably not all that useful to most people anyway, only to those interested in change ringing, a small audience). There's really only one, tiny corner of
> functionality in Roan it supports. While sqlite3 is easily obtained, it would seem polite to allow use of Roan without it, simply disabling  the two functions that depend upon it. The "disable if the cl-sqlite stuff is not present"
> part I can easily do. What's causing me to scratch my head is figuring out how to set things up so Roan will still install whether or not the sqlite3 binary library is present.
> 
> Three unattractive possibilities that come to mind are
> 
> - Define two different roan ASDF systems: one consisting of most stuff, without the SQLite-dependent bits, and the other the "real" one that loads the first plus cl-sqlite. Then the user picks which one to load based on need.
> 
> - Define something to push onto *features* that, if present, comments out in roan.asd the dependency on cl-sqlite, etc. Then the user has to push the appropriate keyword onto *features* before loading roan, if she doesn't have
> sqlite3.
> 
> - I'm guessing I could probably include some code in the roan.asd file that does something or other nasty looking for the binary library, and then adjusts the defsystem appropriately or something (ugh).
> 
> None of these is particularly appealing. The first two seem to put too much on the shoulders of the poor luser who just wants to load the darn thing without having to think about lower level stuff, and the last (a) seems to confound
> too much actually doing complex, kludgy stuff with what should be just declarative, and (b) is something I seriously doubt I'd get right, especially in a portable way.
> 
> It seems there has got to be A Better Way. Has anyone dealt with a similar situation? Or, even if not, might you have a good idea?
> 
> I suppose an attractive possibility might be to wrap some sort of condition handler around the place inside ASDF that's trying to load the cl-sqlite dependency, but I don't know my way around the ASDF internals enough to have a sense
> of where that should be done, or if it's even practical. Is there a good place inside of ASDF to do such a thing? It seems fairly modular, so it seems likely. Maybe it's as easy as defining a specialized method of some sort, or an
> :around method on something or other?
> 
> Thanks!
> 
> 
> 
> 
> -- 
> Don Morrison <dfm2 at cmu.edu>
> "There were two cultures, as far as he was concerned. One was the real
> one, the other was occupied by people who liked machinery and ate
> pizza at unreasonable hours."          -- Terry Pratchett, _The Last Hero_
> 
> 
>