[Asdf-devel] Need to do uiop:setup-temporary-directory in image built with uiop

[Robert P. Goldman]

Faré wrote:
> On Tue, Jun 17, 2014 at 4:33 PM, Robert P. Goldman <rpgoldman at sift.info> wrote:
>>> It is probably a bug that there isn't a uiop/stream:clear-temporary-directory
>>> that gets registered via uiop/configuration:register-clear-configuration-hook
>>> in uiop/configuration. I can do it, though I'd rather someone else do it.
>> I can do this, but I need some help understanding what's going on here.
>>  The following is a little circular:
>>
>>   (defun clear-configuration ()
>>     "Call the functions in *CLEAR-CONFIGURATION-HOOK*"
>>
>> What is the configuration?  ASDF has at least two configurations: the
>> configuration of source locations and the configuration of output-file
>> translations.  Is this one of those, or something else?  I'm a bit
>> confused because uiop/configuration.lisp looks like it provides a
>> *general* utility for configuring things.  And TEMPORARY-FILE is
>> something that is yet another aspect of configuration, outside both of
>> these.
>>
> clear-configuration is the function you call
> when configuration files may have been modified and
> you want to invalidate any current configuration loaded from them.
> All it does is call the functions in the *clear-configuration-hook*,
> so you can extend what "the configuration" means.
> UIOP itself has none (so far), 

I'm confused by this given what David Cooper says.  Now, it's maybe not
a *user-managed* configuration, but isn't the cached temp directory
setting a configuration?

but ASDF has configuration for
> source-registry and output-translations.

Right, and that's what confuses me.  For the source-registry, we have
CLEAR-SOURCE-REGISTRY, and for output-translations we have
CLEAR-OUTPUT-TRANSLATIONS, so what's a one-sentence description of what
UIOP:CLEAR-CONFIGURATION is supposed to do?  Does it subsume those two,
so that calling UIOP:CLEAR-CONFIGURATION effectively clears both
source-registry and output-translations?

Do you see what I mean?  Without a crisp definition of "configuration,"
I don't know what "clear-configuration" should do...

> clear-configuration is itself registered as an image-dump-hook,
> so when you save an image, you don't get configuration
> from whichever user, process, machine dumped the image.
> 
> I'm not completely sure whether you should
> (register-configuration-hook 'clear-temporary-directory)
> or more directly
> (register-image-dump-hook 'clear-temporary-directory)
> 
> Since default-temporary-directory (at least for now)
> doesn't consult any configuration file, only an environment variable,
> that is unlikely to change during execution,
> an image-dump-hook is probably all you need.
> It's a subtle difference in the intent of future possible modifications.
> 
>> I assume that when I understand this, I'll be able to understand how to
>> test it.
> Testing this functionality involves dumping an image
> while TMPDIR is something and running it while it is something else.
> Sigh. Could be done in test-program.script, I suppose.
> 
>>> That said, even with that hook function defined and registered,
>>> you still need to explicitly (call-image-dump-hook)
>>> before you dump an image using implementation-specific tools, and
>>> you need to (call-image-restore-hook) when you restart.
>>>
> 
>>>> I'd encourage you to do that, and CC me when you submit the bugreport to
>>>> Franz.
>>>>
>>> I don't believe there's any Franz bug.
>>> It's a bug in how UIOP is not used properly.
>>> These hooks probably need to be documented better, too.
>> I don't agree.  If a lisp implementation ships with ASDF, and a lisp
>> image dumping facility, then to the maximum extent possible the lisp
>> image dumping facility should not require additional work on the user's
>> part.  Also, it's likely to make the implementation maintainers happier
>> if they don't have to get complaints from users about cryptic errors
>> from the combination of these two features.
>>
> Well, so far UIOP is defined as a layer on top of what the
> implementation provides
> and not the other way around. That said, it might be great if the documentation
> for these implementations would remind the users that if they use
> these functions
> while building their software with ASDF (or similar software),
> they may want to explicitly  (uiop:call-image-dump-hook) and
> (uiop:call-image-restore-hook) (or the equivalent for their software).

Well, maybe UIOP is a layer on top, but it's a layer on top that seeps
down (in a way that seems very much analogous to the way that the
readtable seeps down from the REPL into the compiler...).
> 
>> Suggestion: if there are bits of configuration that can leak out of ASDF
>> and UIOP across image dumpings, let's try to document them, and make a
>> note for implementers that they may wish to appropriately modify their
>> image-dumping facilities to behave properly with their ASDF without
>> requiring additional user actions.
>>
> If there's a leak, it's probably a bug, to be fixed with
> an appropriate image-dump-hook.
> 
>> If worst comes to worst, and the implementors don't feel like handling
>> this, the documentation will be useful to the ASDF user who has to
>> handle it him or herself.
>>
>> Follow-up question: is it getting to be time to add a separate manual
>> for UIOP?
>>
> That would be great. I was kind of hoping for one of the many automatic
> documentation generators to be used based on the many docstrings.
> There are just too many functions to hope to maintain the documentation
> outside of said docstrings and other source comments.

Maybe I'll see what I can manage with Didier's library.  I think it
allows us to wrap structure around docstrings (like the way the SBCL doc
gen does).  I like documentation generators for exactly the reason you
do -- it keeps the documentation for functions close to their
definitions so they don't stray.  But I *dislike* them when, like
doxygen and javadoc, they just give you a big heap of unstructured mess.
 I'd like to structure the docstrings into a manual-shaped skeleton.

Wish I was going to ILC to talk about this with Didier.... :-(

Best,
r