[asdf-devel] ABCL packaging ASDF system in jars (was Re: Disabling "built-in" output translation)

[Mark Evenson]

On 5/17/11 10:01 PM, Faré wrote:
>> I'm gonna need a bit more time to chew over the ASDF code, but thanks for
>> the general direction.  I must confess that in spite of the reasonable
>> looking documentation and having contributed the function translation code
>> to ASDF, its whole output translation API has never really gelled in my
>> understanding as a totality.  Maybe I can contribute some examples to the
>> texinfo file when the fuzz in my understanding resolves a bit.
>>
> output-translations is not THAT hard to understand,
> if you read the source code with my previous comments in mind.
> Patches to the docs are welcome, too.

I agree it shouldn't be that hard.  And every time I read the code and 
docs I go "Uh-huh, now I understand."  But somehow, like Perl code, when 
I come back to it two weeks later, I have to start from the beginning. 
I guess I'm just putting the proverbial stake in the ether to mark my 
frustration in case others have the same problem, as a) I'm old enough 
not to care that much if other people think I'm an idiot and b) maybe 
it's not just me.  In amplifying my concern here, I definitely owe you a 
diff or two to asdf.texinfo to reward your patient explanations…

[…]

>> 1)  You can't immediately load FASL out of the jar.
> Is that an ABCL limitation?

Certainly not:  abcl.jar contains FASLs, after all.

>>   In my current hackish
>> way—i.e. without comprehending your advice yet—one has first ASDF compile
>> the system with output translations disabled, and then
>>
>>   (defmethod asdf:perform ((o asdf:compile-op) (c asdf:cl-source-file)))
>>   (setf (asdf::output-translations) '((t t))))
>>
>> in the target JVM to load the ABCL FASLS from the jar.
>>
> Why that pain? Doesn't the zip process preserve the timestamps?

Timestamps should be preserved, but the current output translation code 
looks for FASLs in the user cache.  Well, by that reasoning including 
both statements should be redundant.  I need to look at this again…

[…]

> You want CL systems to be in a variety of jars,
> rather than just a one jar with everything in it?

Yep.  I wouldn't mind being able to include multiple systems in a single 
jar, but I want to keep the core abcl separate from the systems in order 
to be able to consistently rev the core (and, of course, the systems) 
via checksums.

>
>> 4)  The extremely nice use of [JSS][jss] and [ABCLD's slight
>> modification][abcld] of ASDF to also refer to jar files to dynamically load
>> into the JVM probably needs to be rewritten, otherwise we run into the
>> situation whereby we have jars (i.e. the packaged Java code) within the ASDF
>> packaged jar which A) needs changes within ABCL to completely work and B)
>> would be rather inefficient in that the naive implementation  each request
>> for a new entry in the JAR within a JAR would require a complete "reseek"
>> through the enclosed ZIP file.
>>
>> [jss]: http://code.google.com/p/lsw2/source/browse/#svn%2Ftrunk%2Fjss
>> [abcld]:
>> http://code.google.com/p/abcl-dynamic-install/source/browse?repo=abcld
>>
> I welcome patches to ASDF and/or implementation-specific contributions,
> just like our asdf-ecl.lisp.

I have my eye on that, and when I have mature patches to ASDF that work 
with ABCL out of the box, I will submit them for consideration.  But 
currently, one needs a specialized build of ABCL with the jscheme and 
bsh jars in the classpath in order to use the jar-file extensions so 
this isn't practical.  Given the amount of ABCL's lunch that Clojure has 
eaten, maybe it's time to make some post-ANSI CL decisions in our 
implementation wrt. ABCL Java/CL interop.

>> 5)  A fear of mine: if I enable all this, I presume that people would start
>> going around creating 'abcl.jar' files with different inclusions of
>> different ASDF packagings.  Without a real smart dynamic introspection
>> system that essentially solves the problems in JVM's classpath we would end
>> up with, at the minimum, a rather hostile situation for the end user.  "Put
>> 'abcl.jar' in your classpath."  "I did, and it still didn't load Maxima."
>>   "Well, what's the checksum of your abcl.jar?" "c48d359a23ee"  "Oh, you need
>> 846f78c279cb".  Ideally, I would like to come up with a mechanism that would
>> require that 'abcl.jar' come from "official" ABCL packaging, but would
>> somehow be able to introspect the JVM classpath to include ASDF definitions.
>>
> Maybe make it hard for anything but the original to be named abcl.jar?

Not really possible as far as I can figure things to avoid the ability 
to name things 'abcl.jar' (or are you being sarcastic?)  Solving the 
dynamic introspection problem seems more tractable, at the corresponding 
runtime cost of building a cache of the contents jars that the JVM knows 
about, but memory is getting cheaper by the ns, n'est-ce pas?

-- 
"A screaming comes across the sky.  It has happened before, but there
is nothing to compare to it now."