Following up on this,<div><br></div><div>I have some code by modifying some of the call chain for compile, jvm-compile etc in compiler-pass2.lisp that lets me generate class files where the compiled code for top level functions are stored. I have checked that I can load this and call this (and managed to do this without initializing the interpreter) from pure java land. One case that is not covered is a set of defuns within a let binding, for example</div>
<div><br></div><div>(let (y) (defun get-y() y) (defun set-y (x) (setq y x)) (defun add-x (x) (+ x y)))</div><div><br></div><div>right now compile-to-file only lets me compile these separately, and creates separate symbols static fields in each file separately and tries to initialize them in the static initialization blocks using a Lisp.recall function call. There is something funky going on in the compilation re whether this is in memory or not, but each function gets its own class and there does not seem an easy way to compile just the let form in the top level.</div>
<div><br></div><div>I will write and tests to see if my changes to jnew-runtime-class can work as well, and would like to co-ordinate on how to do more extensive testing and merging.</div><div><br></div><div>Archi</div><div>
<br><br><div class="gmail_quote">On Fri, Sep 21, 2012 at 11:52 AM, Alessio Stalla <span dir="ltr"><<a href="mailto:alessiostalla@gmail.com" target="_blank">alessiostalla@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Fri, Sep 21, 2012 at 4:39 PM, archisman rudra <<a href="mailto:archi.rudra@gmail.com">archi.rudra@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> hopefully this is the correct list. If not please let me know.<br>
<br>
</div>Hi,<br>
<br>
yes, you are writing to the correct list - welcome!<br>
<div class="im"><br>
> Background:<br>
> I am trying to use abcl to generate jars to send off to a hadoop cluster for<br>
> execution.<br>
<br>
</div>Interesting! I'm answering because I wrote the current (incomplete)<br>
version of jnew-runtime-class. Just FYI, you can find some notes about<br>
it at <<a href="http://trac.common-lisp.net/armedbear/wiki/JavaFfi/RuntimeClass" target="_blank">http://trac.common-lisp.net/armedbear/wiki/JavaFfi/RuntimeClass</a>>.<br>
<div class="im"><br>
> The svn version of the jnew-runtime-class does not store the<br>
> actual method bodies in the generated class file, but rather loads the<br>
> generated class file and sets the relevant field. Here is the relevant code<br>
> :<br>
><br>
> (multiple-value-bind (class-file method-implementation-fields)<br>
> (apply #'java::%jnew-runtime-class class-name stream args)<br>
> (sys::put-memory-function memory-class-loader<br>
> class-name (sys::%get-output-stream-bytes<br>
> stream))<br>
> (let ((jclass (java:jcall "loadClass" memory-class-loader<br>
> class-name)))<br>
> (dolist (method method-implementation-fields)<br>
> (setf (java:jfield jclass (car method)) (cdr method)))<br>
> jclass))))<br>
><br>
><br>
> This means that the generated jar file cannot be easily used from standard<br>
> java code.<br>
<br>
</div>This is a byproduct of the design - although the design is mine, and<br>
hasn't been discussed much among the other developers, so I'm not<br>
representing anyone but myself. Let me clarify below.<br>
<div class="im"><br>
> I would like to add some code in the static initialization block<br>
> of generated class file to do something equivalent. I will have to save the<br>
> lambda expression for the method and have been experimenting as follows:<br>
><br>
> (defvar *cf*)<br>
> (defvar *mif*)<br>
><br>
> (with-open-file (stream "test.class" :direction :output :element-type<br>
> '(unsigned-byte 8))<br>
> (multiple-value-bind (class-file method-implementation-field)<br>
> (java::%jnew-runtime-class "test" stream :methods<br>
> `(("forty_two_method1" :int () ,(lambda (this) 42))))<br>
> (setf *cf* class-file)<br>
> (setf *mif* method-implementation-field)))<br>
><br>
> (with-open-file (stream "test_forms" :direction :output)<br>
> (dolist (m *mif*)<br>
> (system:dump-form (list (car m) (function-lambda-expression (cdr m)))<br>
> stream)))<br>
><br>
><br>
> This assumes that the lambda expression does not refer to any variable or<br>
> user defined function, but perhaps the asdf-jar contrib might help here.<br>
><br>
> I was going to write the relevant code and start using it for my project,<br>
> but I was wondering if I am duplicating work that others have already done<br>
> or are doing and whether there are other libraries (jffli perhaps, I am not<br>
> very familiar with it) that have similar functionality. Also if there are<br>
> other interactions with other parts of the system that I am perhaps missing.<br>
><br>
> I would e grateful for any advice.<br>
<br>
</div>What I wanted to achieve is to have methods that can be:<br>
* arbitrary functions, including closures<br>
* possibly redefined after the class has been loaded. Redefinition is<br>
possible by using symbols as method implementations.<br>
<br>
There's also a hidden goal that is to keep the implementation simple<br>
enough, and compiling Lisp code to the body of methods in an arbitrary<br>
class is not easy at the moment.<br>
<br>
What explicitly is NOT a requirement is to have the generated class be<br>
independent from ABCL (the methods are implemented in Lisp and will<br>
always need runtime support). So, no matter how you arrange things,<br>
you will never be able to distribute your class without abcl.jar. Note<br>
that it is the same for Clojure, Scala, Groovy, etc.<br>
<br>
I don't know specifically how Hadoop works, but, as I see things, you need to<br>
1. provide abcl.jar + your class to it<br>
2. have Hadoop execute enough code to load ABCL and initialize your class.<br>
<br>
Now, probably the key point is who loads your class.<br>
* if you can arrange to have ABCL load it and pass it to Hadoop, you<br>
should be good with the current design.<br>
* if Hadoop wants to load it by itself from a jar, then yes, you have<br>
to somehow dump the method implementations somewhere and have the<br>
class auto-load them from there when it is initialized.<br>
<br>
For the second case, I propose an intermediate design that is, imho,<br>
simpler than dumping arbitrary forms: special-case the situation when<br>
a method implementation is a symbol. In that case, directly generate<br>
an instruction in the class file to find that symbol in its package<br>
and assign it to the appropriate field. If the implementation of a<br>
method is not a symbol nothing changes. That way, you will still need<br>
to init ABCL before loading your class, but, when all your method<br>
implementations are symbols, the class will be able to initialize<br>
itself without external support. If some methods are proper functions,<br>
instead, the class won't be self-contained, but will need some Lisp<br>
code to be executed to be properly initialized.<br>
<br>
Does it make sense to you? Do you find it acceptable? If yes, I can<br>
give a shot at implementing it, if you want.<br>
<br>
Peace,<br>
Alessio<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Some gratuitous spam:<br>
<br>
<a href="http://ripple-project.org" target="_blank">http://ripple-project.org</a> Ripple, social credit system<br>
<a href="http://villages.cc" target="_blank">http://villages.cc</a> Villages.cc, Ripple-powered community economy<br>
<a href="http://common-lisp.net/project/armedbear" target="_blank">http://common-lisp.net/project/armedbear</a> ABCL, Common Lisp on the JVM<br>
<a href="http://code.google.com/p/tapulli" target="_blank">http://code.google.com/p/tapulli</a> my current open source projects<br>
<a href="http://www.manydesigns.com/" target="_blank">http://www.manydesigns.com/</a> ManyDesigns Portofino, open source<br>
model-driven Java web application framework<br>
</font></span></blockquote></div><br></div>