[armedbear-devel] Question about jnew-runtime-class

[archisman rudra]

Following up on this,

I have some code by modifying some of the call chain for compile,
jvm-compile etc in compiler-pass2.lisp that lets me generate class files
where the compiled code for top level functions are stored. I have checked
that I can load this and call this (and managed to do this without
initializing the interpreter) from pure java land. One case that is not
covered is a set of defuns within a let binding, for example

(let (y) (defun get-y() y) (defun set-y (x) (setq y x)) (defun add-x (x) (+
x y)))

right now compile-to-file only lets me compile these separately, and
creates separate symbols static fields in each file separately and tries to
initialize them in the static initialization blocks using a Lisp.recall
function call. There is something funky going on in the compilation re
whether this is in memory or not, but each function gets its own class and
there does not seem an easy way to compile just the let form  in the top
level.

I will write and tests to see if my changes to jnew-runtime-class can work
as well, and would like to co-ordinate on how to do more extensive testing
and merging.

Archi


On Fri, Sep 21, 2012 at 11:52 AM, Alessio Stalla <alessiostalla at gmail.com>wrote:

> On Fri, Sep 21, 2012 at 4:39 PM, archisman rudra <archi.rudra at gmail.com>
> wrote:
> > Hi,
> >
> > hopefully this is the correct list. If not please let me know.
>
> Hi,
>
> yes, you are writing to the correct list - welcome!
>
> > Background:
> > I am trying to use abcl to generate jars to send off to a hadoop cluster
> for
> > execution.
>
> Interesting! I'm answering because I wrote the current (incomplete)
> version of jnew-runtime-class. Just FYI, you can find some notes about
> it at <http://trac.common-lisp.net/armedbear/wiki/JavaFfi/RuntimeClass>.
>
> > The svn version of the jnew-runtime-class does not store the
> > actual method bodies in the generated class file, but rather loads the
> > generated class file and sets the relevant field. Here is the relevant
> code
> > :
> >
> >     (multiple-value-bind (class-file method-implementation-fields)
> >         (apply #'java::%jnew-runtime-class class-name stream args)
> >       (sys::put-memory-function memory-class-loader
> >                                 class-name (sys::%get-output-stream-bytes
> > stream))
> >       (let ((jclass (java:jcall "loadClass" memory-class-loader
> > class-name)))
> >         (dolist (method method-implementation-fields)
> >           (setf (java:jfield jclass (car method)) (cdr method)))
> >         jclass))))
> >
> >
> > This means that the generated jar file cannot be easily used from
> standard
> > java code.
>
> This is a byproduct of the design - although the design is mine, and
> hasn't been discussed much among the other developers, so I'm not
> representing anyone but myself. Let me clarify below.
>
> > I would like to add some code in the static initialization block
> > of generated class file to do something equivalent. I will have to save
> the
> > lambda expression for the method and have been experimenting as follows:
> >
> > (defvar *cf*)
> > (defvar *mif*)
> >
> > (with-open-file (stream "test.class" :direction :output :element-type
> > '(unsigned-byte 8))
> >   (multiple-value-bind (class-file method-implementation-field)
> >       (java::%jnew-runtime-class "test" stream :methods
> > `(("forty_two_method1" :int () ,(lambda (this) 42))))
> >     (setf *cf* class-file)
> >     (setf *mif* method-implementation-field)))
> >
> > (with-open-file (stream "test_forms" :direction :output)
> >   (dolist (m *mif*)
> >     (system:dump-form (list (car m) (function-lambda-expression (cdr m)))
> > stream)))
> >
> >
> > This assumes that the lambda expression does not refer to any variable or
> > user defined function, but perhaps the asdf-jar contrib might help here.
> >
> > I was going to write the relevant code and start using it for my project,
> > but I was wondering if I am duplicating work that others have already
> done
> > or are doing and whether there are other libraries (jffli perhaps, I am
> not
> > very familiar with it) that have similar functionality. Also if there are
> > other interactions with other parts of the system that I am perhaps
> missing.
> >
> > I would e grateful for any advice.
>
> What I wanted to achieve is to have methods that can be:
>  * arbitrary functions, including closures
>  * possibly redefined after the class has been loaded. Redefinition is
> possible by using symbols as method implementations.
>
> There's also a hidden goal that is to keep the implementation simple
> enough, and compiling Lisp code to the body of methods in an arbitrary
> class is not easy at the moment.
>
> What explicitly is NOT a requirement is to have the generated class be
> independent from ABCL (the methods are implemented in Lisp and will
> always need runtime support). So, no matter how you arrange things,
> you will never be able to distribute your class without abcl.jar. Note
> that it is the same for Clojure, Scala, Groovy, etc.
>
> I don't know specifically how Hadoop works, but, as I see things, you need
> to
>  1. provide abcl.jar + your class to it
>  2. have Hadoop execute enough code to load ABCL and initialize your class.
>
> Now, probably the key point is who loads your class.
>  * if you can arrange to have ABCL load it and pass it to Hadoop, you
> should be good with the current design.
>  * if Hadoop wants to load it by itself from a jar, then yes, you have
> to somehow dump the method implementations somewhere and have the
> class auto-load them from there when it is initialized.
>
> For the second case, I propose an intermediate design that is, imho,
> simpler than dumping arbitrary forms: special-case the situation when
> a method implementation is a symbol. In that case, directly generate
> an instruction in the class file to find that symbol in its package
> and assign it to the appropriate field. If the implementation of a
> method is not a symbol nothing changes. That way, you will still need
> to init ABCL before loading your class, but, when all your method
> implementations are symbols, the class will be able to initialize
> itself without external support. If some methods are proper functions,
> instead, the class won't be self-contained, but will need some Lisp
> code to be executed to be properly initialized.
>
> Does it make sense to you? Do you find it acceptable? If yes, I can
> give a shot at implementing it, if you want.
>
> Peace,
> Alessio
>
> --
> Some gratuitous spam:
>
> http://ripple-project.org Ripple, social credit system
> http://villages.cc Villages.cc, Ripple-powered community economy
> http://common-lisp.net/project/armedbear ABCL, Common Lisp on the JVM
> http://code.google.com/p/tapulli my current open source projects
> http://www.manydesigns.com/ ManyDesigns Portofino, open source
> model-driven Java web application framework
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20121001/b01dc49b/attachment.html>