JDK8 Style Lambda Functions and ABCL

[Alessio Stalla]

Java lambdas are syntax sugar to implement interfaces with a single
non-default method. ABCL doesn't translate them automatically and doing so
dynamically would be slow (creating a proxy each time is costly) without
some advanced optimizations, but it could be done.
You could use some functions and macros to hide the complexity of creating
the proxy.

On Wed, 29 Jul 2020 at 09:30, Steven Nunez <steve_nunez at yahoo.com> wrote:

> Greetings all,
>
> I'm trying to convert the following Java code in the "Basics" section of
> the Spark Programming Guide
> <http://spark.apache.org/docs/latest/rdd-programming-guide.html> to ABCL:
>
> JavaRDD<Integer> lineLengths = lines.map(s -> s.length());
>
> I know that "s -> s.length" is a JDK8 style lambda function with one
> parameter, returning the result of calling length() on 's'. What I'd like
> to be able to do is write:
>
> (let ((line-lengths (#"map" *lines* (lambda (s) (#"length" s)))))
>
> but this isn't getting me anywhere, with Java saying there is no
> applicable method 'map
> <https://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaRDDLike.html#map-org.apache.spark.api.java.function.Function->'
> on *lines* (an instance of JavaRDD
> <https://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaRDD.html>).
> There is such a method (if it matters, it is inherited by JavaRDD from
> interface JavaRDDLike
> <https://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaRDDLike.html>).
> Investigating that map method a bit further, it seems to want an
> org.apache.spark.api.java.function
> <https://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/function/Function.html>.
> Here's a clip from the Spark description:
>
> *Spark’s API relies heavily on passing functions in the driver program to
> run on the cluster. In Java, functions are represented by classes
> implementing the interfaces in the org.apache.spark.api.java.function
> package. There are two ways to create such functions:*
>
>    - *Implement the Function interfaces in your own class, either as an
>    anonymous inner class or a named one, and pass an instance of it to Spark.*
>    - *Use lambda expressions
>    <http://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html>
>    to concisely define an implementation.*
>
> I think what I'm getting from the ABCL lambda expression is a
> java.util.function:
>
> SPARK> (describe (lambda (s) #"length" s))
> #<FUNCTION #<FUNCTION (LAMBDA (S)) {70B82AD0}> {70B82AD0}> is an object of
> type FUNCTION.
> The function's lambda list is:
>   (S)
>
> I do wonder though how the Java lambda "s -> s.length()" manages to
> produce the correct result, so this theory may not be correct.
>
> The Spark guide goes on to say:
>
> *While much of this guide uses lambda syntax for conciseness, it is easy
> to use all the same APIs in long-form. For example, we could have written
> our code above as follows:*
>
> *JavaRDD<String> lines = sc.textFile("data.txt");*
> *JavaRDD<Integer> lineLengths = lines.map(new Function<String, Integer>()
> {*
> *   public Integer call(String s) { return s.length(); }*
> *});*
> *int totalLength = lineLengths.reduce(new Function2<Integer, Integer,
> Integer>() {*
> *  public Integer call(Integer a, Integer b) { return a + b; }*
> *});*
>
> *Or, if writing the functions inline is unwieldy:*
>
> *class GetLength implements Function<String, Integer> {*
> *    public Integer call(String s) { return s.length(); }*
> *}*
> *class Sum implements Function2<Integer, Integer, Integer> {*
> *   public Integer call(Integer a, Integer b) { return a + b; }*
> *}*
>
> *JavaRDD<String> lines = sc.textFile("data.txt");*
> *JavaRDD<Integer> lineLengths = lines.map(new GetLength());*
> *int totalLength = lineLengths.reduce(new Sum());*
>
>
> To me, both of those examples look unwieldy. There was a stack overflow
> discussion on creating Java classes with ABCL
> <https://stackoverflow.com/questions/4785969/can-you-write-a-java-class-with-abcl>
> from 9 years ago, and I've read the java interface examples
> <https://abcl.org/trac/browser/trunk/abcl/examples/java-interface> on
> abcl.org, but both of those techniques look like they will produce code
> just as unwieldy as the Java syntax above.
>
> Is there any way to use a lisp-style lambda syntax to produce a function
> that will satisfy Spark's requirement to implement the
> org.apache.spark.api.java.function interface? If not, the obvious route
> would be some macrology to create a defsparkfun, defsparkfun2, etc. that
> wrap the ABCL function I want to use with something that implements 'call'
> in the org.apache.spark.api.java.function interface. I'm hoping there's a
> better way.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20200729/2c5e42ef/attachment.htm>