<div dir="ltr">Hmm so Spark being a distributed computing library I guess you need full serialization of functions/closures. Well, I can give it a shot, but don't expect anything too soon.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 30 Jul 2020 at 13:26, Steven Nunez <<a href="mailto:steve_nunez@yahoo.com">steve_nunez@yahoo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif;font-size:13px"><div></div>
<div dir="ltr"><div><div dir="ltr">Sorry, meant to send to the ABCL-dev list the first time.<br></div><div dir="ltr"><br></div><div dir="ltr">The use case is Spark lambda functions. I couldn't do a better job than the <a shape="rect" href="http://spark.apache.org/docs/latest/rdd-programming-guide.html" rel="nofollow" target="_blank">Spark RDD Programming Guide</a>
does at explaining the use case. It starts on the Basics heading. The
ideal case would be the ability to take Java code like this:</div><div dir="ltr"><span></span><pre><code><span>JavaRDD</span><span><</span><span>Integer</span><span>></span> <span>lineLengths</span> <span>=</span> <span>lines</span><span>.</span><span>map</span><span>(</span><span>s</span> <span>-></span> <span>s</span><span>.</span><span>length</span><span>());</span></code></pre></div><div dir="ltr">and write it in ABCL like this:</div><div dir="ltr"><br clear="none"></div><div dir="ltr"><span><font style="background-color:inherit" face="lucida console, sans-serif">(let ((line-lengths (#"map" *lines* (lambda (s) (length s)))))</font></span></div><div dir="ltr"><span><font style="background-color:inherit" face="lucida console, sans-serif"><br clear="none"></font></span></div><font face="Helvetica Neue, Helvetica, Arial, sans-serif"><span>This
uses the ABCL length function, which would be a huge win if we can use
Lisp functions to map across data structure. I've already got abcl.jar
accessible to Spark on all the nodes of a cluster. I'd probably shadow the cl:lambda with a spark:lambda to make the syntax natural.<br></span></font></div><div><br></div></div><div><br></div>
</div><div id="gmail-m_7837994359633009899yahoo_quoted_6710089963">
<div style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;color:rgb(38,40,42)">
<div>
On Thursday, July 30, 2020, 7:08:35 PM GMT+8, Alessio Stalla <<a href="mailto:alessiostalla@gmail.com" target="_blank">alessiostalla@gmail.com</a>> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="gmail-m_7837994359633009899yiv0600516597"><div><div><div>Somewhere in between.<div>I could give a shot at it. It would be useful if Steven detailed his use case a bit more.</div><br clear="none"><br clear="none"><div id="gmail-m_7837994359633009899yiv0600516597yqt05471"><div><div dir="ltr">On Thu, Jul 30, 2020, 13:04 Mark Evenson <<a rel="nofollow" shape="rect" href="mailto:evenson@panix.com" target="_blank">evenson@panix.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br clear="none">
<br clear="none">
> On Jul 30, 2020, at 10:00, Alessio Stalla <<a rel="nofollow" shape="rect" href="mailto:alessiostalla@gmail.com" target="_blank">alessiostalla@gmail.com</a>> wrote:<br clear="none">
> <br clear="none">
> Correction: indeed it was merged, but I didn't go as far as to make closures serializable.<br clear="none">
<br clear="none">
Any idea how much work for someone (i.e. me)to be able to serialize closures?<br clear="none">
Just a bit of elbow-grease, or major implementation?<br clear="none">
<br clear="none">
-- <br clear="none">
"A screaming comes across the sky. It has happened before but there is nothing <br clear="none">
to compare to it now."<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
<br clear="none">
</blockquote></div></div></div></div></div></div></div>
</div>
</div></div></blockquote></div>