<html><head></head><body><div class="ydpd7993f61yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div></div>
<div dir="ltr" data-setdir="false">24 hours later and little progress. I have determined that moving the properties file into the ABCL project directory enables me to get an inputstream on it from ABCL, but the application library still fails to load.<span></span><br></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">It (still) looks like a class loader issue. What I'd really like is a macro along lines of:</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">(with-class-loader 'foo</div><div dir="ltr" data-setdir="false"> ...</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">which would quickly confirm or eliminate that hypothesis. Anyone know if one exists, or something similar?</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Cheers,</div><div dir="ltr" data-setdir="false"> Steve<br></div><div><br></div>
</div><div id="ydpa1dc6962yahoo_quoted_6179434685" class="ydpa1dc6962yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div><br>
</div><div><br></div><div>On Tuesday, July 21, 2020, 3:22:13 PM GMT+8, Steven Nunez <steve_nunez@yahoo.com> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id="ydpa1dc6962yiv9046726048"><div><div class="ydpa1dc6962yiv9046726048yahoo-style-wrap" style="font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div dir="ltr">Greetings all,</div><div dir="ltr"><br></div><div dir="ltr">I have what I think is a problem with the ABCL class loader. I am working with a 'big data' library, Spark, but run into an issue on line 2 of the <a href="http://spark.apache.org/docs/latest/rdd-programming-guide.html" class="ydpa1dc6962yiv9046726048" rel="nofollow" target="_blank">programming guide</a> example. I am able to load the JARs from Maven with the ASDF system definition:</div><div dir="ltr"><font face="lucida console, sans-serif"><br></font></div><div dir="ltr"><div><font face="lucida console, sans-serif">(asdf:defsystem #:spark<br> :description "Wrapper for Spark 3.0"<br> :serial t<br> :defsystem-depends-on (abcl-asdf)<br> :depends-on (#:jss #:javaparser)<br> :components ((:mvn "org.apache.spark/spark-core_2.12" :version "3.0.0")<br> (:file "package")<br> (:file "spark")))</font></div><div><br></div></div><div><br></div><div dir="ltr">and can create a SparkConf object:</div><div dir="ltr"><br></div><div dir="ltr"><div><font face="lucida console, sans-serif">(defvar *spark-conf*<br> #1"new SparkConf()<br> .setAppName("abcl-app")<br> .setMaster("local")" )</font></div><div><br></div></div><div dir="ltr"><br></div><div dir="ltr">But when I try to create a 'context'</div><div dir="ltr"><br></div><div dir="ltr"><font face="lucida console, sans-serif"><span>(defvar *sc* (new 'JavaSparkContext *spark-conf*))</span></font><br></div><div dir="ltr"><br></div><div dir="ltr">I get an error in the initialisation:</div><div dir="ltr"><br></div><div dir="ltr"><font face="lucida console, sans-serif"><span>Java exception 'java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.package$'.</span></font><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">There isn't much on this except from the Jenkins guys, who have attempted to put Spark and Spark applications into a CI system. They seem to think that it's related to a call to <a href="https://github.com/apache/spark/blob/a4ca355af8556e8c5948e492ef70ef0b48416dc4/core/src/main/scala/org/apache/spark/package.scala#L59" class="ydpa1dc6962yiv9046726048" rel="nofollow" target="_blank">get a properties file</a> in the package class and on a <a href="https://stackoverflow.com/questions/42751816/spark-version-info-properties-not-found-in-jenkins" class="ydpa1dc6962yiv9046726048" rel="nofollow" target="_blank">StackOverflow discussion</a> suggested that "<span>you should make sure that you set the classloader that Spark was loaded through using the <code>Thread.currentThread().setContextClassLoader(myCustomLoader)</code> call".<br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span>I've verified with <span>(java:dump-classpath) that the JAR is on the ABCL classpath, and the JAR file does contain the <span><code>spark-version-info.properties</code></span> file. I've also tried getting the file myself with:</span></span></div><div dir="ltr"><span><span><br></span></span></div><div dir="ltr"><font face="lucida console, sans-serif"><span><div>(defvar rs<br> #1"Thread.currentThread()<br> .getContextClassLoader()<br> .getResourceAsStream("spark-version-info.properties")" )</div></span></font><div><br></div><div dir="ltr">which returns nil<span><span>, so their theory may be correct.</span></span></div></div><div dir="ltr"><span><span><br></span></span></div><div dir="ltr"><span><span>Messing around with class loaders is a bit beyond my 20 year old Java knowledge so I thought I'd ask here if anyone has any ideas on how I can load Spark in way to use the default Java class loader. Alternatively it occurs to me to ask <i>why </i>the ABCL class loader isn't able to find the properties file if the JAR is on the classpath and then to correct whatever that problem is.</span></span><br></div><div><br></div><div dir="ltr">Cheers,</div><div dir="ltr"> Steve<br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div></div></div></div>
</div>
</div></body></html>