[armedbear-devel] Space leak due to threads stuck in LispThread map

[Pendergrass, James A.]

On Jul 20, 2012, at 10:14 AM, Pendergrass, James A. wrote:

> Hi Erik,
> Thanks for the response.
>
> On Jul 19, 2012, at 6:37 PM, Erik Huelsmann wrote:
>
>> Hi James,
>>
>> On Thu, Jul 19, 2012 at 10:16 PM, Pendergrass, James A. <James.Pendergrass at jhuapl.edu> wrote:
>> Hello all,
>> I'm having some trouble with space leakage that looks to be because the static map in LispThread isn't letting go of completed JavaThreads.  Basically what I'm doing on the Java side looks like:
>>
>>> class Foo {
>>>      public static int main(){
>>>              manager = new Manager();
>>>              fun     = getLispFunction("processResults");
>>>
>>>              new Thread() {
>>>                      public void run() {
>>>                              LispObject res = generateSomeLisp();
>>>                              fun.execute(manager, res);
>>>                      }.run();
>>>              manager = null;
>>>              while(1);
>>>      }
>>> }
>>
>> The lisp side looks like:
>>
>>> (defun processResults (manager res)
>>>      (let ((*manager* manager))
>>>         (declare (special *manager*))
>>>         (eval res)))
>>
>> But the manager never gets destroyed.  I used the eclipse Java Memory Analysis tool to track this down to the created JavaThread being referenced as a key in the static map variable in LispThread.
>>
>> Any suggestions on how to get around this?  The whole code base is pretty memory intensive so garbage collecting that manager is really a must (and cleaning up the thread is probably a good plan too).
>>
>>
>> One way around this would be to use the LispThread class from the org.armedbear.lisp package: it removes the thread it creates from the map when the thread.run() function terminates.
>
> I'm looking in to this, but there seem to be two problems with it:
> 1) the constructors for LispThread aren't externally visible.
> 2) even if it was visible the constructor LispThread(Function fun,LispObject name) requires I create a 0-arity closure with the java manager object bound,  which always seems to involve LispThread.currentThread() which sticks the current JavaThread in the hash map.
>
>> The other option would be for someone to rewrite the map to use a weak hash table algorithm.
>>
>> The latter is probably the best long term solution anyway....
>
> This does seem like the best long term solution.  There seem to be a few WeakConcurrentHashMap implementations around, but I don't see anything in the core jdk (I could easily be missing something; I'm not really a java guy).  The choices seem to be Weak or Concurrent but not both.

The attached patch is my attempt at modifying LispThread to only hang on to Java Threads using weak references.  I haven't tested this extensively, but it solves my problem with out blowing up.
-Aaron


>
> -aaron
>
>>
>> Bye,
>>
>>
>> Erik.
>>
>
>
> _______________________________________________
> armedbear-devel mailing list
> armedbear-devel at common-lisp.net
> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/armedbear-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20120724/a2a7df4e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LispThread-WeakReferences.patch
Type: application/octet-stream
Size: 3470 bytes
Desc: LispThread-WeakReferences.patch
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20120724/a2a7df4e/attachment.obj>