[armedbear-devel] Recovering "special slots" after package deletion

[Erik Huelsmann]

The other day, Ville and I were discussing the issue with recovery of
"special slots" after package deletion on IRC. Let me start by
describing the problem for you first. After that I'll describe the
solution I see, however, your feedback on the solution is very much
appreciated.


Special symbol slots?
ABCL - like SBCL and CCL - uses a thread-local array containing the
'current' values of its special variables. All symbols which have been
declared or declaimed special at one point or another have been
assigned an index in this array.
This method allows very quick access to the special value (and hence
the symbol value) with only a single level of added indirection: the
symbols hold the number of their index in the array, meaning that
having a reference to the symbol and the current thread is enough to
find its value.  When I implemented it, it was a huge difference with
respect to the method we were using before.

Do those need release?
Since packages can be deleted, symbols be garbage collected.
In the current implementation, GC-ing a special symbol will leave the
slot it was assigned unused. There's no way the slot gets reassigned,
because nothing keeps track of these 'dead' slots. To make matters
worse: there may be huge lisp structures referenced in the dead slot,
because it's also not cleared out.
In the very long run, this will cause issues in scenarios as Blake
describes where one creates, deletes and re-creates packages. The
symbols in the packages are new symbols on every re-creation. Yet
there are only 4096 spots in the special-symbols table.

The solution?
Well, the solution I see is to use the Object.finalize() method to
release the index back to a freelist. There's a downside here though:
the 4096 slots may run out before a multi-gb system sees its first GC
run. There are two ways out here: grow the table once we run out of
free slots (even though many potentially free slots may be pending
GC). Or force a gc/runFinalization() run when running out of indices.
Surely a combination of both would also be an option.

Ville partially objects to the above because he's used to the pattern
"you can't depend on finalize() to be invoked". Reading its
documentation and googling the topic a bit, I see that finalize()
can't be depended upon especially on JVM termination. However, that
particular case isn't interesting here: we're looking at releasing
"memory" related resources, which seems to be the exact purpose of
finalize() as described in all the web docs I found.


Can you comment? Is my approach as sane as I think it is?


Bye,


Erik.