[rucksack-devel] GC problems

[Arthur Lemmens]

Edi Weitz wrote:

>> Would you agree that this problem can also be solved by writing
>> dirty objects to disk when the scanner sees them?  Or am I missing
>> something?
>
> I'm not sure.  When you're in the DOTIMES loop above (which is
> supposed to take a /long/ time), the only reference to INNER is from
> an object which is in RAM.  Whatever you write to disk, you won't be
> able to reach INNER unless you have left the loop.  But at that point
> it'll probably be too late.

Yeah, maybe.

Anyway, after some more thinking I've decided to solve the GC problem
for now by not doing any GC while a transaction is writing objects to
disk.  Instead I keep track of the amount of disk space allocated by
the committing transaction.  Then I do a (partial) GC immediately after
committing the transaction.

This turns out to be good enough to pass your CHECK-GC test (and
TEST-BTREE & friends now finally work for large values of N).

I also added a SERIAL-TRANSACTION-RUCKSACK class that allows for only
one transaction at a time (by using a transaction lock).  This allows
for a fast track towards a working Rucksack implementation.  Once we
get all the basics working, we can add parallel transactions.  At that
point, we may also have to find a more fine-grained solution for the
GC problem.

And for you non-Lispworks users of Rucksack (if there are any left):
to implement serial transactions I had to use PROCESS-LOCK and
PROCESS-UNLOCK.  If you don't use Lispworks you'll have to add your own
implementation-dependent definitions of these functions.  But that
shouldn't be too difficult for any implementation that supports threads,
I hope.  Patches welcome, of course.

Arthur