[rucksack-devel] Re: Rucksack, ECLM

[Nikodemus Siivola]

"Arthur Lemmens" <alemmens at xs4all.nl> writes:

>> If X was initialized as (make-instance 'my-p-thing :slot
>> (make-instance 'my-p-thing-2)), then by the time we tried to add the
>> root neither Lisp nor Rucksack would have the P-THING-2 around...
>
> Well... Lisp would still have the P-THING-2 around, if I understand
> your example correctly.

My intention was that the SLOT was proxied -- so Lisp would not have
a direct reference to it.

>>> Probably the best approach would be to have the GC automatically mark
>>> the in-memory copy of X as dead.  Then we could signal an error (how
>>> about NECROPHILIA-NOT-ALLOWED-HERE?) for code that tries to do something
>>> with a dead object.
>>
>> I assume you mean "committing the transaction", not GC?
>
> No, I meant GC.  The transaction has no way of knowing if X is dead or not.
> The only way to know that for sure is by doing the complete trace phase of
> the garbage collector (just checking the roots is not enough).  That's
> obviously not something you want to do for each transaction commit.

Right. (Unless we maintained back-pointers for all pointers on disk,
but that sounds like a horrible idea.)

>> If legality of the add-root bit depends on the amount of garbage
>> collected in between.
>
> Yeah, that's unfortunate but there's nothing we can do about that.  That's
> why I really think it's a "don't do that then" scenario.  We can try to
> do some cheap checks to prevent this kind of stuff from happening, but
> we can't give any guarantee that you're protected from this.  It's just
> too expensive.  Unless I'm missing something, of course.

I'm not sure.

Starting from the top: to do anything at all with a persistent object
(ignoring non-persistent slots) you need to

 1. Have the rucksack the object belongs to open.
 2. Be in a transaction.

Could this "anything" be extended to holding a reference?

So that when a transaction exits, all in-memory objects with that
transaction-id become invalid, and trying to use them in the context
of another transaction would signal an error.

Then, the only way another transaction can legally get a hold of the
same object is by getting it afresh from the Rucksack.

That provides for the clear semantics and determinism, but is somewhat
inconvenient. No problem: indexed objects can be exempted from this
invalidation, as they are always reachable (and it is cheap to check
if an object is directly indexed). For other cases P-LET (call WITH-ROOTS)
can provide the same guarantee and convenience:

 (WITH-ROOTS (X Y) ...) =>

 (LET (X Y)
   (LET ((#:TAIL-THUNK *ROOT-THUNK*)
         (*ROOT-THUNK* (LAMBDA () (LIST* X Y (FUNCALL #:TAIL-THUNK)))))
      ...))

Now, both GC and transactions have access to additional in-memory
roots through the *ROOT-THUNK*. GC can use them just as regular roots.
Transactions just need to check if an object with an old
transaction-id is in the in-memory roots: if so, then using it is
fine. If not, an error is signalled.

Does that make sense?

(Strictly speaking, I think WITH-ROOTS needs to jump through a couple
of extra hoops to ensure single-assignment semantics, as otherwise you
can still get non-deterministic behaviour by removing an object from
the roots for the duration of GC and then putting it back. That won't
be too hard, though: nothing a symbol-macro can't handle.)

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."