[elephant-devel] txn

[Alex Mizrahi]

hello

i'm now going to improve transaction handling in db-postmodern.

first of all, i'm going to use "serializable" isolation level instead of 
default "read commited". this means: transaction can fail because of 
concurrent slot updates.
with default isolation level concurrent updates were OK, last value 
persisted.
so with default isolation level operations like INCF could be just ignored, 
that's not safe in some cases.

another difference is that with default isolation level it was possible to 
read data that was commited after txn started -- this artefact is also 
fixed.

they say there should be no performance impact, but now we should be ready 
to actually retry transaction body (actually deadlocks were possible before, 
but since they are infrequent we ignored them).

first of all, some aspects of execute-transaction implementation:

 * in BDB backend retries count is actually "tries count", so retries = 1 
means "no retries".

   i find it counter-logical: there is always one first try, and then N 
retries.
   0 = retries means that there still should be one TRY, but no REtries.

   i'm not only one interpreting parameter in this way, so please fix this 
in bdb implementation.

 * transaction-retry-count-exceeded condition does not have slot for 
original cause. that is weird.
   application catches retry-count-exceeded. but what in a hell happened??
   i can extend this condition for db-postmodern, but i think it will be 
better if all backends would handle it in this way.

 * for a case retries = 0 i think it would be better to throw original 
condition rather than transaction-retry-count-exceeded,
   since there were no retries.. also this can help debugging in some way.


however, i found it's problematic to integrate txn retrying in complex 
application having side effects (i.e. web framework).
at same time using low level primitives in this case is ugly.

i think it's possible to make it much more flexible with single callback 
function, that is called when transaction is about to be restarted.
so applications has chance to:
 * log error, so later it would be possible to diagnose performance problems 
etc.
 * do cleanup of side-effects
 * implement some other restarting mechanism

this will be called retry-cleanup-fn key parameter.

additionally, i've found some problems with "separatist" error handlers 
i.e.:

(ele:with-transaction ()
 (handler-case (work-with-database)
   (error () (pring "we have a no-go")))

if work-with-database produced some error, with-transaction will not able to 
know about this and thus could commit erroneous data.

of course this is a problem of application, but sometimes such handlers are 
installed by web framework or dictated by architecture in some way. and it 
will be nice to be able to deal with such cases.

i have some ideas how to handle this: nested with-transaction instead of 
simply calling function (because SQL does not have nested transactions) 
could detect abnormal control flow or conditions signaled and inform parent 
transaction about this -- so it can rollback or retry. but i'm not going to 
implement this right now.

other extensions in db-postmodern's execute-transaction:
 * always-rollback parameter that turns off auto-commit. could be useful if 
you're doing test and do not want results in database..
 * execute-transaction creates restart named 
db-postmodern::retry-transaction, so application can programmatically force 
retry.

comments appreciated.

with best regards, Alex 'killerstorm' Mizrahi.