[elephant-devel] txn

[Ian Eslick]

On Feb 20, 2008, at 5:14 PM, Alex Mizrahi wrote:

> hello
>
> i'm now going to improve transaction handling in db-postmodern.

Great!  We should make sure this doesn't change the API, or if it does  
to create tracking changes in the BDB and CL-SQL stores.

> first of all, i'm going to use "serializable" isolation level  
> instead of
> default "read commited". this means: transaction can fail because of
> concurrent slot updates.

This is the BDB default.  We have keyword options to :read-uncommitted  
(see bdb-transactions.lisp) or the documentation for other options.

> with default isolation level concurrent updates were OK, last value
> persisted.
> so with default isolation level operations like INCF could be just  
> ignored,
> that's not safe in some cases.
>
> another difference is that with default isolation level it was  
> possible to
> read data that was commited after txn started -- this artefact is also
> fixed.
>
> they say there should be no performance impact, but now we should be  
> ready
> to actually retry transaction body (actually deadlocks were possible  
> before,
> but since they are infrequent we ignored them).
>
> first of all, some aspects of execute-transaction implementation:
>
> * in BDB backend retries count is actually "tries count", so retries  
> = 1
> means "no retries".
>
>   i find it counter-logical: there is always one first try, and then N
> retries.
>   0 = retries means that there still should be one TRY, but no  
> REtries.
>
>   i'm not only one interpreting parameter in this way, so please fix  
> this
> in bdb implementation.

I just checked in a patch to correct this, good eye!

> * transaction-retry-count-exceeded condition does not have slot for
> original cause. that is weird.
>   application catches retry-count-exceeded. but what in a hell  
> happened??
>   i can extend this condition for db-postmodern, but i think it will  
> be
> better if all backends would handle it in this way.
>
> * for a case retries = 0 i think it would be better to throw original
> condition rather than transaction-retry-count-exceeded,
>   since there were no retries.. also this can help debugging in some  
> way.
>

In the db-bdb store, transaction-retry-count-exceeded only happens  
when the non-local exist from the transaction closure is a DEADLOCK or  
a LOCK_NOT_GRANTED.  Any other non-local exit results in the  
transaction being aborted and the error being returned.

>
> however, i found it's problematic to integrate txn retrying in complex
> application having side effects (i.e. web framework).
> at same time using low level primitives in this case is ugly.
>
> i think it's possible to make it much more flexible with single  
> callback
> function, that is called when transaction is about to be restarted.
> so applications has chance to:
> * log error, so later it would be possible to diagnose performance  
> problems
> etc.
> * do cleanup of side-effects
> * implement some other restarting mechanism
>
> this will be called retry-cleanup-fn key parameter.

Great!  Let me know when you have it, what assumptions you're making,  
and I'll update db-bdb to match.

> additionally, i've found some problems with "separatist" error  
> handlers
> i.e.:
>
> (ele:with-transaction ()
> (handler-case (work-with-database)
>   (error () (pring "we have a no-go")))
>
> if work-with-database produced some error, with-transaction will not  
> able to
> know about this and thus could commit erroneous data.
>
> of course this is a problem of application, but sometimes such  
> handlers are
> installed by web framework or dictated by architecture in some way.  
> and it
> will be nice to be able to deal with such cases.
>
> i have some ideas how to handle this: nested with-transaction  
> instead of
> simply calling function (because SQL does not have nested  
> transactions)
> could detect abnormal control flow or conditions signaled and inform  
> parent
> transaction about this -- so it can rollback or retry. but i'm not  
> going to
> implement this right now.
>

Too much magic and too much trying to do the work for the developers  
can make the system more complicated and actually make the developer's  
life harder, because architectural errors are hidden until they become  
really complicated.  I'm hesitant about the above due to this principle.

> other extensions in db-postmodern's execute-transaction:
> * always-rollback parameter that turns off auto-commit. could be  
> useful if
> you're doing test and do not want results in database..

What would the usage model be?  I don't know how that makes sense  
since it means you always have an implicit, active transaction.  The  
easiest way to do this today is to explicitly start a transaction  
(controller-start-transaction *sc*) and then do your tests and then  
call (controller-abort-transaction *sc* txnid) when you're done -- as  
long as the tests don't blow out your transaction cache.

>
> * execute-transaction creates restart named
> db-postmodern::retry-transaction, so application can  
> programmatically force
> retry.
>

That's a good idea.  I assume this has an implicit abort tied to it,  
or do you need to abort and then retry?

> comments appreciated.
>
> with best regards, Alex 'killerstorm' Mizrahi.
>
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel