[elephant-devel] Elephant backend performance characteristics

[Ian Eslick]

Unfortunately performance is one thing of several problems that  
violate the persistent object abstraction.  The first big thing to  
sanity check is that a set of operations is wrapped in a transaction.   
This avoids disk syncs after every primitive operation which can speed  
things up tremendously.

Currently each primitive elephant op takes place in its own  
transaction.  For example the slot write to an indexed slot is in a  
transaction with the index updates associated with it. If you write a  
set of indexed slots then each slot write is a separate transaction  
and this can become really expensive.  We often see this kind of  
performance issue reported on the list for data import procedures  
where it's the most obvious.

I highly recommend not using cached slots for the time being as they  
don't interact well with transactions at the present time.   
Associations can be considered to be of beta quality.

Ian

On Oct 28, 2008, at 7:40 AM, Leslie P. Polzer wrote:

>
>> Background: Our application that makes heavy use of elephant/bdb  
>> and the
>> association feature in the unstable branch is too slow by at least  
>> one order
>> of magnitude. We obviously need to profile it first before we start  
>> any
>> refactoring or changing the elephant configuration, and most likely  
>> we will
>> find that our application code used for the data import uses an  
>> inefficient
>> algorithm. But it might be helpful for interpreting the profile  
>> data when we
>> have some background knowledge about the performance  
>> characteristics one can
>> expect from elephant and its various backends.
>
> BDB is the fastest backend currently available. Postmodern
> is about half as fast and CLSQL lags three or four times
> behind.
>
> There are several areas here where a slowdown could occur:
> your algorithm, the association code (maybe Ian knows more
> about this) or maybe it's just the natural slowdown of
> Elephant's disk-backed reads and writes.
>
> Maybe using cached slots would help you, but that's
> work in progress.
>
>  Leslie
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel