[elephant-devel] Optimization

[Elliott Slaughter]

On Sun, Dec 28, 2008 at 2:09 PM, Ian Eslick <eslick at media.mit.edu> wrote:

> I implemented a quick check-in/check-out protocol for persistent

objects.


Nice. Mind sharing?

A standard-object slot allows about 50M updates-sec
> A persistent-object slot (inside a txn) is about 60k updates-sec
> A cached persistent-object slot is about 3M updates-sec


I implemented my own version of caching using Alex's suggestion (with slot
defs caching their values in hash tables).

With a simple loop (code attached), I get about 30M updates per second (Hz)
for standard slots, about 30 kHz for persistent slots, and about 2.4 MHz
using my caching scheme.

The slot access protocol adds about 10x overhead per slot read but the
> cached access is about 50x faster than the pure persistent query.
> I'll see if there are some easy opportunities to speed things up.


I observed about the same performance differences (although I saw closer to
an 100x difference between persistence and caching).

My method uses hash table lookups for slot reads, which makes me wonder if
I'd get better performance by using a slot caching protocol more similar to
the default implementation.

Anyways, using cached slots in my game results in a 2x speed increase,
although it still runs 2x to 3x slower than the completely in-memory
version. I guess I have other performance concerns to think about than just
slot reads.

Ian
>
> On Dec 28, 2008, at 2:05 AM, Elliott Slaughter wrote:
>
> > There are several replies to my original query, so I will attempt to
> > address all of them here.
> >
> > On Wed, Dec 24, 2008 at 2:31 PM, Ian Eslick <eslick at media.mit.edu>
> > wrote:
> > A couple of quick thoughts on your problem:
> >
> > 1) Are you wrapping the critical sections of your code in with-
> > transaction?  This causes all database pages you touch to be cached
> > within the body of the transaction.  This avoids all 'sync' operations
> > and transaction setup/teardown caused by a read/write slot operation
> > that takes place outside a with-transaction body.
> >
> > I currently wrap the contents of each frame update in with-
> > transaction. Each frame consists of updating 100 objects on the
> > screen (which by my estimation ought to be about 20 slot reads and 2
> > writes), so I ought to be doing about 2000 slot reads each frame and
> > 200 writes.
> >
> > Wrapping the entire game loop in with-transaction increases
> > performance by about 10%. I also tried
> >
> > (db-env-set-flags (controller-environment *store-controller*) 1 :txn-
> > nosync t)
> >
> > as suggested in the manual, with a similar (about 10%) performance
> > increase.
> >
> > 2) If you are doing this, have you increased your BDB cache size to
> > ensure you can cache your key working set?
> >
> > It should already be large enough, but just for kicks I increased it
> > from 2 to 10 MB, resulting in a performance increase of about 3%.
> >
> > 3) If yes, do you have significant contention between reads and writes
> > for certain data structures that are causing transactions to be
> > aborted.  Perhaps you can refactor around this.
> >
> > This is a single-threaded (and single-process) app. I don't see how
> > I could possibly have contention for the db.
> >
> > Ian
> >
> > And thanks to both Leslie and Alex for suggesting caching methods. I
> > will probably try to implement something along the lines of Alex's
> > suggestion, as that makes sense and should be simple enough (for the
> > single threaded model, which is sufficient for me).
> >
> >
> > Thanks again for all the help. I'll report back either when I have
> > something working or I've run into another wall.
> >
> > --
> > Elliott Slaughter
> >
> > "Any road followed precisely to its end leads precisely nowhere." -
> > Frank Herbert
> > _______________________________________________
> > elephant-devel site list
> > elephant-devel at common-lisp.net
> > http://common-lisp.net/mailman/listinfo/elephant-devel
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>



-- 
Elliott Slaughter

"Any road followed precisely to its end leads precisely nowhere." - Frank
Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20081229/5f406ea6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: slot-read-cache-performance.lisp
Type: application/octet-stream
Size: 1122 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20081229/5f406ea6/attachment.obj>