[elephant-devel] Optimization

[Elliott Slaughter]

Thanks for your comments. Let me try to provide you with a little more
information.

On Thu, Jan 8, 2009 at 4:38 AM, Ian Eslick <eslick at media.mit.edu> wrote:
>
> On Jan 8, 2009, at 3:22 AM, Elliott Slaughter wrote:
>
> > Another issue is that the cache-style declaration in persistent
> > classes doesn't get inherited by default. This particularly annoying
> > since this throws an error merely by inheriting cached slots from a
> > superclass.
>
> Hmmmm...it should inherit by default unless you are overriding the
> base class slot.  I'll look into this too.


It looks like cache-style is inherited when no new cached slots are
declared. But when new cached slots are declared, it produces a rather
strange error. I have attached a test case for you to examine.

> That said, I could probably avoid reading indexed slots if I used
> > the query language discussed on another thread. (Currently I use get-
> > instances-by-value to cons a list of objects, and then sort by the
> > indexed slot to effectively iterate over two sorted slots.)
>
> The query system would basically do this:
>
> (let ((a (map-inverted-index #'identity 'slot1 :collect t :oids t))
>       (b (map-inverted-index #'identity 'slot2 :collect t :oids t)))
>   (mapcar #'elephant::controller-recreate-instance
>          (merge-unique (sort a #'<) (sort b #'<))


What I need to do is more like

(let ((list (map-inverted-index #'identity 'slot1 :value foo :collect t)))
  (sort list #'< :key #'slot2))

which is effectively what I am currently doing with get-instances-by-value.
Is there a better way to efficiently iterate over one value in the first
slot, with the ordering of the second slot? (Should I look into creating
another level of btree for holding each subset of object in order?)

You can also do map-inverted-index and accumulate only values matching
> the second value as discussed earlier using an accumulation closure as
> an argument.  For small sets on the first slot, this is probably
> faster.  In general mapping is always more efficient than get-
> instances...


Most of the time, the sets will be close in size. Less frequently, the
second set will be much larger.

> I get roughly the same results as you on the micro-benchmarks. When
> > using checkout caching in the game, I get about a 2x speed increase.
> > It is still about 3x slower than no database.
>
> It would be interesting to see some profile data on this to see what,
> if anything, in BDB/Elephant could be improved.


Do you want profile results from my package, or from Elephant internals?
Either way, I have attached to profile reports. I hope you find them
helpful.

On Thu, Jan 8, 2009 at 5:02 AM, Ian Eslick <eslick at media.mit.edu> wrote:

> By the way, a write through mode is complex because if a transaction
> is rolled back your index could differ from your memory state so you
> would then have to keep track if slot writes to undo them on rollback...


I don't currently (or expect to) rollback changes, so this isn't an issue
with my particular application.

Are you caching blocks of objects for short term operations or long
> term ones?
>

I would like to cache all objects in the current room, so the only major
context switches should be when the player moves between rooms. So long
term.

-- 
Elliott Slaughter

"Any road followed precisely to its end leads precisely nowhere." - Frank
Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090113/b37e0438/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: class_cache-style_inheritence_bug.lisp
Type: text/x-lisp-source
Size: 465 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090113/b37e0438/attachment.bin>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: profile_ele.txt
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090113/b37e0438/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: profile_bt-ele.txt
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090113/b37e0438/attachment-0001.txt>