[elephant-devel] berkeley db bulk get

[Gábor Melis]

Do you guys think that the bulk get feature is worth supporting?

http://www.sleepycat.com/docs/ref/am_misc/get_bulk.html

I ran into a number of situations where the db or a huge btree must be scanned 
(gc, creating a site map with a link to user pages, joins) and it takes ages. 
Current thinking is that adding a buffer to the cursor itself may perform 
well:

(with-transaction (:degree-2 t)
  (with-btree-cursor (cursor my-btree :multiple t
                                      :multiple-buffer-size 1000000)
    (loop for (found? key user) = (multiple-value-list
                                    (cursor-next cursor))
          while found?
          collect user)))

The berkeley db doc says multiple get will not work on secondary indices so 
joins will not benefit :-(.

Any thoughts? Is there another way to speed up iteration? Maybe raise the page 
size?

Gabor