[rucksack-devel] continuing the rucksack saga...
Cyrus Harmon
ch-rucksack at bobobeach.com
Fri Jan 12 02:50:02 UTC 2007
I'm not sure whether to lay the blame at rucksacks' feet or at SBCLs,
but now that i've resolved my performance problems to the point where
I can make-instance over 150k objects, I run out of heap space:
Heap exhausted during garbage collection: 0 bytes available, 8
requested.
Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc
Waste Trig WP GCs Mem-age
0: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
1: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
2: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
3: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
4: 251638 251641 0 0 192577 3687 1134 1603 0
814849768 258328 488442912 0 1 1.4186
5: 393214 388673 7009 0 182649 441 1372 4015 266
771849320 152472 2000000 0 0 0.0000
6: 0 0 0 0 5734 0 0 0 0
23486464 0 2000000 5340 0 0.0000
Total bytes allocated=1610185552
fatal error encountered in SBCL pid 289:
Heap exhausted, game over.
LDB monitor
ldb> backtrace
Backtrace:
0: Foreign function ldb_monitor, fp = 0x2205698, ra = 0x70d7
1: Foreign function lose, fp = 0x22056c8, ra = 0x586e
2: Foreign function gc_heap_exhausted_error_or_lose, fp =
0x2205708, ra = 0xf74e
3: Foreign function gc_find_freeish_pages, fp = 0x2205778, ra =
0xf815
4: Foreign function gc_alloc_large, fp = 0x22057e8, ra = 0xfebe
5: Foreign function gc_alloc_with_region, fp = 0x2205828, ra =
0x101ef
6: Foreign function gc_general_alloc, fp = 0x2205848, ra = 0x102a9
7: Foreign function scan_weak_pointers, fp = 0x22058c8, ra = 0x408e
8: Foreign function scavenge, fp = 0x2205938, ra = 0x3bc5
9: Foreign function zero_pages_with_mmap, fp = 0x2205998, ra =
0xec9a
10: Foreign function collect_garbage, fp = 0x2205ad8, ra = 0x11ebc
11: (SB-C::TL-XEP SB-KERNEL::COLLECT-GARBAGE)
12: (COMMON-LISP::FLET WITHOUT-INTERRUPTS-BODY-56)
13: (SB-C::TL-XEP SB-KERNEL::SUB-GC)
14: Foreign function call_into_lisp, fp = 0x2205c88, ra = 0x13ba1
15: Foreign function funcall0, fp = 0x2205ca8, ra = 0xd83f
16: Foreign function interrupt_maybe_gc_int, fp = 0x2205cc8, ra =
0x63f0
17: Foreign function interrupt_handle_pending, fp = 0x2205cf8, ra
= 0x64bc
18: Foreign function _sigtramp, fp = 0x2205d18, ra = 0x9011110c
19: Foreign fp = 0x2206028, ra = 0xffffffff
20: (SB-C::TL-XEP SB-KERNEL::MAKE-RESTART)
Heap exhausted during garbage collection: 0 bytes available, 16
requested.
Gen StaPg UbSta LaSta LUbSt Boxed Unboxed LB LUB !move Alloc
Waste Trig WP GCs Mem-age
0: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
1: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
2: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
3: 0 0 0 0 0 0 0 0 0
0 0 2000000 0 0 0.0000
4: 251638 251641 0 0 192577 3687 1134 1603 0
814849768 258328 488442912 0 1 1.4186
5: 393214 388673 7009 0 182649 441 1372 4015 266
771849320 152472 2000000 0 0 0.0000
6: 0 0 0 0 5734 0 0 0 0
23486464 0 2000000 5340 0 0.0000
Total bytes allocated=1610185552
fatal error encountered in SBCL pid 289:
and this happens when trying to allocate all .5M objects in a single
transaction, also, perhaps, a bad idea. But, still, this seems like a
rather catastrophic failure. I think I got basically the same error
when trying 1 obj/transaction, and figured that using only one
transaction for the whole lot would make things better, but that
doesn't seem to be the case.
Cyrus
On Jan 11, 2007, at 4:03 PM, Cyrus Harmon wrote:
> Ok, well there are probably still some issues here, but removing
> the string-index on the slot that had about 12 distinct values
> (over .5M objects) really seems to help performance! careful with
> that axe, Eugene!
>
> crossing my fingers,
>
> Cyrus
>
> On Jan 11, 2007, at 3:13 PM, Cyrus Harmon wrote:
>
>> you probably already knew this, but it seems as though most of the
>> allocation is happening down in btree-insert.
>>
>> Cyrus
>>
>> On Jan 11, 2007, at 2:15 PM, Cyrus Harmon wrote:
>>
>>>
>>> On Jan 11, 2007, at 12:00 PM, Arthur Lemmens wrote:
>>>
>>>> Cyrus Harmon wrote:
>>>>
>>>>> Following up on my post from yesterday, even when I can coerce the
>>>>> "lots of transactions" into working, at some point performance
>>>>> breaks
>>>>> down rather severely.
>>>>
>>>> I haven't looked at this in detail, but my first guess would be
>>>> that
>>>> the garbage collector settings and/or performance is critical here.
>>>
>>> Hmm... back to the
>>>
>>> The value
>>> #<RUCKSACK:PERSISTENT-ARRAY #178397 in #<STANDARD-CACHE of size
>>> 10000, heap #P"/Users/sly/projects/cyrusharmon.org/cl-bio/
>>> rucksack/heap" and 7481 objects in memory.>>
>>> is not of type
>>> (OR NULL RUCKSACK:PERSISTENT-CONS).
>>> [Condition of type TYPE-ERROR]
>>>
>>> error, which is interesting as it looks like we're trying to do a
>>> p-car of an array, and it's getting the array be accessing the
>>> last element in the other (legitimate) array, but I'm getting
>>> distracted...
>>>
>>>>> yes, along the way there is some fluctuation, as, I imagine, the
>>>>> indices and caches grow, etc... but we reach a threshold where it
>>>>> takes roughly .5 sec and 3M of consing for every object.
>>>>
>>>> 3M of consing per object is ridiculously much of course. I'm
>>>> pretty
>>>> sure that it should be possible to reduce this a lot by tracing
>>>> some
>>>> of the garbage collector routines and looking at how much work
>>>> they do.
>>>>
>>>> One thing you could consider is to turn the garbage collector off
>>>> during the phase where you're creating very many objects
>>>> (initializing
>>>> your database maybe?). In fact, you could just turn it off,
>>>> period.
>>>> As long as your disk is big enough, of course...
>>>>
>>>> Let me know if turning the GC off doesn't help.
>>>
>>> How do I do this? I commented out the collect-some-garbage in
>>> transaction, but that didn't seem to fix the problem.
>>>
>>>
>>>>> And it would be nice if this approach worked as well.
>>>>
>>>> Yes. Having a separate transaction for each created object is
>>>> not the
>>>> most efficient way and should not be necessary, but obviously it
>>>> should
>>>> work and it shouldn't be ridiculously slow.
>>>
>>> Agreed. Of course I only went down this route as performance
>>> became unacceptable with a really big transaction too, so perhaps
>>> the transaction/gc thing is a red herring.
>>>
>>> Cyrus
>>>
>>> _______________________________________________
>>> rucksack-devel mailing list
>>> rucksack-devel at common-lisp.net
>>> http://common-lisp.net/cgi-bin/mailman/listinfo/rucksack-devel
>>
>> _______________________________________________
>> rucksack-devel mailing list
>> rucksack-devel at common-lisp.net
>> http://common-lisp.net/cgi-bin/mailman/listinfo/rucksack-devel
>
> _______________________________________________
> rucksack-devel mailing list
> rucksack-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/rucksack-devel
More information about the rucksack-devel
mailing list