From henrik at evahjelte.com Fri Aug 3 09:17:07 2007 From: henrik at evahjelte.com (Henrik Hjelte) Date: Fri, 3 Aug 2007 11:17:07 +0200 Subject: [rucksack-devel] Fixed the btree stress-test bug Message-ID: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> I have fixed the btree bug that used to show up when stress testing. Now I have run a test with 100 000 iterations which took some days, so I am sure it is gone. Also included in the patch is a new check-bnode-keys function that I used to find the bug. In the patch bundle is also some previous changes from the mailing list that are not in cvs yet. The patch bundle is from darcs, but it is easy to read. A hunk is a change, plus lines are added, the minus lines are removed. I use tailor to get a local mirrored darcs repo of the offical cvs repo. Best wishes, Henrik Hjelte -------------- next part -------------- A non-text attachment was scrubbed... Name: darcs.patch Type: text/x-diff Size: 8889 bytes Desc: not available URL: From alemmens at xs4all.nl Fri Aug 3 11:23:44 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Fri, 03 Aug 2007 13:23:44 +0200 Subject: [rucksack-devel] Fixed the btree stress-test bug In-Reply-To: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> References: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> Message-ID: Henrik Hjelte wrote: > I have fixed the btree bug that used to show up when stress testing. > Now I have run a test with 100 000 iterations which took some days, so > I am sure it is gone. That's great news! > Also included in the patch is a new check-bnode-keys function that I > used to find the bug. In the patch bundle is also some previous > changes from the mailing list that are not in cvs yet. OK. I'll look at the details this weekend. Thanks a lot, Arthur From alemmens at xs4all.nl Sun Aug 12 13:11:21 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Sun, 12 Aug 2007 15:11:21 +0200 Subject: [rucksack-devel] Fixed the btree stress-test bug In-Reply-To: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> References: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> Message-ID: Henrik Hjelte wrote: > The patch bundle is from darcs, but it is easy to read. Thanks a lot, I've commited your fixes to CVS now. See also my comments below. > hunk ./p-btrees.lisp 831 > - (remove-key leaf key) > + (remove-key leaf (binding-key binding)) This is a bug fix that you'd sent earlier already, right? It was in CVS already. > hunk ./p-btrees.lisp 551 > - (unless (typep value (btree-key-type btree)) > + (unless (typep value (btree-value-type btree)) Yes, that looks like a copy & paste error. Thanks. > I think it would be nice if rucksack signals error when I give > a duplicated key to a unique slot. >But then I noticed the following comment in do.txt: > - Check that btrees actually signal an error for duplicate keys. > Handle those errors correctly for slot indexes. > Why does rucksack need to handle the errors? It's been a while since I looked at this, but I suppose my reasoning was that the btree errors may need to be signalled in a slightly different way at the rucksack level. > hunk ./rucksack.lisp 864 > - (index-insert index new-value id)))))) > + (index-insert index new-value id > + :if-exists > + (and (slot-unique slot) :error))))))) I modified this fix a bit. If the slot may have duplicate values, your version passes :IF-EXISTS NIL to INDEX-INSERT. But :IF-EXISTS is specified to take either :OVERWRITE or :ERROR, not NIL. > I think it would be nicer if rucksack signals error for me > when I give an imcompatible value to a slot with a index. > (e.g. giving string to :symbol-index.) >Adding :key-type to index-spec definitions seems to accomplish > it. But I'm not sure this is right. >What do you think? This sounds like a good idea to me. Thanks, I've incorporated this. > +(defun update-parents-for-deleted-key (btree parent-stack old-key new-key) > + (when parent-stack > + (let ((node (first parent-stack))) > + (when node > + (let ((position (key-position old-key node))) > + (when position > + (setf (binding-key (node-binding node position)) > + new-key) > + (update-parents-for-deleted-key btree (rest parent-stack) old-key new-key))))))) > + > hunk ./p-btrees.lisp 874 > - (remove-key leaf (binding-key binding)) > - (unless (node-full-enough-p btree leaf) > - (enlarge-node btree leaf parent-stack)))) > + > + (let ((position (key-position key leaf)) > + (length (btree-node-index-count leaf)) > + (was-biggest-key-p nil)) > + (when (= position (1- length)) > + (setf was-biggest-key-p t)) > + + (remove-key leaf (binding-key binding)) > + > + (unless (node-full-enough-p btree leaf) > + (enlarge-node btree leaf parent-stack)) > + + (when was-biggest-key-p > + (unless (= 0 (btree-node-index-count leaf)) > + (update-parents-for-deleted-key btree parent-stack key (biggest-key leaf))))))) So this was the big bug, right? If we're deleting the biggest key from a leaf, we should update the parents so they'll use the key that has now become the biggest, right? Thanks a lot for finding and fixing this! Arthur From henrik at evahjelte.com Mon Aug 13 14:59:34 2007 From: henrik at evahjelte.com (Henrik Hjelte) Date: Mon, 13 Aug 2007 16:59:34 +0200 Subject: [rucksack-devel] Fixed the btree stress-test bug In-Reply-To: References: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> Message-ID: <50e8e4f60708130759y2186d0a5n9c7a91f69247b9c5@mail.gmail.com> > So this was the big bug, right? If we're deleting the biggest key from > a leaf, we should update the parents so they'll use the key that has now > become the biggest, right? Yes, this was the bug. The patches before were patches submitted to the mailing list by Takehiko Abe, I didn't want to steal the credit for them... Thanks, Henrik From alemmens at xs4all.nl Mon Aug 13 15:16:06 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Mon, 13 Aug 2007 17:16:06 +0200 Subject: [rucksack-devel] Fixed the btree stress-test bug In-Reply-To: <50e8e4f60708130759y2186d0a5n9c7a91f69247b9c5@mail.gmail.com> References: <50e8e4f60708030217q253f340cv801f1183c156ce91@mail.gmail.com> <50e8e4f60708130759y2186d0a5n9c7a91f69247b9c5@mail.gmail.com> Message-ID: > Yes, this was the bug. The patches before were patches submitted to > the mailing list by Takehiko Abe, I didn't want to steal the credit > for them... Oops, I had forgotten about those ;-) Sorry Takehiko, I've fixed the credits now...