[rucksack-devel] Unexpected results from Rucksack (timing dependence?)

[Klaus Harbo]

Hi Everyone -

I was looking at Rucksack yesterday to see if it might be useful for a  
small project I'm considering.  I have very little experience with  
Rucksack and use the material in the tutorial as a starting point for  
my explorations.  However, I'm inconsistent and - to me - surprising  
results, so thought I'd check with a more experienced crowd (i.e. you  
guys, the members of rucksack-devel!)...

What I'm seeing, basically, is that the effect of calling the  
tutorial's MAKE-CONTACT does not seem to show up consistently when  
retrieving instances using RUCKSACK-MAP-CLASS.  I have compiled an  
example to which illustrates (not very pretty, but except for the  
removal of docstrings to keep things short, it's cut-and-paste'd  
straight out of the tutorial):

   (defun try-it (iters &optional (sleep-list '(0)))
     (loop for sleep-period in sleep-list
           for res = (let ((the-dir "/tmp/rs-test/"))
                       (flet ((make-contact (name &optional phone- 
number email address notes)
                                (with-rucksack (rs the-dir)
                                  (with-transaction ()
                                    (make-instance 'contact-details
                                                   :name (or name "")
                                                   :phone-number (or  
phone-number 0)
                                                   :email (or email "")
                                                   :address (or  
address "")
                                                   :notes notes)))))
                         (macrolet ((def-the-class ()
                                      `(with-rucksack (rs the-dir)
                                         (with-transaction ()
                                           (defclass contact-details ()
                                             ((unique- 
id    :initarg :unique-id :accessor unique-id-of
                                                            :index 
  :number-index
                                                            :unique t)
                                               
(name         :initarg :name :accessor name-of
                                                            :index 
  :case-insensitive-string-index)
                                              (phone- 
number :initarg :phone-number :accessor phone-number-of
                                                            :index 
  :number-index)
                                               
(email        :initarg :email :accessor email-of
                                                            :index 
  :case-insensitive-string-index)
                                               
(address      :initarg :address :accessor address-of
                                                            :index 
  :case-insensitive-string-index)
                                               
(notes        :initarg :notes :accessor notes-of))
                                             (:index t)
                                             (:metaclass persistent- 
class))))))
                           (let (ret)
                             (loop repeat iters
                                   do (mapc #'delete-file (directory  
the-dir))
                                   do (def-the-class)
                                   do (tagbody
                                       (loop for count from 1 to 50
                                             do (progn (make-contact  
"name") (sleep sleep-period))
                                             do (with-rucksack (rs the- 
dir)
                                                  (with-transaction ()
                                                    (let ((c 0))
                                                      (rucksack-map- 
class rs 'contact-details (lambda (o) (incf c)))
                                                      (unless (= c  
count)
                                                        (push (list :c  
c :count count) ret)
                                                        (go done))))))
                                       done))
                             (list :sleep sleep-period :ret ret)))))
           do (print res)))

Essentially, the code (is supposed to)

  (1) clear the rucksack directory by deleting everything
  (2) define the CONTACT-DETAILS class in the context of a (thus)  
newly created rucksack
  (3) repeatedly (i) creates a CONTACT-DETAIL and (ii) checks to see  
if the number of created instances is correct

When evaluating (ignoring SLEEP for a moment)

  RUCKSACK 52 > (try-it 5)

  (:SLEEP 0 :RET ((:C 25 :COUNT 26) (:C 28 :COUNT 29) (:C 30 :COUNT  
31) (:C 28 :COUNT 29) (:C 27 :COUNT 28)))
  NIL

  RUCKSACK 53 >

it appears that in the 5 iterations after 26, 29, 31, 29, and 28  
insertions (respectively), one was 'missing' at the least when viewed  
using RUCKSACK-MAP-CLASS.  I would have expected the output

  (:SLEEP 0 :RET NIL)

since I would expect to be able retrieve all the instances that have  
been created.

Due to the apparent variation in the number of insertions and - as far  
as I can see - the lack of variation in the input, I thought that  
perhaps there's some kind of timing issue, so I added calls to SLEEP

  RUCKSACK 54 > (try-it 5 '(0 .01 .1 .25))

  (:SLEEP 0 :RET ((:C 28 :COUNT 29) (:C 29 :COUNT 30) (:C 29 :COUNT  
30) (:C 28 :COUNT 29) (:C 12 :COUNT 13)))
  (:SLEEP 0.01 :RET ((:C 23 :COUNT 24) (:C 23 :COUNT 24) (:C 23 :COUNT  
24) (:C 23 :COUNT 24) (:C 23 :COUNT 24)))
  (:SLEEP 0.1 :RET ((:C 6 :COUNT 7) (:C 6 :COUNT 7) (:C 7 :COUNT 8)  
(:C 6 :COUNT 7) (:C 7 :COUNT 8)))
  (:SLEEP 0.25 :RET ((:C 3 :COUNT 4) (:C 2 :COUNT 3) (:C 2 :COUNT 3)  
(:C 2 :COUNT 3) (:C 4 :COUNT 5)))
  NIL

which does indeed seem to impact the results.  Interestingly, if the  
sleep becomes very long

  RUCKSACK 55 > (try-it 5 '(1))

  (:SLEEP 1 :RET NIL)
  NIL

  RUCKSACK 56 >

I get the expected result.  (The last one is 250+ seconds to evaluate  
- I haven't tried more than a few times but I got the same result in  
all cases).

I do not understand these results.  Offhand, I can think of 4 possible  
explanations

1. The results are correct, I just don't understand Rucksack well  
enough.  I don't think so, but please enlighten me if that's the case...

2. I'm not using Rucksack as intended, leading to undefined results.   
Am I doing something wrong?  If so, where?

3. There's something wrong with my setup - I'm using Rucksack  
downloaded as a tarball from common-lisp.net yesterday and Lispworks  
for Macintosh v5.1 Pro on Mac OSX Leopard.  Can anyone reproduce this  
variation?

4. There's an error in Rucksack.  I do not understand the code very  
well, but the variation in terms of how many insertions are needed and  
the apparent timing dependency surprises me.

Suggestions / insights?

best regards,

Klaus Harbo