[elephant-devel] linux-sbcl-64 report

Mon Feb 12 21:30:46 UTC 2007

On Mon, 2007-02-12 at 15:10 -0500, Ian Eslick wrote:
> I hope you aren't relying on the current HEAD for anything  
> important!  
Well no, I was hoping to use it but I'm affraid I don't trust it yet...
> As you can tell we're not quite there yet!
> 
> There could be several issues.  I assume you are using the with- 
> transaction macro around your transactions?  (Or are you relying on  
> auto-commit?)  Are you using cursors yourself or functions like get- 
> instances-by-value that themselves use cursors manually?
> 
> I recently made some changes to several of these functions so it  
> could be any of them.  If I have some sense of what you're doing I  
> can go inspect the changes for holes.  I should also be able to  
> reproduce this locally if it's an elephant function rather than an  
> issue with usage.
> 
> Thanks!
> Ian
> 
Actually, I have just released this test as a project called grand-prix,
a black-box test framework to compare persistence solutions. The idea
was to save some of my test code instead of throwing it away, maybe It
can grow to be something useful.

http://common-lisp.net/project/grand-prix/
darcs get http://common-lisp.net/project/grand-prix/darcs/grand-prix

grand-prix can probably be a little messy to set up,
the actual elephant code is below, you can probably just paste it into a
buffer and run it.

I first run (my-test-create), which works
Then (my-test-update) which show the problem.

elephant works with *nr-person* set to 2000.

Thanks,
Henrik Hjelte

(defparameter *spec* '(:bdb "/tmp/elephant-test-suite-mp"))

(defparameter *names* '("David" "Jim" "Peter" "Thomas"
                        "Arthur" "Jans" "Klaus" "James" "Martin"))

(defclass person ()
  ((name :initform (elt *names* (random (length *names*)))
         :accessor name
         :index t) 
;; Actually the index t shouldn't be needed, but since elephant
sometimes complained that "person is not an index class", I try if this
fixes it.
   (age :initform (random 100) :accessor age)
   (made-by :initform (grand-prix::current-process-id))
   (updated-by :initform nil :accessor updated-by))
  (:metaclass elephant:persistent-metaclass))

(defparameter *nr-persons* 10000)  ;; Should be 10000, but for me
elephant can't allocate memory after 3000.
;; I think the problem it is becuase the number of locks (999) is = max
1000. see db_stat -e

(defparameter +age+ 50)

;; I have tried different places for with-transaction below
(defun make-persons (nr-objects)
  (loop for i below nr-objects
        do (let ((person (elephant:with-transaction () 
                           (make-instance 'person))))
             (when (zerop (mod i 1000))
               (format t "~D ~a" i (name person)))
             ))) 

(defun ensure-clean-store ()
  (let ((dir (cl-fad:pathname-as-directory (second *spec*))))
    (when (cl-fad:directory-exists-p dir)
      (cl-fad:delete-directory-and-files dir))
    (ensure-directories-exist dir)))

(defun my-test-create ()
  (ensure-clean-store)
  (elephant:with-open-store (*spec*)
    (make-persons *nr-persons*)))

(defun my-test-update (&key (new-age 27))
  "Test updating all persons by changing their age."
  (elephant:with-open-store (*spec*)
    (elephant:with-transaction ()
      (mapcar #'(lambda (person)
                  (setf (age person) new-age))
              (elephant:get-instances-by-class 'person)))))

(defun my-test-load ()
  "Test loading all persons by computing their average age."
  (let ((nr-persons 0)
        (total-age 0)
        (show-first nil))
    (elephant:with-open-store (*spec*)
      (elephant:with-transaction ()
        (mapcar #'(lambda (person)
                    (incf nr-persons)
                    (print nr-persons)
                    (when (and show-first (> show-first))
                      (format t "Sample person ~a~%F" show-first)
                      (describe person)
                      (decf show-first))
                    (incf total-age (age person)))
                (elephant:get-instances-by-class 'person))))
    (values (coerce (/ total-age nr-persons) 'float)
            nr-persons
            total-age)))

(defun check-basic-setup ()
  (my-test-update :new-age +age+)
  (multiple-value-bind (average nr-persons)
      (my-test-load)
    (assert (= +age+ average))
    (assert (= nr-persons *nr-persons*))))

> On Feb 12, 2007, at 12:55 PM, Henrik Hjelte wrote:
> 
> > Minor problems on 64 bit sbcl/linux
> > .
> > serializer2 line 174:
> > 		    (if (< (abs frob) +2^32+)
> > should be
> > 		    (if (< (abs frob) +2^31+)
> 
> Fixed
> 
> > :module utils in elephant.asd should have a serial :t otherwise  
> > lock may
> > be compiled before package.
> 
> Fixed
> 
> > A reminder about compiler switches,
> > -arch x86_64 I think is apple specific,
> > I have
> >    #+(or :X86-64) "-march=x86-64"
> > on linux.
> > Maybe a compromise is
> > "-m64" ?
> >
> > http://gcc.gnu.org/onlinedocs/gcc-3.3.6/gcc/i386-and-x86_002d64- 
> > Options.html#i386-and-x86_002d64-Options
> 
> Hmmm...I think I'll add conditionals on a per-lisp, per-platform  
> basis as the *feature* lists are all very different.  Check out the  
> latest checkin in elephant.asd
> 
> >
> > A bigger problem: I get out-of-memory errors after running elephant a
> > while. I *think* (I am by no means an expert with these things)  
> > this can
> > mean anything, not only out-of-memory. (Because of a low default
> > error-message level)
> >
> > What I strongly suspect is that I run out of locks.
> > Doing dbstat-e reveals:
> > 999     Number of current locks
> > 1000    Maximum number of locks at any one time
> > 13      Number of current lockers
> > 17      Maximum number of lockers at any one time
> > 996     Number of current lock objects
> > 997     Maximum number of lock objects at any one time
> > When I am thrown to the debugger.
> >
> > When quiting the debugger:
> > 0       Number of current locks
> > 1000    Maximum number of locks at any one time
> >
> > I am running the deadlock detector in a shell process: db_deadlock - 
> > t 3
> >
> > Oracle writes about this as point six in the common problems section:
> > http://www.oracle.com/technology/documentation/berkeley-db/db/ref/ 
> > debug/common.html
> >
> > I have tried to use with-locks surrounding the get and put calls in
> > persistent-slot-reader and writer, but it didn't help anything.
> 
> That shouldn't have an effect (where did you get with-locks anyway?   
> Is that an elephant internal function or a threading function?)
> 
> > Any ideas? Has no one else seen this? Have I missed something obvious?
> 
> I haven't seen this on earlier versions, and I used Elephant 0.6.0  
> very heavily so it must be an artifact that was recently introduced.
> 
> > Best wishes,
> > Henrik Hjelte
> >
> > _______________________________________________
> > elephant-devel site list
> > elephant-devel at common-lisp.net
> > http://common-lisp.net/mailman/listinfo/elephant-devel
> 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
> 
> 