[elephant-devel] running out of locks

[Gabor Melis]

I have started performance testing my application for which a few thousand 
users were needed:



(defclass user ()
  ((login :accessor user-login :initarg :login :initform "")
   (login2 :accessor user-login2 :initarg :login :initform "")
   (login3 :accessor user-login3 :initarg :login :initform ""))
  (:metaclass persistent-metaclass))

(defun add-users-to-root ()
  (with-transaction ()
    (dotimes (i 10000)
      (add-to-root i (make-instance 'user)))))



Then I got an error from next-counter:



Berkeley DB error: Unknown error: -5
   [Condition of type SLEEPYCAT:DB-ERROR]

Restarts:
  0: [ABORT] Abort handling SLIME request.
  1: [DESTROY-THREAD] Destroy this thread (6135)

Backtrace:
  0: (SLEEPYCAT::NEXT-COUNTER #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP 
#X082797E0 :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP 
#X08285648 :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP 
#X08282CF0 :TYPE (* T)> #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP 
#X08279490 :TYPE (* (SIGNED 8))> 12 #<SB-ALIEN-INTERNALS:ALIEN-VALUE :SAP 
#X08277BD0 :TYPE (* (SIGNED 8))> 16)
  1: ((INITIALIZE-INSTANCE :BEFORE (ELEPHANT:PERSISTENT)) #<unavailable 
argument> #<unavailable argument> #<ELETEST::USER {9BA05D1}> NIL)
  2: ("#'(LAMBDA (SB-PCL::.PV-CELL. SB-PCL::.NEXT-METHOD-CALL. 
SB-PCL::.ARG0. ...) (DECLARE #) ...)" #<unavailable argument> #<unavailable 
argument> #<ELETEST::USER {9BA05D1}> NIL)
  3: ((MAKE-INSTANCE (CLASS)) #<unavailable argument> #<unavailable argument> 
#<ELEPHANT:PERSISTENT-METACLASS ELETEST::USER> NIL)
  4: (ELETEST::ADD-USERS-TO-ROOT)



It turned out the cause of the error was put in next_counter failing with 
"Berkeley DB error: Cannot allocate memory" that in turn was caused by 
running out of locks. It was verified db_stat:

[eletestdb]$ db4.2_stat -c
60      Last allocated locker ID.
2147M   Current maximum unused locker ID.
9       Number of lock modes.
1000    Maximum number of locks possible.
1000    Maximum number of lockers possible.
1000    Maximum number of lock objects possible.
1098    Number of current locks.
1101    Maximum number of locks at any one time.
10      Number of current lockers.
11      Maximum number of lockers at any one time.
599     Number of current lock objects.
600     Maximum number of lock objects at any one time.
196238  Total number of locks requested.
188556  Total number of locks released.
0       Total number of lock requests failing because DB_LOCK_NOWAIT was set.
0       Total number of locks not immediately available due to conflicts.
0       Number of deadlocks.
0       Lock timeout value.
0       Number of locks that have timed out.
0       Transaction timeout value.
0       Number of transactions that have timed out.
360KB   The size of the lock region..
0       The number of region locks granted after waiting.
315390  The number of region locks granted without waiting.

Adding the users without a transaction works but slow. Adding the same number 
of users within a transaction usually produces different "Maximum number of 
locks at any one time.". Is this seemingly non-deterministic accumulation of 
locks expected?

Thanks, Gábor