[elephant-devel] 'not of type FIXNUM' type-error

[Dmitry V. Gorbatovsky]

Thanks a lot. 
I am not tried it yet, but it seems to me that your design of matrix storage 
by indexed rows "naturally" better. 
 
Regards,
Dmitry

On Wednesday 14 February 2007 18:05, Ian Eslick wrote:
> Geesh...I chased this down to a design decision inside the SBCL sb-
> alien implementation as I couldn't reproduce this on Allegro.
>
> One design decision in Elephant was to serialize all lisp values to a
> foreign byte array so they can be directly submitted to the Berkeley
> DB internal pages as a single foreign byte array (required by BDB
> API).  As values are serialized, the foreign array is resized to
> accommodate longer and longer values so the total serialized size of
> your object has to not violate any of the various fixnum related
> limits in the system.
>
> It turns out that SBCL has an internal limitation that it computes
> foreign array sizes by first computing the # of bits.  Thus when you
> try to allocate a buffer of 80M bytes, it becomes 640M bytes which is
> greater than the SBCL positive fixnum limit of ~512MB (30-bit
> signed).  Perhaps SBCL 64 doesn't have this same limitation, but when
> you allocate extremely large objects you'll eventually run into these
> kinds of limitations.
>
> Elephant itself is currently limited to 2 billion objects (indexes
> and persistent indexes) but there is no fixed limit on # of values
> other than the ultimate depth of the btree which will exhaust your
> locks on a write.
>
> If you chose to store large arrays in a special index by row you
> should have no problem:
>
> ;; psuedo code - you should get the idea
> (defun store-large-array (array index)
>    (setf (get-value 'columns index) (num-array-columns array))
>    (setf (get-value 'rows index) (num-array-rows array))
>    (loop for i from 0 upto (num-array-rows array) do
>         (setf (get-value i index) (array-row array i))))
>
> (defun load-large-array (index)
>     (let* ((columns (get-value 'columns index))
>   	  (rows (get-value 'rows index))
>            (array (make-array (list rows columns))))
>        (loop for i from 0 upto rows do
>            (write-row (get-value i index) array))))
>
> Cheers,
> Ian
>
> On Feb 14, 2007, at 8:45 AM, Dmitry V. Gorbatovsky wrote:
> > Hello , here is debian testing with sbcl 1.02
> > and cvs version of elephant. Intel 32-bit machine with
> > 1gb of ram.
> >
> > The code I am try to run is pretty straitforward,
> > it runs with 2000x2000 array, but returns type-error
> > with anything bigger.
> >
> > (let ((big-array (make-array '(5000 5000)
> >
> >                   			:element-type 'float
> >                   			:initial-element 0.0)))
> >
> >   (time (ele:with-open-store
> >             (*test-bdb-spec*)
> >           (ele:add-to-root 'big-array big-array))))
> >
> > And here is output from the debugger:
> >
> > The value 1000000640 is not of type FIXNUM.
> >    [Condition of type TYPE-ERROR]
> >
> > Restarts:
> >   0: [ABORT-REQUEST] Abort handling SLIME request.
> >   1: [TERMINATE-THREAD] Terminate this thread (#<THREAD
> > "worker" {B2CBE61}>)
> >
> > Backtrace:
> >   0: (ELEPHANT-MEMUTIL:RESIZE-BUFFER-STREAM #<unavailable lambda
> > list>)
> >       Locals:
> >   1: ((LABELS ELEPHANT-SERIALIZER2::%SERIALIZE) #<unavailable
> > lambda list>)
> >       Locals:
> >   2: ((LABELS ELEPHANT-SERIALIZER2::%SERIALIZE) #<unavailable
> > lambda list>)
> >       Locals:
> >   3: (ELEPHANT-SERIALIZER2::SERIALIZE #<unavailable lambda list>)
> >       Locals:
> >   4: ((SB-PCL::FAST-METHOD (SETF GET-VALUE) (T T DB-BDB::BDB-BTREE))
> > #<unavailable lambda list>)
> >       Locals:
> >   5: ((LAMBDA NIL))
> >       Locals:
> >   6: (SB-IMPL::%TIME #<CLOSURE (LAMBDA NIL) {A90C005}>)
> >       Locals:
> >         SB-DEBUG::ARG-0 = #<CLOSURE (LAMBDA NIL) {A90C005}>
> >   7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (LET ((BIG-ARRAY #)) (TIME
> >  (WITH-OPEN-STORE # #))) #<NULL-LEXENV>)
> >       Locals:
> >         SB-DEBUG::ARG-0 = (LET ((BIG-ARRAY (MAKE-ARRAY # :ELEMENT-TYPE
> > # :INITIAL-ELEMENT 0.0))) (TIME (WITH-OPEN-STORE (*TEST-BDB-SPEC*)
> > (ADD-TO-ROOT # BIG-ARRAY))))
> >         SB-DEBUG::ARG-1 = #<NULL-LEXENV>
> >   8: ((LAMBDA NIL))
> >       Locals:
> >   9: ((LAMBDA (SWANK-BACKEND::FN)) #<CLOSURE (LAMBDA NIL) {B2CD03D}>)
> >
> > I would highly appreciate any help.
> > Regards, Dmitry
> >
> > --
> > Dmitry V. Gorbatovsky
> > _______________________________________________
> > elephant-devel site list
> > elephant-devel at common-lisp.net
> > http://common-lisp.net/mailman/listinfo/elephant-devel
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel