[rucksack-devel] rucksack performance

[Cyrus Harmon]

Ok, so if I remove the (:index t) line from the defclass form, it's  
fast, and works, but I can't seem to get the indices to appear by  
redefining the class.

if, OTOH, I delete the :index forms from the slot definitions, but  
leave the (:index t) in the class definition, things start of ok, but  
then I get this familiar error message (oh, and this is when batching  
500 make-instances at a time into a transaction; or at least I'm  
doing a transaction-commit every 500 items, hoping that that's doing  
what I think it is):

There is no applicable method for the generic function
   #<STANDARD-GENERIC-FUNCTION RUCKSACK:P-CAR (1)>
when called with arguments
   (#<RUCKSACK:PERSISTENT-ARRAY #85564 in #<STANDARD-CACHE of size  
10000, heap #P"/Users/sly/projects/cyrusharmon.org/cl-bio/ruc$
    [Condition of type SIMPLE-ERROR]

Restarts:
   0: [ABORT] Abort #<RUCKSACK:STANDARD-TRANSACTION #337757771800000  
with 567 dirty objects>
   1: [RETRY] Retry #<RUCKSACK:STANDARD-TRANSACTION #337757771800000  
with 567 dirty objects>
   2: [ABORT-REQUEST] Abort handling SLIME request.
   3: [ABORT] Exit debugger, returning to top level.

Backtrace:
   0: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #<unused  
argument> #<unused argument> #<STANDARD-GENERIC-FUNCTION RUCKSACK$
   1: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #<unavailable  
argument> #<unavailable argument> #<STANDARD-GENERIC-FUNCTIO$
   2: (RUCKSACK::FIND-BINDING-IN-NODE 85566 #<RUCKSACK:BTREE-NODE  
with 51 bindings {12786E11}> #<RUCKSACK:BTREE #51577 in #<STAN$
   3: (RUCKSACK::LEAF-INSERT #<RUCKSACK:BTREE #51577 in #<STANDARD- 
CACHE of size 10000, heap #P"/Users/sly/projects/cyrusharmon.$
   4: ((SB-PCL::FAST-METHOD RUCKSACK:BTREE-INSERT (RUCKSACK:BTREE  
#1="#<...>" . #1#)) (#(NIL) . #()) #<unused argument> #<RUCKSA$
   5: ((SB-PCL::FAST-METHOD INITIALIZE-INSTANCE :AROUND  
(#1="#<...>" . #1#)) (#(NIL NIL) . #()) #S(SB-PCL::FAST-METHOD- 
CALL :FUN$
   6: ((SB-PCL::FAST-METHOD MAKE-INSTANCE (CLASS)) #<unused argument>  
#<unused argument> #<RUCKSACK:PERSISTENT-CLASS P-TAXON> #<$
   7: (PARSE-TAX-NODES :FILE NIL)
   8: (SB-INT:SIMPLE-EVAL-IN-LEXENV (PARSE-TAX-NODES) #<NULL-LEXENV>)
   9: (SWANK::EVAL-REGION
       "(parse-tax-nodes)
      "
       T)

which is reminiscent of the error I was seeing before when we were  
getting an array back as the last element in another p-array, where  
it really expected a p-cons. Not sure where the underlying bug is.

Thanks,

Cyrus

On Jan 11, 2007, at 11:55 PM, Cyrus Harmon wrote:

>
> On Jan 11, 2007, at 11:23 PM, Arthur Lemmens wrote:
>
>> [Replying to the list.]
>>
>>> Oh, while I'm at it, one thing i was thinking about is the nature of
>>> indices. One of the nice things in RDBMSes is that you can, say,
>>> create a whole bunch of rows and then index them after the fact,
>>> rather than having the index in place from the get go (and, more
>>> importantly, while doing the insertion). Have you thought about the
>>> implications of such a strategy for rucksack? would it make sense to
>>> have unindexed slots and then redefine the class to have indices
>>> after the instances have been created?
>>
>> This should already work now.  Have a look at the function REPLACE- 
>> SLOT-INDEX,
>> for example.  Just add an :INDEX foo option to a slot, and existing
>> instances will be re-indexed automatically.
>
> Ok, now we're getting somewhere. removing the indices seems to help  
> greatly. We'll see how it goes adding them back once all is said  
> and done, but this looks promising.
>
>> At least that's the idea.  If it doesn't work, I'd like to hear about
>> it.
>
> I'll certainly post the results when this is done.
>
> Since the index code seems to be the bottleneck here, it would be  
> good to spend some time optimizing the index code as well. Also,  
> some API support for disabling indexing might be nice. Redefining  
> the defclass forms seems to be a bit "extreme".
>
> Cyrus
>
> _______________________________________________
> rucksack-devel mailing list
> rucksack-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/rucksack-devel