[elephant-devel] secondary indices

[Robert L. Read]

Dear Waldo,
     Thanks for you excellent message.  This problem reaches
the border of my knowledge, but hopefully this answer will 
be somewhat helpful.


> Robert,
> 
> I've been trying to understand the inners of indexed-btree. I have
to  
> admit, this exercise has helped me greatly in my quest for learning  
> Lisp.

I also taught (and is teaching) me a great deal.
> 
> So far, I have been able to narrow down the problem to the transient  
> slot INDICES-CACHE of the BDB-INDEXED-BTREE class.
> 
Good.

> Even though this slot has the same :initform as the non-transient  
> version (e.g. make-hash-table), after you make an instance of
indexed- 
> btree, you can query (indices *instance*) and it returns the hash  
> table object. However, when you (indices-cache *instance*) you get
the:
> 
> Array index 4 out of bounds for #<SLOT-VECTOR  #x89BFD86> .
>     [Condition of type SIMPLE-ERROR]

This must be some kind of direct OpenMCL error.  I don't understand it;
but at least we can use it to debug this problem.

> 
> What I can't figure out yet is that in collections.lisp, you have:
> 
> (defmethod shared-initialize :after ((instance indexed-btree) slot-
names
>                                      &rest rest)
>    (declare (ignore slot-names rest))
>    (setf (indices-cache instance) (indices instance)))
> 
> which, to my understanding, should set the value of the transient  
> slot to point to the same hash-table that the INDICES slot points
to.  
> However, this doesn't seem to do anything. Even if you inspect  
> (indices instance) before the setf, you get the hash-table, but for  
> some reason, whatever (indices-cache instance) points to is not  
> accepting the change.

OK, I think you have found the heart of the problem.  

> 
> I tried tracing it back into classes.lisp, into the:
> 
> (defmethod shared-initialize :around ((instance persistent-object)  
> slot-names &rest initargs &key &allow-other-keys)
>    "Initializes the persistent slots via initargs or forms.
> This seems to be necessary because it is typical for
> implementations to optimize setting the slots via initforms
> and initargs in such a way that slot-value-using-class et al
> aren't used.  Calls the next method for the transient slots."
> 
> In this method, it seems to initialize the persistent and non- 
> persistent slots. At the end of the method, (indices instance) is  
> properly pointing to a hash table. However, the (indices-cache  
> instance) in that method is still returning the same error. Now, at  
> the end of the defmethod, you have:
> 
>           ;; let the implementation initialize the transient slots
>           (apply #'call-next-method instance transient-slot-inits  
> initargs))))))

I didn't write this routine (It's possible by that my extraction 
of the class bdb-indexed-btree is breaking it.

> 
> which is where I'm a bit at a loss. I would assume it's trying to  
> call the next-method during the initialization process. However,  
> transient-slot-inits is internally defined (flet) within this  
> defmethod and it doesn't seem to be doing anything. Also, after  

I think "transient-slot-inits" is not a function but a list of 
the slot-names that are transient that will need to be initialized.
However, you are correct that "initialize-from-initarg" is defined
by an flet here, but it does do something --- it tries to 
set certain slots.  Whether it is correct or not, I don't know.

> inspecting all the values of the parameters passed to this
defmethod,  
> there is no reference to the INDICES-CACHE slot. If you inspect SLOT- 
> NAMES in this method, it only has (INDICES). Could this be the reason?

I can think of two explanations for this:  Possibly it is bad to 
name the accessor for the slot named "indices-cache" the same as the 
slot itself (but I dobut it).  More likely at the time you are 
examinining it you are in the contect of indexed-btree or bdb-btree
and NOT the class bdb-indexed-btree.  (I'm not too sure of this...
it may be implementation dependent, and I'm not a MOP/CLOS expert.)


> 
> I hope this may trigger something that could allow you or others to  
> solve the problem or at least help me further in diagnosing it.

I think you have basically identified the approach I would use
to debugging this problem:  we know invoking indices-cache at
various times gives the above error, and we can't allow that.

So although it is quite complicated, we can potentially sprinkle
"print" statements into things like shared-initialize till we 
identified the precise point where indices-cache is legitimate
and then becomes illegitimate.  I think you have already started
this process---I suggest you just continue it.

I also suggest you organize your code into a specific piece
of test code, something like this:
(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-bdb)
(asdf:operate 'asdf:load-op :elephant-tests)
(in-package "ELEPHANT-TESTS")
(open-store *testdb-path*)
(setq indexed (build-indexed-btree *store-controller*))
(elephant::indices-cache indexed)

For example, this code works for me under SBCL; if it fails
under OpenMCL on the last line, then what we need to 
do is to push something like (format t "IC = ~A~%" (indices-cache X))
into the shared-initialize and other routines until we understand 
it more deeply.

By producing a reproducible piece of text like this, we might 
not solve the problem:  Possibly there is a bug in OpenMCL, or 
more likely I have misunderstood CLOS and MOP in some way that
won't have a one-liner fix.  But if we can get it down to  clear,
reproducible piece of code, I suspect Andrew Blumberg would be
able to solve it from that point, or we could even ask for 
help from some OpenMCL forum.

Clearly, SBCL and OpenMCL are behaving differently; if we can
track down this difference very clearly, we have best hope of 
solving it.

Ross Overbeek once told me:

Momements of confusion are to be cherished, because they
precede moments of enlightment.

I apologize for not understanding this more deeply; like you,
I am still learning.

> 
> Anyway, I will continue hacking as much as I can.
> (asdf:operate 'asdf:load-op :elephant)

> Thanks,
> Waldo
> 
> On Jan 11, 2006, at 7:39 AM, Robert L. Read wrote:
> 
> > OK, thank you.
> >
> > If we can get Elephant into a state that you don't have to be a  
> > LISP hacker to use it,
> > we will be greatly expanding its usefulness.
> >
> > I will try to work with Waldo Rubinstein to solve this problem for  
> > openmcl.
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20060113/8bb49c17/attachment.html>