[elephant-devel] Cross-references and SETF SLOT-VALUE

Fri Feb 22 12:35:08 UTC 2008

The problem lies in an implementation choice.  It can be changed, but  
it's not as easy as it sounds.

1) The class object caches the class index reference.  It currently  
can only maintain a single cached reference.  (from metaclasses.lisp)

(defclass persistent-metaclass (standard-class)
   ((%persistent-slots :accessor %persistent-slots)
    (%indexed-slots :accessor %indexed-slots)
    (%index-cache :accessor %index-cache))
   (:documentation
    "Metaclass for persistent classes.  Use this metaclass to
     define persistent classes.  All slots are persistent by
     default; use the :transient flag otherwise.  Slots can also
     be indexed for by-value retrieval."))

2) An indexed slot writer (in classindex.lisp) is called during writes  
which looks up the class index using find-class-index

(defmethod indexed-slot-writer ((class persistent-metaclass) (instance  
persistent-object) (slot-def persistent-slot-definition) new-value)
   "Anything that side effects a persistent-object slot should call  
this to keep
    the dependant indices in synch.  Only classes with derived indices  
need to
    update on writes to non-indexed slots.  This is a side effect of  
user-managed
    indices in Elephant - a necessity because we allow arbitrary lisp  
expressions to
    determine index value so without bi-directional pointers, the  
indices cannot
    automatically update a changed indexed value in derived slots"
   (let ((slot-name (slot-definition-name slot-def))
	(oid (oid instance))
	(con (get-con instance)))
     (declare (type fixnum oid))
     (if (no-indexing-needed? class instance slot-def oid)
	(persistent-slot-writer con new-value instance slot-name)
	(let ((class-idx (find-class-index class)))
	  (ensure-transaction (:store-controller con)
	    (when (get-value oid class-idx)
	      (remove-kv oid class-idx))
	    (persistent-slot-writer con new-value instance slot-name)
	    (setf (get-value oid class-idx) instance))))))

3) find-class-index (classindex.lisp) looks in the class cache,  
defined above, to avoid doing an additional btree lookup on-disk for  
every write operation.

(defmethod find-class-index ((class persistent-metaclass) &key (sc  
*store-controller*) (errorp t))
   (ensure-finalized class)
   (if (not (indexed class))
       (when errorp
	(signal-class-not-indexed class))
       (if (class-index-cached? class sc)
	  (%index-cache class) ;; we've got a cached reference, just return it
	  (multiple-value-bind (btree found)
	      (get-value (class-name class) (controller-class-root sc))
	    (if found
		(cache-existing-class-index class btree sc)
		(cache-new-class-index class sc))))))

4) Now we could easily make this cache slot be an alist and dispatch  
on the store controller, but there is another wrinkle.  The system  
tries to be smart about not losing references to index data if the  
user happens to change the class definition between connections.  In  
general there is a configurable facility for defining what happens  
when the indexed slot list of the class and the database disagree.

(defun cache-existing-class-index (class btree sc)
   "If we have a persistent index already, assign, synchronize &  
return it"
   (let ((method (determine-synch-method class)))
     (setf (%index-cache class) btree)
     (synchronize-class-to-store class :sc sc :method method)
     btree))

;; from classindex-utils.lisp
(defun synchronize-class-to-store (class &key (sc *store-controller*)
				   (method *default-indexed-class-synch-policy*))
   (let ((slot-records (compute-class-and-ele-status class sc))
	(rule-set (cdr (assoc method *synchronize-rules*))))
     (apply-synch-rules class slot-records rule-set)))

If you can come up with a reasonable policy for N-way disagreement,  
then the rest of the changes probably aren't too hard.  Perhaps this  
could be configurable.  If a store controller has already connected to  
the class, then you just add any indices that are missing in the new  
database.

Remember, though, that the consequence of adding an index is that the  
whole index has to be constructed so for classes with lots of  
instances, this can take awhile.

If this doesn't make sense, you should try skimming the code in  
metaclasses.lisp, classindex.lisp, and classindex-utils.lisp.

Ian

On Feb 22, 2008, at 4:05 AM, Leslie P. Polzer wrote:

>
>
>> Do you need to do indexing operations across stores?  i.e. run a
>> cursor over all objects regardless of which store they are in?
>
> No, I don't do this at the moment. I might later, so let's postpone
> that point.
>
>
>> Ideally, what semantics would you like to see?
>
> Right now all I want is “do what I mean” behaviour for SETF SLOT- 
> VALUE.
> AFAICS the current behaviour for this operation is that Elephant  
> attempts
> to stuff the object into the current sc instead of using the objects
> home controller. Which seems not exactly sensible to me.
>
>
>> Nope, the system only creates one class index independent of which
>> store controller it's in, so you can only map indexed class instances
>> to one store.
>
> I'm really having difficuly correlating what you say here to my
> experience. I can happily have separate sets of indexed objects
> with separate store controllers, or am at least deluded in a
> convincing way :)
>
> So I suppose it's my own dumbness again that's preventing me
> from understanding correctly what you say here.
>
> What you say here is that there's only one index for all controllers,
> but otherwhere you state that the problem is separation of
> controller indexes... I'm confused.
>
>
>> e.g. you create indexed instances in store 1, the class index
>> associated with store one indexes all those instances.  When you call
>> get-instances-by-class in the context of store 2, however, all the
>> objects from store 1 are now missing.  The semantics of get-instance-
>> by-class/value/range are all incorrect when you have instances stored
>> in multiple store controllers.
>
> It makes good sense for me, since I'm used to the model of store
> controllers as objects with a very high grade of separation.
> So when I change the sc, everything is different.
> And this fits my current project well, too.
>
>
>> Using multiple stores starts to break the model of a simple global
>> namespace assumed by the current class indexing mechanism.  Rather
>> than contort the class indexing mechanism to accommodate multiple
>> store operations, my preference is that users write their own  
>> indexing
>> mechanism that has the semantics they care about.
>
> It depends on what you intend here.
>
> If the aim is global indexing across controllers, let's have it your
> way with user-based indexing. It makes sense.
>
> But if we talk about the thing as I see it, i.e. indexing separate
> for all controllers, let's just fix the system so it behaves well
> in this context.
>
>  Leslie
>
>
> -- 
> My personal blog: http://blog.viridian-project.de/
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel