[elephant-devel] Querying Advice

Tue Nov 14 19:50:19 UTC 2006

Ian et al,

Based on my comment to Robert and that of Pierre, could you, or  
anyone, please clarify this for me (and maybe others):

If making a class persistent means that there is no need to add it to  
root or to any persistent collection, when I look at Robert's sample  
code, I see (excerpts):

(defclass User ()
   ((username :type 'string :initform "" :initarg :uname :accessor  
username-of)
    (password :type 'string :initform "" :initarg :pword :accessor  
password-of)
    (email :type 'string :initform "" :initarg :email :accessor email- 
of)
    (fullname :type 'string :initform "" :initarg :fullname :accessor  
fullname-of)
    (balance :type 'integer :initform 0 :initarg :balance :accessor  
balance-of)))

(defun random-users (n)
   (dotimes (x n)
     (let ((u (make-instance
               'User
               :uname (format nil "user~A" x)
               :pword (random-password)
               :email (format nil "user~A at .nowheresville.org" x)
               :fullname (format nil "~A~A ~A~A" (random-password) x  
(random-password) x)
               :balance (random 100))))
       (add-to-root x u))))

There is an explicit add-to-root in random-users. I suppose the  
reason for this is because User class does not inherit from  
persistent-metaclass and in order to be able to "search for" or  
retrieve that object (could this also be the reason for the  
additional storage overhead, as you pointed out yesterday?). Right?  
So, if my understanding is correct, defining User with defpclass  
instead would mean that you don't have to add-to-root because it will  
be automatically persisted. However, after the function exits, there  
will be no reference to that persistent object and will therefore be  
eventually garbage collected (whether or not the persistent space  
will be reclaimed is a different story, as you mentioned in your  
email). Is that right? If so, how could I avoid for the User objects  
to be garbage collected in this case, since there really is no other  
reference to these objects after creating them? Or, if the objects  
are NOT garbage collected, how could I manually "delete" any of them?

Now, if I (or Robert) had defined the User class as:

(defpclass User ()
   ((username :type 'string :initform "" :initarg :uname :accessor  
username-of :index t)
    (password :type 'string :initform "" :initarg :pword :accessor  
password-of)
    (email :type 'string :initform "" :initarg :email :accessor email- 
of)
    (fullname :type 'string :initform "" :initarg :fullname :accessor  
fullname-of)
    (balance :type 'integer :initform 0 :initarg :balance :accessor  
balance-of)))

where (notice how it's defined with defpclass) the username slot is  
indexed, the system would automatically store a reference to the  
object in the slot index, and there would be no need to use the add- 
to-root in random-users. Correct? If that's the case, how would I  
then go about removing this user object from persistence? Would it be  
by setting the indexed slot value to NIL?

On another note, if I want to create a collection of users, I don't  
have to store these users in a collection. Simply making them inherit  
from persistent-metaclass and indexing them would do so automatically  
(just like the example above). Right? How about this, then:

(asdf:operate 'asdf:load-op :elephant-tests)

(in-package :elephant-tests)

(setf *default-spec* *testbdb-spec*)

(open-store *default-spec*)

(defpclass state ()
   ((abbr :type 'string :initform "" :initarg :abbr :accessor abbr- 
of :index t)
    (name :type 'string :initform "" :initarg :name :accessor name-of)))

(defpclass zip-code ()
   ((zip :type 'string :initform "" :initarg :zip :accessor zip- 
of :index t)
    (city :type 'string :initform "" :initarg :city :accessor city-of)
    (county :type 'string :initform "" :initarg :county :accessor  
county-of)
    (state :initform nil :initarg :state :accessor state-of)))

(defmethod print-object ((obj state) stream)
   (format stream "State (abbr, name) = (~A, ~A)" (abbr-of obj) (name- 
of obj)))

(defmethod print-object ((obj zip-code) stream)
   (format stream "Zip (zip, city, county, state) = (~A, ~A, ~A, ~A)"
           (zip-of obj) (city-of obj) (county-of obj) (state-of obj)))

(let* ((s1 (make-instance 'state :abbr "FL" :name "Florida"))
       (s2 (make-instance 'state :abbr "NY" :name "New York"))
       (z1 (make-instance 'zip-code :zip "33015" :city  
"Miami" :county "Dade" :state s1))
       (z2 (make-instance 'zip-code :zip "13605" :city  
"Adams" :county "Jefferson" :state s2))
       (z2 (make-instance 'zip-code :zip "33160" :city "Sunny Isles  
Beach" :county "Dade" :state s1)))
   (print s1)
   (print s2)
   (print z1)
   (print z2)
   (print z3))

Here, I'm creating a couple of state objects and a couple of zip-code  
objects. Since a zip-code can only belong to one state, they have a  
reference back to the state in order to "quickly" determine the state  
they belong to.

A couple of questions/comments here:

1) If I wanted to ask a state to give me all the zip-code(s) within  
it, would I create a slot in the state class to hold a collection of  
zip-code references? Or would I simply create a state-class method like:

(defmethod get-zip-codes ((obj state))
   (get-instances-by-value 'zip-code 'state obj))

This does not work because the state slot in zip-code is not indexed.  
Also, from the code above, I don't know how to get a reference to the  
btree in order to create a cursor so that I can linearly traverse the  
zip-code(s) to return only those zip-code(s) which belong to that state.

Of course, I would probably want to index the state slot of zip-code  
because there are tens of thousands of zip codes and I wouldn't want  
to linearly traverse them, but I just wanted to illustrate the  
problem to get some additional feedback (the overhead of maintaining  
a secondary index on state wouldn't matter too much to me because  
changes to this class/objects are very rare but access is much more  
frequent)

2) Maybe this is a totally different issue and mainly caused by my  
lisp ignorance:

If I have:

(defparameter *s1* (get-instances-by-value 'state 'abbr "FL"))
(defparameter *s2* (get-instances-by-value 'state 'abbr "NY"))
(defparameter *z1* (get-instances-by-value 'zip-code 'zip "33015"))
(defparameter *z2* (get-instances-by-value 'zip-code 'zip "13605"))
(defparameter *z3* (get-instances-by-value 'zip-code 'zip "33160"))

and I want to remove the association of zip code 33160 from the "FL"  
state object, I would think all I have to do is:

(setf (state-of *z3*) nil)

However, when I do so, I get this error message:

There is no applicable method for the generic function
   #<STANDARD-GENERIC-FUNCTION (SETF STATE-OF) (1)>
when called with arguments
   (NIL
    (Zip (zip, city, county, state) = (33160, Sunny Isles Beach,  
Dade, State (abbr, name) = (FL, Florida)))).
    [Condition of type SIMPLE-ERROR]

Restarts:
   0: [ABORT-REQUEST] Abort handling SLIME request.
   1: [ABORT] Exit debugger, returning to top level.

Backtrace:
   0: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #<unavailable  
argument> #<unavailable argument> #<STANDARD-GENERIC-FUNCTION (SETF  
STATE-OF) (1)> (NIL (Zip (zip, city, county, state) = (33160, Sunny  
Isles Beach, Dade, State (abbr, name) = (FL, Florida)))))
   1: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) #<unavailable  
argument> #<unavailable argument> #<STANDARD-GENERIC-FUNCTION (SETF  
STATE-OF) (1)> (NIL (Zip (zip, city, county, state) = (33160, Sunny  
Isles Beach, Dade, State (abbr, name) = (FL, Florida)))))
   2: (SB-INT:EVAL-IN-LEXENV (SETF (STATE-OF *Z3*) NIL) #<NULL-LEXENV>)
   3: (SWANK::EVAL-REGION "(setf (state-of *z3*) nil)" T)

3) So, going back to the previous questions in the email, if I wanted  
to permanently remove zip code "33015", how would I go about it?

Thanks again and thanks for the patience.

- Daniel

On Nov 14, 2006, at 9:04 AM, Ian Eslick wrote:

> Persistent classes in the current model aren't easy to fully delete -
> you can only make the unreachable by having no reference to them
> reachable from the controller or class roots.  You can drop an object
> from the index and there is a way to delete the object from the main
> system BTree - but until 4.4 you couldn't reclaim that space  
> (unless it
> happened to be reused opportunistically).  In 4.4 you can compact the
> DB, but elephant does not take advantage of this yet.
>
> Ideally there would be an automated way to 'migrate' which is a
> stop-and-copy model of GC - everything reachable from the  
> controller and
> class roots gets copied, anything that isn't reachable stays in the  
> old
> DB.  This compacts and cleans the DB.  I hope that we'll clean up the
> model of space reclamation and provide something akin to GC in the not
> too distant future.  It's not a high priority in large part because
> on-disk space has little implication on in-memory space and disk is ~
> free.  (Although I do suffer from too many backup copies of my 6GB  
> DB -
> fortunately I just upgraded my computer and doubled my hard drive  
> space
> so I'm OK for a awhile longer!)
>
> Ian
>
> Pierre THIERRY wrote:
>> Scribit Daniel Salama dies 13/11/2006 hora 21:00:
>>
>>> So, if I make my class persistent, it's persistent... period! I  
>>> don't
>>> need to add it to a collection or to the root.
>>>
>>
>> Well, I did not know that before that discussion. I think it's not
>> clearly indicated in the docs...
>>
>> And then I wonder: if I use persistent classes, how do I remove an
>> object from the store?
>>
>> Curiously,
>> Nowhere man
>>
>> --------------------------------------------------------------------- 
>> ---
>>
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel