From keke at gol.com Sat Mar 10 02:52:33 2007 From: keke at gol.com (Takehiko Abe) Date: Sat, 10 Mar 2007 11:52:33 +0900 Subject: [rucksack-devel] Patch for openmcl Message-ID: <20070310025233.30223@roaming.gol.com> Attached is a patch for OpenMCL 1.1 (and MCL 5.1). (MCL's loop does not work for loop in rs::serialize-dotted-list . The loop.lisp that comes with openmcl has no problem and it seems to work fine with MCL.) regards, T. -------------- next part -------------- A non-text attachment was scrubbed... Name: openmcl.diff Type: application/applefile Size: 528 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: openmcl.diff Type: application/octet-stream Size: 3032 bytes Desc: not available URL: From csr21 at cantab.net Mon Mar 12 16:40:14 2007 From: csr21 at cantab.net (Christophe Rhodes) Date: Mon, 12 Mar 2007 16:40:14 +0000 Subject: [rucksack-devel] How do you use rucksack? Message-ID: <87fy8aqutt.fsf@cantab.net> Hi, I began to investigate building a simple application over the weekend, along the lines of an addressbook, and I thought I'd give rucksack a try for the persistent storage facility. However, I quickly ran up against the problem of wanting to delete objects from indices. For instance, one way of managing the information might be something along the lines of (defclass person () ((surname :index ) (other-names) (address)) (:metaclass persistent-class) (:index t)) but I rapidly discovered, as I built toy commands for adding, visualising and editing entries, that I couldn't remove anything. I understand the rationale given in the ECLM talk -- you get dangling pointers if you let users delete stuff -- but then I simply don't understand how you're meant to use rucksack. Anything which is indexed is a root for the garbage collector, so it would appear that I can't have both indexing over some property of a class and also have instances of that class ever be reclaimed by the garbage collector. Is that right? If so, how would you suggest that something like an addressbook be implemented using rucksack for storage. (If you do, that is; maybe rucksack just isn't designed for that, but if it isn't I'd be interested in knowing in what circumstances it is used.) I can imagine using, say, a persistent array for my root set, but as I say, to get collection of my instances I must not index anything... Thanks, Christophe From alemmens at xs4all.nl Tue Mar 13 13:16:59 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Tue, 13 Mar 2007 14:16:59 +0100 Subject: [rucksack-devel] Bug in leaf-delete-key In-Reply-To: <1170438022.8265.397.camel@trinidad> References: <1170438022.8265.397.camel@trinidad> Message-ID: Henrik Hjelte wrote: > While trying out this beautiful source-code, I found a little bug in > leaf-delete-key, see below. Good catch! Thanks a lot (and sorry for the absurdly late reply). I've committed your fix. > While writing this I saw a recent thread about deleting and index > problems, and I think this fixes these problems. I don't think it does, unfortunately. To verify that, you can run the BTREE-STRESS-TEST function. Arthur From alemmens at xs4all.nl Tue Mar 13 13:19:28 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Tue, 13 Mar 2007 14:19:28 +0100 Subject: [rucksack-devel] delete-instances method In-Reply-To: <1170438906.8265.405.camel@trinidad> References: <1170438906.8265.405.camel@trinidad> Message-ID: Henrik Hjelte wrote: > May I propose adding an easier way to remove instances from the > database, something like the method below? Thanks, I followed your suggestion and added the functions RUCKSACK-DELETE-OBJECT and RUCKSACK-DELETE-OBJECTS, based on your example implementation. I committed these functions to CVS, but I haven't tested them yet. Arthur From alemmens at xs4all.nl Tue Mar 13 13:27:59 2007 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Tue, 13 Mar 2007 14:27:59 +0100 Subject: [rucksack-devel] How do you use rucksack? In-Reply-To: <87fy8aqutt.fsf@cantab.net> References: <87fy8aqutt.fsf@cantab.net> Message-ID: Hi Christophe, > I began to investigate building a simple application over the weekend, > along the lines of an addressbook, and I thought I'd give rucksack a > try for the persistent storage facility. However, I quickly ran up > against the problem of wanting to delete objects from indices. Yeah, that's becoming something of a FAQ. You may want to try out the brand new RUCKSACK-DELETE-OBJECT function, which removes an object from the root set (if necessary) and from all class and slot indexes in which it appears. I committed this function to the CVS repository today. I haven't tested this function yet, and I'd be interested to know if it works for you. > Anything which is indexed is a root for the garbage collector, so it > would appear that I can't have both indexing over some property of a > class and also have instances of that class ever be reclaimed by the > garbage collector. Is that right? Yes, that's right. But you can remove an object from indexes; see the implementation of RUCKSACK-DELETE-OBJECT. If this doesn't answer your question, let me know. Arthur From attila.lendvai at gmail.com Tue Mar 13 13:43:58 2007 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Tue, 13 Mar 2007 14:43:58 +0100 Subject: [rucksack-devel] How do you use rucksack? In-Reply-To: References: <87fy8aqutt.fsf@cantab.net> Message-ID: > > I began to investigate building a simple application over the weekend, > > along the lines of an addressbook, and I thought I'd give rucksack a > > try for the persistent storage facility. However, I quickly ran up > > against the problem of wanting to delete objects from indices. > > Yeah, that's becoming something of a FAQ. You may want to try out the > brand new RUCKSACK-DELETE-OBJECT function, which removes an object from > the root set (if necessary) and from all class and slot indexes in which > it appears. I committed this function to the CVS repository today. you probably already consider adding weakness support for persistent collections and references, but i throw it in just in case... it could be another piece in the puzzle to mimic the way the VM and the GC is managing the heap: you keep one or more strong collections and/or references that represent ownership or something that should keep the object alive, and use weak collections/indices/references where you are just using them for optimization or at a semantically weak connection between two objects (if there is such thing at all which does not qualify as an optimization). for slots this could be either a slot option (probably simpler) or a wrapper struct that is automatically stripped by slot-value-using-class from the slot values. just some 0.02, -- - attila "- The truth is that I've been too considerate, and so became unintentionally cruel... - I understand. - No, you don't understand! We don't speak the same language!" (Ingmar Bergman - Smultronst?llet) From csr21 at cantab.net Tue Mar 13 15:50:32 2007 From: csr21 at cantab.net (Christophe Rhodes) Date: Tue, 13 Mar 2007 15:50:32 +0000 Subject: [rucksack-devel] How do you use rucksack? In-Reply-To: (Arthur Lemmens's message of "Tue, 13 Mar 2007 14:27:59 +0100") References: <87fy8aqutt.fsf@cantab.net> Message-ID: <87ps7d16t3.fsf@cantab.net> "Arthur Lemmens" writes: > Hi Christophe, > >> I began to investigate building a simple application over the weekend, >> along the lines of an addressbook, and I thought I'd give rucksack a >> try for the persistent storage facility. However, I quickly ran up >> against the problem of wanting to delete objects from indices. > > Yeah, that's becoming something of a FAQ. You may want to try out the > brand new RUCKSACK-DELETE-OBJECT function, which removes an object from > the root set (if necessary) and from all class and slot indexes in which > it appears. I committed this function to the CVS repository today. > > I haven't tested this function yet, and I'd be interested to know if it > works for you. OK. I'll continue exploring. >> Anything which is indexed is a root for the garbage collector, so it >> would appear that I can't have both indexing over some property of a >> class and also have instances of that class ever be reclaimed by the >> garbage collector. Is that right? > > Yes, that's right. But you can remove an object from indexes; see the > implementation of RUCKSACK-DELETE-OBJECT. > > If this doesn't answer your question, let me know. Well, I think it does up to a point, in that it removes the immediate roadblock I ran up against; I don't think that it explains how other people are using rucksack's exported functionality -- presumably you yourself haven't felt the need for delete-object, but maybe that's because you don't worry about using internal symbols? I don't know, that's kind of why I asked. Given the acceptability of delete-object functionality, I wonder if there's a case for separating the object initialization and object indexing protocols slightly. Currently, I think that the initialization of an instance of persistent-object performs a number of steps involging adding the new object to the applicable indices. If that's so, maybe this is the natural inverse to what is currently called delete-object? What I'm thinking about is if, having deleted an object from all the applicable indices, it could be convenient to add it back again. If so, maybe there should be a defined pair of functions, conceptually add-object and delete-object, and that add-object should be called from the initialize-instance :around method, but also callable by users. Cheers, Christophe From uvl at htg1.de Tue Mar 13 16:54:31 2007 From: uvl at htg1.de (Uwe von Loh) Date: Tue, 13 Mar 2007 17:54:31 +0100 Subject: [rucksack-devel] delete-instances method In-Reply-To: References: <1170438906.8265.405.camel@trinidad> Message-ID: <45F6D747.1030108@htg1.de> Arthur Lemmens wrote: > Thanks, I followed your suggestion and added the functions > RUCKSACK-DELETE-OBJECT and RUCKSACK-DELETE-OBJECTS, based on > your example implementation. I committed these functions to > CVS, but I haven't tested them yet. Henriks DELETE-function saved me two weeks of guessing but it works for classes with direct slots only. Indexes of derived slots are indexed with the classes they are defined in. I assume this is the more general way of indexing slots because all are in one place then. Here is my "disimprovement" for deleting instances with direct and derived slots from all indexes. Caution, this is slow code. I'm not so familiar with real programming yet. Uwe 8-) (defmethod delete-instances ((rucksack standard-rucksack) instances) (let ((rucksack-roots (rucksack-roots rucksack))) (flet ((delete-instance (instance) (let ((class-index (rucksack-class-index rucksack (class-of instance) :errorp nil)) (slot-indexes (slot-indexes-of (class-of instance))) (oid (object-id instance))) (when class-index (index-delete class-index oid oid)) (when slot-indexes (loop for slot-index in slot-indexes do (when (slot-boundp instance (cadr slot-index)) (index-delete (rucksack-slot-index rucksack (car slot-index) (cadr slot-index)) (slot-value instance (cadr slot-index)) oid :if-does-not-exist :ignore)))) (when (member oid rucksack-roots) (delete-rucksack-root instance rucksack))))) (mapcar #'delete-instance instances)))) (defun slot-indexes-of (myclass) (mapcan #'indexed-direct-slots (maybe-indexed-superclasses-of myclass))) (defun indexed-direct-slots (myclass) (with-rucksack (rs *uwes-rs*) (with-transaction () (let ((slots (rucksack-indexed-slots-for-class rs myclass))) (if slots (mapcar #'(lambda (x) (list (class-name myclass) x))slots)))))) (defun maybe-indexed-superclasses-of (myclass) (let ((superclasses (class-direct-superclasses myclass))) (delete-duplicates (adjoin myclass (mapcan #'maybe-indexed-superclasses-of superclasses))))) From keke at gol.com Wed Mar 14 08:40:55 2007 From: keke at gol.com (Takehiko Abe) Date: Wed, 14 Mar 2007 17:40:55 +0900 Subject: [rucksack-devel] btree-insert Message-ID: <20070314084055.28977@roaming.gol.com> btree-insert compares value with btree-key-type . Please examine. regards, T. Index: p-btrees.lisp =================================================================== RCS file: /project/rucksack/cvsroot/rucksack/p-btrees.lisp,v retrieving revision 1.14 diff -u -r1.14 p-btrees.lisp --- p-btrees.lisp 13 Mar 2007 13:13:00 -0000 1.14 +++ p-btrees.lisp 14 Mar 2007 08:36:20 -0000 @@ -548,7 +548,7 @@ :btree btree :datum key :expected-type (btree-key-type btree))) - (unless (typep value (btree-key-type btree)) + (unless (typep value (btree-value-type btree)) (error 'btree-type-error :btree btree :datum value From keke at gol.com Wed Mar 14 09:33:20 2007 From: keke at gol.com (Takehiko Abe) Date: Wed, 14 Mar 2007 18:33:20 +0900 Subject: [rucksack-devel] unique slot Message-ID: <20070314093320.12782@roaming.gol.com> I think it would be nice if rucksack signals error when I give a duplicated key to a unique slot. But then I noticed the following comment in do.txt: - Check that btrees actually signal an error for duplicate keys. Handle those errors correctly for slot indexes. Why does rucksack need to handle the errors? Thanks, T. Index: rucksack.lisp =================================================================== RCS file: /project/rucksack/cvsroot/rucksack/rucksack.lisp,v retrieving revision 1.20 diff -u -r1.20 rucksack.lisp --- rucksack.lisp 13 Mar 2007 13:13:00 -0000 1.20 +++ rucksack.lisp 14 Mar 2007 09:23:15 -0000 @@ -876,7 +886,9 @@ (when old-boundp (index-delete index old-value id :if-does-not-exist :ignore)) (when new-boundp - (index-insert index new-value id)))))) + (index-insert index new-value id + :if-exists + (and (slot-unique slot) :error))))))) (defmethod rucksack-slot-index ((rucksack standard-rucksack) class slot From keke at gol.com Wed Mar 14 09:33:12 2007 From: keke at gol.com (Takehiko Abe) Date: Wed, 14 Mar 2007 18:33:12 +0900 Subject: [rucksack-devel] slot-index Message-ID: <20070314093312.11312@roaming.gol.com> I think it would be nicer if rucksack signals error for me when I give an imcompatible value to a slot with a index. (e.g. giving string to :symbol-index.) Adding :key-type to index-spec definitions seems to accomplish it. But I'm not sure this is right. What do you think? regards, T. Index: index.lisp =================================================================== RCS file: /project/rucksack/cvsroot/rucksack/index.lisp,v retrieving revision 1.9 diff -u -r1.9 index.lisp --- index.lisp 20 Jan 2007 18:17:55 -0000 1.9 +++ index.lisp 14 Mar 2007 08:41:58 -0000 @@ -193,20 +193,21 @@ (eval-when (:compile-toplevel :load-toplevel :execute) (define-index-spec :number-index - '(btree :key< < :value= p-eql)) + '(btree :key< < :value= p-eql :key-type number)) (define-index-spec :string-index - '(btree :key< string< :value p-eql)) + '(btree :key< string< :value p-eql :key-type string)) (define-index-spec :symbol-index - '(btree :key< string< :value p-eql)) + '(btree :key< string< :value p-eql :key-type symbol)) (define-index-spec :case-insensitive-string-index - '(btree :key< string-lessp :value p-eql)) + '(btree :key< string-lessp :value p-eql :key-type string)) (define-index-spec :trimmed-string-index ;; Like :STRING-INDEX, but with whitespace trimmed left ;; and right. '(btree :key< string< :key-key trim-whitespace - :value p-eql))) + :value p-eql + :key-type string))) From csr21 at cantab.net Sat Mar 17 19:45:42 2007 From: csr21 at cantab.net (Christophe Rhodes) Date: Sat, 17 Mar 2007 19:45:42 +0000 Subject: [rucksack-devel] How do you use rucksack? In-Reply-To: (Arthur Lemmens's message of "Tue, 13 Mar 2007 14:27:59 +0100") References: <87fy8aqutt.fsf@cantab.net> Message-ID: <87abybr6vt.fsf@cantab.net> "Arthur Lemmens" writes: > [RUCKSACK-DELETE-OBJECT] > I haven't tested this function yet, and I'd be interested to know if it > works for you. My limited testing suggests that this function is behaving as advertised. > If this doesn't answer your question, let me know. (I'm still somewhat in the dark as to how one is supposed to use rucksack, but I am fumbling my way there in my toy application) Thanks, Christophe