From ch-rucksack at bobobeach.com Fri Dec 1 00:14:09 2006 From: ch-rucksack at bobobeach.com (Cyrus Harmon) Date: Thu, 30 Nov 2006 16:14:09 -0800 Subject: [rucksack-devel] indexing issue? In-Reply-To: References: <42002F55-6253-465C-BC6F-33CE9A84DA80@bobobeach.com> <15FC081E-4C24-4CE6-8C5D-D6C3AE30AFA6@bobobeach.com> <0B87D3FA-6352-4804-B3D3-1CB93A373888@bobobeach.com> Message-ID: <9192B6E3-BC68-483C-83CD-46F8BF3DDBE9@bobobeach.com> Following up on my previous email, the following works as expected: (defun test-persistent-class-cycle () (rucksack:with-rucksack (rucksack *test-rucksack*) (rucksack:with-transaction () (let ((p (make-instance 'a-persistent-class :name "ape"))) (setf (info p) p))))) (defun test-persistent-class-cycle-lookup () (rucksack:with-rucksack (rucksack *test-rucksack*) (rucksack:with-transaction () (rucksack:rucksack-map-slot rucksack 'a-persistent-class 'name (lambda (x) (print (list (info (info x)) (name x) x))) :equal "ape")))) but when the cycle is in a non-persistent class reachable from the persistent class, bad things happen (i.e. it hangs). Cyrus On Nov 30, 2006, at 3:49 PM, Cyrus Harmon wrote: > > On Nov 30, 2006, at 12:26 PM, Arthur Lemmens wrote: > >> Cyrus Harmon wrote: >>> Oh?0 is using the index the only way to find it? >> ... >> This is not a relational database, it's persistent Lisp objects ;-) > > Indeed. Still trying to get my head around that concept fully. > > So does this mean that when I have an object of an indexed class I > don't need to explicitly call add-rucksack-root? > >> The easiest way is probably to use SLOT-MAKUNBOUND. This should >> sooner or later call RUCKSACK-MAYBE-INDEX-CHANGED-SLOT, which >> will call INDEX-DELETE to remove the association between the >> old slot value and the object instance from the index. > > Hmm... I'm confused, but I'll keep digging. > >>> Finally, one further question... what does rucksack do about cycles? >> >> Nothing special, really. I'm not sure that I understand your >> question. > > Well, for starters there are cyclic lists and it looks like > rucksacks answer to that is: > > Serializing circular lists isn't implemented yet. > [Condition of type SIMPLE-ERROR] > > Restarts: > 0: [ABORT] Abort # with 8 dirty objects> > 1: [RETRY] Retry # with 8 dirty objects> > 2: [ABORT-REQUEST] Abort handling SLIME request. > 3: [TERMINATE-THREAD] Terminate this thread (# thread" {36147FF9}>) > > Backtrace: > 0: ((SB-PCL::FAST-METHOD RUCKSACK::SERIALIZE (CONS T)) > # # #1=(1 . #1#) > #) > 1: ((SB-PCL::FAST-METHOD RUCKSACK::SAVE-OBJECT (T #1="#<...>" . > #1#)) # # # # cyrusharmon.org/research/discs/rucksack/heap" and 45 objects in > memory.>> 58 # Users/sly/projects/cyrusharmon.org/research/discs/rucksack/heap" > and 45 objects in memory.> 337391780600000 NIL :SCHEMA NIL) > 2: ((SB-PCL::FAST-METHOD RUCKSACK::SAVE-DIRTY-OBJECT (T > #1="#<...>" . #1#)) # # # TEST #58 in # projects/cyrusharmon.org/research/discs/rucksack/heap" and 45 > objects in memory.>> # #P"/Users/sly/projects/cyrusharmon.org/research/discs/rucksack/ > heap" and 45 objects in memory.> # #337391780600000 with 8 dirty objects> 58 :SCHEMA NIL) > 3: ((SB-PCL::FAST-METHOD RUCKSACK:TRANSACTION-COMMIT-1 > (RUCKSACK:STANDARD-TRANSACTION RUCKSACK:STANDARD-CACHE > RUCKSACK:STANDARD-RUCKSACK)) # # argument> # dirty objects> # Users/sly/projects/cyrusharmon.org/research/discs/rucksack/heap" > and 45 objects in memory.> # in #P"/Users/sly/projects/cyrusharmon.org/research/discs/rucksack/" > with 3 roots {175E4AF9}>) > 4: ((LAMBDA (SB-PCL::.PV-CELL. SB-PCL::.NEXT-METHOD-CALL. SB- > PCL::.ARG0. SB-PCL::.ARG1. SB-PCL::.ARG2.)) # > # # #337391780600000 with 8 dirty objects> # size 10000, heap #P"/Users/sly/projects/cyrusharmon.org/research/ > discs/rucksack/heap" and 45 objects in memory.> # TRANSACTION-RUCKSACK in #P"/Users/sly/projects/cyrusharmon.org/ > research/discs/rucksack/" with 3 roots {175E4AF9}>) > 5: (TEST-CYCLE) > 6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (TEST-CYCLE) #) > 7: (SWANK::EVAL-REGION "(test-cycle) > > which is better than spinning for ever. > >>> Is there a way to break cycles >> >> What kind of cycles would you want to break, and why? > > Well, breaking them isn't the right answer, but somehow > representing them in persistent storage would be nice. > >>> When I try to add an object that points to itself rucksack seems >>> to spin forever. >> >> If it does, that's a bug. Can you show me the code that does this? > > Working on a test case. What I have is an object that contains a > (non-persistent) object that points to itself. Which leads me to > what is probably another fundamental misunderstanding on my part. > What does rucksack do with objects of non-persistent classes? > Here's my test: > > > (defclass a-non-persistent-class () > ((bogosity :accessor bogosity :initarg :bogosity))) > > (eval-when (:compile-toplevel :load-toplevel) > (rucksack:with-rucksack (rucksack *test-rucksack* #+nil :if- > exists #+nil :supersede) > (rucksack:with-transaction () > > (defclass a-persistent-class () > ((name :accessor name :initarg :name :index :string-index) > (info :accessor info :initarg :info)) > (:metaclass persistent-class) > (:index t))))) > > > (defun test-not-persistent-data-1 () > (let ((np (make-instance 'a-non-persistent-class :bogosity 412))) > (rucksack:with-rucksack (rucksack *test-rucksack*) > (rucksack:with-transaction () > (let ((p (make-instance 'a-persistent-class :name > "moose" :info np))) > (print (name p))))))) > > (defun test-not-persistent-data-lookup-1 () > (rucksack:with-rucksack (rucksack *test-rucksack*) > (rucksack:with-transaction () > (rucksack:rucksack-map-slot > rucksack 'a-persistent-class 'name > (lambda (x) > (print (list (bogosity (info x)) (name x) x))))))) > > (defun test-not-persistent-data-2 () > (let ((np (make-instance 'a-non-persistent-class :bogosity 412))) > (setf (bogosity np) np) > (rucksack:with-rucksack (rucksack *test-rucksack*) > (rucksack:with-transaction () > (let ((p (make-instance 'a-persistent-class :name > "moose" :info np))) > (print (name p))))))) > > > > test-persistent-data-1 seems to work fine and the slot in a-non- > persistent-class is even preserved, it seems. yay! > > but, when you go to run test-not-persistent-data-2 you get stuck on > a loop. yes, perhaps this is a contrived situation, but it's the > situation I find myself when trying to shoe-horn my existing > classes into rucksack. > > So, in summary, there are two separate issues it seems, cyclic > lists, which are not supported but which throw an error and > persistent objects that contain objects that point to themselves > which just hang. > > Thanks again for the insightful discussion and for rucksack in > general! It's starting to get clearer. > > Cyrus > > _______________________________________________ > rucksack-devel mailing list > rucksack-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/rucksack-devel From alemmens at xs4all.nl Fri Dec 1 20:50:51 2006 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Fri, 01 Dec 2006 21:50:51 +0100 Subject: [rucksack-devel] indexing issue? In-Reply-To: <9192B6E3-BC68-483C-83CD-46F8BF3DDBE9@bobobeach.com> References: <42002F55-6253-465C-BC6F-33CE9A84DA80@bobobeach.com> <15FC081E-4C24-4CE6-8C5D-D6C3AE30AFA6@bobobeach.com> <0B87D3FA-6352-4804-B3D3-1CB93A373888@bobobeach.com> <9192B6E3-BC68-483C-83CD-46F8BF3DDBE9@bobobeach.com> Message-ID: Cyrus Harmon wrote: > but when the cycle is in a non-persistent class reachable from the > persistent class, bad things happen (i.e. it hangs) Yes, that's expected behavior. ;-) With Rucksack, you can have as many cycles as you want in persistent objects, but Rucksack won't attempt to detect cycles when serializing objects that are 'outside Rucksack's control'. The reason is that this is too expensive in general. For circular lists it's cheap and I suppose that could be added to Rucksack, but I wouldn't want to add circularity detection for arbitrary data structures. Arthur From alemmens at xs4all.nl Fri Dec 1 21:17:13 2006 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Fri, 01 Dec 2006 22:17:13 +0100 Subject: [rucksack-devel] indexing issue? In-Reply-To: References: <42002F55-6253-465C-BC6F-33CE9A84DA80@bobobeach.com> <15FC081E-4C24-4CE6-8C5D-D6C3AE30AFA6@bobobeach.com> <0B87D3FA-6352-4804-B3D3-1CB93A373888@bobobeach.com> Message-ID: Cyrus Harmon wrote: > So does this mean that when I have an object of an indexed class I > don't need to explicitly call add-rucksack-root? Yes, that's right. The class index itself is a Rucksack root, and that index points to all instances of the class, so you don't need to add it explicitly to the root set. > Well, for starters there are cyclic lists and it looks like rucksacks > answer to that is: > > Serializing circular lists isn't implemented yet. Yes. See my other reply. >> What kind of cycles would you want to break, and why? > > Well, breaking them isn't the right answer, but somehow representing > them in persistent storage would be nice. See my other mail: cycles are OK, but only as long as the cycles 'pass through' objects that were explicitly declared to be persistent. > Working on a test case. What I have is an object that contains a (non- > persistent) object that points to itself. Which leads me to what is > probably another fundamental misunderstanding on my part. What does > rucksack do with objects of non-persistent classes? It serializes them, assuming that they don't contain cycles. > (defclass a-non-persistent-class () > ((bogosity :accessor bogosity :initarg :bogosity))) > > (eval-when (:compile-toplevel :load-toplevel) > (rucksack:with-rucksack (rucksack *test-rucksack* #+nil :if-exists > #+nil :supersede) > (rucksack:with-transaction () > > (defclass a-persistent-class () > ((name :accessor name :initarg :name :index :string-index) > (info :accessor info :initarg :info)) > (:metaclass persistent-class) > (:index t))))) > > > (defun test-not-persistent-data-1 () > (let ((np (make-instance 'a-non-persistent-class :bogosity 412))) > (rucksack:with-rucksack (rucksack *test-rucksack*) > (rucksack:with-transaction () > (let ((p (make-instance 'a-persistent-class :name > "moose" :info np))) > (print (name p))))))) > [SNIP] > > (defun test-not-persistent-data-2 () > (let ((np (make-instance 'a-non-persistent-class :bogosity 412))) > (setf (bogosity np) np) > (rucksack:with-rucksack (rucksack *test-rucksack*) > (rucksack:with-transaction () > (let ((p (make-instance 'a-persistent-class :name > "moose" :info np))) > (print (name p))))))) > > > > test-persistent-data-1 seems to work fine and the slot in a-non- > persistent-class is even preserved, it seems. yay! Yes, that's expected. > but, when you go to run test-not-persistent-data-2 you get stuck on a > loop. And this is expected too. > yes, perhaps this is a contrived situation, but it's the situation I > find myself when trying to shoe-horn my existing classes into rucksack. I understand. But I think that the best approach for you would be to add (:METACLASS PERSISTENT-CLASS) to your existing DEFCLASS forms. I would think that isn't too much work, even if you have lots of class definitions. > So, in summary, there are two separate issues it seems, cyclic lists, > which are not supported but which throw an error This could be fixed quite easily. Have a look at serialize.lisp to get a feeling for how this could be done. > and persistent objects that contain objects that point to themselves > which just hang. Yes. That's not very likely to change. Arthur