From haragx at gmail.com Wed Jun 1 00:47:35 2011 From: haragx at gmail.com (Phil Marneweck) Date: Wed, 01 Jun 2011 02:47:35 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> Message-ID: <1306889260.8463.25.camel@mohadib> Well the first thing I need is a: 1. disconnected transaction that is explicitly commited. or 2. way to update slots outside of a transaction and a method to reintreduce the object to a new transaction. This is because the webframe work that I use does understand or use the concept of a db (never mind a transaction) so if the framework tries to change the object it fails. The disconnected transaction is most likely the best solution but it looked like implementing the update of slots outside of a transaction would be a easier quickfix. I had a quick try at stopping the marking of the object to dirty when a slot is set if there is no (current-transaction) but that failed the object still ends up in the dirty que. Anny suggestions? -------------- next part -------------- An HTML attachment was scrubbed... URL: From haragx at gmail.com Wed Jun 1 00:48:29 2011 From: haragx at gmail.com (Phil Marneweck) Date: Wed, 01 Jun 2011 02:48:29 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> Message-ID: <1306889314.9186.0.camel@mohadib> Well the first thing I need is a: 1. disconnected transaction that is explicitly commited. or 2. way to update slots outside of a transaction and a method to reintreduce the object to a new transaction. This is because the webframe work that I use does understand or use the concept of a db 9 so if the framework tries to change the object it fails. The disconnected transaction is most likely the best solution but it looked like implementing the update of slots outside of a transaction would be a easier quickfix. I had a quick try at stopping the marking of the object to dirty when a slot is set if there is no (current-transaction) but that failed the object still ends up in the dirty que. Any advise on this would be appreciated. -------------- next part -------------- An HTML attachment was scrubbed... URL: From haragx at gmail.com Wed Jun 1 08:26:41 2011 From: haragx at gmail.com (Phil Marneweck) Date: Wed, 01 Jun 2011 10:26:41 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> Message-ID: <1306916806.11416.1.camel@mohadib> Is this (taken from the comments in searialize.lisp) still true? That option 3 is used? What do we do when we serialize an object and it turns out to contain other objects? There are a few options: 1. Don't allow it: this should be dealt with at a higher level 2. Automatically add the child object to the cache: that means it will be saved and we'll get an object-id for the child. But what if the child was already in the cache? We have no way of knowing that and we'll probably create a mess. 3. Just serialize the contents. This basically assumes that this is the only reference to this objects; or, if it isn't, that it doesn't matter if we create more than one copy of this object when we deserialize it (and that object identity is irrelevant). I think I'll go for option 3. From alemmens at xs4all.nl Wed Jun 1 11:41:45 2011 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Wed, 01 Jun 2011 13:41:45 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: <1306916806.11416.1.camel@mohadib> References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> <1306916806.11416.1.camel@mohadib> Message-ID: On Wed, 01 Jun 2011 10:26:41 +0200, Phil Marneweck wrote: > Is this (taken from the comments in searialize.lisp) still true? That > option 3 is used? Yes, that's still true. (In this context, 'object' means some kind of CLOS instance that's not an instance of PERSISTENT-OBJECT or PERSISTENT-DATA.) Arthur > What do we do when we serialize an object and it turns out to contain > other objects? There are a few options: > 1. Don't allow it: this should be dealt with at a higher level > 2. Automatically add the child object to the cache: that means it > will be saved and we'll get an object-id for the child. But what if > the child was already in the cache? We have no way of knowing that > and we'll probably create a mess. > 3. Just serialize the contents. This basically assumes that this is the > only reference to this objects; or, if it isn't, that it doesn't > matter > if we create more than one copy of this object when we deserialize > it (and that object identity is irrelevant). > I think I'll go for option 3. From haragx at gmail.com Wed Jun 1 14:21:06 2011 From: haragx at gmail.com (Phil Marneweck) Date: Wed, 01 Jun 2011 16:21:06 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> Message-ID: <1306938071.3151.1.camel@mohadib> This is taken from the comments in searialize.lisp, is it still true? What do we do when we serialize an object and it turns out to contain other objects? There are a few options: 1. Don't allow it: this should be dealt with at a higher level 2. Automatically add the child object to the cache: that means it will be saved and we'll get an object-id for the child. But what if the child was already in the cache? We have no way of knowing that and we'll probably create a mess. 3. Just serialize the contents. This basically assumes that this is the only reference to this objects; or, if it isn't, that it doesn't matter if we create more than one copy of this object when we deserialize it (and that object identity is irrelevant). I think I'll go for option 3. -------------- next part -------------- An HTML attachment was scrubbed... URL: From haragx at gmail.com Wed Jun 1 14:26:01 2011 From: haragx at gmail.com (Phil Marneweck) Date: Wed, 01 Jun 2011 16:26:01 +0200 Subject: [rucksack-devel] Enhancements - Code? In-Reply-To: References: <1306747942.13921.5.camel@mohadib> <7C57C0CA-685F-4543-BD59-AB27A9F08F29@crispylogics.com> <1306764792.15399.12.camel@mohadib> Message-ID: <1306938366.3339.3.camel@mohadib> Sorry about the double posting of the previous message but I kid you not evolution crashes each time I try to send a reply to a message on this list!!!! -------- This is taken from the comments in searialize.lisp, is this still true? What do we do when we serialize an object and it turns out to contain other objects? There are a few options: 1. Don't allow it: this should be dealt with at a higher level 2. Automatically add the child object to the cache: that means it will be saved and we'll get an object-id for the child. But what if the child was already in the cache? We have no way of knowing that and we'll probably create a mess. 3. Just serialize the contents. This basically assumes that this is the only reference to this objects; or, if it isn't, that it doesn't matter if we create more than one copy of this object when we deserialize it (and that object identity is irrelevant). I think I'll go for option 3. From phil at dataxware.co.za Wed Jun 1 14:27:43 2011 From: phil at dataxware.co.za (Phil) Date: Wed, 01 Jun 2011 16:27:43 +0200 Subject: [rucksack-devel] Serialization of chinld objects? Message-ID: <1306938465.3455.0.camel@mohadib> Sorry about the double posting of the previous message but I kid you not evolution crashes each time I try to send a reply to a message on this list!!!! -------- The following is taken from the comments in searialize.lisp, is this still true What do we do when we serialize an object and it turns out to contain other objects? There are a few options: 1. Don't allow it: this should be dealt with at a higher level 2. Automatically add the child object to the cache: that means it will be saved and we'll get an object-id for the child. But what if the child was already in the cache? We have no way of knowing that and we'll probably create a mess. 3. Just serialize the contents. This basically assumes that this is the only reference to this objects; or, if it isn't, that it doesn't matter if we create more than one copy of this object when we deserialize it (and that object identity is irrelevant). I think I'll go for option 3. -- This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. From pixel at kepibu.org Wed Jun 15 10:25:29 2011 From: pixel at kepibu.org (Pixel B. Interface, III) Date: Wed, 15 Jun 2011 04:25:29 -0600 Subject: [rucksack-devel] non-simple-strings and fill-pointers Message-ID: <026901cc2b46$8dc9a7d0$3100a8c0@trinity> Greetings! I've been using rucksack for the past, uh, two days. I may have found a bug. I happened to be adding some on-disk caching of HTTP responses to my cliki-spam-reverting-bot, and hit a tiny, easily-worked-around snag. What I'm getting from Drakma is an adjustable (and hence non-simple) string /without/ a fill-pointer. Rucksack, on the other hand, assumes non-simple-strings always have fill-pointers. To wit: (defmethod serialize ((string string) stream) ... (unless simple-p (serialize (fill-pointer string) stream) (serialize (adjustable-array-p string) stream)) ...) Creating a non-simple string without a fill pointer: (setq str (make-array '(0) :element-type 'character :adjustable t :fill-pointer nil)) (fill-pointer str) => # (simple-string-p str) => nil (array-has-fill-pointer-p str) => nil (rs::serialize str stream) => # Fortunately, the non-simple-arrays-have-fill-pointers assumption is avoided when dealing with arrays in general: (defmethod serialize ((array array) stream) ... (when (= 1 (array-rank array)) ;; Arrays with rank > 1 can't have fill-pointers. (serialize (array-has-fill-pointer-p array) stream) (when (array-has-fill-pointer-p array) (serialize (fill-pointer array) stream))) ...) Unfortunately, adjusting string serialization to match array's [] format would break file-compatibility between rucksack versions. However, I believe a fairly simple change would allow for fill-pointer-less non-simple strings while maintaining the existing format: (defmethod serialize ((string string) stream) ... (unless simple-p - (serialize (fill-pointer string) stream) + (serialize (and (array-has-fill-pointer-p string) + (fill-pointer string)) stream) (serialize (adjustable-array-p string) stream)) ...) (Apologies for not sending a proper patch, but I'm mostly pontificating anyway; haven't tested it.) If I understand rucksack's serialization and deserialization correctly, that should work fine, resulting in deserializing the fill-pointer as nil, which is a perfectly acceptable value and does the right thing. Or maybe I'm nuts and wrong entirely. Either way, for now, I'm working around the issue by just coercing to a simple-string. Other than that little snag, Rucksack has been remarkably easy to work with. So, uh, thanks! -- -pix What? Manually reverting spam on cliki got old. From alemmens at xs4all.nl Mon Jun 20 17:28:10 2011 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Mon, 20 Jun 2011 19:28:10 +0200 Subject: [rucksack-devel] non-simple-strings and fill-pointers In-Reply-To: <026901cc2b46$8dc9a7d0$3100a8c0@trinity> References: <026901cc2b46$8dc9a7d0$3100a8c0@trinity> Message-ID: Pixel B. Interface, III wrote: > What I'm getting from Drakma is an adjustable (and hence non-simple) > string /without/ a fill-pointer. Rucksack, on the other hand, assumes > non-simple-strings always have fill-pointers. > [...] > I believe a fairly simple change would allow for fill-pointer-less > non-simple strings while maintaining the existing format: > (defmethod serialize ((string string) stream) > ... > (unless simple-p > - (serialize (fill-pointer string) stream) > + (serialize (and (array-has-fill-pointer-p string) > + (fill-pointer string)) stream) > (serialize (adjustable-array-p string) stream)) > ...) Thanks for the report. Your analysis and your proposed solution look OK to me. I applied your patch and added a little test to make sure we're not hallucinating. This is now in the CVS repository as Rucksack version 0.1.21. Arthur