From hraban at 0brg.net Sat Oct 1 01:37:58 2011 From: hraban at 0brg.net (Hraban Luyat) Date: Sat, 1 Oct 2011 03:37:58 +0200 Subject: [cl-store-devel] numeric code In-Reply-To: References: Message-ID: Or use both: whenever output-type-code receives a byte, use the classic efficient model. Whenever it receives something else, encode that code itself with store-object and return it verbatim on restore. It is up to the user to make sure that such objects are compatible with generic method dispatch through eql (e.g. symbols and larger numbers). This alleviates the 0-255 constraint (larger numbers now also possible), keeps the old behaviour for backwards compatibility and efficiency reasons and alleviates the need to do manual bookkeeping of codes. Only tested briefly against clisp, but the patch is pretty simple. Comments welcome, let me know what you think. Greetings, Hraban 2011/10/1 Nikolaus Demmel : > > Am 30.09.2011 um 23:32 schrieb Hraban Luyat: >> Is there a reason for the necessity of codes being numeric? I thought >> it would be nice to use symbols. Specifically, the class name of an >> object. That way each class has an obvious unique code and I do not >> need to manually keep track of which ones are in use and which ones >> are not. Then I can forget about register-code entirely. > > Without knowing too much detail about the implementation, I guess it is mainly an implementation reason. The code is directly used for serializing. IIRC you can only use 256 different values anyway, at least with the default backend. > > I like your idea though. I guess one would have to maintain a table of codes and corresponding symbols and use codes still for actual serialization, otherwise there could be a very significant (space) overhead if you dump big data. > >> I would like to comment further on cl-store in this otherwise quite >> abrupt e-mail, but I have not really used it yet. So far the API looks >> cool, though. Thanks for maintaining and writing cl-store. > > > As Sean is looking for a new owner for cl-store I guess you will probably have to implement this yourself. > > Best, > Nikolaus -------------- next part -------------- A non-text attachment was scrubbed... Name: default-package.diff Type: application/octet-stream Size: 1539 bytes Desc: not available URL: From hraban at 0brg.net Sat Oct 1 09:17:38 2011 From: hraban at 0brg.net (Hraban Luyat) Date: Sat, 1 Oct 2011 11:17:38 +0200 Subject: [cl-store-devel] numeric code In-Reply-To: References: Message-ID: It turns out that my patch breaks when referred objects come into the picture because (store-object) is called recursively from within (output-type-code). This patch adds one line to the latter function, decreasing *stored-counter* before entering recursion. Hopefully this makes the behaviour added by this patch transparent. Comments and suggestions welcome. Greetings, Hraban 2011/10/1 Hraban Luyat : > Or use both: whenever output-type-code receives a byte, use the > classic efficient model. Whenever it receives something else, encode > that code itself with store-object and return it verbatim on restore. > It is up to the user to make sure that such objects are compatible > with generic method dispatch through eql (e.g. symbols and larger > numbers). > > This alleviates the 0-255 constraint (larger numbers now also > possible), keeps the old behaviour for backwards compatibility and > efficiency reasons and alleviates the need to do manual bookkeeping of > codes. > > Only tested briefly against clisp, but the patch is pretty simple. > > Comments welcome, let me know what you think. > > Greetings, > > Hraban > > 2011/10/1 Nikolaus Demmel : >> >> Am 30.09.2011 um 23:32 schrieb Hraban Luyat: >>> Is there a reason for the necessity of codes being numeric? I thought >>> it would be nice to use symbols. Specifically, the class name of an >>> object. That way each class has an obvious unique code and I do not >>> need to manually keep track of which ones are in use and which ones >>> are not. Then I can forget about register-code entirely. >> >> Without knowing too much detail about the implementation, I guess it is mainly an implementation reason. The code is directly used for serializing. IIRC you can only use 256 different values anyway, at least with the default backend. >> >> I like your idea though. I guess one would have to maintain a table of codes and corresponding symbols and use codes still for actual serialization, otherwise there could be a very significant (space) overhead if you dump big data. >> >>> I would like to comment further on cl-store in this otherwise quite >>> abrupt e-mail, but I have not really used it yet. So far the API looks >>> cool, though. Thanks for maintaining and writing cl-store. >> >> >> As Sean is looking for a new owner for cl-store I guess you will probably have to implement this yourself. >> >> Best, >> Nikolaus > From hraban at 0brg.net Sat Oct 1 09:18:18 2011 From: hraban at 0brg.net (Hraban Luyat) Date: Sat, 1 Oct 2011 11:18:18 +0200 Subject: [cl-store-devel] numeric code In-Reply-To: References: Message-ID: .. and here is the new patch. 2011/10/1 Hraban Luyat : > It turns out that my patch breaks when referred objects come into the > picture because (store-object) is called recursively from within > (output-type-code). This patch adds one line to the latter function, > decreasing *stored-counter* before entering recursion. Hopefully this > makes the behaviour added by this patch transparent. > > Comments and suggestions welcome. > > Greetings, > > Hraban > > 2011/10/1 Hraban Luyat : >> Or use both: whenever output-type-code receives a byte, use the >> classic efficient model. Whenever it receives something else, encode >> that code itself with store-object and return it verbatim on restore. >> It is up to the user to make sure that such objects are compatible >> with generic method dispatch through eql (e.g. symbols and larger >> numbers). >> >> This alleviates the 0-255 constraint (larger numbers now also >> possible), keeps the old behaviour for backwards compatibility and >> efficiency reasons and alleviates the need to do manual bookkeeping of >> codes. >> >> Only tested briefly against clisp, but the patch is pretty simple. >> >> Comments welcome, let me know what you think. >> >> Greetings, >> >> Hraban >> >> 2011/10/1 Nikolaus Demmel : >>> >>> Am 30.09.2011 um 23:32 schrieb Hraban Luyat: >>>> Is there a reason for the necessity of codes being numeric? I thought >>>> it would be nice to use symbols. Specifically, the class name of an >>>> object. That way each class has an obvious unique code and I do not >>>> need to manually keep track of which ones are in use and which ones >>>> are not. Then I can forget about register-code entirely. >>> >>> Without knowing too much detail about the implementation, I guess it is mainly an implementation reason. The code is directly used for serializing. IIRC you can only use 256 different values anyway, at least with the default backend. >>> >>> I like your idea though. I guess one would have to maintain a table of codes and corresponding symbols and use codes still for actual serialization, otherwise there could be a very significant (space) overhead if you dump big data. >>> >>>> I would like to comment further on cl-store in this otherwise quite >>>> abrupt e-mail, but I have not really used it yet. So far the API looks >>>> cool, though. Thanks for maintaining and writing cl-store. >>> >>> >>> As Sean is looking for a new owner for cl-store I guess you will probably have to implement this yourself. >>> >>> Best, >>> Nikolaus >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: default-package-2.diff Type: application/octet-stream Size: 1567 bytes Desc: not available URL: From hraban at 0brg.net Sat Oct 1 21:48:30 2011 From: hraban at 0brg.net (Hraban Luyat) Date: Sat, 1 Oct 2011 23:48:30 +0200 Subject: [cl-store-devel] numeric code In-Reply-To: References: Message-ID: Well, I kept on struggling with this issue, and I think I finally have something reasonable for my purposes. It is still not perfect: circular references in classes, for example, are not properly serialised. However, I do not use those. At this point, I am satisfied enough with the patch to continue what I was working on originally, so you may consider this my last patch until some new bug comes up. This patch includes a long comment that explains the problem arising from recursive storage. I put it in there because at least it was better than nothing, but the wording could benefit from some editing. Of course, this is all (intended to be) backwards compatible; if you only use codes between 0 and 255 this patch should not affect you. See e.g. tests.lisp, whose results are unaffected. Greetings, Hraban Luyat 2011/10/1 Hraban Luyat : > .. and here is the new patch. > > 2011/10/1 Hraban Luyat : >> It turns out that my patch breaks when referred objects come into the >> picture because (store-object) is called recursively from within >> (output-type-code). This patch adds one line to the latter function, >> decreasing *stored-counter* before entering recursion. Hopefully this >> makes the behaviour added by this patch transparent. >> >> Comments and suggestions welcome. >> >> Greetings, >> >> Hraban >> >> 2011/10/1 Hraban Luyat : >>> Or use both: whenever output-type-code receives a byte, use the >>> classic efficient model. Whenever it receives something else, encode >>> that code itself with store-object and return it verbatim on restore. >>> It is up to the user to make sure that such objects are compatible >>> with generic method dispatch through eql (e.g. symbols and larger >>> numbers). >>> >>> This alleviates the 0-255 constraint (larger numbers now also >>> possible), keeps the old behaviour for backwards compatibility and >>> efficiency reasons and alleviates the need to do manual bookkeeping of >>> codes. >>> >>> Only tested briefly against clisp, but the patch is pretty simple. >>> >>> Comments welcome, let me know what you think. >>> >>> Greetings, >>> >>> Hraban >>> >>> 2011/10/1 Nikolaus Demmel : >>>> >>>> Am 30.09.2011 um 23:32 schrieb Hraban Luyat: >>>>> Is there a reason for the necessity of codes being numeric? I thought >>>>> it would be nice to use symbols. Specifically, the class name of an >>>>> object. That way each class has an obvious unique code and I do not >>>>> need to manually keep track of which ones are in use and which ones >>>>> are not. Then I can forget about register-code entirely. >>>> >>>> Without knowing too much detail about the implementation, I guess it is mainly an implementation reason. The code is directly used for serializing. IIRC you can only use 256 different values anyway, at least with the default backend. >>>> >>>> I like your idea though. I guess one would have to maintain a table of codes and corresponding symbols and use codes still for actual serialization, otherwise there could be a very significant (space) overhead if you dump big data. >>>> >>>>> I would like to comment further on cl-store in this otherwise quite >>>>> abrupt e-mail, but I have not really used it yet. So far the API looks >>>>> cool, though. Thanks for maintaining and writing cl-store. >>>> >>>> >>>> As Sean is looking for a new owner for cl-store I guess you will probably have to implement this yourself. >>>> >>>> Best, >>>> Nikolaus >>> >> > -------------- next part -------------- A non-text attachment was scrubbed... Name: default-package-3.diff Type: application/octet-stream Size: 2260 bytes Desc: not available URL: