From elliottslaughter at gmail.com Sun Mar 7 05:45:49 2010 From: elliottslaughter at gmail.com (Elliott Slaughter) Date: Sat, 6 Mar 2010 21:45:49 -0800 Subject: [cl-store-devel] On the security of cl-store Message-ID: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> Hi, I would like to use cl-store to serialize messages sent over a network connection and am wondering if cl-store has any known security issues. I know, for example, that pickle in python can be used to execute arbitrary code when deserializing an object [1]. Does cl-store have any obvious issues like this? Thanks. [1] http://nadiana.com/python-pickle-insecure -- Elliott Slaughter "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosssd at gmail.com Sun Mar 7 11:34:36 2010 From: rosssd at gmail.com (Sean Ross) Date: Sun, 7 Mar 2010 11:34:36 +0000 Subject: [cl-store-devel] On the security of cl-store In-Reply-To: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> References: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> Message-ID: On 7 Mar 2010, at 05:45, Elliott Slaughter wrote: > Hi, > > I would like to use cl-store to serialize messages sent over a network connection and am wondering if cl-store has any known security issues. I know, for example, that pickle in python can be used to execute arbitrary code when deserializing an object [1]. Does cl-store have any obvious issues like this? Hi Elliot, All deserialization in the default backend is done via read-byte which is then fed into custom constructors so I would be quite confident in saying that cl-store isn't vulnerable to arbitrary code execution. One possible source of concern would be hash-table & function restoration; A carefully crafted data stream could create a data structure with a malicious function. As an example if you had the following function defined in your image (defun clear-all-data (a b) ... ; some code which deletes all data from your database ) then a stream could be crafted which, when loaded, would create a hash-table with this function as the hash-table-test. If your lisp allows arbitrary hash table tests then this would run clear-all-data when using the hash-table. If you are concerned about this, a work around would be to create a new backend which extends the cl-store-backend and override the hash-table restore to only allow symbols and/or only allow a certain set of tests. Additionally CLISP has support for restoration of arbitrary closures which would allow the same attack but without the requirement that the function be predefined. If you are using CLISP I would suggest extending the default backend and using the following definitions for function storing and restoring. (defstore-cl-store (obj function stream) (output-type-code +function-code+ stream) (store-object (get-function-name obj) stream)) (defrestore-cl-store (function stream) (fdefinition (restore-object stream))) To my knowledge the only other vulnerabilities would revolve around uses of eval, read or read-from-string which cl-store does not use. Regards, Sean. > > Thanks. > > [1] http://nadiana.com/python-pickle-insecure > > -- > Elliott Slaughter > > "Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay > _______________________________________________ > cl-store-devel mailing list > cl-store-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/cl-store-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.mizrahi at gmail.com Sun Mar 7 19:20:12 2010 From: alex.mizrahi at gmail.com (Alex Mizrahi) Date: Sun, 7 Mar 2010 21:20:12 +0200 Subject: [cl-store-devel] On the security of cl-store References: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> Message-ID: <91F81FA0B5E746C384A317E9C10CE7A7@killer> SR> To my knowledge the only other vulnerabilities would revolve SR> around uses of eval, read or read-from-string which cl-store If denial-of-service (e.g. crashing lisp server) qualifies as a successful attack, spectrum of issues to deal with is much larger. Perhaps simplies way to DoS the server is force it to use large quantities of memory, so it would be thrashing and eventually will collapse. As I understand, with cl-store binary format output is proportional to serialized input and can't be much large. So, to start with, one can simply reject messages which are too long. More sophisticate strategy is to make server leaking memory. For example, force it to intern new symbols in each packet, then packets will be small, but with each packet server will leak some memory. Obvious fix to this problem is to forbid interning new symbol. Finally, one can DoS server by putting it into an infinite loop. This can be achieved by sending a cyclical data structure where server code expects simple one. Then iteration or recursive traversal will either hang in indefinite loop or cause a stack overflow. From rosssd at gmail.com Sun Mar 7 23:06:03 2010 From: rosssd at gmail.com (Sean Ross) Date: Sun, 7 Mar 2010 23:06:03 +0000 Subject: [cl-store-devel] On the security of cl-store In-Reply-To: <91F81FA0B5E746C384A317E9C10CE7A7@killer> References: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> <91F81FA0B5E746C384A317E9C10CE7A7@killer> Message-ID: <40DE96E9-569C-4E3E-95A1-90241B2E3100@gmail.com> On 7 Mar 2010, at 19:20, Alex Mizrahi wrote: > SR> To my knowledge the only other vulnerabilities would revolve > SR> around uses of eval, read or read-from-string which cl-store > > If denial-of-service (e.g. crashing lisp server) qualifies as a successful > attack, spectrum of issues to deal with is much larger. Perhaps simplies way > to DoS the server is force it to use large quantities of memory, so it would > be thrashing and eventually will collapse. As I understand, with cl-store > binary format output is proportional to serialized input and can't be much > large. So, to start with, one can simply reject messages which are too long. > > More sophisticate strategy is to make server leaking memory. For example, > force it to intern new symbols in each packet, then packets will be small, > but with each packet server will leak some memory. Obvious fix to this > problem is to forbid interning new symbol. Quite right. As always, accepting input from untrusted sources is always going to open yourself to a number of potential security issues. The most cl-store will do is try not to execute arbitrary code. DOS attacks are part and parcel of accepting requests from all and sundry and is only recommended if absolutely necessary. > > Finally, one can DoS server by putting it into an infinite loop. This can be > achieved by sending a cyclical data structure where server code expects > simple one. Then iteration or recursive traversal will either hang in > indefinite loop or cause a stack overflow. This shouldn't be possible in cl-store as all cyclic data structures are detected and serialized/deserialized correctly (custom serialization/deserializations notwithstanding) Regards, Sean. From demmeln at in.tum.de Mon Mar 8 09:07:02 2010 From: demmeln at in.tum.de (Nikolaus Demmel) Date: Mon, 8 Mar 2010 10:07:02 +0100 Subject: [cl-store-devel] On the security of cl-store In-Reply-To: <40DE96E9-569C-4E3E-95A1-90241B2E3100@gmail.com> References: <42c0ab791003062145t756a565eg2f89106f41c266c2@mail.gmail.com> <91F81FA0B5E746C384A317E9C10CE7A7@killer> <40DE96E9-569C-4E3E-95A1-90241B2E3100@gmail.com> Message-ID: Am 08.03.2010 um 00:06 schrieb Sean Ross: > On 7 Mar 2010, at 19:20, Alex Mizrahi wrote: > > Quite right. As always, accepting input from untrusted sources is always going to open yourself > to a number of potential security issues. The most cl-store will do is try not to > execute arbitrary code. DOS attacks are part and parcel of accepting requests > from all and sundry and is only recommended if absolutely necessary. > >> >> Finally, one can DoS server by putting it into an infinite loop. This can be >> achieved by sending a cyclical data structure where server code expects >> simple one. Then iteration or recursive traversal will either hang in >> indefinite loop or cause a stack overflow. > > This shouldn't be possible in cl-store as all cyclic data structures are detected > and serialized/deserialized correctly (custom serialization/deserializations notwithstanding) I guess what Alex is referring to is the possibility, that an attacker sends a packet with a cyclic list/data-structure in a place where the server code expects a simple one. If the server code dosn't check for cyclic data-structures and simply traverses them, it might loop forever. Here the problem is precisely the fact that cl-store does recover cycles correctly, so a solution might be to either disable that, or make sure the server doesn't choke on cycles in any possible places. Regards, Niko -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4623 bytes Desc: not available URL: From jcornez at ravenpack.com Mon Mar 15 11:54:41 2010 From: jcornez at ravenpack.com (Jason S. Cornez) Date: Mon, 15 Mar 2010 12:54:41 +0100 Subject: [cl-store-devel] Serializing structure instances with Allegro CL Message-ID: <4B9E2001.5060405@ravenpack.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The doc and code for cl-store claim that Allegro CL structure instances cannot be stored. But enabling the functionality seems to work just fine with current versions of Allegro CL, 8.1 and 8.2. The patch is just the obvious changes and it is included below. Thanks, - -Jason $ git diff -p -w diff --git a/world/site/cl-store-darcs/acl/custom.lisp b/world/site/cl-store-darcs/acl/custom.lisp index 9f11bd2..fdb75f0 100644 - --- a/world/site/cl-store-darcs/acl/custom.lisp +++ b/world/site/cl-store-darcs/acl/custom.lisp @@ -26,4 +26,11 @@ (double-float-values) (long-float-values))))) +(defstore-cl-store (obj structure-object stream) + (output-type-code +structure-object-code+ stream) + (store-type-object obj stream)) + +(defrestore-cl-store (structure-object stream) + (restore-type-object stream)) + ;; EOF diff --git a/world/site/cl-store-darcs/allegrocl/custom.lisp b/world/site/cl-store-darcs/allegrocl/custom.lisp index daa29eb..4896a2d 100644 - --- a/world/site/cl-store-darcs/allegrocl/custom.lisp +++ b/world/site/cl-store-darcs/allegrocl/custom.lisp @@ -26,4 +26,11 @@ (double-float-values) (long-float-values))))) +(defstore-cl-store (obj structure-object stream) + (output-type-code +structure-object-code+ stream) + (store-type-object obj stream)) + +(defrestore-cl-store (structure-object stream) + (restore-type-object stream)) + ;; EOF diff --git a/world/site/cl-store-darcs/utils.lisp b/world/site/cl-store-darcs/utils.lisp index 0457f60..a2a822b 100644 - --- a/world/site/cl-store-darcs/utils.lisp +++ b/world/site/cl-store-darcs/utils.lisp @@ -20,7 +20,7 @@ and the objects class") (:method ((object standard-object)) (serializable-slots-using-class object (class-of object))) - -#+(or sbcl cmu openmcl) +#+(or sbcl cmu openmcl allegro) (:method ((object structure-object)) (serializable-slots-using-class object (class-of object))) (:method ((object condition)) @@ -35,7 +35,7 @@ The default calls compute slots with class") (:method ((object t) (class standard-class)) (class-slots class)) - -#+(or sbcl cmu openmcl) +#+(or sbcl cmu openmcl allegro) (:method ((object t) (class structure-class)) (class-slots class)) #+sbcl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkueIAEACgkQQlm6HDTMLyOKsQCg/vK+yD6cp4Bk3EkzmeExfpDr lbYAn3d6ykCsvTI8BrUBdyRq6AsaDw8G =2ogX -----END PGP SIGNATURE----- From kishor_saitwal at yahoo.com Mon Mar 15 17:15:46 2010 From: kishor_saitwal at yahoo.com (kishor saitwal) Date: Mon, 15 Mar 2010 10:15:46 -0700 (PDT) Subject: [cl-store-devel] 10 fell through ECASE expression. Wanted one of ((1) (0)). Message-ID: <528801.10703.qm@web53106.mail.re2.yahoo.com> Hi there: Sometimes (about 20% of the times) restore fails with the following error message: 10 fell through ECASE expression. Wanted one of ((1) (0)). Upon initial investigation, it seems that it is occurring while restoring the lists. I will start to check for the root cause, but I thought I would ask the question here to see if anybody has come across such problem and/or has easy fix for it. Thanks, Kishor -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosssd at gmail.com Tue Mar 23 09:36:33 2010 From: rosssd at gmail.com (Sean Ross) Date: Tue, 23 Mar 2010 09:36:33 +0000 Subject: [cl-store-devel] Serializing structure instances with Allegro CL In-Reply-To: <4B9E2001.5060405@ravenpack.com> References: <4B9E2001.5060405@ravenpack.com> Message-ID: <8D12FC4D-0448-448D-822D-EE010FE2EBC0@gmail.com> Hi Jason, This looks good, can you please provide me with a darcs patch which I can apply. Thanks, Sean. On 15 Mar 2010, at 11:54, Jason S. Cornez wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > The doc and code for cl-store claim that Allegro CL structure instances > cannot be stored. But enabling the functionality seems to work just > fine with current versions of Allegro CL, 8.1 and 8.2. The patch is > just the obvious changes and it is included below. > > Thanks, > - -Jason > > $ git diff -p -w > diff --git a/world/site/cl-store-darcs/acl/custom.lisp > b/world/site/cl-store-darcs/acl/custom.lisp > index 9f11bd2..fdb75f0 100644 > - --- a/world/site/cl-store-darcs/acl/custom.lisp > +++ b/world/site/cl-store-darcs/acl/custom.lisp > @@ -26,4 +26,11 @@ > (double-float-values) > (long-float-values))))) > > +(defstore-cl-store (obj structure-object stream) > + (output-type-code +structure-object-code+ stream) > + (store-type-object obj stream)) > + > +(defrestore-cl-store (structure-object stream) > + (restore-type-object stream)) > + > ;; EOF > diff --git a/world/site/cl-store-darcs/allegrocl/custom.lisp > b/world/site/cl-store-darcs/allegrocl/custom.lisp > index daa29eb..4896a2d 100644 > - --- a/world/site/cl-store-darcs/allegrocl/custom.lisp > +++ b/world/site/cl-store-darcs/allegrocl/custom.lisp > @@ -26,4 +26,11 @@ > (double-float-values) > (long-float-values))))) > > +(defstore-cl-store (obj structure-object stream) > + (output-type-code +structure-object-code+ stream) > + (store-type-object obj stream)) > + > +(defrestore-cl-store (structure-object stream) > + (restore-type-object stream)) > + > ;; EOF > diff --git a/world/site/cl-store-darcs/utils.lisp > b/world/site/cl-store-darcs/utils.lisp > index 0457f60..a2a822b 100644 > - --- a/world/site/cl-store-darcs/utils.lisp > +++ b/world/site/cl-store-darcs/utils.lisp > @@ -20,7 +20,7 @@ > and the objects class") > (:method ((object standard-object)) > (serializable-slots-using-class object (class-of object))) > - -#+(or sbcl cmu openmcl) > +#+(or sbcl cmu openmcl allegro) > (:method ((object structure-object)) > (serializable-slots-using-class object (class-of object))) > (:method ((object condition)) > @@ -35,7 +35,7 @@ > The default calls compute slots with class") > (:method ((object t) (class standard-class)) > (class-slots class)) > - -#+(or sbcl cmu openmcl) > +#+(or sbcl cmu openmcl allegro) > (:method ((object t) (class structure-class)) > (class-slots class)) > #+sbcl > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkueIAEACgkQQlm6HDTMLyOKsQCg/vK+yD6cp4Bk3EkzmeExfpDr > lbYAn3d6ykCsvTI8BrUBdyRq6AsaDw8G > =2ogX > -----END PGP SIGNATURE----- > > _______________________________________________ > cl-store-devel mailing list > cl-store-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/cl-store-devel From rosssd at gmail.com Tue Mar 23 10:11:22 2010 From: rosssd at gmail.com (Sean Ross) Date: Tue, 23 Mar 2010 10:11:22 +0000 Subject: [cl-store-devel] Serializing structure instances with Allegro CL In-Reply-To: <4BA892EC.2000607@ravenpack.com> References: <4B9E2001.5060405@ravenpack.com> <8D12FC4D-0448-448D-822D-EE010FE2EBC0@gmail.com> <4BA88FD5.5010706@ravenpack.com> <10683E73-24A0-4810-AD58-BA6936A2DDA4@gmail.com> <4BA892EC.2000607@ravenpack.com> Message-ID: <33D2777A-F8D9-4A60-9DAB-C1AB1F3571FB@gmail.com> Hi Jason, Thanks for the patch, I've applied it to darcs repository and, provided I am able to find the time, will look to doing a new release sometime in the next week. Regards, Sean. On 23 Mar 2010, at 10:07, Jason S. Cornez wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Sean, > > OK, I've attached the allegro.patch I produced via > $ darcs record > $ darcs send -o allegro.patch > > Hope it works. > - -Jason > > On 03/23/2010 11:01 AM, Sean Ross wrote: >> Hi Jason, >> >> If you run commit your patch to your local darcs repository you can `darcs send -o allegro.patch` and then email the file to me and I will >> apply it to the repository. >> >> >> Thanks, >> Sean. >> >> On 23 Mar 2010, at 09:54, Jason S. Cornez wrote: >> >> Hi Sean, >> >> Apologies: I'm not familiar with darcs. If you can give me the commands >> I'll be happy to try it out. Or of course, as you can see, the patch >> itself is really minor - to the point of trivial. It should take just a >> few minutes to apply the changes by hand. >> >> I've tried darcs diff - maybe that's all you need? >> >> If not, let me know. >> -Jason >> >> $ darcs diff >> diff -rN old-cl-store-darcs/acl/custom.lisp >> new-cl-store-darcs/acl/custom.lisp >> 28a29,35 >>>>> (defstore-cl-store (obj structure-object stream) >>>>> (output-type-code +structure-object-code+ stream) >>>>> (store-type-object obj stream)) >>>>> >>>>> (defrestore-cl-store (structure-object stream) >>>>> (restore-type-object stream)) >>>>> >> diff -rN old-cl-store-darcs/allegrocl/custom.lisp >> new-cl-store-darcs/allegrocl/custom.lisp >> 28a29,35 >>>>> (defstore-cl-store (obj structure-object stream) >>>>> (output-type-code +structure-object-code+ stream) >>>>> (store-type-object obj stream)) >>>>> >>>>> (defrestore-cl-store (structure-object stream) >>>>> (restore-type-object stream)) >>>>> >> diff -rN old-cl-store-darcs/utils.lisp new-cl-store-darcs/utils.lisp >> 23c23 >> < #+(or sbcl cmu openmcl) >> --- >>>>> #+(or sbcl cmu openmcl allegro) >> 38c38 >> < #+(or sbcl cmu openmcl) >> --- >>>>> #+(or sbcl cmu openmcl allegro) > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkuokusACgkQQlm6HDTMLyOddwCdGfFLUd4sqgd6qoaRLm5i/VAN > 7oUAoKREbaAh4MnsrXWpFp5JJMG4w3Vw > =DOCR > -----END PGP SIGNATURE----- >