From james.ashley at gmail.com Tue Dec 7 01:44:59 2010 From: james.ashley at gmail.com (James Ashley) Date: Mon, 6 Dec 2010 19:44:59 -0600 Subject: [rucksack-devel] Stability Message-ID: Howdy! I blogged yesterday about data persistence in common lisp (http://www.dotnetmafia.com/blogs/jamesashley/archive/2010/12/05/4241.aspx if anyone cares). One of the members of #lispcafe (schmrkc) convinced me to take another look at rucksack. It seems to be *exactly* what I'm looking for. Before I update the blog entry and recommend it (to all 5 people who actually read the blog), I figured I'd check here about its status/maturity. I've run across way too many CL libraries where the maintainers recommended I ignore it and roll my own. Is this something I can "safely" (for some value of "safe") recommend to end-users? The documentation seems pretty self-deprecating, which is why I gave it a pass the first time I noticed it. BTW, here's a patch I used to get it working on clozure. Not sure that it's "correct," but it looked pretty straightforward: diff --unified rucksack.original.lisp rucksack.lisp --- rucksack.original.lisp 2010-12-06 19:24:19.000000000 -0600 +++ rucksack.lisp 2010-12-06 19:26:28.000000000 -0600 @@ -258,7 +258,9 @@ (mp:process-lock lock) #+sbcl (sb-thread:get-mutex lock) - #-(or sbcl lispworks) + #+ccl-1.4 + (ccl:grab-lock lock) + #-(or sbcl lispworks ccl-1.4) (not-implemented 'process-lock)) @@ -267,7 +269,9 @@ (mp:process-unlock lock) #+sbcl (sb-thread:release-mutex lock) - #-(or sbcl lispworks) + #+ccl-1.4 + (ccl:release-lock lock) + #-(or sbcl lispworks ccl-1.4) (not-implemented 'process-unlock)) (Yeah, I know. I've been bitten numerous times by changes that "looked" simple). Thank you, James From henrik at evahjelte.com Tue Dec 7 11:32:54 2010 From: henrik at evahjelte.com (Henrik Hjelte) Date: Tue, 7 Dec 2010 12:32:54 +0100 Subject: [rucksack-devel] Stability In-Reply-To: References: Message-ID: On Tue, Dec 7, 2010 at 2:44 AM, James Ashley wrote: > Howdy! > > I blogged yesterday about data persistence in common lisp > (http://www.dotnetmafia.com/blogs/jamesashley/archive/2010/12/05/4241.aspx > if anyone cares). > One of the members of #lispcafe (schmrkc) convinced me to take another > look at rucksack. It seems to be *exactly* what I'm looking for. > > Before I update the blog entry and recommend it (to all 5 people who > actually read the blog), I figured I'd check here about its > status/maturity. I've run across way too many CL libraries where the > maintainers recommended I ignore it and roll my own. > > Is this something I can "safely" (for some value of "safe") recommend > to end-users? The documentation seems pretty self-deprecating, which > is why I gave it a pass the first time I noticed it. > > It was some time ago since I experimented with rucksack. The code was very nice, it has a testsuite and I fixed the only error that was detected by the stress-test. So I think it is safe in some sort of way: no failing tests in a decent test-suite. But: I don't think many use it. And why I am not using it is that it is only for one process at a time. Are you building an application where only one process will reach the database? For a web-application we built and use elephant with the postgresql backend. It is scalable, built for multiple processes (even on different servers) and multiple threads and has caching so performance is ok. It has nothing to do with SQL, just uses postgres for storage to get advantage of the scalability. I read from your blog it is not installable with quicklisp, but for some applications the easiness of install is not the most important issue to consider. Good luck, Henrik Hjelte -------------- next part -------------- An HTML attachment was scrubbed... URL: From js at crispylogics.com Tue Dec 7 11:49:31 2010 From: js at crispylogics.com (Jochen Schmidt) Date: Tue, 7 Dec 2010 12:49:31 +0100 Subject: [rucksack-devel] Stability In-Reply-To: References: Message-ID: <0DCEF434-FC0B-412D-908E-84793CECF076@crispylogics.com> Am 07.12.2010 um 12:32 schrieb Henrik Hjelte: > > It was some time ago since I experimented with rucksack. The code was very nice, it has a testsuite and I fixed the only error that was detected by the stress-test. So I think it is safe in some sort of way: no failing tests in a decent test-suite. But: I don't think many use it. And why I am not using it is that it is only for one process at a time. Are you building an application where only one process will reach the database? Do you have the fix for the error shown by the stress test? Or is it actually already in current rucksack? I've a modified snapshot of rucksack here which makes parallel reader transactions possible. The current official rucksack isn't particularily well suited for web applications, because it serializes all transactions. I've implemented a multi-reader single-writer transaction for rucksack and deployed several simple web apps using it (1-n rucksack accesses per request). In about a year running those apps I didn't have any lost data or other problems. Perhaps we should throw our code together and put it up somewhere (Github?) for others to use. > For a web-application we built and use elephant with the postgresql backend. It is scalable, built for multiple processes (even on different servers) and multiple threads and has caching so performance is ok. It has nothing to do with SQL, just uses postgres for storage to get advantage of the scalability. I read from your blog it is not installable with quicklisp, but for some applications the easiness of install is not the most important issue to consider. It's a pity that elephant still seems to depend heavily on external libraries or systems. Last time I looked elephant still used FFI for serializing lisp-data. Rucksack is cool because it is actually nearly 100% just CL - its a really simple drop in persistence solution where others are always a PITA to install and maintain. I'm currently quite deep in other works, but wanted to investigate the merits of "NOSQL" ideas in a CL only (or at least "mostly CL") implementation. -- Jochen Schmidt CRISPYLOGICS Cross Media Solutions Uhlandstr. 9, 90408 N?rnberg Telefon +49 (0)911 517 999 82 Telefax +49 (0)911 517 999 83 mailto:info at crispylogics.com http://www.crispylogics.com From henrik at evahjelte.com Tue Dec 7 12:37:12 2010 From: henrik at evahjelte.com (Henrik Hjelte) Date: Tue, 7 Dec 2010 13:37:12 +0100 Subject: [rucksack-devel] Stability In-Reply-To: <0DCEF434-FC0B-412D-908E-84793CECF076@crispylogics.com> References: <0DCEF434-FC0B-412D-908E-84793CECF076@crispylogics.com> Message-ID: On Tue, Dec 7, 2010 at 12:49 PM, Jochen Schmidt wrote: > Am 07.12.2010 um 12:32 schrieb Henrik Hjelte: > Do you have the fix for the error shown by the stress test? Or is it > actually already in current rucksack? > I checked, it is in rucksack since 3 years. How fast life goes by... I've a modified snapshot of rucksack here which makes parallel reader > transactions possible. The current official rucksack isn't particularily > well suited for web applications, because it serializes all transactions. > I've implemented a multi-reader single-writer transaction for rucksack and > deployed several simple web apps using it (1-n rucksack accesses per > request). In about a year running those apps I didn't have any lost data or > other problems. > > Perhaps we should throw our code together and put it up somewhere (Github?) > for others to use. > Or rather your code, it sounds very interesting, I suggest talking with Arthur Lemmens about it. It's a pity that elephant still seems to depend heavily on external > libraries or systems. Last time I looked elephant still used FFI for > serializing lisp-data. Rucksack is cool because it is actually nearly 100% > just CL - its a really simple drop in persistence solution where others are > always a PITA to install and maintain. > I agree it is cool with a pure CL solution, and rucksack is nice. But for me actual usefulness often beats cool. There are several people dabbling with elephant, and patches and improvements are welcome. There is some work going on in making it more pure-CL. I would really like if there was a project where everyone could contribute to do something that was both more pure CL and scalable across servers/fast/nice. I am sort a lot of people besides me have been dreaming of various solutions. /Henrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From alemmens at xs4all.nl Mon Dec 13 09:38:50 2010 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Mon, 13 Dec 2010 10:38:50 +0100 Subject: [rucksack-devel] Stability In-Reply-To: <0DCEF434-FC0B-412D-908E-84793CECF076@crispylogics.com> References: <0DCEF434-FC0B-412D-908E-84793CECF076@crispylogics.com> Message-ID: Jochen Schmidt wrote: > Perhaps we should throw our code together and put it up > somewhere (Github?) for others to use. If you can send me your diffs for the multi-reader transactions, I'll try to integrate them with the current CVS repository. But if you prefer to set up some other repository for Rucksack and take over the maintenance, that's fine with me too. Or you can get write access to the CVS repository, if you want. Arthur From alemmens at xs4all.nl Mon Dec 13 10:24:23 2010 From: alemmens at xs4all.nl (Arthur Lemmens) Date: Mon, 13 Dec 2010 11:24:23 +0100 Subject: [rucksack-devel] Stability In-Reply-To: References: Message-ID: James Ashley wrote: > Is this something I can "safely" (for some value of "safe") recommend > to end-users? Difficult question. I'm not aware of any bugs, but I still haven't used Rucksack for real-world applications myself. Apart from Jochen there don't seem to be many real-world users, so it's not impossible that there are still some undetected bugs lurking around. Then again, this is a chicken and egg situation. If people don't use it because nobody else is using it, this is not very likely to change. > BTW, here's a patch I used to get it working on clozure. Thanks, I'll look into this. Arthur From bobbie at ua.fm Fri Dec 17 16:12:30 2010 From: bobbie at ua.fm (Victor) Date: Fri, 17 Dec 2010 18:12:30 +0200 Subject: [rucksack-devel] Stability Message-ID: On Tue Dec 7 06:49:31 EST 2010 Jochen Schmidt wrote: > Am 07.12.2010 um 12:32 schrieb Henrik Hjelte: > > It was some time ago since I experimented with rucksack. The code > > was very nice, it has a testsuite and I fixed the only error that > > was detected by the stress-test. So I think it is safe in some > > sort of way: no failing tests in a decent test-suite. But: I don't > > think many use it. And why I am not using it is that it is only > > for one process at a time. Are you building an application where > > only one process will reach the database? > Do you have the fix for the error shown by the stress test? Or is it > actually already in current rucksack? I've a modified snapshot of > rucksack here which makes parallel reader transactions possible. The > current official rucksack isn't particularily well suited for web > applications, because it serializes all transactions. I've > implemented a multi-reader single-writer transaction for rucksack > and deployed several simple web apps using it (1-n rucksack accesses > per request). In about a year running those apps I didn't have any > lost data or other problems. > Perhaps we should throw our code together and put it up somewhere > (Github?) for others to use. I would be interested in this feature too. And I think that making it possible to have single writer multiple readers would benefit others as well. The application I use rucksack for has one long-running writer transaction and multiple very short reader transactions. When the writers starts running the application is really stalled. It would be great if you share your sources. Thanks, Victor P.S. I am new on this list and found these messages in the archives