From dsears at shout.net Tue Feb 20 22:11:36 2007 From: dsears at shout.net (Duane Searsmith) Date: Tue, 20 Feb 2007 16:11:36 -0600 Subject: [cells-devel] Cells and Threads Message-ID: <45DB7218.3030801@shout.net> Hi -- Quick question. I skimmed through the archives and didn't see any obvious answer to this question. Is cells thread safe? If not, is there a particular pattern of use that one can follow to avoid problems in a multiprocess scenario? I have an application that is multithreaded and I would like to explore how to use cells to drive the dataflow. I've read through the examples and looked over the code a bit. Seems like I could get in trouble if multiple threads tried to change a cell with dependencies at the same time. If this is a silly question and I have missed some obvious explanation somewhere, please just point me in the right direction. Thanks, -- Duane From kentilton at gmail.com Wed Feb 21 02:18:12 2007 From: kentilton at gmail.com (Ken Tilton) Date: Tue, 20 Feb 2007 21:18:12 -0500 Subject: [Fwd: Re: [cells-devel] Cells and Threads] Message-ID: <45DBABE4.4070907@gmail.com> Duane Searsmith wrote: > Hi -- > > Quick question. I skimmed through the archives and didn't see any > obvious answer to this question. Is cells thread safe? If not, is > there a particular pattern of use that one can follow to avoid > problems in a multiprocess scenario? > > I have an application that is multithreaded and I would like to > explore how to use cells to drive the dataflow. I've read through the > examples and looked over the code a bit. Seems like I could get in > trouble if multiple threads tried to change a cell with dependencies > at the same time. > > If this is a silly question and I have missed some obvious explanation > somewhere, please just point me in the right direction. Not at all, good question, I have been waiting for it. :) I am afraid I have never programmed a threaded application, so I do not know what to say. Can threads enqueue on some resource (in the case of Cells, that would be the "data pulse", just a sequential counter of state changes that drives data integrity)? That would do the trick (I think!), but then does that defeat the whole point of threads? If each thread operates on a discrete subset of the application universe, well, I /did/ start to mess with allowing such a beast to have its own pulse -- then we just have to figure out the boundary (assuming one wants at least /some/ dependency to reach thru the wormhole, if you will). Hmmm, I guess if you are worried about two threads setting the same input cell then clearly they can enqueue on some resource. I guess the next problem is if you want to have the Cells engine itself running in multiple threads, which could be tough. And in /my/ Cells apps, Cells tend to ineluctably take over the whole application, so if Cells runs in one thread only then again -- well, like I said, I have never programmed with threads. I guess I will leave the implications to you. Any help? ken From attila.lendvai at gmail.com Wed Feb 21 09:25:04 2007 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Wed, 21 Feb 2007 10:25:04 +0100 Subject: [Fwd: Re: [cells-devel] Cells and Threads] In-Reply-To: <45DBABE4.4070907@gmail.com> References: <45DBABE4.4070907@gmail.com> Message-ID: > > I have an application that is multithreaded and I would like to > > explore how to use cells to drive the dataflow. I've read through the > > examples and looked over the code a bit. Seems like I could get in > > trouble if multiple threads tried to change a cell with dependencies > > at the same time. i think a global read-write lock could work, which is acquired for read whenever a cell is checked for validity and upgraded to writing when the cell is invalid and needs recalculation. a write lock locks out all other readers and writers, while readers can operate paralel. interesting questions arise when the calculations themselves have to acquire other locks in the application... then it's not trivial to ensure the proper locking order everywhere that avoids random deadlocks. but at first i would just create a big-lock-of-the-world which is acquired whenever anything is used that uses Cells in your app. but this kills paralelism more and more as the Cells-using part of the app is bigger and bigger. hope i said something useful, -- - attila "- The truth is that I've been too considerate, and so became unintentionally cruel... - I understand. - No, you don't understand! We don't speak the same language!" (Ingmar Bergman - Smultronst?llet) From dsears at shout.net Wed Feb 21 12:38:38 2007 From: dsears at shout.net (Duane Searsmith) Date: Wed, 21 Feb 2007 06:38:38 -0600 Subject: [Fwd: Re: [cells-devel] Cells and Threads] In-Reply-To: References: <45DBABE4.4070907@gmail.com> Message-ID: <45DC3D4E.70209@shout.net> Thank you Ken and Attila for your feedback. I think I have a fair understanding of the issues now. I'm going to do some brainstorming over exactly how I would like the threads to interact with the Cells engine and then try some experiments with different locking strategies and see what happens. I'll report back on what I discover. Best, -- Duane Attila Lendvai wrote: >> > I have an application that is multithreaded and I would like to >> > explore how to use cells to drive the dataflow. I've read through the >> > examples and looked over the code a bit. Seems like I could get in >> > trouble if multiple threads tried to change a cell with dependencies >> > at the same time. > > i think a global read-write lock could work, which is acquired for > read whenever a cell is checked for validity and upgraded to writing > when the cell is invalid and needs recalculation. a write lock locks > out all other readers and writers, while readers can operate paralel. > > interesting questions arise when the calculations themselves have to > acquire other locks in the application... then it's not trivial to > ensure the proper locking order everywhere that avoids random > deadlocks. > > but at first i would just create a big-lock-of-the-world which is > acquired whenever anything is used that uses Cells in your app. but > this kills paralelism more and more as the Cells-using part of the app > is bigger and bigger. > > hope i said something useful, > > ------------------------------------------------------------------------ > > _______________________________________________ > cells-devel site list > cells-devel at common-lisp.net > http://common-lisp.net/mailman/listinfo/cells-devel From kentilton at gmail.com Wed Feb 21 16:46:48 2007 From: kentilton at gmail.com (Ken Tilton) Date: Wed, 21 Feb 2007 11:46:48 -0500 Subject: [Fwd: Re: [cells-devel] Cells and Threads] In-Reply-To: <45DC3D4E.70209@shout.net> References: <45DBABE4.4070907@gmail.com> <45DC3D4E.70209@shout.net> Message-ID: <45DC7778.308@gmail.com> Duane Searsmith wrote: > Thank you Ken and Attila for your feedback. I think I have a fair > understanding of the issues now. I'm going to do some brainstorming > over exactly how I would like the threads to interact with the Cells > engine and then try some experiments with different locking strategies > and see what happens. > I'll report back on what I discover. OK, don't be afraid to ask. And don't forget about lazy cells. :) If you use those, that creates other issues. I don't much. One thing that might help (or hurt) is that in Cells3 I started forcing the programmer to single-thread model perturbations: (with-integrity (:change ...) (setf ...) ....and there are multiple queues for follow-up work and propagation arising from any change, including a "client" queue for application work, and one can specify either or both a client queue sort function and client queue handler. So there might be some place to wire in a little thread awareness and let things keep moving despite activity by other threads. And with all that information around, there may also be a way for the programmer to indicate they are taking responsibility for data integrity so Just Let Me Do This. ie, They may know X is against the rules but X is quite safe so Just Do It. kt From peter.denno at nist.gov Wed Feb 21 17:33:30 2007 From: peter.denno at nist.gov (Peter Denno) Date: Wed, 21 Feb 2007 12:33:30 -0500 Subject: [cells-devel] cells and cells-gtk Message-ID: <200702211233.30414.peter.denno@nist.gov> Hi ken, It has been a while (2006-06-07 to be exact) since I have updated the cells code in cells-gtk. Is there something in this new version, cells3, that would make an upgrade worthwhile? Bug fixes? -- Best regards, - Peter From kentilton at gmail.com Wed Feb 21 19:18:20 2007 From: kentilton at gmail.com (Ken Tilton) Date: Wed, 21 Feb 2007 14:18:20 -0500 Subject: [cells-devel] cells and cells-gtk In-Reply-To: <200702211233.30414.peter.denno@nist.gov> References: <200702211233.30414.peter.denno@nist.gov> Message-ID: <45DC9AFC.1070104@gmail.com> [resending, Peter, to include the list] Peter Denno wrote: >Hi ken, > >It has been a while (2006-06-07 to be exact) since I have updated the cells >code in cells-gtk. Is there something in this new version, cells3, that would >make an upgrade worthwhile? Bug fixes? > > > Yes, some bug fixes, and a new concept: data integrity. Sounds important, right? :) But I wrote an astonishing amount of reliable code before getting serious about it, sooo... ah, but it was real-world apps that forced me to get serious, so.... bottom line is that I think you need to do it at some point or people will eventually hit the cases where it matters, and I would not even call those edge cases. The new scheme definitely made it easy to interface to Tk, which was needlessly fussy about the order in which certain things happen, not good for a declarative package like Cells that does things in whatever order it happens to get to them. The bad news is that Cells is not as transparent as it used to be, because now if I want to SETF some cell in an observer I have to say: (with-integrity (:change :id42) (setf 42)). The good news is the data integrity and a formal model one can describe and understand and plan for etc etc. But it might be a big overhaul, especially because you too are talking to an interesting package (GTk), so expect some excitement. If you decide to do it I will see if I can quickly reinstall Cells-Gtk and play along at home, but I am four weeks away from a big annual conference where I will be showcasing my Algebra software and I am pretty busy. cheers, ken ps. I doubt the source is diff-able so you can find /just/ the things I fixed that would also apply to your version, but mebbe. kt From BlogBlaster at common-lisp.net Sat Feb 24 23:33:55 2007 From: BlogBlaster at common-lisp.net (BlogBlaster at common-lisp.net) Date: 25 Feb 2007 01:33:55 +0200 Subject: [cells-devel] How would you like 2 Million Sites linking to your ad ? Message-ID: <20070225013353.1D44627FB49DEF75@from.header.has.no.domain> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: blogblaster.htm Type: application/octet-stream Size: 470 bytes Desc: not available URL: From Instant at common-lisp.net Tue Feb 27 10:31:40 2007 From: Instant at common-lisp.net (Instant at common-lisp.net) Date: 27 Feb 2007 12:31:40 +0200 Subject: [cells-devel] Get 1000's of Highly Targeted visitors to your web site or affiliate web site overnight Message-ID: <20070227123139.5C20AE347DE9867E@from.header.has.no.domain> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: instantbooster.htm Type: application/octet-stream Size: 518 bytes Desc: not available URL: