From ckimyt at gmail.com Sat Mar 5 08:22:56 2005 From: ckimyt at gmail.com (Mike J. Bell) Date: Sat, 5 Mar 2005 03:22:56 -0500 Subject: [cells-devel] Newbie questions Message-ID: I suppose some of this might be in a FAQ or something, but I'm somewhat lazy and I've got some questions burning. I also apologize for A) not knowing CL very well B) not knowing very much about CLOS at all (except what it stands for). Please pardon any strange or misused terminology. That said, I've got some questions about how Cells works. 1) Synchronicity, or the currency conversion problem: Let's say I define a slot in an object that represents the amount of US Dollars I have in a bank account. Let's further say that in my app I also want to represent the amount of Japanese Yen in my account. If someone "sets" the amount of dollars, I want the yen slot to be updated. If someone sets the yen, I want the dollars to be updated. Is there a mechanism to create such co-dependent cells? (Obviously this is addressing the update cycle problem, and at the shortest path level: A depends on B depends on A (length 1), which forms a very useful group of calculations). 2) Equals: It looks like, from reading 01-Cell-basics.lisp, that setting a slot to an "equivalent" object does not cause all its dependencies to fire again (in order to save substantial useless recalculations). What does "equivalent" mean? Can I install my own equivalency operator, per slot (or per class would be fine) so that Cells knows when computations should be rerun? 3) Lack of values, or triggers: It's easy to come up with typed slots that participate in calculations. However, what about untyped events? For instance, a press of a button conveys no useful state information. Is it possible in Cells to make slots dependent upon the "change" of a stateless slot? (This sort of ties back into the equals business...I understand you could implement this sort of functionality by setting a slot to, for example, a gensym uninterned symbol, so that there is no useful value and it's sure to cause everything to recalculate by any useful definition of equals, but this solution is...well, kind of ugly. It would be nice to be able to define the recalculation or "firing" semantics of a slot, which is a superset of the "equals" issue above). 4) Transactions: Let's say you have 50 or 60 slots that are connected in a complicated graph of calculation. A change to one slot (let's call this "initiation") causes all of these guys to start running until finally the system reaches a steady state and no more changes occur. We could conceptualize this initiation, subsequent calculations and then collapse into steady state as a single "transaction." Now, imagine that at the second-to-last calculation of this propagation of values, the system enters a state of inconsistency. In other words, the initiation of that one state change way back in the beginning is actually invalid. However, we don't know that (given that this is a nice event-driven framework) until step 52, and we've got 40 slots that are all now different than they were, and *wrong*. Ideally, it would be nice to have mechanism that A) notifies the programmer, similar to a "catch" construct, so that s/he can note it in an appropriate fashion, and B) restores all the slots in the system to the last known good steady state; i.e., what they were before the initiation occured. Does Cells have a transaction model similar to this? If so, can someone explain it? Are there separate phases of execution in the transaction? Are there separate "guards" that are responsible for determining the validity of system state? Is there a way to specify that slots get updated only once the system state becomes steady; i.e. they don't participate in the middle of the transaction? (These beasts have quite a few constraining properties, I realize, but they could (are) be very useful to reduce the amount of "spinning" the recalculation has to do). Thanks in advance for all your comments. Mike -- Mike J. Bell on gmail From ckimyt at gmail.com Sat Mar 5 15:18:24 2005 From: ckimyt at gmail.com (Mike J. Bell) Date: Sat, 5 Mar 2005 10:18:24 -0500 Subject: [cells-devel] More newbie questions Message-ID: I wasn't thinking totally clearly at 3am, so around 4:30 I thought of some things that I didn't ask. =) (You may notice that I'm coming from an event-driven paradigm...I'm trying to map my understanding of how mine works onto Cells and "make sure" Cells is a superset of what I'm using). 5) Multi-method (?) calculations: I've seen simple examples in the Cells docs that show a slot being dependent on another slot. It looks like you install this computation in the "destination" slot, i.e. the one that's dependent. What if you've got a situation where you have five slots, A B C D and E, that are parts of say three different objects, O1 O2 and O3. There's one computation that needs A B C D and E, and ends up modifying slots F G H and I in objects O4 and O5. Can it be done, making a single computation that depends on many slots in different instances that then updates slots in many instances? Or do you need to decompose this computation into (complicated!) pieces to be installed in each of the destination slots (F G H and I)? Can a change in one or more of A B C D and E cause a computation that updates C D F and G? (I.e. C and D are both inputs *and* outputs of the calculation. Note that self-cycling must be avoided!). 6) Multiple "owners" of responsibility, or aspect separation: (this is related to #5 above) What if you have a normal calculation that happens 99% of the time that changes slot A into slot B (B is dependent on A). You write the formula into the slot code for B, and you're happy. But then, because of some strange requirement or other complicated design issue, you realize that B also depends, every once in a while (say 1%), on C D and E. But you don't want to stick this new computation and dependency code with the original code, because it will not only clutter reading, but also the complicated process may change in the future. So you want to keep the concerns separate, i.e. the two different "times" when B can change. Is there a way in Cells to make two different pieces of code that show two different ways that slot B is dependent on other slots? 7) "Lazy" updates: Probably a bad term for this, not to be confused with strict and non-strict argument evaluation. Nonetheless, let's say you have a slot A that depends on B C D and E. B and C are absolutely critical to A's computation, and every single time they change, you need to enforce that A is recomputed. My guess is that Cells works like this already, automatically. However, let's say D and E change their values inordinately fast, like maybe they're mouse X and Y coordinates on the screen. Let's also say that the computation for A is non-trivial, and you want to run it only when absolutely necessary. Further, although A needs the current mouse X and Y coordinates, it doesn't make sense to recompute A every time they change. Is there a way to tell Cells that A should be recomputed *every* time that B and C change, but also needs access to cells D and E while not "firing" every time they change? Sorry if this sounds stupidly simple, but remember, I'm projecting my domain onto yours, so I'm pretty new at this. 8) Phase-based calculation, or data "wave" propagation: If a slot A depends on slots B C and D...when will A get recomputed? Every time that B C or D change, right? What if two other computations change both C and D as a result of a single slot change in E...does Cells defer the running of A until *both* C and D have been recalculated (i.e. enforced breadth-first dependency graph traversal), or will the calculation of A happen twice, once when C changes and once when D changes (i.e. random, depth-first, or other traversal)? 9) Introspection: How easy is it to determine the full dependency graph of interrelated computation given a system coded using Cells? I.e., other than of course looking at the code, can I use a programmatic method to determine the topology of the dependencies in order to, for instance, produce a visual graph of the system? Is this introspection system static, or dynamic (i.e. allow callback hooks or the like to update changes in the system as they happen)? Thanks again for any comments. Mike -- Mike J. Bell on gmail From ktilton at nyc.rr.com Sat Mar 5 15:24:17 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sat, 05 Mar 2005 10:24:17 -0500 Subject: [cells-devel] Newbie questions In-Reply-To: References: Message-ID: <4229CF21.4070008@nyc.rr.com> Mike J. Bell wrote: >I suppose some of this might be in a FAQ or something, but I'm >somewhat lazy and I've got some questions burning. > >I also apologize for A) not knowing CL very well B) not knowing very >much about CLOS at all (except what it stands for). Please pardon any >strange or misused terminology. > >That said, I've got some questions about how Cells works. > >1) Synchronicity, or the currency conversion problem: Let's say I >define a slot in an object that represents the amount of US Dollars I >have in a bank account. Let's further say that in my app I also want >to represent the amount of Japanese Yen in my account. If someone >"sets" the amount of dollars, I want the yen slot to be updated. If >someone sets the yen, I want the dollars to be updated. Is there a >mechanism to create such co-dependent cells? > I did a foreign exchange system once, so I would say always update the dollars. But to answer your question, the way I handle things like scroll bars (the thumb can be used to move the text, but scrolling thru the text with the arrow keys moves the thumb) is to have DEF-C-OUTPUT methods on both the text offset and (really) the scrollbar ratio, with each one setf-ing the other, meaning both get initialized to (c-input 0). I used to have a more elaborate scheme in Cells which handled this sort of thing by swapping in a different pair of rules when the user moused-down on the thumb, but I decided dual outputs, while losing the declarative thing, were better because it let me eliminate the whole dynamic rule-swapping section of Cells. > (Obviously this is >addressing the update cycle problem, and at the shortest path level: >A depends on B depends on A (length 1), which forms a very useful >group of calculations). > Cells have always grown in response to application requirements, so if someone comes up with a good example I will gladly look at implementing cycles in Cells. The question is, how can a = f(b) and b=f(a) ever be computed? In your example, there is no real cycle, because you are saying: "dollars is a function of yen /when changes/, and yen is a function of dollar /when dollar changes/." Only one can be changing at a time, so there is no cycle as the problem is conceived. But how is an instance initialized? I can supply conversion rules for dollar-balance and yen-balance, but how do I get the intial balance in there? I would need to specify a starting value, but for which one? Both? What if I specify 1 for both? That conversion would be wrong, the dollar is (for now) worth more than 1 yen. Conceivably the answer is to let the programmer worry about it. I could specify a value (and the conversion rule) for dollar-balance and specify "unbound" and a rule for all other currencies. Then if the others get accessed, the system just calculates as usual. If all values are unbound a runtime error results. If the programmer specifies inconsistent initial values they get a wrong result (which we might be able to detect at some point of runtime) until some SETF perturbs the model and the formulas kick in to make things consistent. Thoughts on this? > >2) Equals: It looks like, from reading 01-Cell-basics.lisp, that >setting a slot to an "equivalent" object does not cause all its >dependencies to fire again (in order to save substantial useless >recalculations). What does "equivalent" mean? Can I install my own >equivalency operator, per slot (or per class would be fine) so that >Cells knows when computations should be rerun? > Yes. There is an :unchanged-if option (default EQL (supplied by Cells internals)) on slot definitions in DEFMODEL: Example from Cello: (defmodel image (ogl-node model) ((clipped :cell nil :initarg :clipped :initform nil :reader clipped) (inset :initarg :inset :unchanged-if 'v2= :initform (mkv2 0 0) :reader inset) ....etc.... [Actually, I just noticed that I also had ":cell nil" on that! "inset" was not a Cell for a lonnnggg time, and I think when I finally wanted to make it a Cell I was smart enough to specify unchanged-if, but dumb enough to leave behind :cell nil. Looks like I never tested... perhaps before I got done I decided against playing with inset? Anyway, that is neither there nor there.] (defstruct v2 (h 0 ) (v 0 ) ) (defun v2= (a b) (and a b (= (v2-h a)(v2-h b)) (= (v2-v a)(v2-v b)))) > >3) Lack of values, or triggers: It's easy to come up with typed slots >that participate in calculations. However, what about untyped events? > For instance, a press of a button conveys no useful state >information. Is it possible in Cells to make slots dependent upon the >"change" of a stateless slot? (This sort of ties back into the equals >business...I understand you could implement this sort of functionality >by setting a slot to, for example, a gensym uninterned symbol, so that >there is no useful value and it's sure to cause everything to >recalculate by any useful definition of equals, but this solution >is...well, kind of ugly. It would be nice to be able to define the >recalculation or "firing" semantics of a slot, which is a superset of >the "equals" issue above). > I am afraid I do not follow you. My GUIs all manage buttons and other standard GUI components via Cells. A button has a "click event" slot (because clicking is surprisingly complicated) which Cello mouse-handling SETFs (so it is initialized to (c-input nil). Anyway, I am not answering your question... could you explain again what you mean by a stateless slot? By the way, in case this helps, as a steady-state, declarative approach, Cells do make handling events tricky. I came up with the :ephemeral option to the :cell option of a slot definition: (defmodel window .... (mouse-up-evt :cell :ephemeral :initarg :mouse-up-evt :initform (c-in nil) :accessor mouse-up-evt) ..... The way it works is this: when I SETF such a slot, the engine takes care of all propagation and then silently (not via any accessor) resets the slot value to nil. Not sure, but I think the internals yell at you if you try to initialize to anything other than nil, such as (c-in 42). Close? > >4) Transactions: Let's say you have 50 or 60 slots that are connected >in a complicated graph of calculation. A change to one slot (let's >call this "initiation") causes all of these guys to start running >until finally the system reaches a steady state and no more changes >occur. We could conceptualize this initiation, subsequent >calculations and then collapse into steady state as a single >"transaction." > >Now, imagine that at the second-to-last calculation of this >propagation of values, the system enters a state of inconsistency. In >other words, the initiation of that one state change way back in the >beginning is actually invalid. However, we don't know that (given >that this is a nice event-driven framework) until step 52, and we've >got 40 slots that are all now different than they were, and *wrong*. >Ideally, it would be nice to have mechanism that A) notifies the >programmer, similar to a "catch" construct, so that s/he can note it >in an appropriate fashion, and B) restores all the slots in the system >to the last known good steady state; i.e., what they were before the >initiation occured. > >Does Cells have a transaction model similar to this? If so, can >someone explain it? Are there separate phases of execution in the >transaction? Are there separate "guards" that are responsible for >determining the validity of system state? Is there a way to specify >that slots get updated only once the system state becomes steady; i.e. >they don't participate in the middle of the transaction? (These >beasts have quite a few constraining properties, I realize, but they >could (are) be very useful to reduce the amount of "spinning" the >recalculation has to do). > No, there is no such rollback mechanism in Cells, but it is not a bad idea. If you check out the function finish-business, you might be able to make out the flow: - first, recalculate all "users" (direct dependents) of the original changed cell, recursively propagating to users of users as necessary. - second, invoke any output methods defined on the changed cells - third, invoke SETF's of cells deferred during outputs (that is indeed legal) note that the processing of each SETF is handled by executing those same steps. Anyway, rollback would take a little work since there is no dynamic scope encompassing all the state changes -- then we could just restore the slot value in unwind-protect -- but it would be feasible. As for other questions, the latest revision of Cells (I call it Cells II) guarantees the following (I really should write this up!): - during propagation, each cell calculates only once - no slot accessor returns a "stale" value, by which I mean a value which has yet to be re-calculated after some dependency direct or indirect has changed. kt -- Cells? Cello? Cells-Gtk?: http://www.common-lisp.net/project/cells/ Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "Doctor, I wrestled with reality for forty years, and I am happy to state that I finally won out over it." -- Elwood P. Dowd From ktilton at nyc.rr.com Sat Mar 5 17:14:19 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sat, 05 Mar 2005 12:14:19 -0500 Subject: [cells-gtk-devel] Re: [cells-devel] Newbie questions In-Reply-To: <200503051122.43249.peter.denno@nist.gov> References: <4229CF21.4070008@nyc.rr.com> <200503051122.43249.peter.denno@nist.gov> Message-ID: <4229E8EB.8060300@nyc.rr.com> Peter Denno wrote: >On Saturday 05 March 2005 10:24, Kenny Tilton wrote: > > >>Yes. There is an :unchanged-if option (default EQL (supplied by Cells >>internals)) on slot definitions in DEFMODEL: >> >>Example from Cello: >>(defmodel image (ogl-node model) >> ((clipped :cell nil :initarg :clipped :initform nil :reader clipped) >> (inset :initarg :inset >> :unchanged-if 'v2= >> >> > >I would have thought that :unchanged-if would take a function of one value. >Are you saying here that it still uses EQL, but instead of checking against >old-value #'(lambda (x) (eql old-value new-value)) it is doing #'(lambda (x) >(eql new-value 'v2=)) ? > > > A function of one value? No, the question is whether the prior value calculated by a rule is the same as the latest value calculated because some dependency changed. The prototype would be: (defun custom-change-test (latest-value prior-value).... Here is the relevant code, massively edited/simplified here to make the flow apparent: (defmethod md-slot-value-assume (c raw-value) (let ((prior-state (c-value-state c)) (prior-value (c-value c)) (absorbed-value (c-absorb-value c raw-value))) (unless (and (eql prior-state :valid) (c-no-news c absorbed-value prior-value)) (setf (c-changed c) t) (c-propagate c prior-value (not (eql :unbound prior-state))))) absorbed-value))) ;----------------- change detection --------------------------------- (defun c-no-news (c new-value old-value) (bif (test (c-unchanged-test (c-model c) (c-slot-name c))) (funcall test new-value old-value) (eql new-value old-value))) kt -- Cells? Cello? Cells-Gtk?: http://www.common-lisp.net/project/cells/ Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "Doctor, I wrestled with reality for forty years, and I am happy to state that I finally won out over it." -- Elwood P. Dowd From ktilton at nyc.rr.com Sat Mar 5 17:45:35 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sat, 05 Mar 2005 12:45:35 -0500 Subject: [cells-devel] More newbie questions In-Reply-To: References: Message-ID: <4229F03F.2050100@nyc.rr.com> Mike J. Bell wrote: >I wasn't thinking totally clearly at 3am, so around 4:30 I thought of >some things that I didn't ask. =) > >(You may notice that I'm coming from an event-driven paradigm... > No problem. I am usually developing GUI apps--all events all the time. >I'm >trying to map my understanding of how mine works onto Cells and "make >sure" Cells is a superset of what I'm using). > >5) Multi-method (?) calculations: I've seen simple examples in the >Cells docs that show a slot being dependent on another slot. It looks >like you install this computation in the "destination" slot, i.e. the >one that's dependent. What if you've got a situation where you have >five slots, A B C D and E, that are parts of say three different >objects, O1 O2 and O3. There's one computation that needs A B C D and >E, and ends up modifying slots F G H and I in objects O4 and O5. Can >it be done, making a single computation that depends on many slots in >different instances that then updates slots in many instances? Or do >you need to decompose this computation into (complicated!) pieces to >be installed in each of the destination slots (F G H and I)? > You say complicated, I say "divide and conquer". Have you ever developed a finite state machine, breaking some complex problem down into a kazillion states? It seems painfully slow at first, until you realize how effortlessly the approach deals with the overall computation, which had you so confused that you resorted to using a finite state machine. :) Cells is all about exposing the semantics of a slot in the one formula which determines the value of a slot. What you are describing is "classic" imperative programming, where this point in the code decides FGHI, another decides GH, another decides GHZ, etc etc. At which point no one can really tell us the semantics of H. Now if you promise me that there is one and only one computation which decides FGHI, what I do in cases like that is simply create a new slot called FGHI (I wager the real name will be quite meaningful to anyone reading your code) and then F, G, H, and I can have extremely simple rules: :f (c? (get-f (^fghi))) :g (c? (get-g (^fghi))) ..etc... > Can a >change in one or more of A B C D and E cause a computation that >updates C D F and G? (I.e. C and D are both inputs *and* outputs of >the calculation. Note that self-cycling must be avoided!). > Again, sounds like you are resisting "divide and conquer". I have been doing cells for about nine years now, and I have never had a problem decomposing big computations into so many slots mediated by so many cells. The cyclicity thing rarely materializes, but see my discussion of scrollbars in an earlier reply for how I handle that cyclic case. > >6) Multiple "owners" of responsibility, or aspect separation: (this >is related to #5 above) What if you have a normal calculation that >happens 99% of the time that changes slot A into slot B (B is >dependent on A). You write the formula into the slot code for B, and >you're happy. But then, because of some strange requirement or other >complicated design issue, you realize that B also depends, every once >in a while (say 1%), on C D and E. But you don't want to stick this >new computation and dependency code with the original code, because it >will not only clutter reading, but also the complicated process may >change in the future. > You say "clutter", I say "completely specified". What is wrong with: (c? (if (one-chance-in-a-million self) (big-hairy-mess (^c) (^d) (e))) (nice-and-easy (^a)))) And who cares if it changes in the future? Just maintain the formula, which will always fully document the semantics of a slot. One of the nice things about not using GOTOs is just /knowing/ without worrying that any given point in the code is always reached via visible control flow. > So you want to keep the concerns separate, i.e. >the two different "times" when B can change. Is there a way in Cells >to make two different pieces of code that show two different ways that >slot B is dependent on other slots? > Not sure what you mean. Are you aware that different instances of the same class can have different rules for the same slot. This might help with your question above. Perhaps the rare situation is known at make-instance time? In which case you can indeed supply a different rule which goes after CDE. > >7) "Lazy" updates: Probably a bad term for this, not to be confused >with strict and non-strict argument evaluation. Nonetheless, let's >say you have a slot A that depends on B C D and E. B and C are >absolutely critical to A's computation, and every single time they >change, you need to enforce that A is recomputed. My guess is that >Cells works like this already, automatically. However, let's say D >and E change their values inordinately fast, like maybe they're mouse >X and Y coordinates on the screen. Let's also say that the >computation for A is non-trivial, and you want to run it only when >absolutely necessary. Further, although A needs the current mouse X >and Y coordinates, it doesn't make sense to recompute A every time >they change. Is there a way to tell Cells that A should be recomputed >*every* time that B and C change, but also needs access to cells D and >E while not "firing" every time they change? > Yes. See "synapses". I believe Bill's blog on Cells covers that, as well as some of my awful doc. > >Sorry if this sounds stupidly simple, but remember, I'm projecting my >domain onto yours, so I'm pretty new at this. > >8) Phase-based calculation, or data "wave" propagation: If a slot A >depends on slots B C and D...when will A get recomputed? Every time >that B C or D change, right? What if two other computations change >both C and D as a result of a single slot change in E...does Cells >defer the running of A until *both* C and D have been recalculated >(i.e. enforced breadth-first dependency graph traversal), or will the >calculation of A happen twice, once when C changes and once when D >changes (i.e. random, depth-first, or other traversal)? > Some recent work I playfully christened "Cells II" addressed this concern. Now A will be calculated only once. Each wave gets an ID (just an incremented count) and, if A gets calculated too early, no problem. The engine sees that C and/or D are obsolete (we are calculating wave N, but C and/or D are stamped N-1) and recomputes them Just In Time before completeing the calculation of A. > >9) Introspection: How easy is it to determine the full dependency >graph of interrelated computation given a system coded using Cells? >I.e., other than of course looking at the code, can I use a >programmatic method to determine the topology of the dependencies in >order to, for instance, produce a visual graph of the system? Is this >introspection system static, or dynamic (i.e. allow callback hooks or >the like to update changes in the system as they happen)? > A visual graph would be easy to do and probably great fun to watch. I imagine little sparks shooting around, like an animation of the brain. I have long been tempted to do something like that. I guess the thing that holds me back is the problem of how to lay out the visual elements so the whole thing looks good. I always hate the way graphical class browsers end up looking. kt From tfb at OCF.Berkeley.EDU Sat Mar 5 18:06:58 2005 From: tfb at OCF.Berkeley.EDU (Thomas F. Burdick) Date: Sat, 5 Mar 2005 10:06:58 -0800 Subject: [cells-devel] Release engineering, and relationship with LTK In-Reply-To: <4220F1AB.60506@nyc.rr.com> References: <16928.58024.13267.800370@conquest.OCF.Berkeley.EDU> <4220F1AB.60506@nyc.rr.com> Message-ID: <16937.62786.835115.100533@conquest.OCF.Berkeley.EDU> Kenny Tilton writes: > > > Thomas F. Burdick wrote: > > >It's been a pretty long time since the last Cells release, and in > >particular, what asdf-install gets you is the old Cells-I codebase. > >So, Kenny, do you have any problems with labeling the current cvs as > >Cells 2.0? > > "labelling the current CVS"? whassat? Do you mean making c2 explicit, > and keeping c1 around? Bah, I was so pleased with the silly phrasing of my mail that I didn't bother to make sure it was comprehensible. I meant, if I take the cvs version of cell-cultures/cells and package it up as cells-2.0, do you have any objections? I wasn't expecting any, buy maybe you had some fix for something that was half-finished. > Raised above (if I understood you). But I am all in favor of fixing > asdf-install to point to cells2. That's exactly what I was attemptiong to communicate :-) > I am pretty sure the two were incompatible. I certainly got no value > from LTk's higher-level mechanism for exchanging info with Tk, since the > output (ne echo) mechanism handled that automatically (given suitable > macrology). Only some low-level stuff from LTk survives in Celtic. Then > I dashed ahead and did a ton more widgets than LTk had at the time, or I > should say with all Tk attributes available to the Celtic user, while > Peter was pulling them in one by one. > > It sounds as if you want to work out a marriage of Cells with LTk, which > is really a different project. Go for it. I would consider erasing > Celtic, but it /was/ Vasilis's inspiration for Cells-gtk, so I think it > should be spared the glue factory. Maybe call your project Cells-Ltk, or > just add some code to LTk which will happen to require Cells and > Utils-kt? Could that be done with a separate ASDF module underneath LTk? What I'm trying to do is get LTK to depend on Cells. I hadn't thought of using echos for the communication mechanism -- sounds slick. Maybe I should have just looked at the code. Talking to Peter, it sounds like the LTK mechanism has improved since Celtic's genesis, so that makes sense. > fwiw, I certainly plan no further work on Celtic (which was never > commercial, just a way to pump Cells). (Huh, I thought you were considering using it for ed software). Okay, I'll go ahead with my plans to marry Cells and Ltk, then raid Celtic for good widgets with no qualms or worries. > ps. hey, guess what? looks like we will be attempting some form of > literate programming to handle vicious FDA requirements for system > documentation in a sensible way. kt Ha ha, that's great! FWIW, I liked using noweb with static languages, and with Cells I think it'd be pretty easy to reimplement a subset of it dynamically in Lisp, so you can C-M-x forms like: (defun some-function (x y z) <> (values x (foo y z))) And somewhere else: ... text explaining to the FDA why this works correctly ... <>= ... @ Actually, I once got partway through such a beast for Hemlock using KR, before realizing I had no real need for it that justified finishing it. If I were you, I'd do something like that, so you could make a for-the-FDA-auditor chapter. And if you use noweb syntax, you only need to build the tangle step, you can just reuse the existing noweave. From peter.denno at nist.gov Sat Mar 5 20:25:02 2005 From: peter.denno at nist.gov (Peter Denno) Date: Sat, 5 Mar 2005 15:25:02 -0500 Subject: [cells-devel] fm-descendant-named Message-ID: <200503051525.02802.peter.denno@nist.gov> Hi, I was wondering why fm-descendant-named isn't exported. Is there something else I should be using for that purpose? -- Best Regards, - Peter From ktilton at nyc.rr.com Sat Mar 5 21:34:19 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sat, 05 Mar 2005 16:34:19 -0500 Subject: [cells-devel] fm-descendant-named In-Reply-To: <200503051525.02802.peter.denno@nist.gov> References: <200503051525.02802.peter.denno@nist.gov> Message-ID: <422A25DB.7080606@nyc.rr.com> Peter Denno wrote: >Hi, > >I was wondering why fm-descendant-named isn't exported. > I forget why I even created that. Perhaps I meant to have an exported macro which expanded into that? I do not use that function anywhere. > Is there something >else I should be using for that purpose? > well, fm-other /by default/ searches down the tree to descendants first. I think I had a problem once where there was a widget of that name higher in the tree and it was not the one I wanted, so I created an api entry for an fm-find call with :global-search nil. then maybe I found a better way and avoided the whole problem and never used the entry point. I could export that if you like. Or just use fm-other and count on the widget being there to get the same effect, if that is the case. kt From ktilton at nyc.rr.com Sun Mar 6 17:07:00 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 06 Mar 2005 12:07:00 -0500 Subject: [cells-devel] Newbie questions In-Reply-To: References: <4229CF21.4070008@nyc.rr.com> Message-ID: <422B38B4.2010002@nyc.rr.com> Mike J. Bell wrote: >On Sat, 05 Mar 2005 10:24:17 -0500, Kenny Tilton wrote: > > >>Mike J. Bell wrote: >> >> >>> (Obviously this is >>>addressing the update cycle problem, and at the shortest path level: >>>A depends on B depends on A (length 1), which forms a very useful >>>group of calculations). >>> >>> >>> >>Cells have always grown in response to application requirements, so if >>someone comes up with a good example I will gladly look at implementing >>cycles in Cells. The question is, how can a = f(b) and b=f(a) ever be >>computed? In your example, there is no real cycle, because you are >>saying: "dollars is a function of yen /when changes/, and yen is a >>function of dollar /when dollar changes/." Only one can be changing at a >>time, so there is no cycle as the problem is conceived. >> >> > >If you were to model calculations and slots *separately*, i.e. have >data objects to hold the values you're worried about, and function >objects that respond to changes in these data objects by changing >other data objects, then you can get into update cycles. > >For instance, given data objects A and B, and function objects F1 and >F2, if F1 responds to changes in A and then sets B, and F2 responds to >changes in B and then sets A, you could cycle infinitely without >reaching steady state. A system designed with F1 and F2 is designed >incorrectly. In the paradigm I'm currently using, there's a special >type of function that maintains changes between A and B so as to >eliminate this infinite cycle. It's actually quite simple, and you >elude to the solution above: this special function is really just a >tuple of the original F1 and F2, with the added semantic that the >calculation code knows that if A changed, run function F1; if B >changed, run function F2; if both changed, signal an error. > Yes. It is a cycle only if the system is ignorant of the full spec. By the way, with Cells it is possible to declare a formula as "cyclic". But this is a bit of a hack. The way it works for your scenario is this: Define an output method on slot A which sets slot B. Define an output method on slot B which sets slot A. During propagation, keep track of who is being set during any one data "pulse". If I discover, say, that A is already being set when we get another attempt to set A: If A has been defined to be cyclic, simply abandon the subsequent set operation else produce a runtime error The bad news is that we lose the declarative thing, because the rule for each calculation is now in an output method (which is furthermore only slot-specific, where Cell rules can be instance-specific). Hmmm, maybe I should go back to the old scheme. Well, I always wait for an issue to get truly troublesome before fixing it, just to have as clear an idea as possible of the problem I am fixing. > >(This is starting to expose some of the differences between what I'm >using and Cells...in my system, there are distinct phases of updates >apparently like in your Cells II). > > > >>But how is an instance initialized? I can supply conversion rules for >>dollar-balance and yen-balance, but how do I get the intial balance in >>there? I would need to specify a starting value, but for which one? >>Both? What if I specify 1 for both? That conversion would be wrong, the >>dollar is (for now) worth more than 1 yen. >> >>Conceivably the answer is to let the programmer worry about it. I could >>specify a value (and the conversion rule) for dollar-balance and specify >>"unbound" and a rule for all other currencies. Then if the others get >>accessed, the system just calculates as usual. If all values are unbound >>a runtime error results. If the programmer specifies inconsistent >>initial values they get a wrong result (which we might be able to detect >>at some point of runtime) until some SETF perturbs the model and the >>formulas kick in to make things consistent. >> >>Thoughts on this? >> >> > >In the system I currently use, the programmer worries about it, and is >only allowed to set one initial value. The system recognizes that the >other slot has no value, that it *could* have a value due to this >two-way conversion, and runs the appropriate function to find what it >is. If the programmer sets an initial value for both slots, even if >they are consistent with the function, the system signals an error. > Cool. > > > >>>3) Lack of values, or triggers: It's easy to come up with typed slots >>>that participate in calculations. However, what about untyped events? >>>For instance, a press of a button conveys no useful state >>>information. Is it possible in Cells to make slots dependent upon the >>>"change" of a stateless slot? (This sort of ties back into the equals >>>business...I understand you could implement this sort of functionality >>>by setting a slot to, for example, a gensym uninterned symbol, so that >>>there is no useful value and it's sure to cause everything to >>>recalculate by any useful definition of equals, but this solution >>>is...well, kind of ugly. It would be nice to be able to define the >>>recalculation or "firing" semantics of a slot, which is a superset of >>>the "equals" issue above). >>> >>> >>> >>I am afraid I do not follow you. My GUIs all manage buttons and other >>standard GUI components via Cells. A button has a "click event" slot >>(because clicking is surprisingly complicated) which Cello >>mouse-handling SETFs (so it is initialized to (c-input nil). Anyway, I >>am not answering your question... could you explain again what you mean >>by a stateless slot? >> >>By the way, in case this helps, as a steady-state, declarative approach, >>Cells do make handling events tricky. I came up with the :ephemeral >>option to the :cell option of a slot definition: >> >>(defmodel window .... >> (mouse-up-evt :cell :ephemeral :initarg :mouse-up-evt :initform (c-in >>nil) :accessor mouse-up-evt) >> ..... >> >>The way it works is this: when I SETF such a slot, the engine takes care >>of all propagation and then silently (not via any accessor) resets the >>slot value to nil. >> >>Not sure, but I think the internals yell at you if you try to initialize >>to anything other than nil, such as (c-in 42). >> >>Close? >> >> > >I think I understand this...the reason you set the ephemeral slot back >to nil is so that when its value changes again in the future, any >slots dependent on it will definitely fire. > Not exactly. I have delta slots for that. They are understood to express a change, so each time I set a slot to any value other than a user-definable identity value, propagation takes place. Besides, events are distinct Lisp objects (not EQL) so a new event may be EQUAL if you will, but it will be a new Lisp object and cause the rule to fire. The reason I have ephemerals is, simply, that events are ephemeral! Consider this partial rule: (c? (let ((evt (^mouse-down (window self)))) (when (and evt (rect-contains (^my-rect self) (mouse-pos evt))).... Now this sequence occurs: a mouse-down in the window, in or out of my-rect, no matter. a mouse-up occurs my-rect changes in response to any other event besides mouse-down Well, the rule fires, because my-rect changed. The mouse button is no longer down, but we still have an old event sitting in the mouse-down slot. We could have a nice WITH-EPHEMERAL-SETF (place value) macro, but that would end with (setf place nil), which would kick off propagation (hmm, not the end of the world, and it /would/ clear dependencies. hmmm...) or we could end with (setf (slot-value ) nil), but then that is what ephemrals do. Gosh, maybe :ephemeral was a mistake? If I do propagate via (setf ,place nil), then the rule fires and the dependency on MY-RECT gets cleared, as it should. And then I can eliminate the whole complexification of the ephemeral option. The downside is that ephemerality is a property of the slot (anything known to hold an OS event, for example), so if I eliminate that I then have to worry about using WITH-EPHEMRAL-SETF consistently. Maybe the right thing is for Cells internals to handle ephemerality by propagating the reset instead of doing it silently. >In the system I use, since the data is separate from the functions, >it's not *really* object-oriented. And we end up with quite a few of >these data that represent "something happening." In our system, the >identity of the data (i.e. which slot in which object) conveys all the >information that is necessary to know when a button is pressed. >Notice this is at an application level, not the GUI level. I'm sure >that in the GUI internals, there's a lot of complicated information >that goes into a button press...but at the system level, when you want >to know, "hey, this button was pressed!"...that's essentially >stateless (excepting, like I said, identity). > >There are also frequent other times this comes into effect, and it all >falls into a command-pattern type of deal. "I have collected data X, >Y and Z...now DO this." The DO is essentially stateless. > >I would imagine since we have implicit transactions and automated >rollback capabilities (and all that implies...meaning phased execution >cycles to propogate the "waves" of data modifications) that this >"staging" of values to perform a calculation is 100% the fault of the >imperative event model. If Cells is really completely declarative >(like Haskell), then it's going to take me a while to wrap my brain >into that model to see how they compare. > > > >>As for other questions, the latest revision of Cells (I call it Cells >>II) guarantees the following (I really should write this up!): >> >>- during propagation, each cell calculates only once >>- no slot accessor returns a "stale" value, by which I mean a value >>which has yet to be re-calculated after some dependency direct or >>indirect has changed. >> >> > >Interesting. That's definitely different from what I'm using. Often >times data updates hundreds of times in each transaction before >settling on a new value. > I got away with that for years (with many different kinds of applications) before it broke down (while doing a RoboCup soccer simulator of all things). kt From ktilton at nyc.rr.com Sun Mar 6 20:56:00 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 06 Mar 2005 15:56:00 -0500 Subject: [Fwd: Re: [cells-devel] More newbie questions] Message-ID: <422B6E60.7080406@nyc.rr.com> -------- Original Message -------- Subject: Re: [cells-devel] More newbie questions Date: Sun, 06 Mar 2005 15:55:29 -0500 From: Kenny Tilton To: Mike J. Bell References: <4229F03F.2050100 at nyc.rr.com> Mike J. Bell wrote: >I'm quoting a bunch of stuff, sorry so long, but I think it's all >related...comments below. > >On Sat, 05 Mar 2005 12:45:35 -0500, Kenny Tilton wrote: > > >>Mike J. Bell wrote: >> >> >>>5) Multi-method (?) calculations: I've seen simple examples in the >>>Cells docs that show a slot being dependent on another slot. It looks >>>like you install this computation in the "destination" slot, i.e. the >>>one that's dependent. What if you've got a situation where you have >>>five slots, A B C D and E, that are parts of say three different >>>objects, O1 O2 and O3. There's one computation that needs A B C D and >>>E, and ends up modifying slots F G H and I in objects O4 and O5. Can >>>it be done, making a single computation that depends on many slots in >>>different instances that then updates slots in many instances? Or do >>>you need to decompose this computation into (complicated!) pieces to >>>be installed in each of the destination slots (F G H and I)? >>> >>> >>> >>You say complicated, I say "divide and conquer". Have you ever developed >>a finite state machine, breaking some complex problem down into a >>kazillion states? It seems painfully slow at first, until you realize >>how effortlessly the approach deals with the overall computation, which >>had you so confused that you resorted to using a finite state machine. :) >> >>Cells is all about exposing the semantics of a slot in the one formula >>which determines the value of a slot. What you are describing is >>"classic" imperative programming, where this point in the code decides >>FGHI, another decides GH, another decides GHZ, etc etc. At which point >>no one can really tell us the semantics of H. >> >>Now if you promise me that there is one and only one computation which >>decides FGHI, what I do in cases like that is simply create a new slot >>called FGHI (I wager the real name will be quite meaningful to anyone >>reading your code) and then F, G, H, and I can have extremely simple rules: >> >> :f (c? (get-f (^fghi))) >> :g (c? (get-g (^fghi))) >> ..etc... >> >> >> >>> Can a >>>change in one or more of A B C D and E cause a computation that >>>updates C D F and G? (I.e. C and D are both inputs *and* outputs of >>>the calculation. Note that self-cycling must be avoided!). >>> >>> >>> >>Again, sounds like you are resisting "divide and conquer". I have >>been doing cells for about nine years now, and I have never had a >>problem decomposing big computations into so many slots mediated by so >>many cells. The cyclicity thing rarely materializes, but see my >>discussion of scrollbars in an earlier reply for how I handle that >>cyclic case. >> >> >> >>>6) Multiple "owners" of responsibility, or aspect separation: (this >>>is related to #5 above) What if you have a normal calculation that >>>happens 99% of the time that changes slot A into slot B (B is >>>dependent on A). You write the formula into the slot code for B, and >>>you're happy. But then, because of some strange requirement or other >>>complicated design issue, you realize that B also depends, every once >>>in a while (say 1%), on C D and E. But you don't want to stick this >>>new computation and dependency code with the original code, because it >>>will not only clutter reading, but also the complicated process may >>>change in the future. >>> >>> >>> >>You say "clutter", I say "completely specified". What is wrong with: >> >> (c? (if (one-chance-in-a-million self) >> (big-hairy-mess (^c) (^d) (e))) >> (nice-and-easy (^a)))) >> >>And who cares if it changes in the future? Just maintain the formula, >>which will always fully document the semantics of a slot. One of the >>nice things about not using GOTOs is just /knowing/ without worrying >>that any given point in the code is always reached via visible control flow. >> >> > >I think these comments strike into the heart of the differences >between Cells and what I'm using. It seems to me they're both >event-driven paradigms; Cells is declarative, and mine is imperative. > >I guess my biggest concern about using Cells at this point (and I >think I'm going to have to just try it and see!) is scalability, from >a design perspective. > >In the system I'm using right now, because the data and functions that >operate on it are separate (and many to many), you get a really nice >modularization of major aspects of an application. For instance, I >wrote a program that loads and displays audio waveform data visually. >The original code allowed integer multiples of zooming. The zoom >factor was represented by one of these pieces of data. The drawing >code listened for changes in this data in order to repaint the screen >at the appropriate zoom level. The GUI widget (slider) posted changes >to this zoom factor. This exemplifies the extreme separation of >model, view and controller (as I'm sure Cells does as well). > >Now, the really nice thing about the system I'm using is that when a >new requirement came along about zoom levels, I didn't have to touch a >single bit of old code. It went like this: a ruler was added to the >screen to show a timecode display with the waveform. Because of >strange "evenness" or regularity constraints on the design, the zoom >level couldn't just be any old integer...it had to satisfy a >divisibility property involving the sample rate of the audio. This >contraint was written as a new function that took the sample rate and >the zoom level as inputs, and published (possibly) a new zoom level >that was "correct." The old code remained exactly the same, and the >new code was added to the application, and everything just works. > >In the declarative model that I think Cells uses, I would have had to >change code that's already written, tested, debugged, etc. in order to >effect the change in requirements. > Not sure what code you are referring to. I would not have the slider set the zoom, I would give the zoom a rule which watched the slider value, which would just go from zero to one. When the new requirement came along I would simply have the zoom rule look at the slider /and/ the sample rate. Seems easy enough to me, and more self-documenting than the approach you described. If the zoom value is coming out wrong, I like knowing without doubt exactly where to look. Hell, in the inspector I can even look at the code, which for debugging purposes I store in symbolic form in the Cell for those cases where a slot gets different rules for different instances so often that it is painful to work out which rule applies to some instance I am debugging. I am not saying I do not find constraint programming interesting or that it would not be fun to simply drop a new rule in the basket as a way of modifying a program. On the other hand, I have heard nothing but nightmares about what it is like to program with multi-way and partial constraints, so I have not been tempted to add anything like that to Cells (so far ). > Granted this is a simple example, >picked for its ease of describing, but I've had this pattern occur >quite a bit coding in the system I'm using...i.e. 95% of my >application stands as is, with most changes just introducing new code >which operates on existing defined data in the application. > >Please don't take any of my comments as negative criticism. > Not at all. We are both working the same very productive field, constraints of one kind or another, and we see the same benefits. kt -- Cells? Cello? Cells-Gtk?: http://www.common-lisp.net/project/cells/ Why Lisp? http://lisp.tech.coop/RtL%20Highlight%20Film "Doctor, I wrestled with reality for forty years, and I am happy to state that I finally won out over it." -- Elwood P. Dowd From ktilton at nyc.rr.com Sun Mar 6 23:57:29 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 06 Mar 2005 18:57:29 -0500 Subject: [cells-devel] Newbie questions In-Reply-To: <16939.38005.590392.903414@gargle.gargle.HOWL> References: <4229CF21.4070008@nyc.rr.com> <422B38B4.2010002@nyc.rr.com> <16939.38005.590392.903414@gargle.gargle.HOWL> Message-ID: <422B98E9.8080906@nyc.rr.com> rpgoldman at real-time.com wrote: >I've just been following up on the cells-devel discussion and wanted >to ask a quick question. Were you involved in the Garnet project, or >was cells just inspired by it? > Neither. :) I was ignorant of all the substantial prior art. I just stumbled on the idea of doing something like this. And now as I contemplate all the many other hacks that work along the same lines, I have to conclude that the idea is just one of those inevitables that folks will fall into if they code long enough. In fact, once I had it working pretty well, I just knew that someone else must have done it before. That is when I started googling and found all the prior art. > I notice you discuss the possibility >of making stuff with KR. I assume that this is the same KR that was >part of Garnet, right? > Yes. kt From ckimyt at gmail.com Mon Mar 7 04:14:52 2005 From: ckimyt at gmail.com (Mike J. Bell) Date: Sun, 6 Mar 2005 23:14:52 -0500 Subject: [cells-devel] More newbie questions In-Reply-To: <422B6E41.4060803@nyc.rr.com> References: <4229F03F.2050100@nyc.rr.com> <422B6E41.4060803@nyc.rr.com> Message-ID: Kenny Tilton On Sun, 06 Mar 2005 15:55:29 -0500, Kenny Tilton wrote: > Not sure what code you are referring to. I would not have the slider set > the zoom, I would give the zoom a rule which watched the slider value, > which would just go from zero to one. When the new requirement came > along I would simply have the zoom rule look at the slider /and/ the > sample rate. Seems easy enough to me, and more self-documenting than the > approach you described. If the zoom value is coming out wrong, I like > knowing without doubt exactly where to look. Hell, in the inspector I > can even look at the code, which for debugging purposes I store in > symbolic form in the Cell for those cases where a slot gets different > rules for different instances so often that it is painful to work out > which rule applies to some instance I am debugging. Now this I have to disagree with, from a design perspective. The zoom level in my application is used in at least 8 different unrelated subsystems. It's also controlled (changed) by 4 different GUI widgets. Having the zoom be calculated based on the value of a GUI slider (oh, and I was wrong, I was actually thinking of a combobox/list now, but close enough...sorry) really points the arrow in the wrong direction, in my opinion. The current zoom is the leader of zooming! The actual current zoom level is the piece of data that "feeds" any part of the rest of the system...including the GUI widgets that set (or attempt to set in some cases) its value, the GUI panels that actually display the audio data at the appropriate resolution, to the scaling functions that allow time markers to be created based on mouse clicks. The other pieces of related data (current selection in the combobox/list, slider pixel value, etc. etc.) are all dependent. I think this is the spirit of what I was getting at: when the data is decoupled from the functions that act upon it, it's much easier to realize a true model-view-controller architecture. If you decide to replace the slider in your example, the zoom code has to be rewritten to be calculated based on whatever widget you decided to change it to. If your functions are separate from the data, all you change is the GUI widget function...all the other pieces stay intact, and still work as originally designed. Of course, that's the key...now your application is (more than ever) tied to the proper design of your data model. If you didn't design it properly to begin with, or your original cut wasn't resilient enough to cope with change, or your changes are overwhelming and incompatible with your original design...then you've got to change a bunch of things. But that's true for every programming style. Now it's entirely possible that what you described could be refactored into a much more loosely coupled version, I just haven't thought about it enough and I'm not familiar enough with Cells to know how to do that. > I am not saying I do not find constraint programming interesting or that > it would not be fun to simply drop a new rule in the basket as a way of > modifying a program. On the other hand, I have heard nothing but > nightmares about what it is like to program with multi-way and partial > constraints, so I have not been tempted to add anything like that to > Cells (so far ). What I've (we've, I guess) found with the system I'm using now is that having tools that can display to you your application topology (not necessarily in a nice beautiful GUI app) and also categorize/classify the interactions between the data and functions that act upon it is absolutely essential. I would imagine that any application of sufficient complexity would benefit from this of course, but since ours is imperative (to a certain degree) and forms a very complex publish subscribe network, it's mandatory. There are interactions that are impossible to reason about without program introspection. I'm not sure if this is a really bad thing, considering that most applications written in the world (without the benefit of a framework like Cells or ours) have tons of undiscovered bugs that are thousands of times harder to find. I'm going to keep thinking about declarative event handling vs. imperative event handling. Thanks again Kenny... -- Mike J. Bell on gmail From ktilton at nyc.rr.com Mon Mar 7 04:38:39 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Sun, 06 Mar 2005 23:38:39 -0500 Subject: [cells-devel] More newbie questions In-Reply-To: References: <4229F03F.2050100@nyc.rr.com> <422B6E41.4060803@nyc.rr.com> Message-ID: <422BDACF.9040309@nyc.rr.com> Mike J. Bell wrote: >Hey Kenny...I've got sort of a philosophical question for you. (I'm >much more interested in the intent of Cells rather than the mechanics, >even though the mechanics define the intent...I'm sure you know what I >mean). > >The spirit of Cells seems to be to very much like the declarative >functional world of spreadsheets (ok, I cheated and actually *read* >that in the docs. =). If I were to start a new project (and thus >legacy code/app is irrelevant), why would I use Cells and not Haskell >or some other pure lazy functional language? > >If the answer is along the lines of "CL is super cool 1337 d00d, why >use anything else" that's fine. What I'm digging for here is any >major philosophical differences between using Cells (and I mean >*using*, like building your entire app on as much Cells functionality >as possible and not mixing paradigms) > aside: actually, I think it would be hard to mix paradigms. I always point out that Cells is a slippery slope: use them a little and pretty soon you have to use them for everything. the paradigm is not "lazy functional", it is "dataflow". In my apps, all input gets fed into c-input cells, and all application behavior is manifested by def-c-output methods. The web of cells is a working model where slots have causal power over other slots. The feel of the /programming/ is spreadsheet-y, but the result is, again, a working model. Working in the sense that one simply SETfs an input cell and the whole model reacts /and/ behaves automatically, according to the supplied rules. Of course, a spreadsheet is also a working model. getting back to the slippery slope, I am not sure how one could combine imperative, programmer-oriented control with the automatic dataflow engine -- where would one put the hooks? > and purely functional, >declarative, lazy languages. What's the discriminator? > > Sorry, I have never used anything like Haskell, so I have no idea what it feels like or how to compare, other than what I said above. But maybe this will be close (my apologies if I am muddying the waters): if you have a lazy functional language, the control lies outside the declarative authoring, because, as I understand lazy, nothing happens until someone asks. So there must be a control flow programmed some other way (imperative?) which finally does the asking. In which case one is not really getting all the benefit possible out of the declarative approach. kt From ckimyt at gmail.com Tue Mar 8 20:00:14 2005 From: ckimyt at gmail.com (Mike J. Bell) Date: Tue, 8 Mar 2005 15:00:14 -0500 Subject: [cells-devel] Imperative vs. declarative Message-ID: Hey Kenny, I did a lot of thinking about what you said about declarative vs. imperative. I'm not sure what's the right terminology, but I did finally see the light. We've had problems in applications written in our framework when there are multiple producers of one datum. In a lot of cases (benign and simple), it's been OK. However, as soon as things get bigger, we put a repeating pattern into our designs where a single controller function will act as a gatekeeper, reading from multiple channels of data, and forming the single producer for the difficult datum in the previously intractable design. I've thought about it quite a bit, and I've proved to myself that this is equivalent to your declarative model, even if there are some slight structural differences*. So, in order for me to get from where I am to where Cells is, all I need to do is instead of waiting for the multiple producers issue to get out of hand and uncontrollable...just nip it in the bud and don't allow it to begin with. Then every piece of data has only one producer. Thanks again for talking with me about it! Mike * Speaking of structure, question for you. Let's say you're declaring a slot to be a function of a couple other slots in Cells. How do you know they exist? I.e., where do you find your objects? Do you have like a database somewhere (hash of name->object or the like) that you lookup to find things? Are they all passed in at construction time? (I would imagine this wouldn't scale well at all). Is there some other scoping mechanism? I'm not sure if Cells has any builtin support for finding the other objects/slots that participate in Cells, or if you roll your own per application...but I'd love to hear about how you solve it when *you* write code. -- Mike J. Bell on gmail From rpgoldman at real-time.com Tue Mar 8 19:40:00 2005 From: rpgoldman at real-time.com (rpgoldman at real-time.com) Date: Tue, 8 Mar 2005 13:40:00 -0600 Subject: [cells-devel] Newbie questions In-Reply-To: <4229CF21.4070008@nyc.rr.com> References: <4229CF21.4070008@nyc.rr.com> Message-ID: <16941.65424.144212.575270@gargle.gargle.HOWL> >>>>> "Kenny" == Kenny Tilton writes: Kenny> Mike J. Bell wrote: [...snip...] Kenny> Cells have always grown in response to application Kenny> requirements, so if someone comes up with a good example I Kenny> will gladly look at implementing cycles in Cells. The Kenny> question is, how can a = f(b) and b=f(a) ever be computed? Kenny> In your example, there is no real cycle, because you are Kenny> saying: "dollars is a function of yen /when changes/, and Kenny> yen is a function of dollar /when dollar changes/." Only Kenny> one can be changing at a time, so there is no cycle as the Kenny> problem is conceived. Kenny> But how is an instance initialized? I can supply conversion Kenny> rules for dollar-balance and yen-balance, but how do I get Kenny> the intial balance in there? I would need to specify a Kenny> starting value, but for which one? Both? What if I specify Kenny> 1 for both? That conversion would be wrong, the dollar is Kenny> (for now) worth more than 1 yen. Kenny> Conceivably the answer is to let the programmer worry about Kenny> it. I could specify a value (and the conversion rule) for Kenny> dollar-balance and specify "unbound" and a rule for all Kenny> other currencies. Then if the others get accessed, the Kenny> system just calculates as usual. If all values are unbound Kenny> a runtime error results. If the programmer specifies Kenny> inconsistent initial values they get a wrong result (which Kenny> we might be able to detect at some point of runtime) until Kenny> some SETF perturbs the model and the formulas kick in to Kenny> make things consistent. Kenny> Thoughts on this? IIRC, Garnet's KR has a mechanism (one might call it "a cheap hack") to solve the problem of cyclic dependencies, which was to allow you to "plonk" an initial value into a slot that would be used to kick the whole mess off. Best, Robert From ktilton at nyc.rr.com Tue Mar 8 21:07:19 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Tue, 08 Mar 2005 16:07:19 -0500 Subject: [cells-devel] Newbie questions In-Reply-To: <16941.65424.144212.575270@gargle.gargle.HOWL> References: <4229CF21.4070008@nyc.rr.com> <16941.65424.144212.575270@gargle.gargle.HOWL> Message-ID: <422E1407.7060006@nyc.rr.com> rpgoldman at real-time.com wrote: >>>>>>"Kenny" == Kenny Tilton writes: >>>>>> >>>>>> > > Kenny> Mike J. Bell wrote: > >[...snip...] > > Kenny> Cells have always grown in response to application > Kenny> requirements, so if someone comes up with a good example I > Kenny> will gladly look at implementing cycles in Cells. The > Kenny> question is, how can a = f(b) and b=f(a) ever be computed? > Kenny> In your example, there is no real cycle, because you are > Kenny> saying: "dollars is a function of yen /when changes/, and > Kenny> yen is a function of dollar /when dollar changes/." Only > Kenny> one can be changing at a time, so there is no cycle as the > Kenny> problem is conceived. > > Kenny> But how is an instance initialized? I can supply conversion > Kenny> rules for dollar-balance and yen-balance, but how do I get > Kenny> the intial balance in there? I would need to specify a > Kenny> starting value, but for which one? Both? What if I specify > Kenny> 1 for both? That conversion would be wrong, the dollar is > Kenny> (for now) worth more than 1 yen. > > Kenny> Conceivably the answer is to let the programmer worry about > Kenny> it. I could specify a value (and the conversion rule) for > Kenny> dollar-balance and specify "unbound" and a rule for all > Kenny> other currencies. Then if the others get accessed, the > Kenny> system just calculates as usual. If all values are unbound > Kenny> a runtime error results. If the programmer specifies > Kenny> inconsistent initial values they get a wrong result (which > Kenny> we might be able to detect at some point of runtime) until > Kenny> some SETF perturbs the model and the formulas kick in to > Kenny> make things consistent. > > Kenny> Thoughts on this? > >IIRC, Garnet's KR has a mechanism (one might call it "a cheap hack") >to solve the problem of cyclic dependencies, which was to allow you to >"plonk" an initial value into a slot that would be used to kick the >whole mess off. > Yes. I was frankly horrified by all the backdoor hacks KR allowed. I must be getting old and stodgy. :) kenny From ktilton at nyc.rr.com Tue Mar 8 21:25:39 2005 From: ktilton at nyc.rr.com (Kenny Tilton) Date: Tue, 08 Mar 2005 16:25:39 -0500 Subject: [cells-devel] Re: Imperative vs. declarative In-Reply-To: References: Message-ID: <422E1853.1060505@nyc.rr.com> Mike J. Bell wrote: >Hey Kenny, I did a lot of thinking about what you said about >declarative vs. imperative. I'm not sure what's the right >terminology, but I did finally see the light. > >We've had problems in applications written in our framework when there >are multiple producers of one datum. In a lot of cases (benign and >simple), it's been OK. However, as soon as things get bigger, we put >a repeating pattern into our designs where a single controller >function will act as a gatekeeper,... > Yep. I even cite such things as "prior art" of Cells, or at least other ways to address the issue (and serve as an existence proof that this is a real problem for complex applications). > reading from multiple channels of >data, and forming the single producer for the difficult datum in the >previously intractable design. > >I've thought about it quite a bit, and I've proved to myself that this >is equivalent to your declarative model, even if there are some slight >structural differences*. So, in order for me to get from where I am >to where Cells is, all I need to do is instead of waiting for the >multiple producers issue to get out of hand and uncontrollable...just >nip it in the bud and don't allow it to begin with. Then every piece >of data has only one producer. > >Thanks again for talking with me about it! > >Mike > >* Speaking of structure, question for you. Let's say you're declaring >a slot to be a function of a couple other slots in Cells. How do you >know they exist? I.e., where do you find your objects? Do you have >like a database somewhere (hash of name->object or the like) that you >lookup to find things? Are they all passed in at construction time? >(I would imagine this wouldn't scale well at all). Is there some >other scoping mechanism? > The Family class is an integral part of all my applications. i even have special handling for the kids slot (the other slot forming the DAG is fm-parent), though I wonder if the fixes in Cells II obviate the need for that. Anyway.... A suite of routines will search the Family tree of objects in various ways. One is by name (the md-name of a model object) and lately I have found myself searching more often by type "gimme the nearest Selector above me so I can find out if I am part of the current selection". And I /have/ been tempted to do something with a hash table of md-names, but one can get a lot of duplicates on that and one is usually searching relative to oneself. The nice thing about this is that it turns out that a given sought object need not exist when a rule requiring one of its cells fires. Cells would be Hell to program is that were the case. Instead, what happens is that the traversing finder function keeps sampling the kids slots of this Family instance and that. If the kids are a rule and we are dealing with a brand new tree of widgets, a lot of these rules will not yet have fired (to produce the actual kid instances). No problemo. Rules fire on demand, so the instances pop into existence JIT. If your declarative code specifies the kid being sought, it will get created JIT before the original rule seeking it completes execution. I was amazed when I saw that work, btw, many moons ago. > >I'm not sure if Cells has any builtin support for finding the other >objects/slots that participate in Cells, or if you roll your own per >application...but I'd love to hear about how you solve it when *you* >write code. > Check out the def-c-output KIDS method. (It might be .KIDS.) and the TO-BE function. You are right: nothing says some other way of managing an object namespace could not be used. (Well, I /do/ have to make sure that Cells internals do not really need to be giving KIDS special handling... that may even have been an optimization or something to make debugging simpler "get the model population into existence first rather than have JIT model creation kicked off during rule fires." kt From ckimyt at gmail.com Thu Mar 10 13:57:26 2005 From: ckimyt at gmail.com (Mike J. Bell) Date: Thu, 10 Mar 2005 08:57:26 -0500 Subject: [cells-devel] More on the details... Message-ID: Ok, so I did a survey of the design in one of the applications I built, and I'm pretty convinced I could move to a completely-declarative model, with not *too* much effort. One thing that baffles me though, is how you would go about implementing undo/redo functionality. In my current application, just about everything can be undone/redone, with a theoretically infinite list of snapshots. I had to put in some weird mechanisms to shield the undo/redo system from saving state *too* often, like for example if the user is dragging an object on the screen, it only remembers where it was and where it ended up, not the whole path. It's unbelievably simple to implement this in our imperative world...the undo/redo system (which, incidentally, in written in user-land and requires no support from the architecture) listens for changes in the specified slots, records everything when they change, and when you tell it, will walk the stack and publish saved data at a later time. Obviously, when you look at most of the "core" data slots in my application, you'll find at least three publishers: the thing that really calculates what it should be at runtime, and then undo, and redo. How do you solve this in Cells? It sounds like it would be really annoying to have to write "dispatch" slots that marshall the three different possible values for every cell that falls into this pattern...but I guess that's what macros are for. What about integrating it into the Cells engine itself? I mean, since Cells is declarative, theoretically you should be able to record the value of all the independent cells (the "roots" of your DAG of slot dependencies) and then just reset them all accordingly to get the whole system into the same state as it was...right? Thanks for your comments... -- Mike J. Bell on gmail