From hoan at ton-that.org Sat Jul 1 11:22:52 2006 From: hoan at ton-that.org (Hoan Ton-That) Date: Sat, 1 Jul 2006 21:22:52 +1000 Subject: [cl-stm-devel] Binding variables Message-ID: Hey everyone, Today, I added `tlet' a There is a slight problem with our previous higher-order function: (deftransaction sleep-after (seconds transaction) transaction (trans (sleep seconds))) The return value of the new transaction is not the same as the return value of the input transaction. Instead what `sleep-after' does is always return NIL, because `sleep' returns NIL. What we need is a way to bind the value of `transaction' to a variable. Hence `tlet'. Here is how we could write `sleep-after' correctly: (deftransaction sleep-after (seconds transaction) (tlet ((val transaction)) (trans (sleep seconds)) (trans val))) Now if you looked through the debugging output when typing `(perform (sleep-after 2 (increment *c*)))', with both versions, you will see a line looking like: 20:33 STM-LOGGER/+DRIBBLE+: Transaction finished executing with: 6 or, with the old version: 20:33 STM-LOGGER/+DRIBBLE+: Transaction finished executing with: NIL If you were very observant, you would have noticed that the `tlet' in `sleep-after' could be replaced with a `prog1'-like construct. We can define `progt1' in terms of `tlet': (defmacro progt1 (&body body) (with-unique-names (val) `(tlet ((,val (trans ,(car body)))) ,@(cdr body) (trans ,val)))) And thus `sleep-after' becomes: (deftransaction sleep-after (seconds transaction) (progt1 transaction (trans (sleep seconds)))) which is much nicer. However, we quickly run into a problem. We now have to define a whole slew of transactional equivalents of `prog2', `multiple-value-prog1', and so on. When translating an the MVar example from Haskell to Common Lisp, I even had to write a new version of `if'! The next step is to write a code-walker that converts Common Lisp forms into the transactional equivalent. Hoan From hoan at ton-that.org Sun Jul 2 06:57:42 2006 From: hoan at ton-that.org (Hoan Ton-That) Date: Sun, 2 Jul 2006 16:57:42 +1000 Subject: [cl-stm-devel] Try Message-ID: Here is an example of the `try' operator. From the documentation: "Return a transaction that executes the transactions in BODY atomically until one succeeds. The return value of the transaction is the value of the transaction that succeeds." Here we have two counters *c1* and *c2*. (defvar *c1* (new 'counter)) (defvar *c2* (new 'counter))) STM> (perform (try (increment *c1*) (increment *c2*))) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 1 0 Here, the first transaction (increment *c1*) succeeds, and we're done. Thus the values of the counters are 1,0 respectively. This is what happens when we can't increment *c1*. STM> (acquire-lock (lock-of (count-of *c1*))) T STM> (perform (try (increment *c1*) (increment *c2*))) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 1 1 Since the first transaction (increment *c1*) failed, the second one (increment *c2*) was run and succeeded. So far the behaviour of `try' is very similar to the `or' operator. Now what happens when we can't increment either counter? STM> (acquire-lock (lock-of (count-of *c2*))) T STM> (perform (try (increment *c1*) (increment *c2*))) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 1 1 The values are still the same as before. However the transaction is still waiting for either *c1* or *c2* to be freed. Here is what happens when we free *c2* and notify the threads waiting on it: STM> (release-lock (lock-of (count-of *c2*))) NIL STM> (unwait (count-of *c2*)) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 1 2 The whole transaction has been retried from the beginning. The first counter was still locked so the transaction (increment *c1*) failed. The second transaction (increment *c2*), succeeded. Whenever a variable is written to memory (when a log is committed), all threads waiting on that variable are woken up automatically with `unwait'. See the code in the `commit' function for more details. That guarantees that transactions eventually commit. Hoan From hoan at ton-that.org Mon Jul 3 12:05:54 2006 From: hoan at ton-that.org (Hoan Ton-That) Date: Mon, 3 Jul 2006 22:05:54 +1000 Subject: [cl-stm-devel] Retry Message-ID: Yesterday I gave an example of the operator `try'. Today, you'll see how `retry' nicely complements it. The previous code tried to increment one of two counters. It didn't matter which one we incremented as long as we incremented! Lets generalize from that example to increment any counter in a list of counters: (deftransaction increment-any (counters) (tif (trans (null counters)) (retry) (try (increment (car counters)) (increment-any (cdr counters))))) Here `tif' is a macro just like `if', but it takes transactions as its test and branches. That's why the (null counters) is wrapped up in a `trans'. As you can infer, that means the return value of `retry' and `try' are transactions. Looking at the second clause tells us that this is a standard car/cdr recursion. We try to increment the first counter, and if we fail, we try the rest of them. The base case is where we call retry. So if the list is empty, the transaction is *aborted* and starts from scratch again. The transaction waits on all the variables that have been read until `retry' was called. This guarantees that the transaction will eventually commit. Here is an example: STM> (progn (defvar *c1* (new 'counter)) (defvar *c2* (new 'counter))) *C2* STM> (acquire-lock (lock-of (count-of *c1*))) T STM> (acquire-lock (lock-of (count-of *c2*))) T STM> (perform (increment-any (list *c1* *c2*))) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 0 0 STM> (release-lock (lock-of (count-of *c1*))) NIL STM> (unwait (count-of *c1*)) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 1 0 So that was just the same as yesterday. But we can do better. `increment-any' is just a special case of trying any transaction down a list, just like `map' is a special case of apply a function down a list. So the equivalent would be: (deftransaction try-list (transaction list) (tif (trans (null list)) (retry) (try (funcall transaction (car list)) (try-list transaction (cdr list))))) And we could now define `increment-any' as: (deftransaction increment-any (counters) (try-list #'increment counters)) Now running it gives us: STM> (perform (increment-any (list *c1* *c2*))) [debugging output elided...] STM> (values (value-of (count-of *c1*)) (value-of (count-of *c2*))) 2 0 And indeed it works the same as before. Indeed we could even use `try-list' in many other situations, like reading from a socket like Unix's select. Now I am in a bit of a conundrum with regards to the interface that CL-STM should have. At the moment I'm not happy with using all these new macros like `progt', `progt1', `tif' and so on because it means the programmer has to learn all these new names that are just variations of the standard CL ones. Its also a pain to figure out where to use `trans' and `untrans'. I've started some work on a code walker that transforms CL forms into STM ones. I have the equivalents of `tlet', `progt' and `tif' implemented. Even better is the code walker handles macro expansion, so that `cond', and `prog1' will be taken care of. BTW, this code walker will only traverse forms inside a `deftransaction'. Ideally we'd like to write code without explicit `trans' and `untrans'. Even in the simple `increment-any' example, we had to use `trans'. It would be nice if we could tell in advance whether or not a particular expression is a transaction or not. But Common Lisp doesn't have a type system. But here is what we know: Forms like `(trans ...)' are transactions. A form is a transaction if its car is a symbol defined with `deftransaction'. So (increment-any ...) is a transaction. Assuming all transactions are take the form above, we can infer that anything else is not a transaction. So (null ...) is not a transaction, and thus the walker can wrap it with `trans'. The above analysis doesn't work for higher-order functions. It assumes incorrectly that (reduce ...) is doesn't return a transaction. But it could return a transaction depending on the function passed to it. So the analysis only work on first-order functions. I'm wondering if this is the best we can do? I've still got more thoughts on this but now I'm asking you. What interface would you like? Do you mind writing `trans' and `untrans'? Would you accept a half-solution? Whatever, just tell me whats on your mind. Did you have a good day? Are you following the world cup? Hoan From hoan at ton-that.org Wed Jul 5 01:55:00 2006 From: hoan at ton-that.org (Hoan Ton-That) Date: Wed, 5 Jul 2006 11:55:00 +1000 Subject: [cl-stm-devel] Documentation Message-ID: This is a quick note letting you know that there is now an online API reference to CL-STM: http://www.common-lisp.net/project/cl-stm/doc The documentation is generated by qbook, which extracts it from the source code. qbook is childly simple to use. Hoan From hoan at ton-that.org Sat Jul 15 07:54:22 2006 From: hoan at ton-that.org (Hoan Ton-That) Date: Sat, 15 Jul 2006 17:54:22 +1000 Subject: [cl-stm-devel] What next for CL-STM? Message-ID: Hello everyone! I'm wondering what we should work on next for CL-STM. I haven't worked on it for a few days. What would you like to see done with CL-STM? Has anyone here got it working? I'd be happy if you have, and if you haven't I can help out if you would like. I'm currently working on the code walker, and I think I'll be done with it by Wednesday. But besides that, I have worked on easy things like documentation. Hoan