[cells-devel] Re: [PyCells] caching in models

[Ken Tilton]

Sorry, the Python was so hard for me to read that I punted on the original
query.

[Lispniks: the idea is to have a counter object that counts and caches
counts of a substring in a text object known to the counter, and for the
count cache to get cleared when the text known to the counter gets changed.
Do we do it in a bogus cell and rule that watches the text cell and
imperatively clears the countcache (a non-cell) or do we do it in an
observer on the text? Andrew, the OP, did not like the bogus cell/rule, but
also did not like defining the observer separately from the counter (a more
general observation on observers)]

One cutesy approach would be to make count_cache a cell and (in Lisp):

   :count_cache (c? (when (text self) (make-hash-table)))

Or if it really seems faster to save the dictioanry:

    :count_cache (c? (when (text self)
                                     (if .cache
                                           (clrhash .cache) ;; clrhash
returns its input
                                         (make-hash-table))))

 I use the self-deprecating "cutesy" above because what we are doing here is
achieving an imperative effect using our understanding of how the
declarative cells engine works, so it is a little non-obvious, aka "cutesy".
Okay, the conditional "when" (a one-branch IF for you Pythonistas) is a tad
bogus. Like the original code (not shown, Lispniks) it could have just been
an unused read of the text slot just to get the dependency. With Andrew I
hate myself in the morning after writing code like that. :)

That said, the more I use Cells for real-world application stuff the more I
appreciate the net win of the declarative paradigm, even when I have to
think a little harder to get imperative behavior ("hey, the text is
changing, clear the cache") out of it.

I also would not feel guilty about writing ugly code like that because, hey,
the cache is an optimization, optimizing always leads to ugly code. And,
speaking of unnecessary optimization, why not just make a new LetterCounter
for the new text? Too easy? Or do not even use Cells to clear the cache,
just create a function to set the text that first clears the cache. But this
last brings me back to preferring ugly Cells code to clean otherwise code:
having a fishy rule that clears the cache at least keeps all the important
code in one place, and to the extent that the code gets complicated (not
this simple case) it does guarantee correctness, whereas without Cells
someonbe might set the text directly instead of going thru the bottleneck
function which also clears the cache.

That last bit argues against using the observer at all. In a sense, the
count_cache is indeed a function of the search text, whether one wants its
existence to depend on the text, or its mutable contents to. And the "WHEN"
can be defended as "hey, there is no text so there are no counts cached". I
agree this is best represented as an empty dictionary so other code can just
look things up without worrying about whether the dictionary exists, but
then we come to a new issue: Cells do not really work well with mutable
contents. But we could write:

   :count_cache (c? (if (text self)
                                 (if .cache (clrhash
.cache)(make-hash-table))
                               (make-hash-table)))

Bit of a waste, but if one is really worried about there always being a
dictionary, there ya go.

Now this may look like a top-post, but it is not......

On 9/6/06, Ryan Forsythe <ryan at pdxcb.net> wrote:
>
> On Sep 5, 2006, at 6:57 PM, Andrew Dalke wrote:
> > Is the above the right way to do that?  It feels somewhat wrong
> > because the returned result is never used.  Though it does work.
>
> I hesitate to call any solution "wrong," but here's a version with as
> few changes as possible that uses an observer:
>
> ----
> class LetterCounter(cells.Model):
>      text = cells.makecell(value="")
>      case_sensitive = cells.makecell(value=False)
>
>      @cells.fun2cell(celltype=cells.AlwaysLazyCell)
>      def _text(self, prev):
>          if not self.case_sensitive:
>              return self.text.lower()
>          return self.text
>
>      def count(self, s):
>          try:
>              count = self._count_cache[s]
>              print "Returned from cache"
>              return count
>          except KeyError:
>              if not self.case_sensitive:
>                  s = s.lower()
>              i = self._text.count(s)
>              self._count_cache[s] = i
>              print "Computed and saved in cache"
>              return i
>
>      def __init__(self, *args, **kwargs):
>          self._count_cache = {}
>          cells.Model.__init__(self, *args, **kwargs)
>
> @LetterCounter.observer("_text")
> def cache_clearer(model):
>      model._count_cache.clear()
> ----
>
> >>> lc = LetterCounter()
> >>> lc.text = "Now is the time for all good men to come to the aid
> of their country"
> >>> lc.count("the")
> Computed and saved in cache
> 3
> >>> lc.count("o")
> Computed and saved in cache
> 9
> >>> lc.text = "Life! Don't talk to me about life!"
> >>> lc.count("life")
> Computed and saved in cache
> 2
> >>> lc._count_cache
> {'life': 2}
>
> So, the observer needs to be added to the model after it's built.
> I've never been totally satisfied with the way this works, but it
> seems to more closely match how Lisp Cells works than having a
> mechanism to add an observer within a class def.


I do not do it much any more, but Lisp supports dispatch on multiple
arguments, and early on I defined quite a few observers that specialized on
things other than the class of the instance, or in addition to the class of
the instance. That said...

Perhaps it'd be good to have it available both ways?


Sure. That actually satisfies better the ideal of having everything in one
place. To keep things simple, maybe forget the Lisp Cells approach and just
make it part of the class defintion (I guess also changing the
implementation mechanics so it can happen along with the class definition).

> I tried
> >
> >     @_text.observer()
> >     def _reset_count_cache(self):
> >        ...
>
> This does seem really natural.



I will keep thinking about this, but FWIW I think this would be a fine
divergence for PyCells from Cells. I might even look at adding similar
syntactic sugar to Lisp Cells for those many cases where it could usefully
be defined along with the slot.

OK, here is one new thought. In LispCells, if observers are define for a
slot for classes X and SubclassOfX, both observers run for instances of the
subclass. Well, I guess that is the same issue either way. Never mind. :)

kt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cells-devel/attachments/20060906/ce8c046b/attachment.html>