[cells-devel] Re: [PyCells] caching in models

[Ken Tilton]

On 9/6/06, Andrew Dalke <dalke at dalkescientific.com> wrote:
>
> Me [Andrew Dalke]:
> > I developed a simple dependency system which I've used for several
> > clients.  The 2nd one was simpler because no modifications occurred
> > after a property was set or computed.  The heart was something like
> > this
>
> One thing it couldn't handle is query optimization.
>
> In two cases I computed properties using an external program.
> If I wanted property A it took 1 minute, if I wanted property B
> it took 1 minute, if I wanted properties A and B computed at the
> same time then it took 1 minute 30 seconds.
>
> My system was all lazy.  Set it up with the rules and ask for
> the desired properties.  But that meant asking for them one-at-a-time.
> There was no way to figure out that the user wanted A and B so go ahead
> and do the combined request instead of 2 separate requests.
>
> I don't think PyCells can do this either.  It's a hard problem
> without explicit knowledge of the dependencies.


The way I would phrase it is not that PyCells can or cannot do something,
rather: Right, PyCells cannot help with any dependency it does not "see".
And all that means is that, right, Cells/PyCells is not a free lunch after
all; we must work around the things it can do to get what we want, and I
must say I spent quite a while learning to think fluently in the dataflow
paradigm, and even now hard problems (the ones involving events) I have to
think pretty hard to come up with declarative solutions. The only saving
grace is that it is that fun kind of hard thinking.

My solution is to provide a hint - when I know I'll do something
> which needs A+B I'll look up a special property which computes
> A+B and then sets A and B.


That would work. Or buy into the apparent potential for composite queries
lock stock and barrel, by creating an "and" operator in your burgeoning
language and encouraging users to divide-and-conquer queries as much as
possible to get re-use. This might mean moving the query-result cache out
into a separate supervisory class. Then you have a query class that gets a
pointer to the supervisor to see the search space and cache. Queries can be
atomic (A, B) or "operation queries" which have an opcode "+" and so many
"opnd" queries as an attribute. Hopefully you can figure out how to cache
composites as well as atomics if the operations themselves are expensive
enough (and reuse of those is to be expected). Then you are not doing
anything special to cache subqueries.

hth,kt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/cells-devel/attachments/20060906/f66290f6/attachment.html>