[cells-devel] Added cells-store to cells

[Kenny Tilton]

Peter Hildebrandt wrote:
> Ken,
> 
> thanks for the quick response.  I'll try to illuminate things a bit.
> 
>>  Cool. I am curious what application requirement led to this.
> 
> I've been looking for a while for a way to store a (possibly large)
> number of model objects in a way that makes it easy and fast both to
> iterate over them and to access individual ones.  In principle this
> can all be done with the fm-utilities, e.g. kids and fm-other.
> However, when things get big, I feel bad constantly traversing trees
> with fm-other and keeping tons of instances in the kids slot.

Understood. I have been thinking along the same lines lately, but I 
generally wait until I have a use case because (a) I am lazy and (b) I 
think without an actual use case one puts in too many bells and whistles 
or guesses wrong at what is needed and how it should work or all of the 
above.

Sounds like you have a Real Application to do. A nice extension, btw, 
would be an option for JIT creation should the thing not yet exist, but 
I gather you have Actual Work(tm) to do. When it comes up. I guess you 
would want the lookup then to be the key plus the argument list for 
make-instance.

btw, one of the things I sometimes think to point out is that Cells 
merely benefits (albeit greatly) from being used within a namespace of 
other objects to depend on, and even more when that namespace has Cells 
Inside(tm). I probably did Family only because I started with a GUI 
hierarchy, tho on second thought the tree is generally a powerful data 
structure.

All just a long way of saying "good idea". :)


> 
> There are three reasons I was looking for an alternative:
> - My current research app models some type of human learning, which
> involves huge numbers of interlinked nodes in a network.  I was
> looking for a data structure which has O(1) access.
> - fm-other traverses the tree at initialization time looking for the
> target.  I was looking for a way to reference instances to be created
> in the future. (For example, this is interesting when loading a
> network from file - either you have an acyclic graph and save it in a
> topological order, or you need some way to forward reference
> to-be-loaded nodes)
> - I am afraid (correct me if this is wrong) that ruled cells using
> fm-other are calculated more often than necessary -- e.g. you have a
> cell depending on (fm-other :something).  Then every time a new kids
> is added somewhere in the tree, this rule should be run to check
> whether there is a :something now somewhere.  My cells-store hash
> table makes sure that only those rules are run that are actually
> necessary.

The /potential/ for kicking off kids rules too often is there, but 
generally when rules are written sensibly the dependency on :something's 
existence ends up in the rule for an attribute, not in the rule that 
would make kids elsewhere get recalculated. In a sense we have the 
typical proble with Lisp: one false step and you can just hose your 
application. But as with Lisp, one quickly develops an instinct for 
things that are slow by mistake.

> 
> The immediate reason is that when debugging some cells-gtk stuff, I
> was confronted with a thick knot of fm-other initiated tree
> traversals, and I wished I had something more linear and predictable.

<g> Understood.

> So I tried to use simple hash table look ups, but that did not work
> since the cells did not learn when new items were added to the table
> (i.e. forward references broke).  Thus I needed a cells-aware hash
> table.   And now I'll use that in my app and in cells-ode, too.
> 
>> Not that it is
>> not a good idea. Being able to reach inside other structures for dependency
>> is a natural fleshing-out of Cells -- why stop at the slot, or force
>> everything to /be/ a slot to depend on it?
> 
> Yep, there might be other cases -- even though I can't think of
> anything besides a hash table where I ever missed cells-awareness.
> 
>>  Meta-cool is that you are the first Cells user to do something like this,
>> IIRC, altho there are several who took the more typical Lisper approach of
>> dashing off to do their own versions of the entire package. :)
> 
> Thanks, I take it as a compliment.
> 
>>> The (c?-with-stored (var key store &optional default) &body body)
>>>
>>  I have not looked at the code so maybe what I am about to suggest is
>> already possible, but would this also be useful (just to confuse things I
>> changed with-stored to bwhen-c-gethash:
>>
>>    (c? (if (^whatever)
>>           42
>>          (bwhen-c-gethash (user login-id *user-logins*)
>>              .....)))
> 
> Funny you picked that name.  At first, I called it c?-bwhen-hash.
> Then I figured that with fit the semantics better.  Maybe I was right
> the first time, tho.
> 
> As to you point, no, this is not possible right now since
> c?-with-stored expands into (c? ...).  This is not necessary, though,
> because all my macro does is establish a dependency on an intermediate
> object which gets modified if the corresponding hash table entry is
> manipulated (i.e. an object added/removed).  So indeed we could have
> this (w/o the c? part) as a standalone macro.
> 
> I changed it in CVS.  The following works now, too:
> 
> 	 (make-instance 'test-store-item :value (c? (if (value bypass-lookup?)
> 							     'no-lookup
> 							     (bwhen-gethash (v :bar store 'nothing)
> 							       (value v)))))
> 
> I included it in the test func at the bottom of md-utilities.  If you
> feel like it, you can integrate it with the test suite.  (You're the
> maintainer, after all *g*)

One of my blog articles details my enthusiasm for testing. :)

> 
> I added a few macros for easier writing of unit tests, btw.  There is
> assert-values, which takes instance-value pairs and
> with-assert-observers, which checks whether all and only the specified
> observers run.
> 
>>  The above might be easily done (just a tweak or two) by looking at how
>> synapses work. They are like local and/or anonymous Cells. "Anonymous"
>> because they do not mediate a slot, they mediate an arbitrary sub-form of a
>> rule. "Local" because only the Cell envalued by the containing form depends
>> on the synaptic Cell.
> 
> Ooops, I should read ahead.  Haven't looked at synapses.  Well, next time. ;-)
> 
>>  Historical note: synapses used to be a completely different data structure
>> with distinct code supporting them, now we just have a "synaptic" attribute
>> to guide the general Cells logic in a few places.
>>
>>  Anyway, then c?-with-stored is just the special case where this appears as
>> the entire form:
>>
>>   (c? (bwhen-c-gethash (...) ...))
> 
> Yep, I'm with you.  Just reimplemnented c?-with-stored in terms of
> bwhen-gethash.

I think you need a -c- in there because i can imagine a bwhen-gethash as 
pure CL, nothing cells-y.

btw, using bwhen-store vs gethash might be a good idea (not expose 
implementation) but sometimes I like to do exactly that just to make 
code more approachable. This one is a coin-toss for me.

> 
>>  Hmmm, you know, I almost never use synapses and yet I have always suspected
>> that a lot of fun would be had with them in re expanding the semantics one
>> could express with Cells. But this exchange has me wondering if I missed
>> something: will new kinds of Cells tend to be synapses because the semantics
>> will be useful within rules?
> 
> Ok, now you lost me.  I'll have to look at synapses before I can tell.
>  For now, I need to go back to my actual work and put the new store to
> work.
> 
>>  Or maybe I am just confused: should all cells be synapses now? I might be
>> in the land of distinctions without differences now. :)
> 
> So I guess you are well prepared for Amsterdam :)
> 
>>  Well, I am off to Amsterdam shortly and will play with this when I get
>> back. Not sure how much time I will be on-line, btw, until Monday.
> 
> Ok, have fun over here in old Europe :)
> 
>>  Thanks for a cool contrib.
> 
> You're welcome.  Thanks for a cool library :)
> 
> Cheers,
> Peter
> 
>