[elephant-devel] Query System

Wed May 14 19:15:46 UTC 2008

On May 14, 2008, at 3:06 PM, lists at infoway.net wrote:

> Ian/Glenn,
>
> Thanks for the feedback. I knew it should be somewhere buried within  
> the MOP and last week I ordered The Art of the MOP but am waiting  
> for it to arrive. I also know it will take me some time to grasp it.  
> I just wanted to get some familiarity for this particular need (more  
> specifically, I was thinking of allowing the query to ask for things  
> like Person.name or something like that, but wanted to know if it  
> was possible to "pre-compile" the query and make sure that the name  
> slot exists on the Person class; BTW, sorry for my non-Lisp syntax).

If you download cl-weblocks and look in src/stores/elephant/ 
elephant.lisp you'll see a simple query implementation supporting the  
weblocks query semantics for paged table views that uses indices to  
perform ordering when available and also does reasonably efficient  
range queries (you walk the whole index, but only create the instances  
you need and perform early termination.

If you want to get into query compilation, you can generate closures  
that are highly efficient, but check their preconditions prior to  
execution, recompiling on violations.  I'd wait on this performance  
step and get an interpreter to work first.

Don't sweat syntax too much, I'm sure it will evolve as we play with  
it so better to get something working and evolve and extend it and  
learn as we go.

It's great to see so much energy on the list these days!

> Anyway, I am reading OnLisp. Definitely impressed and perplexed as  
> to what can be done with macros, but I'm getting it slowly :)
>
> I hope to be able to submit a draft proposal by tomorrow. Maybe I'll  
> send to you privately for review and comment before posting it to  
> the list - and possibly ridicule myself :)
>
> Anyway, will look into your suggestions right away.
>
> Thanks,
> Daniel
>
> On May 14, 2008, at 2:25 PM, V. Glenn Tarcea wrote:
>
>>
>> Some great books and PDFs:
>>
>> The Art of the Meta Object Protocol (available on Amazon)
>> Object-Oriented Programming in COMMON LISP (Keene, also on Amazon)
>>
>> For Advance mind twisting Lisp, get Paul Graham's OnLisp (available  
>> as a
>> downloadable PDF). My brain hurts just trying to figure out some of  
>> the
>> macros (and even more advance features) he rolls out:
>> http://www.paulgraham.com/onlisptext.html
>>
>> I would have to say that "The Art of the Meta Object Protocol" is  
>> one of the
>> best comp sci books in my collection.
>>
>> Glenn
>>
>> -----Original Message-----
>> From: elephant-devel-bounces at common-lisp.net
>> [mailto:elephant-devel-bounces at common-lisp.net] On Behalf Of Ian  
>> Eslick
>> Sent: Wednesday, May 14, 2008 2:16 PM
>> To: Elephant bugs and development
>> Subject: Re: [elephant-devel] Query System
>>
>> Hi Daniel,
>>
>> Welcome to the Metaobject Protocol, aka the MOP!  There is a rich
>> embedded language for introspecting over and manipulating the class
>> system in lisp that Elephant uses heavily.  You could have a  
>> procedure
>> extract the and caches the essential information from the class slot
>> definitions into a little structure.  You can do this the first time
>> the query system encounters a class, and add a hook into the class
>> instantiation code that invalidates the cache on class redefinition.
>> It's worth diving into it, but it can take awhile to wrap your head
>> around the concepts, not to mention the idiosyncrasies of the MOP.
>>
>> I'm happy to review your proposal offline or online as you prefer.
>>
>> If you look at src/elephant/package.lisp we import symbols  
>> selectively
>> from the metaobject protocol package specific to each lisp.  The
>> functions you are looking for are:
>>
>> class-slots - return a list of slot-definition-objects
>>
>> Each of the special elephant slots returned by this function is a
>> subtype of the class standard-effective-slot-definition and  
>> persistent-
>> effective-slot-definition.  Direct slots are representations of the
>> arguments to the defclass form.  Effective slots are computed when  
>> the
>> class is 'finalized' (i.e. the first instance is instantiated and  
>> it's
>> inheritance hierarchy is fully defined).  A generic function called
>> slot-definition-using-class dispatches on the type of the effective
>> slot to implement the common-lisp primitive (slot-value obj slotname)
>>
>> The special types you'll care about are:
>> - persistent-effective-slot-definition
>>  persistent-p is a predicate on the slot definition that tells you
>> if it is an instance of or subclass of a persistent slot.
>> - indexed-effective-slot-definition
>>  (has special slots named: indices and base-class; predicate is
>> indexed-p)
>> - derived-effective-slot-definition
>>  (derived from the index slot also contains a slot: fn for
>> determining the value to index on)
>> - set-valued-effective-slot-definition
>>  (slot stores a reference to a pset; set-valued-p)
>> - association-effective-slot-definition
>>  (association-p; this is complicated so we should handle this as a
>> special case in the query system later)
>>
>> I have accessors to get these slots from the class object such as
>> association-slot-defs, indexed-slot-defs, and persistent-slot-defs.
>>
>> Read metaclasses.lisp for more information on these definitions.
>>
>> If you know an index exists on a class slot, you can simply call  
>> (find-
>> inverted-index class-object slotname) to have the system fetch it for
>> you.
>>
>> This should be enough for you to define a structure like:
>>
>> (defstruct class-info classname slotypes)
>>
>> Where slottypes is an alist:
>> - ((slotname . :indexed) (slotname . :persistent)
>> (slotname . :transient))
>>
>> or something like that.
>>
>> Ian
>>
>> On May 14, 2008, at 1:45 PM, lists at infoway.net wrote:
>>
>>> I've digested Ian and Leslie's comments on this thread and think
>>> we're all on the same page now. I've been reading different papers
>>> and other OODBMs in trying to propose a querying syntax and am
>>> pretty close to completing this now. However, I have some doubts for
>>> which I'll need someone's help with, which will allow me to continue
>>> drafting my proposal.
>>>
>>> I believe Ian (if it was someone else, I apologize for not quoting
>>> you) mentioned that we could, as a later phase, introspect the class
>>> and ask for the indices available in order to do some query
>>> optimizations. So my question goes somewhere along those lines and
>>> I'm wondering (please excuse my limited knowledge of Lisp and thank
>>> you for helping my learning curve) if it's possible to obtain the
>>> list of slots and/or slot-accessor functions for a given class.
>>>
>>> Thanks,
>>> Daniel
>>> _______________________________________________
>>> elephant-devel site list
>>> elephant-devel at common-lisp.net
>>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>>
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel