[elephant-devel] Lisp-only data store (prototype)

[Ian Eslick]

The idea is (over time) to move to a higher level of abstraction that  
gives more freedom to data store implementations.  Currently we  
require all data stores to implement the cursor API and I was thinking  
that eventually that may not be a necessary restriction - or we could  
export a tree walking API at a higher level and be less btree-specific.

But this is still in the realm of idle speculation - we have other  
fish to fry first!

Ian

On Feb 6, 2009, at 6:40 PM, John wrote:

> I was looking over at the map-inverted-index code and I see that it  
> calls map-btree method, which in turns calls map-btree-from-start or  
> map-btree-from-end functions. These last two functions seem to use  
> the cursor API.
>
> So the question I have is, are the intentions to "hide" the cursor  
> API and access them via these higher level functions or are the  
> intentions to make them disappear altogether and replace their  
> functionality with the query system?
>
> JD
>
> On Fri, Feb 6, 2009 at 5:03 PM, Ian Eslick <eslick at media.mit.edu>  
> wrote:
> Cursors are nice low level tools, for sure.  They also allows for more
> complex iteration over the index than the pure mapping function.  I'm
> hoping that we can hide most of these cases in the query system,
> eventually.  For 1.0 at least we'll be leaving them in and simply
> encourage most people to use the higher level abstractions.
>
> However, for this example you can hack it up with map-inverted-index
>
> (defun get-min-log-value (type) ;; type = { :min | :max }
>    (map-inverted-index
>       (lambda (x)
>        (return-from get-min-log-value x))
>       'log 'timestamp :from-end (when :max t)))
>
> Ian
>
>
>
>
> On Feb 6, 2009, at 4:21 PM, John wrote:
>
> > As we're still learning Elephant, we are not currently using
> > cursors. However, one thing I'm thinking is that we will definitely
> > have the following scenario:
> >
> > We define a class for storing log information, which is timestamped
> > and we create an index on the timestamp. The purpose for this is so
> > that we could "easily" get the MIN and MAX timestamp from a list of
> > a few million objects. Without looking at the implementation of
> > cursors and just looking at the documentation, I think it would be
> > easier (and possibly more efficient) to set a cursor on the index
> > and get cursor-first and cursor-last, instead of having to map-index
> > and the entire set of millions of objects.
> >
> > Would that qualify as a reasonable need of cursors?
> >
> > Thanks
> > JD
> >
> > On Thu, Feb 5, 2009 at 8:44 AM, Ian Eslick <eslick at media.mit.edu>
> > wrote:
> > Would map functions, as described by leslie, accomplish the same  
> thing
> > with a little rewrite?
> >
> > I would say that we want some way for users of a current store in
> > elephant-unstable to migrate to elephant-1.0 - we do have a solid  
> set
> > of postmodern users.  I'd be surprised if anyone is still using the
> > original 0.9.1 - but let's give it a few days and see if anyone
> > shouts.
> >
> > If you like I can look at migration in 0.9.1, but it will be a  
> little
> > bit before I can get to it.
> >
> > Would it make sense to include gp-export into elephant proper or  
> make
> > it an easy option to load/use?  The online migration is a bit of a
> > pain to maintain over certain changes to the way the stores work; it
> > would be nice if we had a path that was less prone to error to  
> bypass
> > code for complex backwards compatibility in the future if needed.
> >
> > Ian
> >
> > On Feb 5, 2009, at 6:36 AM, Alex Mizrahi wrote:
> >
> > > IE> How many people currently directly use the Btree/cursor
> > interface?
> > >
> > > we use cursors to iterate through multiple sequences in order,
> > e.g. we
> > > want, say 10 latest items that satisfy certain criteria, and
> > > candidates
> > > come from different indices.
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > elephant-devel site list
> > > elephant-devel at common-lisp.net
> > > http://common-lisp.net/mailman/listinfo/elephant-devel
> >
> >
> > _______________________________________________
> > elephant-devel site list
> > elephant-devel at common-lisp.net
> > http://common-lisp.net/mailman/listinfo/elephant-devel
> >
> > _______________________________________________
> > elephant-devel site list
> > elephant-devel at common-lisp.net
> > http://common-lisp.net/mailman/listinfo/elephant-devel
>
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel