[elephant-devel] Re: Derived Indicies

[Alex Mizrahi]

      IE>  :index t is not necessary - in fact it is ignored.  :slot-deps 
are
      also not required, but the derived index is updated on any slot write
      if that slot is not transient, set-valued or an association.  We can
      add those last three slot types into the mix if necessary, but I'm
      trying to avoid too much complex computation taking place during slot
      writes (self-deadlock, etc) for the time being.


seems to be fine..
actually we are using derived indices in quite special way -- to get index
that is ordered in special way, to do efficient lookups of some kind.
suppose we have a messaging system with two persistent classes -- user and
message:

(defpclass user ()
     ((username :index t)
     ...)

(defpclass message ()
    ((from-user :accessor from-user :index t)
     (to-user :accessor to-user :index t)
     (text :accessor text-of :index t)
     (modification-time :accessor modification-time-of :index t)))

suppose we'd like to get inbox and outbox views for user, i.e. list of 10
latest messages to user or from user.
with considerable amounts of users and messages it is not efficient to get
these latest messages from any of normal indices, as it requires scanning
potentially large number of messages.

it's possible to make efficient lookups via derived indices -- messages
ordered first by user, then my modification time. iirc cons sorting in
elephant has desired characteristics, but unfortunately postmodern backend
does not support complex type sorting.

but we happily can reduce problem to sorting strings, e.g. "13_31321433"
where 13 is oid of from-user  and 31321433 is modification time (universal
time). (actually we have user-id field instead of oid to preserve identity
across multiple stores etc).

thus, via two derived indices we can efficiently list messages for inbox and
outbox. (via cursor operations, that's not very easy, but works fine).

as i understant you're going to make elephant more high level, and possibly
such low level index operations can be replaced with some high level
concept.
are you planning something like these dual indices?

if there will be no special functionality for these, it would make sense to
make derived indices somewhat more flexible. for example, i suspect some
uses might require "foreign slot dependency".
like if user definition above had group field, and we'd like to build
messages-by-group index. when user-group changes, all messages of this user
should update derived index.
probably there's no need in this stuff right now, but it would be nice if it
will be possible to add it in future, so advanced indices can be built