[elephant-devel] questions about persistent classes

[Robert L. Read]

On Wed, 2008-04-16 at 02:37 +0200, Ryszard Szopa wrote:
> Hi,
> 
> I've started using Elephant and I must say I am really surprised how
> easy and nice to use it is. However, there are some areas where I feel
> a bit lost, and I couldn't find the answers in the manual. I am using
> Elephant 0.9.1 with a BDB backend.

Thanks.

> 
> 1) What is the right way of doing something similar to a UNIQUE
> constraint in a relational database? The solution on top of my head is
> to write a method for `initialize-instance' that will check the
> existence of an object with the same value as the provided for the
> unique slot (which should of course be indexed), and raise a condition
> if yes.

We don't have a way to do a unique constraint, and I personally would
not want to see one.

The basic idea behind Elephant, IMHO, is that such things are generally
more easily, and certainly more powerfully, handled by LISP itself.  I
come from an "enterprise" background, where you typically are very
strict about separating the persistence layer (implemented via Elephant)
from the business rule layer.  So my normal style would be to write
something like:

(defun create-account (name initial-balance) ....)

which would signal "that name is already in use".  You would implement
that in LISP by just querying elephant first to see if the name really
is in use.  Thus it is the programmer's responsibility to maintain
constraints.  Some would argue they would prefer to have this done by
the DB.  If you prefer that, you might prefer using CL-SQL, which is
more of an "ORM" and works directly with a relational database.
Elephant is more of an "orm-zilla" --- an "orm killer".  The philosophy
behind Elephant is that you don't want to do object relational mapping
--- you just want persistent objects.

> 
> 2) What is the right way to do many-to-many relationships? At the
> moment I have slot with a pset, and add object A to the pset of object
> B and vice versa, in a transaction.

This is something that Ian has implemented in the "unstable" branch,
which is pretty stable under BDB but completely unusable (right now)
under any other back-end.

However, what I would do it (and have done, as an example in the
test-associations suite in the unstable branch) is to just create a new
object to represent the relationship, and then make that object a
persitent class, or poke it into a pset.  So you have:

(defpclass association ()
((from ...)
 (to ...))

Then you create one association object for each link in your graph.

> 
> 3) How do I implement something to do efficient (i.e. not O(n)) LIMIT
> and OFFSET queries? I was thinking of maintaining a counter (my
> intuition would be to use a class allocated slot, but I think that
> class objects are not serialized---am I right?) for classes where I
> need it, and using some method to update it. For decreasing the
> counter my candidate is drop-pobject, for increasing
> initialize-instance, but I am not sure about it (btw Elephant
> complains if you try to trace `initialize-instance').

Sine initialize-instance is used by lots of stuff, if you trace it it
will break your repl (at least in SBCL).  Class objects can be
serialized (I think), although since classes themselves can be
persistent via defpclass, I'm not sure what your planning there.  I
think closures, streams, threads, etc. are the only things that we can't
serialize.

One way to implement:
SELECT * FROM `student` LIMIT 20, 10 
is to open a cursor on "student", call cursor-next 20 times, then call
cursor-next and use the resulting value 10 times.  This is indeed O(n)
-- but don't knock it until you have tried it.

(If the query is really complicated, you can't do this because you can't
really create a cursor on a query -- only on btree, psets, classes, and
slots that you have indexed, I think.)

The other way is similar to what you are hinting at in your email ---
maintain your own mapping from "nth position" to the "key for the nth
position".  Then you can create a cursor at that value efficiently, and
start reading from there.  You could implement this as you are
suggesting, by implementing an :after method on shared-initialize or
initialize instance, but I'm not sure why it wouldn't be cleaner to
force object creating through your own function "create-student" that
would doing the counting or indexing you need.

This may seem like a lame answer --- but don't forget in elephant you
can create indices via ARBITRARY functions, and you have the elegance of
(mapclass #'(lambda (x) whateverICanthinkOf) 'mygnarlyclass)



> 
> 4) What is the intended use of `add-class-derived-index'? I must say
> that the documentation isn't very clear about it (specially figuring
> out what is a valid `derived-defun' wasn't trivial). Tight now I think
> I am supposed to run it once (it complains about a duplicate index if
> run a second time). This isn't exactly handy, because I would prefer
> that defining a derived index to left a clear trace in my code. Right
> now I am calling it wrapped with `handler-case'.

Uhhh....Ian will have to answer that.

> 
> Finally, how stable is the unstable version of Elephant and what are
> the most important changes that have already been implemented? I am
> specially interested in the query language part.

I would say it's stable for experimentation but not stable for actually
kicking off a project where you expect the data to survive the upgrade
to the formal release; and only the BDB back-end works.  However, you
get associations for free there, as Ian has done them very elegantly.
We are not planning any features before the next formal release, we are
just trying to get the other backends to work.  In that sense it is in
"feature-freeze".

I can't really speak to the query language part; I'm a died-in-the-wool
"Prevalence" model advocate.  However, Ian and others like the idea of a
query language, and are doing some work to develop one.  I personally
would rather see an IO efficient implementation of map-reduce,
map-class, etc., and would argue that the query language should work
with LISP rather than in place of it wherever possible.

> 
> Thanks in advance,
> 
>     -- Richard