[elephant-devel] Representational Question

Thu Mar 6 05:50:37 UTC 2008

I'd like to add my two cents, since I'm in a somewhat similar position
- RoR programmer looking at replacing RoR with an object database and
Lisp.

Ruby (and PHP, Java, and almost all other languages commonly used for
web programming) are object-oriented, while relational database
systems are not. I find that (as a wild estimate) perhaps 30-40% of
the time I spend writing the average web app is spent working out
tedious, repetitive bugs in the relational-to-object mapping layer -
the classic http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch
.

The main draw for me with a system like Elephant is that I can make
objects that (from the perspective of the application) Just Work(tm).
I make them and they're saved. That sounds brilliant and beautiful. No
more dorking around with the object-relational bridge; I can focus my
time on the logic required by the application, rather than
housekeeping chores related to the database.

However, like Waldo, it's slow going wrapping my head around the
object database paradigm, as we've been steeped in RDBMS thinking for
so long. As I read this discussion (glossing over the details), the
correct way to find "all users who live in ZIP code 12345 with Health
Plan A who are male and have a male cardiologist" is to simply check
every single user object and its linked subobjects to see whether they
satisfy the conditions. This is perhaps less efficient than an RDBMS
query, but I am perfectly willing to take that kind of performance hit
in many cases, as my databases are often not huge, and CPU time is
cheaper than engineer time - if it saves me a lot of time coding the
app, I can wait a few microseconds more for the result.

I think the thing is not that we need professional consulting, but we
are trying to become the professional consultants. :) Could you more
knowledgable ODBMS coders recommend some good books or papers on the
topic of ODBMS care and feeding that we should read to get a better
handle on all this?

Cheers,
Paul

On Mon, Mar 3, 2008 at 6:52 AM, lists at infoway.net <lists at infoway.net> wrote:
> Leslie,
>
>  Thanks for your answers and the clarifications. I think you're right
>  in that we may need some consulting here. However, we're trying really
>  hard to get into this and become proficient at it. As such, we would
>  probably be interested in some consulting to help us get started so as
>  to give us still a challenge to move forward. Sort of like kicking us
>  off in the right direction. Since this is not a requirements for our
>  client but more of an internal desire to move in this direction, we
>  don't really have a budget (or remunerable need) to do this.
>  Nonetheless, I'd like to discuss this offline with whomever is
>  interested in lending a hand (for a fee of course) at this stage, and
>  potentially be available as we move along the project.
>
>  Thanks,
>  Waldo
>
>
>
>  On Mar 3, 2008, at 4:44 AM, Leslie P. Polzer wrote:
>
>  >
>  >> Since we all have a very strong and hard-headed background on MySQL
>  >> and relational models, it's been extremely difficult for us to
>  >> migrate
>  >> away from that mentality and think of objects and some of Elephant's
>  >> terminology such as class indexes, which kind of confuse us into
>  >> thinking that a class index allows us to look at a set of objects
>  >> in a
>  >> similar way as a MySQL table.
>  >
>  > No. A class itself does that. The indexing mechanism is just a way
>  > of automatically storing every new instance and gathering those
>  > instances.
>  >
>  > A class slot index is actually pretty much the same as a column index
>  > in SQL.
>  >
>  >
>  >> I've read and seen in the src the beginning efforts to building a
>  >> query system into Elephant. That would be great and as our efforts
>  >> approach that phase, we hope to contribute to it.
>  >
>  > You can use MAP-* with appropriate functional queries until then.
>  > It's different from SQL queries. More than one leven of sorting
>  > would have to be implemented by you.
>  >
>  >
>  >> So, in this email, first I will ask for advise as to how to best
>  >> represent the structure of our objects/classes and indices in
>  >> Elephant
>  >> in order to ultimately be able to query the data. Again, I'm not
>  >> going
>  >> to ask for the querying strategy (just yet) but ultimately, we will
>  >> need to be able to answer queries like this. Obviously I don't expect
>  >> anyone to give me the full representation of this, but any advise/
>  >> hints as to best represent them will help greatly.
>  >
>  > Looks like you could make use of some professional consulting here.
>  >
>  >
>  >> The way we see it, the concept of tables disappears
>  >
>  > As such yes. But you can model classes with tables and vice versa.
>  >
>  >
>  >> and so do the
>  >> tables that provide many-to-many joins. So, we end up with some
>  >> classes such as "Person" which contains a reference to a list of
>  >> "Address" objects, and a list of preferred "Medical-Office" objects,
>  >> where each Medical-Office object has a list of Doctor objects and
>  >> each
>  >> Doctor has a list of Specialty objects, etc, etc.
>  >
>  > Yes.
>  >
>  >
>  >> Now, we assume that each of these classes will need to maintain
>  >> multiple indices, such as the Person class being index on first name,
>  >> last name, dob, gender, among others. The Address class indexed on
>  >> zip
>  >> code, county name, among others, and so on and so forth.
>  >
>  > If you like it simple (and non-performant), you don't need any indices
>  > at all. Just use MAP-CLASS.
>  >
>  >
>  >> The querying is one problem. The data representation is another. We
>  >> think it's clear that we should have, as an example, a Person class.
>  >> However, the representation of the links between a Person and its
>  >> Addresses or Medical-Offices is not 100% clear. If we represent them
>  >> as a slot in the Person class, where this slot would be a List or a
>  >> set of references to the Address class, then in order for us to query
>  >> on those, means that we always need to fetch all objects in those
>  >> slots in order to apply any search criteria, which seems like a
>  >> bottleneck. If that was the solution, I assume we could implement
>  >> logic such that Addresses are pushed into the list, so that the most
>  >> recent address is in the CAR, so we wouldn't necessarily need to read
>  >> the entire list of Addresses for each member, but just fetch the CAR
>  >> of the slot.
>  >
>  > Now here you should use btree indices [1].
>  >
>  >
>  >> Now, onto the second question.
>  >
>  > Yuk. Seems there's a bunch of separate issues in this "second
>  > question".
>  > I'll try to handle them separately.
>  >
>  >
>  >> One of the other requirements we have
>  >> is that we need to keep an audit log of data changes.
>  >
>  > You can have a simple log with some trivial MOP changes.
>  >
>  >
>  >> From what we've read in Elephant's manual, this seems harder because
>  >> we don't want to work directly off the Elephant object but a memory
>  >> copy while the user takes his/her time in the browser and after
>  >> submitting, we would take the changes and commit them to the Elephant
>  >> object.
>  >
>  > Sounds sensible, yes.
>  >
>  >
>  >> Makes me think that we would need to classes for each object
>  >> (one with and one without the persistent metaclass).
>  >
>  > No, not necessarily. New instances only get added to the persistent
>  > storage when they are of indexed classes. Otherwise you need
>  > an explicit ADD-TO-ROOT.
>  >
>  > If you need both indexing and this fine-grained behaviour,
>  > use your own indexing stuff with non-indexed objects.
>  >
>  >
>  >> The other problem
>  >> would be how to "easily" have two objects introspect themselves and
>  >> spit out the slots that changed between the two.
>  >
>  > Use the MOP facilities.
>  >
>  > An alternative solution to the whole problem is off-loading this
>  > to the client. Have JavaScript record which fields changed.
>  >
>  >  Leslie
>  >
>  >
>  >> Are we looking at this incorrectly? Any advise would be greatly
>  >> appreciated.
>  >
>  > You seem to be a bit confused. Try hands-on experiments.
>  >
>  >  :) Leslie
>  >
>  >
>  > [1] http://common-lisp.net/project/elephant/doc/BTree-Indexing.html#BTree-Indexing
>  >
>  > _______________________________________________
>  > elephant-devel site list
>  > elephant-devel at common-lisp.net
>  > http://common-lisp.net/mailman/listinfo/elephant-devel
>
>  _______________________________________________
>  elephant-devel site list
>  elephant-devel at common-lisp.net
>  http://common-lisp.net/mailman/listinfo/elephant-devel
>