[elephant-devel] Quick sketch of type evolution system

[Robert L. Read]

I personally think something like what you are proposing is the
direction in which we need to go.  I build a schema-revisioning system
for a SQL schema system at my previous work; it works along the lines
that you suggest, although in LISP it could be a lot better.

Of course, there are schema changes which are invisible to the class
system, that also need to be accounted for.  For example, you represent
user status as an integer (representing an enumerated type.)  You decide
to remove a status.  The types haven't changed, but the entire schema
has to be massaged into the new set of enumerated types.

I think the lack of schema evolution is Elephant's biggest problem; but
is there any system that has solved this problem?  If you use an ORM,
your problems are even worse---you explicitly have to change the
object-relational mapping with an class change, and you still have to
have a way of updating the system.  If the basic problem of schema
evolution has been solved by somebody, I would be happy to use their
solution as inspiration.

Like you, I don't have time to seriously tackle this problem right
now---in fact I am still (sigh) debugging the "allocate-instance" patch
under postmodern that was submitted, which has very subtle symptoms.

On Fri, 2008-01-04 at 13:27 +0100, Henrik Hjelte wrote:
> I am dreaming up something I would like to see, it is perhaps obvious
> and I haven't thought out all details but anyway:
> 
> Schemas are stored in the elephant database. Each class has an
> associated schema class that stores a number of class-versions. Each
> class-version has a number and a list of slot definitions. 
> 
> Each persistent class instance has a way to identify the schema and
> actual class-version. For example a class-version-nr, and the schema
> can be identified from the class-name. (Optional feature: it would be
> nice if there was a way to rename classes and keep the persistent
> data, so link the version 1 of my-renamed-class to version 7 of
> my-old-class) 
> 
> When a class definition is updated, a new class-version is created.
> New instances get the latest class-version-nr.
> For each retrieval of an instance, or slot-value of an instance, the
> version-nr of the instance is checked. If it is outdated, a generic
> function is called to update it. Something along class-evolve, but I
> like long names like
> update-persistent-instance-for-class-redefinition. It is up to the
> user to keep code to update between versions. If you want to do lazy
> updates, you will have to keep the old update code in your system for
> ever. 
> 
> There could be a convenience function called update-all-instances that
> takes a class-name and does upgrades, after calling it the
> class-versions objects of older versions may get a
> known-to-be-obsolete value so
> you will be able to know that you can drop update code safely.
> 
> With this schema, you would never actually read from old versions.
> Every time you query an instance of an old version, the whole instance
> should be updated slot for slot. So the slot-value-using-class
> suggested would be unnecessary, there would be no need to upgrade slot
> types in this function, it should already be done using
> update-persistent-instance-for-class-redefinition. 
> 
> But, since I personally don't have the time to do this grand plan (now
> at least), I don't object to the slot-value-using-class updates Leslie
> proposes. It does not feel like a complete and perfect solution, but
> won't do no big harm either. But for me, until Elephant gets a
> complete and proper schema evolution facility I won't trust it to
> handle lazy updates to slots or objects. So a performance switch to
> turn it off will be nice. Proper schema handling is perhaps elephants
> greatest weakness? 
> 
> Best wishes, 
> Henrik Hjelte
> 
> 
> 
> On Jan 4, 2008 11:43 AM, Leslie P. Polzer <leslie.polzer at gmx.net>
> wrote:
>         
>         > to clarify one of the points that are important to me,
>         > I have prepared this working sketch:
>         >
>         
>         > [...]
>         
>         Any reason why no one commented on this?
>         I'd appreciate some feedback. 
>         
>         
>          Leslie
>         
>         --
>         My personal blog: http://blog.viridian-project.de/
>         
>         _______________________________________________ 
>         elephant-devel site list
>         elephant-devel at common-lisp.net
>         http://common-lisp.net/mailman/listinfo/elephant-devel
>         
>         
>         
> 
> 
> 
> -- 
> Henrik Hjelte
> henrik.hjelte at stix.to
> +46703993945
> http://stix.to 
> Lästmakarg 18-20 (IQube)
> Box 7438
> S-103 91 Stockholm
> Sweden 
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel