[elephant-devel] Quick sketch of type evolution system

Ian Eslick eslick at csail.mit.edu
Fri Jan 4 17:01:01 UTC 2008 

I'm dreaming the same dream as Henrik.  As I mentioned earlier some of  
this was captured in the Trak wiki and tickets (I referenced which  
ones in an earlier e-mail).

I'm completely swamped for a couple of weeks, but I might be motivated  
to work on this later in the month if one or two someone others  
volunteer to participate.

I don't think it's a ton of code, but like most things using the MOP,  
there are a ton of subtle issues to be thought through.

Ian

PS - Just to put this idea into the ether:

One thought for solving the offline repository problem where I change  
a schema while the repository is disconnected such that no schema  
version is created.  One approach is to create a signature hash using  
all the data available in the MOP.  When a class object is created by  
the MOP, it stores this signature.  Then, when a repository connects,  
it verifies that all the latest schemas in the database match the  
signatures of the classes in memory.  If not, it can warn the user or  
take some other configurable action.

update-instance-for-redefined-class would need to call some subsidiary  
function defined by the user on schema version ids, or signatures,  
because the generic function dispatch doesn't know about schema  
versions, just about the prior and the current class objects in memory.



On Jan 4, 2008, at 11:39 AM, Robert L. Read wrote:

> I personally think something like what you are proposing is the
> direction in which we need to go.  I build a schema-revisioning system
> for a SQL schema system at my previous work; it works along the lines
> that you suggest, although in LISP it could be a lot better.
>
> Of course, there are schema changes which are invisible to the class
> system, that also need to be accounted for.  For example, you  
> represent
> user status as an integer (representing an enumerated type.)  You  
> decide
> to remove a status.  The types haven't changed, but the entire schema
> has to be massaged into the new set of enumerated types.
>
> I think the lack of schema evolution is Elephant's biggest problem;  
> but
> is there any system that has solved this problem?  If you use an ORM,
> your problems are even worse---you explicitly have to change the
> object-relational mapping with an class change, and you still have to
> have a way of updating the system.  If the basic problem of schema
> evolution has been solved by somebody, I would be happy to use their
> solution as inspiration.
>
> Like you, I don't have time to seriously tackle this problem right
> now---in fact I am still (sigh) debugging the "allocate-instance"  
> patch
> under postmodern that was submitted, which has very subtle symptoms.
>
> On Fri, 2008-01-04 at 13:27 +0100, Henrik Hjelte wrote:
>> I am dreaming up something I would like to see, it is perhaps obvious
>> and I haven't thought out all details but anyway:
>>
>> Schemas are stored in the elephant database. Each class has an
>> associated schema class that stores a number of class-versions. Each
>> class-version has a number and a list of slot definitions.
>>
>> Each persistent class instance has a way to identify the schema and
>> actual class-version. For example a class-version-nr, and the schema
>> can be identified from the class-name. (Optional feature: it would be
>> nice if there was a way to rename classes and keep the persistent
>> data, so link the version 1 of my-renamed-class to version 7 of
>> my-old-class)
>>
>> When a class definition is updated, a new class-version is created.
>> New instances get the latest class-version-nr.
>> For each retrieval of an instance, or slot-value of an instance, the
>> version-nr of the instance is checked. If it is outdated, a generic
>> function is called to update it. Something along class-evolve, but I
>> like long names like
>> update-persistent-instance-for-class-redefinition. It is up to the
>> user to keep code to update between versions. If you want to do lazy
>> updates, you will have to keep the old update code in your system for
>> ever.
>>
>> There could be a convenience function called update-all-instances  
>> that
>> takes a class-name and does upgrades, after calling it the
>> class-versions objects of older versions may get a
>> known-to-be-obsolete value so
>> you will be able to know that you can drop update code safely.
>>
>> With this schema, you would never actually read from old versions.
>> Every time you query an instance of an old version, the whole  
>> instance
>> should be updated slot for slot. So the slot-value-using-class
>> suggested would be unnecessary, there would be no need to upgrade  
>> slot
>> types in this function, it should already be done using
>> update-persistent-instance-for-class-redefinition.
>>
>> But, since I personally don't have the time to do this grand plan  
>> (now
>> at least), I don't object to the slot-value-using-class updates  
>> Leslie
>> proposes. It does not feel like a complete and perfect solution, but
>> won't do no big harm either. But for me, until Elephant gets a
>> complete and proper schema evolution facility I won't trust it to
>> handle lazy updates to slots or objects. So a performance switch to
>> turn it off will be nice. Proper schema handling is perhaps elephants
>> greatest weakness?
>>
>> Best wishes,
>> Henrik Hjelte
>>
>>
>>
>> On Jan 4, 2008 11:43 AM, Leslie P. Polzer <leslie.polzer at gmx.net>
>> wrote:
>>
>>> to clarify one of the points that are important to me,
>>> I have prepared this working sketch:
>>>
>>
>>> [...]
>>
>>        Any reason why no one commented on this?
>>        I'd appreciate some feedback.
>>
>>
>>         Leslie
>>
>>        --
>>        My personal blog: http://blog.viridian-project.de/
>>
>>        _______________________________________________
>>        elephant-devel site list
>>        elephant-devel at common-lisp.net
>>        http://common-lisp.net/mailman/listinfo/elephant-devel
>>
>>
>>
>>
>>
>>
>> -- 
>> Henrik Hjelte
>> henrik.hjelte at stix.to
>> +46703993945
>> http://stix.to
>> Lästmakarg 18-20 (IQube)
>> Box 7438
>> S-103 91 Stockholm
>> Sweden
>> _______________________________________________
>> elephant-devel site list
>> elephant-devel at common-lisp.net
>> http://common-lisp.net/mailman/listinfo/elephant-devel
>
> _______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel

More information about the elephant-devel mailing list