[elephant-devel] patches for 1.0

[Ian Eslick]

Pushed to elephant-1.0.  -Ian

On Feb 6, 2009, at 6:49 AM, Alex Mizrahi wrote:

> ;; gmane was eating message with patch attached without forwarding  
> it to mailing list, so
> ;; i'm now trying with email
>
> here's bunch of patches for 1.0:
>
> * postmodern documentation update
>
> i've updated documentation in postmodern-backend.texinfo.
> i'm not really good at texinfo, so it might be not pretty, but at  
> least it
> covers important aspects.
> upgrade is not yet documented.
>
> by the way, i've noticed that postmodern-backend.texinfo is not  
> included
> into documentation, and neither does sql-backend.texinfo, by the  
> way, so
> this stuff is not included into documentation. weird.
> i did not fix it because i'm not sure what were intentions of having
> different pieces of documentation. (and also i'm not sure how to  
> cope with
> links).
>
> also i've found i cannot build documentation on my machine, includes
> directory is empty, and makeinfo complains, is it ok?
>
> * db-postmodern: proper transaction handling
>
> *current-transaction* was not bound properly in execute-transaction
>
> * db-postmodern: dup btree value type
>  there were some issues with value types in dup btrees..
>
> * db-postmodern: btree upgrade handling, btree wrappers. WARNING:
> incompatible store format!
> * tests for mixed-type btrees
>  tests for former patch
> * db-postmodern: hardened type handling
>  there was some mess with types where db-postmodern could accept some
> values where it should not.
>  now it's pretty strict with this, but it might signal errors if you  
> were
> using it in messy
>  way (like looking for a string where all values are integers), be
> careful.
> * db-postmodern: cursor semantics
>  cursor-delete now does not close cursor
>
> -----
>
>
>
> btree upgrade handling is what i was fighting with. the thing is  
> that in
> db-postmodern btree keys and values are (separately) typed, key-type  
> can be
> one of four values:
> * nil -- btree is empty
> * integer -- all keys are integers
> * string -- all keys are strings
> * object -- keys are stored indirectly and can be of any type (in  
> this case
> ranged queries and sorting do not work).
>
> sometimes btree key type might need to be upgraded -- from nil to  
> integer or
> string or object, and from integer or string to object. second type of
> upgrades were not working at all, but after i've fixed that, i've  
> found out
> there are more problems..
>
> the main problem is that state of btree (its table, queries and  
> types) is
> loaded only when btree is loaded and never updated in process. this  
> is a
> problem if multiple processes are connected -- one process could  
> upgrade
> btree, and another would not know about it. this got fixed via  
> catching
> errors in execute-prepared-statement function and updating btree  
> state when
> errors of certain type are detected. (there are some more subtleties  
> in
> this.)
>
> another problem is that btree state is shared by all threads in  
> process, and
> if one thread tries to update state, other might be confused. i've  
> solved
> this problem like this -- now btree states are recorded per  
> connection (that
> is associated with thread) rather than globally, so it is ok to  
> update this
> state.
>
> unfortunately, to implement this correctly i had to change store  
> format a
> bit -- now metatree table has additional bt_oid column.
> it should be easy to upgrade from old format to new with a few SQL
> statements, but i dunno if anybody needs this, so i did not
> bother checking this so far.
>
> there were lots of changes and there are lots of different cases in  
> this
> issues, so i'd appreciate if somebody helps testing this.
> particularly it would be great to test btree upgrades -- that is,  
> you create
> btrees in one process and write some values of different
> types into it, while other processes/threads load these btrees and  
> try to
> use it in various way.. stuff like this.
> or it would be at least nice to know that new changes did not broke  
> some
> functionality -- it passes all tests, but who knows.
> <patchbundle.hunks.gz>_______________________________________________
> elephant-devel site list
> elephant-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/elephant-devel