[elephant-devel] patches for 1.0

[Alex Mizrahi]

;; gmane was eating message with patch attached without forwarding it to 
mailing list, so
;; i'm now trying with email

here's bunch of patches for 1.0:

* postmodern documentation update

i've updated documentation in postmodern-backend.texinfo.
i'm not really good at texinfo, so it might be not pretty, but at least it
covers important aspects.
upgrade is not yet documented.

by the way, i've noticed that postmodern-backend.texinfo is not included
into documentation, and neither does sql-backend.texinfo, by the way, so
this stuff is not included into documentation. weird.
i did not fix it because i'm not sure what were intentions of having
different pieces of documentation. (and also i'm not sure how to cope with
links).

also i've found i cannot build documentation on my machine, includes
directory is empty, and makeinfo complains, is it ok?

 * db-postmodern: proper transaction handling

  *current-transaction* was not bound properly in execute-transaction

 * db-postmodern: dup btree value type
   there were some issues with value types in dup btrees..

 * db-postmodern: btree upgrade handling, btree wrappers. WARNING:
incompatible store format!
 * tests for mixed-type btrees
   tests for former patch
 * db-postmodern: hardened type handling
   there was some mess with types where db-postmodern could accept some
values where it should not.
   now it's pretty strict with this, but it might signal errors if you were
using it in messy
   way (like looking for a string where all values are integers), be
careful.
 * db-postmodern: cursor semantics
   cursor-delete now does not close cursor

-----



btree upgrade handling is what i was fighting with. the thing is that in
db-postmodern btree keys and values are (separately) typed, key-type can be
one of four values:
 * nil -- btree is empty
 * integer -- all keys are integers
 * string -- all keys are strings
 * object -- keys are stored indirectly and can be of any type (in this case
ranged queries and sorting do not work).

sometimes btree key type might need to be upgraded -- from nil to integer or
string or object, and from integer or string to object. second type of
upgrades were not working at all, but after i've fixed that, i've found out
there are more problems..

the main problem is that state of btree (its table, queries and types) is
loaded only when btree is loaded and never updated in process. this is a
problem if multiple processes are connected -- one process could upgrade
btree, and another would not know about it. this got fixed via catching
errors in execute-prepared-statement function and updating btree state when
errors of certain type are detected. (there are some more subtleties in
this.)

another problem is that btree state is shared by all threads in process, and
if one thread tries to update state, other might be confused. i've solved
this problem like this -- now btree states are recorded per connection (that
is associated with thread) rather than globally, so it is ok to update this
state.

unfortunately, to implement this correctly i had to change store format a
bit -- now metatree table has additional bt_oid column.
it should be easy to upgrade from old format to new with a few SQL
statements, but i dunno if anybody needs this, so i did not
bother checking this so far.

there were lots of changes and there are lots of different cases in this
issues, so i'd appreciate if somebody helps testing this.
particularly it would be great to test btree upgrades -- that is, you create
btrees in one process and write some values of different
types into it, while other processes/threads load these btrees and try to
use it in various way.. stuff like this.
or it would be at least nice to know that new changes did not broke some
functionality -- it passes all tests, but who knows. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patchbundle.hunks.gz
Type: application/octet-stream
Size: 14787 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/elephant-devel/attachments/20090206/1e276f6e/attachment.obj>