<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b>My fellow Elephants,</b></span></font></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">Unstable isn't unstable anymore! All BDB tests, including migration, for BDB/Mac/Allegro and BDB/Mac/SBCL are green as of today's checkin.</span></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">All major new features are implemented, including:</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><span class="Apple-style-span" style="font-family: Helvetica; font-size: 12px; font-weight: normal; "><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; ">- Instance map, class schema evolution and MOP compliance</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b>- New slot types</b></span></font></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b> - Cached read, write-through slots</b></span></font></div></span></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Hierarchical indexed slots</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Virtual, hierarchical derived indices</span></font></b></div><div></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Set-valued slots</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Many-to-1 and many-to-many slot associations</span></font></b></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Trivial query interface example (query.lisp)</span></div><div></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Migration and upgrade </span></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Partial test suite (basic association, indexing, migration, basic schema-evolution)</span></div><div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br></span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b>There are definitely holes in the test suite that need to be plugged and I'm sure that this will uncover bugs, particularly in the schema evolution, upgrade or association infrastructure. The steps needed to prepare this branch for the next release are:</b></span></font></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Integrate patches from the main repository </span></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> (Leslie's patch is the only one that I haven't already integrated into unstable, I think) </span></font></b></div><div></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">- Evaluate multi-threading issues for schema evolution </span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> (only one thread should be able to manipulate class objects at a time)</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">- Upgrade Postmodern and CLSQL data stores</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Support btrees with duplicate keys</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Some minor API additions for upgrade & bootstrapping</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">- Testing</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; "><b> - Expand testing for schema evolution (most complex/subtle bugs were there)</b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; "> - Validate upgrade procedure 0.9.1 -> 0.9.2</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> - Verify referential integrity (delete object, what happens to stale refs?)</span></font></b></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; "> - Standard tests for new features</span></div><div><br></div><div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Documentation of new features</span></div></div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br></span></font></b><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; ">I am tied up with work for the next two weeks. I'm happy to support bug fixes, lisp compatibility issues, etc - but progress will only be made for the remainder of March and early April if others step in to help.</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">Robert and I hope to integrate this work into another 0.9.x release in late April. I think this new functionality makes Elephant sufficiently feature-rich and robust that after some burn-in time we should consider packaging this into a 1.0 release that we can commit to support for the longer term. We can have a 1.1 development branch in which add major new features like an all-lisp data store or a query compiler as longer term projects.</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">There are a few features that could use attention that could, but need not, make it into the upcoming release:</span></div><div><br></div></div><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">- Online GC strategy</span></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b> Now that we have an oid table that maintains information for each object and is used to de-serialize a reference, we can implement facilities such as forwarding pointers, counts or marks that makes it possible to build an online persistent heap GC facility without an overly significant cost or code impact.</b></span></font></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; ">- Query language/interpreter</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; "><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px; "> Daniel Salama is thinking about the query syntax and is motivated to help implement something there. I'd be psyched to see an interpreter that extends my sketch to take good advantage of indices and associations.</span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">- System-level schema evolution</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> Robert is thinking through some system-level schema versioning and evolution ideas akin to the Postgresql notion of schemas, but neither of us has the bandwidth to implement this right now. The basic idea is to group a set of class schemas into a version set and to use these version tags to dispatch a generic-function that can override the default transformation of an instance from one schema version to the next. This would allow you to connect to an old DB with new code, call a global upgrade fn, and have everything converted in one go. </span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"> This would be an independent application layer so would not impact an upcoming release either way.</span></font></b></div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br></span></font></b><div><span class="Apple-style-span" style="font-family: 'Bitstream Vera Sans Mono'; font-size: 13px; font-weight: bold; ">Regards,</span></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">Ian</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><b><br></b></span></font></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">PS - I did some profiling of the unstable branch on BDB/Mac to see what effects different query strategies might have. It though some of you would be interested in this. This is preliminary and not well controlled, but the order of magnitude should be about right.</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">The objects described below are 5-slot objects with a mix of indexed, cached, transient, etc.</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">Persistent object creation: 3000 objects per second </span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">Persistent object reference deserialization w/ object instantiation: 10k per second</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">Persistent object reference deserialization of oids only: 40k per second</span></font></b></div><div><br></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;">This last # would be the key factor in handling queries over large object databases. Since we can instantiate using only an oid, we only need to instantiate objects we need. This should make things like counts and paging pretty efficient for moderately sized databases. Indexing, of course, will have a significant impact on the performance of query by reducing the number of manipulated OIDs.</span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><b><font class="Apple-style-span" face="'Bitstream Vera Sans Mono'" size="3"><span class="Apple-style-span" style="font-size: 13px;"><br class="webkit-block-placeholder"></span></font></b></div><div><br class="webkit-block-placeholder"></div></body></html>