<div dir="ltr">On Thu, Dec 20, 2012 at 3:59 PM, Anton Vodonosov <span dir="ltr"><<a href="mailto:avodonosov@yandex.ru" target="_blank">avodonosov@yandex.ru</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">29.11.2012, 01:21, "Juan Jose Garcia-Ripoll" <<a href="mailto:juanjose.garciaripoll@gmail.com">juanjose.garciaripoll@gmail.com</a>>:<br>
<div class="im">> On Mon, Nov 26, 2012 at 11:06 PM, Anton Vodonosov <<a href="mailto:avodonosov@yandex.ru">avodonosov@yandex.ru</a>> wrote:<br>
>> I see that according reports situation is not always improving with commits.<br>
><br>
> It turns out that the code I used for sorting reports was broken! Now I fixed the code that separates dates from file names and the comparison is way better :-)<br>
><br>
> <a href="http://ecls.sourceforge.net/reports-generated/ecl/index.html" target="_blank">http://ecls.sourceforge.net/reports-generated/ecl/index.html</a><br>
<br>
</div>I look at the reports and see that changes between two consecutive versions<br>
sometimes have regressions, sometimes things are improved.<br></blockquote><div><br></div><div style>Indeed. There are reappearances which are some mysterious and I believe there is some non-reproducibility of some results. There is also the fact that along the table a new quicklisp version was introduced and this broke a lot of things. There was also another bug in cffi which broke further things. In such circumstance the utility of the tests is questionable.</div>
<div style><br></div><div style>And there are sometimes when I can hardly interpret the tables generated. See for instance <a href="http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121207231628.html">http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121207231628.html</a> which comes right after <a href="http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121206231812.html">http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121206231812.html</a></div>
<div style>I have the impression that the code that generates the tables is still somehow broken.</div><div style><br></div><div style>There are also sometimes when cl-test-grid reports OK and this is not so. See <a href="http://cl-test-grid.appspot.com/blob?key=854116">http://cl-test-grid.appspot.com/blob?key=854116</a> I believe this is another source of "regressions"</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">But it's difficult for me to see the overall direction of quality change.<br>
How do you monitor it?<br></blockquote><div><br></div><div style>Right now I don't. The last month I have not monitored the cl-test-grid tests because I have focused on open bug reports.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
I.e. as soon as we reach new level of quality - next dev version without<br>
regressios but with improvements - we capture this quality level by shifting<br>
base version to it, and do not allow the quality go back.<br>
Thus trying to keep quality a non-decreasing function of time.<br></blockquote><div><br></div><div style>In an ideal world, indeed this would be true. In practice I am finding that the evolution of libraries introduces itself the possibility of going back. Moreover, some bug solutions lead to regressions but in that situation I cannot afford not to fix the bug.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">I've have tried this approach when monitored the ABCL development<br>
towards the new release. [...] If you want, I can help you to setup this procedure for ECL.<br></blockquote><div><br></div><div style>Help is always welcome.</div><div style><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
Probably the table may be extended also with columns for bytecode version of ECL.<br></blockquote><div><br></div><div style>Right now this is not of the greatest priority to me. Getting quicklisp to work with the compiled version is _the_ priority, specially when there are many libraries that cannot work with the interpreter because of the features they use.</div>
<div style><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">P.S.<br>
I've fixed small bug in test-grid-reporting code, so that this report will have two columns:<br>
<a href="http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121207231628.html" target="_blank">http://ecls.sourceforge.net/reports-generated/ecl/ecl-diff-lisp-to-c-20121207231628.html</a><br>You may want to git pull to get this fix.<br>
</blockquote></div><br>The reports do "git pull" every night.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Best</div><div class="gmail_extra"><br></div><div class="gmail_extra">Juanjo<br clear="all">
<div><br></div>-- <br>Instituto de Física Fundamental, CSIC<br>c/ Serrano, 113b, Madrid 28006 (Spain) <br><a href="http://juanjose.garciaripoll.googlepages.com" target="_blank">http://juanjose.garciaripoll.googlepages.com</a>
</div></div>