ASDF CI (was Re: Jenkins expertise?)

[Mark Evenson]

> On Oct 16, 2020, at 06:21, Eric Timmons <etimmons at mit.edu> wrote:

[…]

> I'm a huge fan of Gitlab CI and was getting tired of running all tests
> locally whenever I made a change, so I started a Gitlab CI configuration
> for ASDF at
> https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/146 and also
> modified it a bit more today. The recent modifications are marking the
> upgrade tests as `allow_failure: true` (since in a previous email it
> sounded like the previous Jenkins config didn't even run them),
> increasing some timeouts in tests (I don't know what happened, but some
> tests started failing under CCL when run on cl.net runners), reordering
> things such that all regression tests ran first, and addressing a bug in
> the latest SBCL that was triggered by the cl.net infrastructure.
> 
> If you want to take some radically different approach than I did, that's
> great! I just wanted to make sure everyone knew that an 80% solution
> exists and didn't waste their time coming up with something
> substantially similar. I think the biggest things left on !146 are
> deciding how often to run the tests (every push? only on MRs?) and
> adding more runners for licensed implementations or OSes.

Wow!  This looks like much more than 80%, and its working today!

> 
> Speaking of OSes, my research group has a Mac mini we've been using for
> our own CI purposes. By no means is it a speed demon, but it's lightly
> used so I can set up another runner on it for ASDF.
> 
> As for the licensed implementations: I'm not sure of what the cl.net
> infrastructure looks like and how paranoid you want to be about
> protecting the license, but it seems the easiest way forward would be to
> add a new VM based runner with the implementation and license already
> installed in the VM, register that runner so that only the asdf/asdf
> project can use it, and tag it with the implementation name. We can then
> add a job that is triggered only if it's run in the asdf/asdf project
> and have it require that tag.
> 
> As I said in another message, this would mean that forks can't use that
> runner (and MR tests are, unfortunately, run in the CI context of the
> fork). If it's OK for master to occasionally break, the changes from
> forks could be merged directly to master and breakage dealt with after
> that. If you want master to never break because of licensed
> implementations, then someone with write bits on asdf/asdf would need to
> pull the branch from the fork into asdf/asdf first.
> 
> Either way, the person pulling into asdf/asdf would have to do a sanity
> check to make sure the license isn't being exfiltrated, but I don't
> think that's any change from the previous approach (unless you were
> previously running the tests without external network access!).

Thanks for the suggested way forward on Allegro/LispWorks.  We (i.e. CLF)
certainly want to be fairly thorough on protecting the licenses for the
commercial implementations.  If we do a good enough job with ASDF, we would
like to try to provide them as a potential service for any
gitlab.common-lisp.net project.

-- 
"A screaming comes across the sky.  It has happened before but there is nothing 
to compare to it now."