Jenkins expertise?

[Eric Timmons]

Erik Huelsmann <ehuels at gmail.com> writes:

> Hi,
>
> On Wed, Oct 14, 2020 at 5:51 PM Eric Timmons <etimmons at mit.edu> wrote:
>
>> https://gitlab.common-lisp.net/asdf/asdf/-/merge_requests/146 uses
>> cl.net's shared Gitlab CI runners. It appears that the shared runner can
>> only run one job at a time, so the pipeline that MR introduces takes
>> close to an hour to run (and that's only testing with four
>> implementations). I also suspect no pipelines from other projects were
>> running at the time.
>>
>
> I've increased the number of jobs this runner can run concurrently to 3,
> because the VM currently has 3 cores configured. If we need to configure
> more, let me know and I'll have a look at it.
>

Awesome! That seems to have helped, just not as much as I had hoped
(down to 45 minutes). I think at this point the issue is there's one job
that takes a long time (ABCL upgrade tests @ 33 minutes) and it's one of
the later jobs to run. I'll talk about that more in another reply,
though.

>> If the CLF or someone else can provide additional runners for ASDF's
>> use, that would be a nice way to cut down on pipeline time and avoid
>> impacting other projects.
>>
>
> There is also a machine provisioned for cl-test-grid. It's idle often, just
> not when a new Quicklisp release has been released. If cl-test-grid could
> be driven by GitLab CI,  it would be easy to share the resources this
> machine has to spare. In that case, I think 2 extra cores would be
> available for running tests. It would be even better if others would step
> up to donate their cores.
>
>
>> However, as much as I love Gitlab CI, the "core" tier of it (what I
>> /believe/ cl.net runs) does have some limitations that, given my limited
>> Jenkins knowledge, a properly configured Jenkins setup wouldn't have.
>>
>> First, the core tier is unable to test against the merged result of an
>> MR.
>
>
> I'm not sure what you mean by this. Do you mean that the CI wouldn't run
> tests/builds on the master branch after merging an MR?
>
> What I do know that works: The current common-lisp.net site is being built
> after each merge to the master branch of that repository and the artifacts
> of the build used to publish the site.
>

Sorry, I should have been more clear. I meant you can't test against the
merged result before actually performing the merge (this feature:
https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/).
Again, I don't think it's critical to have, but it is super annoying if
a test passes for the MR but then immediately breaks once it hits
master.

>> I don't think that one's a deal breaker as any CI is better than
>> none at all, issues are unlikely if the merge has no conflicts, and any
>> issues would be quickly found after the merge completed (and either
>> fixed or rolled back). Alternatively, ASDF could enforce (via Gitlab)
>> that MRs need to be rebased on to the target branch, effectively making
>> the source branch equivalent to the merged result.
>>
>> The tougher one to work around is caused by the forking model of
>> contributing code. Forks are unable to use their parent's CI runners and
>> the pipelines for MRs from forks are run in the CI context of the
>> fork. So if ASDF has special runners for the licensed implementations
>> they can't be used until the MR is merged. Additionally, the pipelines
>> before merging will be slow as they'll be using the cl.net shared CI
>> runners.
>>
>
> If we can disable artifacts on the ASDF runners, we might be able to use a
> somewhat more liberal assignment of runners to project members: if the
> artifacts can't be extracted, the options for abuse will become more
> limited.
>

I haven't looked into this a lot, but I think it's *possible* to disable
artifact uploads for certain runner configurations (basically any
configuration that uses Docker images). It's nasty, though, and would
require overriding the helper image used to set up and tear down the
testing environment.

It's also not clear to me that blocking artifact extraction would be
sufficient (assuming you're concerned about exfiltrating license
info). It's been a while since I had an Allegro license, but IIRC, it's
a relatively small plain text file that could just be printed into the
job log or curl'ed away to some remote computer.

>
>> There are a couple of ways to deal with the second issue, but they all
>> have their drawbacks. The best way would likely be to pull contributions
>> from forks into a separate branch in the parent repo before merging to
>> master. That would allow any ASDF specific runners to do their job, but
>> would require some extra steps from the maintainers. (And contributors
>> would just have to deal with their jobs running slowly before the
>> merge). I'm sure a sufficiently motivated person could even write a
>> Gitlab CI job that auto merged to master from this testing branch if the
>> tests all pass.
>>
>
>
> That would work too. Let me know what more we can research or I can help
> with to help solve this issue.
>
>
>>
>> -Eric
>>
>
> Regards,
>
> Erik.