Re: [libvirt PATCH 4/5] gitlab: add several native CI jobs
On Mon, 2020-03-23 at 15:16 +0000, Daniel P. Berrangé wrote:
On Mon, Mar 23, 2020 at 04:00:17PM +0100, Andrea Bolognani wrote:
> Since we're going to need a FreeBSD machine configured as a GitLab
> runner, wouldn't it make sense for us to also create one or more
> Linux machines that would run jobs on behalf of GitLab CI? That way
> we'd remove the limitation on the number of containers we can have
> running at the same time, and still have everything controlled by
> GitLab.
It is bit more complicated. Custom GitLab runners are associated with
a project (and its git repos). Those runners will be used for any CI jobs
run in the context of repositories owned by that project. Obviously this
covers post-merge build jobs, or regularly scheduled build jobs.
It gets more interesting with merge requests though, because in a normal
project, the merge request is not being submmitted from a branch on the
main repo, instead it is submitted from a branch on the user's forked
repo copy. As such the CI jobs for the merge request run in the contxt
of the forked repo and do not have access to the customer runners owned
by the project.
Where is this documented? I've been looking through the gitlab-runner
documentation but I haven't been able to find anything about this.
This is a intentional security restriction to prevent denial of
service
from arbitrary users from submitting (malicious) pull requests that
trigger builds on 3rd party runners.
It should be possible to prevent a DoS by configuring a limit on the
number of concurrent jobs that are allowed for the runner (see [1]),
so this looks like a moot point.
As for running the build jobs securely, the Docker executor should
take care of that when it comes to Linux; for FreeBSD, I think we'd
have to use a custom executor that drops privileges instead, because
Docker is not a possibility there.
Thus, we want to maximise the amount of build coverage we get on the
shared runners, as this is what merge requests will be using, and what
contributors will be using for tests before submitting code for review.
We'll have custom runners for providing coverage of platforms where
containers aren't viable (non-Linux), as well as providing additional
capacity.
I don't understand how this would work. If the CI configuration
contains say 20 jobs, 18 of which are to be run on Linux and the
remaining two are for FreeBSD, and the shared runners are limited to
10 jobs per build, how would GitLab decide which 10 jobs to ignore
when testing a merge request?
We can selectively grant access to the custom runners to regular
contributors (whom we trust and would benefit from broader access), but
we canot make them freely available to everyone by default.
Can you please point me to the relevant documentation?
There's gitlab RFEs about making custom runners more flexible,
while still
avoiding the security issues. For example, by allowing a project maintainer
to explicitly approve CI start for a merge request. No ETA for this though.
Do you have links to these RFEs handy?
[1]
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#...
--
Andrea Bolognani / Red Hat / Virtualization