[libvirt PATCH 0/7] docs: ci: Update the CI pages with fresh contents
While we have already descriptive articles on our GitLab CI, there's recently been some work on the CI front where a few sections deserve some updates. Erik Skultety (7): docs: ci-runners: Add a note on a new runner registration process docs: ci: Update the description on the integration CI GitLab variables docs: testtck: Tweak the Avocado command to run TCK test suite docs: testtck: Improve the documentation on how to get a VM from lcitool docs: testing: Adjust the docs on how to run container workloads locally docs: testtck: Add a clear note on libvirt + Perl bindings dependency docs: testtck: Expand the 'Run TCK' section on making use of ci/jobs.sh docs/ci-runners.rst | 10 ++++++++ docs/ci.rst | 8 ++++-- docs/testing.rst | 37 ++++++++++++++++++++++------ docs/testtck.rst | 60 ++++++++++++++++++++++++++++++++++++++------- 4 files changed, 96 insertions(+), 19 deletions(-) -- 2.41.0
The documented process should be updated to reflect the new process once GitLab transitions to it completely and drops the old process involving registration tokens as hinted by the note. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/ci-runners.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/ci-runners.rst b/docs/ci-runners.rst index fd5fdd121a..4a93f0e872 100644 --- a/docs/ci-runners.rst +++ b/docs/ci-runners.rst @@ -54,6 +54,16 @@ configurations on the GitLab UI. Navigate to: * *Runners activated for this project*, then * Click on the *Edit* icon (next to the *Lock* Icon) +*Note: GitLab has changed the runner registration process deprecating the use of +registration tokens in future versions, so while the above process is still +applicable (though the settings are now a bit more hidden) at the time of writing +this note (09/2023), GitLab v18.0+ is planned to completely switch to a new +process (see the links below), deleting the use of registration tokens.* + + * https://gitlab.com/gitlab-org/gitlab/-/issues/380872 + * https://docs.gitlab.com/ee/ci/runners/new_creation_workflow.html + * https://docs.gitlab.com/ee/ci/runners/runners_scope.html#create-a-shared-run... + Don't forget to add a tag to your runner as these are used to route specific jobs to specific runners, e.g. if a job in ``ci/integration.yml`` looked like this :: -- 2.41.0
This patch mainly fixes an unfinished sentence that was supposed to describe the LIBVIRT_CI_INTEGRATION_RUNNER_TAG variable, but took the opportunity to update the description of the other variable too. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/ci.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/ci.rst b/docs/ci.rst index ee85018c49..f3911d84ca 100644 --- a/docs/ci.rst +++ b/docs/ci.rst @@ -31,8 +31,12 @@ see below. GitLab CI variables ------------------- -* ``LIBVIRT_CI_INTEGRATION`` - enables integration test runs manually or in forks -* ``LIBVIRT_CI_INTEGRATION_RUNNER_TAG`` - overrides the upstream runner tag on the +* ``LIBVIRT_CI_INTEGRATION`` - enables integration test suite execution as part + of the pipeline (works in forks too if there's a registered shared runner) +* ``LIBVIRT_CI_INTEGRATION_RUNNER_TAG`` - overrides the gitlab-runner tag for + the pipeline; this is needed in forks because the default tag only matches + upstream shared runners which may be (and will be) different from the tags + defined on shared runners in forks Retrieving test logs -------------------- -- 2.41.0
While we may have needed to run TCK through Avocado by explicitly using the '--tap' option (still possible), we can get a nice output from Avocado by default leaving the option out which is exactly what we do inside GitLab CI environment. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testtck.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/testtck.rst b/docs/testtck.rst index 100a278acd..d7f5483b5f 100644 --- a/docs/testtck.rst +++ b/docs/testtck.rst @@ -78,7 +78,7 @@ as root the test suite as root (when running with Avocado): :: - # avocado --config avocado.config run --tap - ./scripts/ + # avocado --config avocado.config run from the TCK's git root. -- 2.41.0
While wording is still correct to this day, we have already added more features to lcitool and documented it properly in its repo. Make sure that we refer the users to lcitool's doc material for further details on how VMs can be installed locally. Use the opportunity to bump the OS distro target from Fedora 36 -> 38. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testtck.rst | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/testtck.rst b/docs/testtck.rst index d7f5483b5f..89760e3f63 100644 --- a/docs/testtck.rst +++ b/docs/testtck.rst @@ -48,20 +48,31 @@ available through the system package manager and some will likely need to be installed from CPAN (Perl's equivalent of Python's PyPI). Here's where `libvirt-ci's <https://gitlab.com/libvirt/libvirt-ci.git>`__ lcitool can help with preparing a test environment in a fresh VM, taking care of the -dependencies along the way: +dependencies along the way. A simple example of getting a machine from lcitool +would be: :: - $ lcitool install --target fedora-36 tck-fedora36 --wait + $ lcitool install --target fedora-38 tck-fedora38 --wait -would get you a new Fedora 36 VM named ``tck-fedora36``. Then +would get you a new Fedora 38 VM named ``tck-fedora38``. There are different +ways of getting a fresh local VM with ``lcitool``, so please refer to +`Installing local VMs <https://gitlab.com/libvirt/libvirt-ci/-/blob/master/docs/vms.rst>`__ +for further details, especially to utilize vendor cloud images for this +purpose. + +Once you have a fresh virtual machine, you need to pre-install it with all +necessary build dependencies to be able to build libvirt, libvirt Perl bindings +and run the TCK test suite inside it. You'd do that by running :: - $ lcitool update tck-fedora36 libvirt,libvirt-perl,libvirt-tck+runtime + $ lcitool update tck-fedora38 libvirt,libvirt-perl,libvirt-tck+runtime + +Again, for further details on how to update ``lcitool`` virtual machines, +please refer to +`Updating VMs with a given project dependencies <https://gitlab.com/libvirt/libvirt-ci/-/blob/master/docs/vms.rst>`__ -will install all the necessary dependencies to build libvirt, the corresponding -Perl bindings and all TCK runtime dependencies to be able to execute the tests. We also recommend executing TCK using the Avocado framework as the test harness engine which means that you'll have to install Avocado in the test environment as well. You can get it either from -- 2.41.0
The fact that we need ci/helper script to run the workloads remains true, but the invocation has changed as of commit eb41e456 . We also extracted GitLab job specs into a standalone ci/jobs.sh script which allows execution of any container job we run in upstream CI locally, unlike the original functionality which only allowed builds, tests and shell (although important to say it could be adjusted with the right meson/ninja args). lcitool also became mandatory as it enables the container execution which replaced a Makefile we used to have for this purpose. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testing.rst | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/docs/testing.rst b/docs/testing.rst index a597c3ed07..9ca47072ba 100644 --- a/docs/testing.rst +++ b/docs/testing.rst @@ -48,11 +48,17 @@ Running container builds with GitLab CI As long as your GitLab account has CI minutes available, pipelines will run automatically on every branch push to your fork. -Running container builds locally -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Running container jobs locally +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In order to run container builds locally, we have a ``helper`` script inside -the ``ci`` directory that can pull, build, and test (if applicable) changes on +GitLab CI configuration file is the only source of truth when it comes to +various job specifications we execute as part of the upstream pipeline. +Luckily, all "script" (i.e. Bash scripts) were extracted to standalone Shell +functions in ``ci/jobs.sh``. This allows users to run any of the container +GitLab job specifications locally by just referencing the job name. + +When it comes to actually running the GitLab jobs locally, we have a +``ci/helper`` script can pull, build, and test (if applicable) changes on your current local branch. It supports both the Docker and Podman runtimes with an automatic selection of whichever runtime is configured on your system. In case neither has been enabled/configured, please go through the following @@ -130,12 +136,27 @@ the default libvirt registry: fedora-rawhide-cross-mingw64 ... -Now let's say one would want to build their local libvirt changes on Alpine -Edge using their own GitLab's registry container. They'd then proceed with +Now, let's say one would want to run the ``website`` job from GitLab on Debian +11. This is how a GitLab job specification can be referenced on ``ci/helper``'s +command line: :: - $ ci/helper build --image-prefix registry.gitlab.com/<user>/libvirt/ci- alpine-edge + $ ci/helper run --job website debian-10 + +What if you want to run an rpmbuild of libvirt on an RPM distro? + +:: + + $ ci/helper run --job rpmbuild fedora-38 + +Want to use your own, say alpine-edge, container image from your GitLab +container registry? +Proceed with the following: + +:: + + $ ci/helper run --job build --image-prefix registry.gitlab.com/<user>/libvirt/ci- alpine-edge Finally, it would be nice if one could get an interactive shell inside the test environment to debug potential build issues. This can be achieved with the @@ -143,7 +164,7 @@ following: :: - $ ci/helper shell alpine-edge + $ ci/helper run --job shell alpine-edge Integration tests -- 2.41.0
On Wed, Sep 27, 2023 at 08:53:46PM +0200, Erik Skultety wrote:
The fact that we need ci/helper script to run the workloads remains true, but the invocation has changed as of commit eb41e456 . We also
Use full commit hash as in the future the short one might not be unique.
extracted GitLab job specs into a standalone ci/jobs.sh script which allows execution of any container job we run in upstream CI locally, unlike the original functionality which only allowed builds, tests and shell (although important to say it could be adjusted with the right meson/ninja args). lcitool also became mandatory as it enables the container execution which replaced a Makefile we used to have for this purpose.
Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testing.rst | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-)
diff --git a/docs/testing.rst b/docs/testing.rst index a597c3ed07..9ca47072ba 100644 --- a/docs/testing.rst +++ b/docs/testing.rst @@ -48,11 +48,17 @@ Running container builds with GitLab CI As long as your GitLab account has CI minutes available, pipelines will run automatically on every branch push to your fork.
-Running container builds locally -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Running container jobs locally +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-In order to run container builds locally, we have a ``helper`` script inside -the ``ci`` directory that can pull, build, and test (if applicable) changes on +GitLab CI configuration file is the only source of truth when it comes to +various job specifications we execute as part of the upstream pipeline. +Luckily, all "script" (i.e. Bash scripts) were extracted to standalone Shell +functions in ``ci/jobs.sh``. This allows users to run any of the container +GitLab job specifications locally by just referencing the job name. + +When it comes to actually running the GitLab jobs locally, we have a +``ci/helper`` script can pull, build, and test (if applicable) changes on
/script/script that/ Pavel
It's mentioned in an earlier paragraph that Perl bindings in correct version are needed, but there's no note about libvirt even though it should be obvious. So make a clear note on that and while at it, do mention the possibility to get upstream libvirt RPMs from GitLab CI artifacts if users don't feel like building everything on their own. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testtck.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/testtck.rst b/docs/testtck.rst index 89760e3f63..f57af61aa8 100644 --- a/docs/testtck.rst +++ b/docs/testtck.rst @@ -73,6 +73,12 @@ Again, for further details on how to update ``lcitool`` virtual machines, please refer to `Updating VMs with a given project dependencies <https://gitlab.com/libvirt/libvirt-ci/-/blob/master/docs/vms.rst>`__ +Note that lcitool only installs build dependencies, so as mentioned above you'll +need both libvirt **and** libvirt Perl bindings installed in order to be able +to run TCK. You can (depending on use case) either build both inside the VM and +install manually or install the corresponding RPMs from GitLab CI build +artifacts. + We also recommend executing TCK using the Avocado framework as the test harness engine which means that you'll have to install Avocado in the test environment as well. You can get it either from -- 2.41.0
Ever since commit 6e9bd600 added a new GitLab job description function handling the integration test suite process to ci/jobs.sh it should be mentioned in the docs. This patch splits the 'Run TCK' section in two, giving user the option to run the integration test suite in their VM environment the same way as we do in GitLab CI or execute everything manually. This patch takes the opportunity to also link to the virtiofs kbase article to give users a different option to get the local libvirt repositories to be used in testing inside a VM. Signed-off-by: Erik Skultety <eskultet@redhat.com> --- docs/testtck.rst | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/docs/testtck.rst b/docs/testtck.rst index f57af61aa8..568899dcdd 100644 --- a/docs/testtck.rst +++ b/docs/testtck.rst @@ -90,8 +90,33 @@ in the future we plan on making the TCK internal coupling with Avocado tighter. Running TCK ~~~~~~~~~~~ -Once you have all the dependencies installed, you can then proceed with running -as root the test suite as root (when running with Avocado): +Once you have all the dependencies installed, you can then proceed with either +of the following procedures to execute the test suite as root. + +Replicating upstream CI test suite execution locally +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Similarly to how local container builds utilize the standalone ``ci/jobs.sh`` +script containing functions describing GitLab job definitions it can be +utilized to run integration test suite as well. In this case, one needs to +get a copy of their libvirt repository containing the changes to be tested +inside the VM (either by cloning it manually or sharing the repo e.g. via +`virtiofs <https://libvirt.org/kbase/virtiofs.html>`__). Make sure that the +user which is going to execute the following has passwordless "sudo" permissions +(lcitool's default "test" user does). Then it's just a matter of running + +:: + + $ source ci/jobs.sh + $ run_integration + +Manual invocation +^^^^^^^^^^^^^^^^^ + +If you want to have more control over the whole procedure or simply don't want +to run the exact same steps as libvirt's upstream CI pipeline does in context +of integration tests then start by cloning the +`TCK <https://gitlab.com/libvirt/libvirt-tck.git>`__ repository and run :: -- 2.41.0
On Wed, Sep 27, 2023 at 08:53:48PM +0200, Erik Skultety wrote:
Ever since commit 6e9bd600 added a new GitLab job description function
Same here, please use full commit hash.
handling the integration test suite process to ci/jobs.sh it should be mentioned in the docs. This patch splits the 'Run TCK' section in two, giving user the option to run the integration test suite in their VM environment the same way as we do in GitLab CI or execute everything manually. This patch takes the opportunity to also link to the virtiofs kbase article to give users a different option to get the local libvirt repositories to be used in testing inside a VM.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Pavel
On Wed, Sep 27, 2023 at 08:53:41PM +0200, Erik Skultety wrote:
While we have already descriptive articles on our GitLab CI, there's recently been some work on the CI front where a few sections deserve some updates.
Erik Skultety (7): docs: ci-runners: Add a note on a new runner registration process docs: ci: Update the description on the integration CI GitLab variables docs: testtck: Tweak the Avocado command to run TCK test suite docs: testtck: Improve the documentation on how to get a VM from lcitool docs: testing: Adjust the docs on how to run container workloads locally docs: testtck: Add a clear note on libvirt + Perl bindings dependency docs: testtck: Expand the 'Run TCK' section on making use of ci/jobs.sh
docs/ci-runners.rst | 10 ++++++++ docs/ci.rst | 8 ++++-- docs/testing.rst | 37 ++++++++++++++++++++++------ docs/testtck.rst | 60 ++++++++++++++++++++++++++++++++++++++------- 4 files changed, 96 insertions(+), 19 deletions(-)
-- 2.41.0
Polite ping. Erik
On Tue, Oct 03, 2023 at 02:56:37PM +0200, Erik Skultety wrote:
On Wed, Sep 27, 2023 at 08:53:41PM +0200, Erik Skultety wrote:
While we have already descriptive articles on our GitLab CI, there's recently been some work on the CI front where a few sections deserve some updates.
Erik Skultety (7): docs: ci-runners: Add a note on a new runner registration process docs: ci: Update the description on the integration CI GitLab variables docs: testtck: Tweak the Avocado command to run TCK test suite docs: testtck: Improve the documentation on how to get a VM from lcitool docs: testing: Adjust the docs on how to run container workloads locally docs: testtck: Add a clear note on libvirt + Perl bindings dependency docs: testtck: Expand the 'Run TCK' section on making use of ci/jobs.sh
docs/ci-runners.rst | 10 ++++++++ docs/ci.rst | 8 ++++-- docs/testing.rst | 37 ++++++++++++++++++++++------ docs/testtck.rst | 60 ++++++++++++++++++++++++++++++++++++++------- 4 files changed, 96 insertions(+), 19 deletions(-)
-- 2.41.0
Polite ping.
Ping. Erik
On Wed, Sep 27, 2023 at 08:53:41PM +0200, Erik Skultety wrote:
While we have already descriptive articles on our GitLab CI, there's recently been some work on the CI front where a few sections deserve some updates.
Erik Skultety (7): docs: ci-runners: Add a note on a new runner registration process docs: ci: Update the description on the integration CI GitLab variables docs: testtck: Tweak the Avocado command to run TCK test suite docs: testtck: Improve the documentation on how to get a VM from lcitool docs: testing: Adjust the docs on how to run container workloads locally docs: testtck: Add a clear note on libvirt + Perl bindings dependency docs: testtck: Expand the 'Run TCK' section on making use of ci/jobs.sh
With the issues fixed: Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
participants (2)
-
Erik Skultety -
Pavel Hrdina