Re: [PATCH] ci: Install python RPMs from libvirt/libvirt-python CI artifacts

On Mon, Oct 10, 2022 at 11:09:51AM +0100, Daniel P. Berrangé wrote: > On Mon, Oct 10, 2022 at 11:46:43AM +0200, Peter Krempa wrote: > > After addition of the new libvirt-client-qemu sub-package which is using > > python bindings (thus creating a circular dependency between the libvirt > > and libvirt-python projects) the integration jobs fail with: > > > > Error: > > Problem: conflicting requests > > - nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64 > > Hmm, I wonder, should we really be trying to install *all* RPMs ? > > IIUC, we're only running integration tests for QEMU, so it ought > to be sufficient to do only have the QEMU driver and virsh client > and Perl APIs. eg can we change ci/integrate-templates.yml to > do > > createrepo libvirt-rpms > createrepo libvirt-perl > dnf install libvirt-daemon-qemu libvirt-client perl-Sys-Virt Sorry for the delay, I was actually integrating ^this suggestion and GitLab is, well, slow to test these changes with. Anyhow, I vote for Peter's original patch as it's the simplest and cleanest way to fix the problem, why? Here's what I ended up with on my branch: diff --git a/ci/integration-template.yml b/ci/integration-template.yml index d4799b2ca8..9c58a1b71c 100644 --- a/ci/integration-template.yml +++ b/ci/integration-template.yml @@ -25,7 +25,31 @@ .install-deps: &install-deps - - sudo dnf install -y libvirt-rpms/* libvirt-perl-rpms/* + - sudo dnf install -y createrepo + - createrepo libvirt-rpms + - createrepo libvirt-perl-rpms + - |- + for repo in libvirt-rpms libvirt-perl-rpms; do + cat << EOF >> "$SCRATCH_DIR/ci.repo" + [CI-$repo] + name=ci-$repo + baseurl=file://$(pwd)/$repo + enabled=1 + gpgcheck=0 + EOF + done + - sudo mv "$SCRATCH_DIR/ci.repo" /etc/yum.repos.d/ + - sudo dnf install -y + libvirt-daemon-config-nwfilter + libvirt-daemon-config-network + libvirt-daemon-driver-interface + libvirt-daemon-driver-network + libvirt-daemon-driver-nodedev + libvirt-daemon-driver-nwfilter + libvirt-daemon-driver-qemu + libvirt-daemon-driver-secret + libvirt-daemon-driver-storage + perl-Sys-Virt - sudo pip3 install --prefix=/usr avocado-framework

Not only does it require more steps, is ugly because of the heredoc combined with YAML's explicit line break preservation (|-), libvirt-daemon-driver-qemu also doesn't pull all the needed bits as it was suggested, e.g. libvirt-daemon-config-nwfilter which is needed by the tests. Although the tests don't need virtinterfaced, I still had to include it because the logic which restarts all the daemons would have to be adjusted as well; and then one day we add a test for virtinterfaced we'd have to add it back, so IMO installing and restarting all the daemons is the cleanest and most transparent way of setting up the CI env even though we're not making use of all of it now (and so I ended up naming all the secondary drivers explicitly).

The other huge inconvenience with the proposed approach compared to Peter's is debugging the failures. What I currently do with any integration CI failures is that I connect to the runner machine and spin up an instance of one of the images from which gitlab provisions the environment, download the RPM artifacts from libvirt and libvirt-perl, install and then debug - again, why? Because unfortunately trying to do this locally is still less convenient even though we've done plenty of work in lcitool to improve the status quo. So, back to your suggestion - I'd have to create the repositories manually, install a few explicit packages and then test; sorry, that maybe fine for the CI, but doesn't beat the following in any way: wget <libvirt-rpms-uri> -O libvirt_rpms.zip wget <libvirt-perl-rpms-uri> -O libvirt_perl_rpms.zip wget <libvirt-python-rpms-uri> -O libvirt_python_rpms.zip unzip <yada yada> dnf install libvirt_rpms/* libvirt_perl_rpms/* libvirt_python_rpms/*