November 2019 - Devel - libvirt List Archives

[libvirt] [PATCH v5 00/23] scripts: convert most perl scripts to python
by Daniel P. Berrangé 04 Dec '19

04 Dec '19

This series is an effort to reduce the number of different languages we use by eliminating most use of perl in favour of python. This aligns with fact that the likely future build system we'll use (meson) is written in python, and that python is much more commonly used/understood by developers these days than perl. With this applied we use perl in a handful of places only: - src/rpc/gendispatch.pl - this is a horrendously large script and very hard to understand/follow. A straight syntax conversion to Python would still leave a hgue and hard to understand/follow script. It really needs a clean room rewrite from scratch, with better structure. - src/rpc/genprotocol.pl - fairly easy to convert, but might be obsolete depending on approach for rewriting gendispatch.pl, so ignored for now - tests/oomtrace.pl - will be purge by the patches that drop OOM handling anyway - tools/wireshark/util/genxdrstub.pl - a very large script, which I haven't got the courage to tackle yet. - cfg.mk/maint.mk - many syntax rules involve regexes which are fed to perl. Decision on what to do with syntax-check rules punted to another time. - build-aux/gitlog-to-changelog - build-aux/useless-if-before-free - Both pulled in from gnulib. Could be rewritten quite easily if desired, but given that we aren't maintainers of them right now, they're ignored as they don't really impact our developers. Note that the check-spacing.py script is significantly slower in Python than in Perl. After researching this it appears there is nothing that can be done. The Perl regex engine is simply much better optimized than the Python one. As previously discussed we need to loook at uncrustify or clang-format or some other tool to validate whitespace formatting. This is ongoing. We can either accept the slow down in the short term or keep the Perl version in the short term. In v5: - Rebased to cope with changes to require VPATH build - Merged the already-ACKd scripts In v4: - Moved all scripts into the scripts/ directory instead of having them scattered around source tree - Use re.search instead of re.match - Don't bother re-compiling regexes In v3: - All scripts comply with all flake8 style rules with exception of E129 visually indented line with same indent as next logical line In v2: - Pulled in patch to hacking file - Converted many more scripts - Forced UTF-8 character set to avoid ascii codec on py3 < 3.7 Daniel P. Berrangé (23): build-aux: rewrite duplicate header checker in Python build-aux: rewrite whitespace checker in Python build-aux: rewrite mock inline checker in Python build-aux: rewrite header ifdef checker in Python src: rewrite ACL permissions checker in Python src: rewrite symfile sorting checker in Python src: rewrite symfile library checker in Python src: rewrite systemtap probe generator in Python src: rewrite systemtap function generator in Python src: rewrite driver name checker in Python src: rewrite driver impl checker in Python src: rewrite ACL rule checker in Python src: rewrite polkit ACL generator in Python src: rewrite remote protocol checker in Python tests: rewrite test argv line wrapper in Python tests: rewrite qemu capability grouper in Python tests: rewrite file access checker in Python docs: rewrite hvsupport.html page generator in python docs: rewrite polkit docs generator in Python docs: move apibuild.py to the scripts/ directory docs: move reformat-news.py to the scripts/ directory docs: move esx_vi_generator.py to the scripts/ directory docs: move hyperv_wmi_generator.py to the scripts/ directory Makefile.am | 33 +- build-aux/check-spacing.pl | 198 ------- build-aux/header-ifdef.pl | 182 ------- build-aux/mock-noinline.pl | 75 --- build-aux/prohibit-duplicate-header.pl | 26 - build-aux/syntax-check.mk | 32 +- docs/Makefile.am | 16 +- docs/genaclperms.pl | 125 ----- docs/hvsupport.pl | 459 ---------------- {docs => scripts}/apibuild.py | 0 scripts/check-aclperms.py | 75 +++ scripts/check-aclrules.py | 263 +++++++++ scripts/check-driverimpls.py | 102 ++++ scripts/check-drivername.py | 114 ++++ scripts/check-file-access.py | 123 +++++ scripts/check-remote-protocol.py | 136 +++++ scripts/check-spacing.py | 229 ++++++++ scripts/check-symfile.py | 80 +++ scripts/check-symsorting.py | 117 ++++ scripts/dtrace2systemtap.py | 143 +++++ {src/esx => scripts}/esx_vi_generator.py | 0 scripts/genaclperms.py | 123 +++++ scripts/genpolkit.py | 122 +++++ scripts/gensystemtap.py | 184 +++++++ scripts/group-qemu-caps.py | 123 +++++ scripts/header-ifdef.py | 231 ++++++++ scripts/hvsupport.py | 504 ++++++++++++++++++ .../hyperv_wmi_generator.py | 0 scripts/mock-noinline.py | 85 +++ scripts/prohibit-duplicate-header.py | 56 ++ {docs => scripts}/reformat-news.py | 0 scripts/test-wrap-argv.py | 170 ++++++ src/Makefile.am | 146 ++--- src/access/Makefile.inc.am | 6 +- src/access/genpolkit.pl | 119 ----- src/admin/Makefile.inc.am | 6 +- src/check-aclperms.pl | 73 --- src/check-aclrules.pl | 252 --------- src/check-driverimpls.pl | 80 --- src/check-drivername.pl | 83 --- src/check-symfile.pl | 70 --- src/check-symsorting.pl | 106 ---- src/dtrace2systemtap.pl | 130 ----- src/esx/Makefile.inc.am | 6 +- src/hyperv/Makefile.inc.am | 5 +- src/rpc/Makefile.inc.am | 1 - src/rpc/gensystemtap.pl | 193 ------- tests/Makefile.am | 3 +- tests/check-file-access.pl | 126 ----- tests/file_access_whitelist.txt | 2 +- tests/group-qemu-caps.pl | 124 ----- tests/test-wrap-argv.pl | 174 ------ tests/testutils.c | 16 +- 53 files changed, 3092 insertions(+), 2755 deletions(-) delete mode 100755 build-aux/check-spacing.pl delete mode 100644 build-aux/header-ifdef.pl delete mode 100644 build-aux/mock-noinline.pl delete mode 100644 build-aux/prohibit-duplicate-header.pl delete mode 100755 docs/genaclperms.pl delete mode 100755 docs/hvsupport.pl rename {docs => scripts}/apibuild.py (100%) create mode 100755 scripts/check-aclperms.py create mode 100755 scripts/check-aclrules.py create mode 100755 scripts/check-driverimpls.py create mode 100644 scripts/check-drivername.py create mode 100755 scripts/check-file-access.py create mode 100644 scripts/check-remote-protocol.py create mode 100755 scripts/check-spacing.py create mode 100755 scripts/check-symfile.py create mode 100755 scripts/check-symsorting.py create mode 100755 scripts/dtrace2systemtap.py rename {src/esx => scripts}/esx_vi_generator.py (100%) create mode 100755 scripts/genaclperms.py create mode 100755 scripts/genpolkit.py create mode 100755 scripts/gensystemtap.py create mode 100755 scripts/group-qemu-caps.py create mode 100644 scripts/header-ifdef.py create mode 100755 scripts/hvsupport.py rename {src/hyperv => scripts}/hyperv_wmi_generator.py (100%) create mode 100644 scripts/mock-noinline.py create mode 100644 scripts/prohibit-duplicate-header.py rename {docs => scripts}/reformat-news.py (100%) create mode 100755 scripts/test-wrap-argv.py delete mode 100755 src/access/genpolkit.pl delete mode 100755 src/check-aclperms.pl delete mode 100755 src/check-aclrules.pl delete mode 100755 src/check-driverimpls.pl delete mode 100755 src/check-drivername.pl delete mode 100755 src/check-symfile.pl delete mode 100755 src/check-symsorting.pl delete mode 100755 src/dtrace2systemtap.pl delete mode 100755 src/rpc/gensystemtap.pl delete mode 100755 tests/check-file-access.pl delete mode 100755 tests/group-qemu-caps.pl delete mode 100755 tests/test-wrap-argv.pl -- 2.21.0

4 65

[libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment
by Daniel P. Berrangé 04 Dec '19

04 Dec '19

This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options. This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST from this point. The POD man pages will also be converted to RST. I'll post this as a different series. Note this series only touches the main HTML content, not the styling / templating using XSL. Tackling that is a separate job I've not attempted. My guess is that we'd use either Sphinx or Pelican to do the templating. Existing HTML docs are candidates for conversion to RST, however, people should *NOT* attempt todo this manually. In this series I've converted docs/kbase/ files, using the pandoc tool. This does a pretty good job in general but does need some manual cleanups. First I post-processed its output to change the heading highlighting to follow the documented style $ cat convert.pl my @in = <>; my @out; for (my $i ; $i <= $#in; $i++) { my $line = $in[$i]; if ($line =~ /^=+$/) { my @newout = splice @out, 0, $i - 1; push @newout, $line; push @newout, $out[$i-1]; push @newout, $out[$i]; push @newout, $line; push @newout, "\n"; push @newout, ".. contents::\n"; @out = @newout; } elsif ($line =~ /^(-|~|^|')+$/) { $line =~ s/-/=/g; $line =~ s/~/-/g; $line =~ s/\^/~/g; $line =~ s/'/^/g; push @out, $line; } else { push @out, $line; } } print @out; So I'm converting with pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst $EDITOR foot.rst git rm -f foot.html.in git add foo.rst After pandoc & the headings convert, there's usually manual fixes needed to deal with a few oddities pandoc gets wrong, or looks ugly. This is still way simpler than converting the doc to rst manually. Changed in v2: - Added a style guide for RST docs - Probe for rst2html's different names - Added make rules for building rst - Fixed permalinks in generated docs - Misc CSS fixes - Auto-converted all of kbase/ directory Daniel P. Berrangé (15): docs: split TLS certificate setup into its own file docs: move docs about remote driver URIs into URI docs docs: introduce rst2html as a mandatory tool for building docs docs: adapt filling of <head> section for rst2html output docs: generate permalinks correctly for rst2html output docs: relax CSS context match for pretty tables docs: add styling for <tt> element docs: add a minimal style guide for writing RST docs docs: convert kbase/domainstatecapture.html.in to RST docs: convert kbase/launch_security_sev.html.in to RST docs: convert kbase/secureusage.html.in to RST docs: convert kbase/locking.html.in to RST docs: convert kbase/locking-lockd.html.in to RST docs: convert kbase/locking-sanlock.html.in to RST docs: add a kbase page about RPM packaging options docs/Makefile.am | 33 +- docs/aclpolkit.html.in | 20 +- docs/docs.html.in | 6 + docs/genaclperms.pl | 2 +- docs/generic.css | 4 +- docs/kbase.html.in | 4 + docs/kbase/domainstatecapture.html.in | 303 ----------- docs/kbase/domainstatecapture.rst | 255 +++++++++ docs/kbase/launch_security_sev.html.in | 533 ------------------- docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++ docs/kbase/locking-lockd.html.in | 160 ------ docs/kbase/locking-lockd.rst | 121 +++++ docs/kbase/locking-sanlock.html.in | 247 --------- docs/kbase/locking-sanlock.rst | 193 +++++++ docs/kbase/locking.html.in | 48 -- docs/kbase/locking.rst | 33 ++ docs/kbase/rpm-deployment.rst | 410 +++++++++++++++ docs/kbase/secureusage.html.in | 171 ------- docs/kbase/secureusage.rst | 131 +++++ docs/libvirt.css | 76 +-- docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +- docs/page.xsl | 9 +- docs/remote.html.in | 684 +------------------------ docs/styleguide.rst | 66 +++ docs/tlscerts.html.in | 413 +++++++++++++++ docs/uri.html.in | 263 ++++++++-- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 5 + mingw-libvirt.spec.in | 1 + 30 files changed, 2478 insertions(+), 2250 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst create mode 100644 docs/kbase/rpm-deployment.rst delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst create mode 100644 docs/styleguide.rst create mode 100644 docs/tlscerts.html.in -- 2.23.0

2 17

[libvirt] [PATCH RFC V2] guests: Add support for openSUSE
by Jim Fehlig 03 Dec '19

03 Dec '19

This change adds support for the latest openSUSE Leap and openSUSE Tumbleweed guests. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- This version of the patch overcomes the 'lci build' issues noted in V1 and works well for openSUSE Leap 15.1 and Tumbleweed. I'm still of the opinion of only supporting the latest Leap since there is only a six month support overlap between versions. Version x-1 is only supported six months after x is released. 'lci update ... libvirt' fails in task "Configure hostname": fatal: [libvirt-opensuse-tumbleweed]: FAILED! => {"changed": false, "msg": "hostname module cannot be used on platform Linux (Opensuse-tumbleweed)"} Not sure why that is the case. But more importantly, even though it doesn't fail, the "Update installed packages" task doesn't install any of the packages required to build libvirt. I suppose that agrees with guests/REAMDME, but how then are the packages required to build $project installed? How/when is the list of packages required to build $project conveyed to $package_manager? 'lcitool build libvirt-opensuse-tumbleweed libvirt' fails in the "Gathering Facts" task: fatal: [libvirt-opensuse-tumbleweed]: UNREACHABLE! => {"changed": false, "msg": "Invalid/incorrect password: ", "unreachable": true} Passing --debug to the command didn't help with determining what needed the password. ssh? I can't find where the "Gathering Facts" task is defined. The $root_password_file passed to /usr/bin/ansible-playbook contains the correct password for the root user. /me needs to figure out how to better debug this stuff :-). guests/configs/autoinst.xml | 75 +++++++++++++++++++ .../host_vars/libvirt-opensuse-15/docker.yml | 2 + .../host_vars/libvirt-opensuse-15/install.yml | 2 + guests/host_vars/libvirt-opensuse-15/main.yml | 22 ++++++ .../libvirt-opensuse-tumbleweed/docker.yml | 2 + .../libvirt-opensuse-tumbleweed/install.yml | 2 + .../libvirt-opensuse-tumbleweed/main.yml | 22 ++++++ guests/inventory | 2 + guests/lcitool | 2 + guests/playbooks/update/tasks/base.yml | 15 ++++ guests/vars/mappings.yml | 41 +++++++++- 11 files changed, 185 insertions(+), 2 deletions(-) diff --git a/guests/configs/autoinst.xml b/guests/configs/autoinst.xml new file mode 100644 index 0000000..42c42f8 --- /dev/null +++ b/guests/configs/autoinst.xml @@ -0,0 +1,75 @@ +<?xml version="1.0"?> +<!DOCTYPE profile> +<profile + xmlns="http://www.suse.com/1.0/yast2ns" + xmlns:config="http://www.suse.com/1.0/configns"> + <general> + <mode> + <confirm config:type="boolean">false</confirm> + </mode> + </general> + <partitioning config:type="list"> + <drive> + <device>/dev/vda</device> + <use>all</use> + <partitions config:type="list"> + <partition> + <filesystem config:type="symbol">swap</filesystem> + <size>256M</size> + <mount>swap</mount> + </partition> + <partition> + <filesystem config:type="symbol">ext4</filesystem> + <mount>/</mount> + <size>max</size> + </partition> + </partitions> + </drive> + </partitioning> + <bootloader> + <global> + <terminal>console serial</terminal> + </global> + </bootloader> + <timezone> + <hwclock>UTC</hwclock> + <timezone>UTC</timezone> + </timezone> + <software> + <install_recommended config:type="boolean">false</install_recommended> + <products config:type="list"> + <product>openSUSE</product> + </products> + <patterns config:type="list"> + <pattern>base</pattern> + <pattern>minimal_base</pattern> + <pattern>yast2_basis</pattern> + </patterns> + <packages config:type="list"> + <package>openssh</package> + </packages> + </software> + <networking> + <keep_install_network config:type="boolean">true</keep_install_network> + </networking> + <users config:type="list"> + <user> + <username>root</username> + <user_password>root</user_password> + <encrypted config:type="boolean">false</encrypted> + <uid>0</uid> + <gid>0</gid> + <home>/root</home> + <shell>/bin/bash</shell> + </user> + </users> + <services-manager> + <default_target>multi-user</default_target> + <services config:type="list"> + <service> + <service_name>sshd</service_name> + <service_status>enable</service_status> + </service> + </services> + </services-manager> +</profile> diff --git a/guests/host_vars/libvirt-opensuse-15/docker.yml b/guests/host_vars/libvirt-opensuse-15/docker.yml new file mode 100644 index 0000000..8f32d06 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: opensuse/leap:15 diff --git a/guests/host_vars/libvirt-opensuse-15/install.yml b/guests/host_vars/libvirt-opensuse-15/install.yml new file mode 100644 index 0000000..d0fdbe5 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://download.opensuse.org/distribution/leap/15.1/repo/oss/ diff --git a/guests/host_vars/libvirt-opensuse-15/main.yml b/guests/host_vars/libvirt-opensuse-15/main.yml new file mode 100644 index 0000000..abd83c5 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/main.yml @@ -0,0 +1,22 @@ +--- +projects: + - libosinfo + - libvirt + - libvirt-cim + - libvirt-dbus + - libvirt-glib + - libvirt-perl + - libvirt-python + - libvirt-sandbox + - libvirt-tck + - osinfo-db + - osinfo-db-tools + - virt-manager + - virt-viewer + +package_format: 'rpm' +package_manager: 'zypper' +os_name: 'OpenSUSE' +os_version: '15' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml new file mode 100644 index 0000000..cc79a22 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: opensuse/tumbleweed diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml new file mode 100644 index 0000000..36cea68 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://download.opensuse.org/tumbleweed/repo/oss/ diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml new file mode 100644 index 0000000..828df81 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml @@ -0,0 +1,22 @@ +--- +projects: + - libosinfo + - libvirt + - libvirt-cim + - libvirt-dbus + - libvirt-glib + - libvirt-perl + - libvirt-python + - libvirt-sandbox + - libvirt-tck + - osinfo-db + - osinfo-db-tools + - virt-manager + - virt-viewer + +package_format: 'rpm' +package_manager: 'zypper' +os_name: 'OpenSUSE' +os_version: 'Tumbleweed' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/inventory b/guests/inventory index 3b15513..71d3c91 100644 --- a/guests/inventory +++ b/guests/inventory @@ -8,5 +8,7 @@ libvirt-fedora-rawhide libvirt-freebsd-11 libvirt-freebsd-12 libvirt-freebsd-current +libvirt-opensuse-15 +libvirt-opensuse-tumbleweed libvirt-ubuntu-16 libvirt-ubuntu-18 diff --git a/guests/lcitool b/guests/lcitool index a630971..7f26731 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -531,6 +531,8 @@ class Application: install_config = "preseed.cfg" elif facts["os_name"] in ["CentOS", "Fedora"]: install_config = "kickstart.cfg" + elif facts["os_name"] == "OpenSUSE": + install_config = "autoinst.xml" else: raise Exception( "Host {} doesn't support installation".format(host) diff --git a/guests/playbooks/update/tasks/base.yml b/guests/playbooks/update/tasks/base.yml index 3d83e78..b6a2cd2 100644 --- a/guests/playbooks/update/tasks/base.yml +++ b/guests/playbooks/update/tasks/base.yml @@ -65,12 +65,27 @@ when: - package_format == 'pkg' +- name: Update installed packages + command: '{{ package_manager }} update -y -l --force-resolution --no-recommends' + args: + warn: no + when: + - os_name == 'OpenSUSE' + - name: Clean up packages after update shell: '{{ package_manager }} clean packages -y && {{ package_manager }} autoremove -y' args: warn: no when: - package_format == 'rpm' + - not os_name == "OpenSUSE" + +- name: Clean up packages after update + shell: '{{ package_manager }} clean' + args: + warn: no + when: + - os_name == "OpenSUSE" - name: Clean up packages after update apt: diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index bd9b161..8359ce2 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -19,10 +19,10 @@ # - deb, pkg, rpm # # Valid OS names are: -# - CentOS, Debian, Fedora, FreeBSD, Ubuntu +# - CentOS, Debian, Fedora, FreeBSD, OpenSUSE, Ubuntu # # Valid OS versions are: -# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on +# - CentOS7, Debian9, FedoraRawhide, OpenSUSE15, Ubuntu18 and so on # # The arch specific rules use a prefix "$ARCH-" where $ARCH # is a libvirt arch name. @@ -70,6 +70,7 @@ mappings: apparmor: deb: libapparmor-dev + OpenSUSE: libapparmor-devel cross-policy-deb: foreign augeas: @@ -117,6 +118,7 @@ mappings: cppi: Fedora: cppi FreeBSD: cppi + OpenSUSE: cppi cyrus-sasl: deb: libsasl2-dev @@ -127,6 +129,7 @@ mappings: dbus-daemon: default: dbus Fedora: dbus-daemon + OpenSUSE: dbus-1 device-mapper: deb: libdevmapper-dev @@ -196,6 +199,7 @@ mappings: deb: libglusterfs-dev rpm: glusterfs-api-devel Debian9: glusterfs-common + OpenSUSE: glusterfs-devel Ubuntu16: glusterfs-common Ubuntu18: glusterfs-common cross-policy-deb: foreign @@ -204,6 +208,7 @@ mappings: deb: libgnutls28-dev pkg: gnutls rpm: gnutls-devel + OpenSUSE: libgnutls-devel cross-policy-deb: foreign go: @@ -228,6 +233,7 @@ mappings: gtk-update-icon-cache: default: gtk-update-icon-cache + OpenSUSE: gtk3-tools Ubuntu16: libgtk2.0-bin gtk-vnc2: @@ -248,14 +254,17 @@ mappings: ip: deb: iproute2 rpm: iproute + OpenSUSE: iproute2 iscsiadm: deb: open-iscsi rpm: iscsi-initiator-utils + OpenSUSE: open-iscsi isoinfo: default: genisoimage FreeBSD: cdrkit + OpenSUSE: mkisofs java: deb: openjdk-11-jre-headless @@ -289,6 +298,7 @@ mappings: libaudit: deb: libaudit-dev rpm: audit-libs-devel + OpenSUSE: audit-devel cross-policy-deb: foreign libblkid: @@ -320,6 +330,7 @@ mappings: deb: libdbus-1-dev pkg: dbus rpm: dbus-devel + OpenSUSE: dbus-1-devel cross-policy-deb: foreign libgovirt: @@ -347,6 +358,7 @@ mappings: rpm: numactl-devel armv6l-deb: armv7l-deb: + OpenSUSE: libnuma-devel cross-policy-deb: foreign libparted: @@ -370,6 +382,7 @@ mappings: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + OpenSUSE: librbd-devel cross-policy-deb: foreign libselinux: @@ -436,6 +449,7 @@ mappings: deb: locales Fedora: glibc-langpack-en FreeBSD: + OpenSUSE: glibc-locale lsof: default: lsof @@ -458,6 +472,7 @@ mappings: ninja: default: ninja-build FreeBSD: ninja + OpenSUSE: ninja mingw32-curl: Fedora: mingw32-curl @@ -639,6 +654,7 @@ mappings: netcf: deb: libnetcf-dev rpm: netcf-devel + OpenSUSE: cross-policy-deb: skip net-tools: @@ -678,6 +694,7 @@ mappings: deb: perl pkg: perl5 rpm: perl-Archive-Tar + OpenSUSE: perl-Archive-Tar-Wrapper perl-CPAN-Changes: deb: libcpan-changes-perl @@ -709,6 +726,7 @@ mappings: deb: libio-compress-perl pkg: p5-IO-Compress rpm: perl-IO-Compress-Bzip2 + OpenSUSE: perl-Compress-Bzip2 perl-IO-String: deb: libio-string-perl @@ -747,6 +765,7 @@ mappings: deb: libtest-lwp-useragent-perl pkg: p5-Test-LWP-UserAgent Fedora: perl-Test-LWP-UserAgent + OpenSUSE: perl-Test-LWP-UserAgent perl-Test-Pod: deb: libtest-pod-perl @@ -809,20 +828,24 @@ mappings: deb: python-dev pkg: python2 rpm: python2-devel + OpenSUSE: python-devel cross-policy-deb: foreign python2-lxml: default: python-lxml Fedora: python2-lxml FreeBSD: py27-lxml + OpenSUSE: python2-lxml python2-nose: default: python-nose Fedora: python2-nose FreeBSD: py27-nose + OpenSUSE: python2-nose python2-setuptools: CentOS7: python2-setuptools + OpenSUSE: python2-setuptools python3: default: python3 @@ -831,6 +854,7 @@ mappings: default: python3-dbus FreeBSD: py36-dbus CentOS7: python36-dbus + OpenSUSE: python3-dbus-python python3-devel: deb: python3-dev @@ -838,17 +862,20 @@ mappings: Fedora: python3-devel CentOS7: python36-devel cross-policy-deb: foreign + OpenSUSE: python3-devel python3-gi: deb: python3-gi pkg: py36-gobject3 rpm: python3-gobject CentOS7: python36-gobject + OpenSUSE: python3-gobject python3-libxml2: default: python3-libxml2 FreeBSD: py36-libxml2 CentOS7: + OpenSUSE: python3-libxml2-python Ubuntu16: python3-lxml: @@ -864,6 +891,7 @@ mappings: python3-pip: CentOS7: python3-pip Debian9: python3-pip + OpenSUSE: python3-pip Ubuntu16: python3-pip Ubuntu18: python3-pip @@ -884,6 +912,7 @@ mappings: qemu-img: default: qemu-utils rpm: qemu-img + OpenSUSE: qemu-tools radvd: default: radvd @@ -929,6 +958,7 @@ mappings: deb: libspice-client-gtk-3.0-dev pkg: spice-gtk rpm: spice-gtk3-devel + OpenSUSE: spice-gtk-devel cross-policy-deb: foreign strace: @@ -942,6 +972,7 @@ mappings: deb: iproute2 rpm: iproute-tc CentOS7: iproute + OpenSUSE: iproute2 unzip: default: unzip @@ -964,10 +995,12 @@ mappings: wireshark: deb: wireshark-dev Fedora: wireshark-devel + OpenSUSE: wireshark-devel cross-policy-deb: skip xen: Fedora: xen-devel + OpenSUSE: xen-devel x86_64-deb: libxen-dev armv7l-deb: libxen-dev aarch64-deb: libxen-dev @@ -994,18 +1027,21 @@ mappings: xz-static: deb: liblzma-dev Fedora: xz-static + OpenSUSE: xz-static-devel cross-policy-deb: foreign yajl: deb: libyajl-dev pkg: yajl rpm: yajl-devel + OpenSUSE: libyajl-devel cross-policy-deb: foreign zfs: default: zfs-fuse CentOS: FreeBSD: + OpenSUSE: zlib: deb: zlib1g-dev @@ -1015,6 +1051,7 @@ mappings: zlib-static: deb: zlib1g-dev rpm: zlib-static + OpenSUSE: zlib-devel-static cross-policy-deb: foreign -- 2.24.0

3 13

[libvirt] [PATCH v3 0/4] qemu: block: implement optional removal of committed snapshot images
by Pavel Mores 03 Dec '19

03 Dec '19

v3 aims to incorporate Peter's feedback to v2, mainly: - patch 1 was split into two - the delete flag propagation stays in patch 1, the actual enabling the feature was moved to what's now patch 4 - patch 2 was modified to handle NFS and to use g_strerror() instead of strerror_r() - the former patch 4 was squashed into patch 3. Pavel Mores (4): qemu: block: propagate the delete flag to where it can actually be used qemu: block: use the delete flag to delete snapshot images if requested qemu: block: store the delete flag in libvirtd's status XML qemu: block: enable the snapshot image deletion feature src/qemu/qemu_blockjob.c | 43 ++++++++++++++++++- src/qemu/qemu_blockjob.h | 4 +- src/qemu/qemu_domain.c | 6 ++- src/qemu/qemu_domain.h | 6 +++ src/qemu/qemu_driver.c | 5 ++- .../blockjob-blockdev-in.xml | 1 + 6 files changed, 60 insertions(+), 5 deletions(-) -- 2.21.0

2 9

[libvirt] [PATCH 00/21] Incremental backup support for qemu
by Peter Krempa 03 Dec '19

03 Dec '19

This is a finished version of my RFC posted here: https://www.redhat.com/archives/libvir-list/2019-October/msg01193.html All cleanups of that series are now merged and this series has new cleanups and depending patches that I posted earlier this week. For convenience everything can be fetched at: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-backup This series requires qemu 4.2 to work properly. Please make sure to fetch the current head. Compared to previous versions: - The API is similar to what was posted last time or in Eric's version - the backup job ID was removed, this can't be integrated with qemu async jobs as there is only one async jobs. - the virDomainBackupEnd API was removed as virDomainAbortJob can be used instead - the usage of domain job APIs is fully implemented as it was documented in previous eric's postings - the domain job completion event and completed job statistics are used to notify about the state and successful finishing of the job (see virDomainGetJobStats/virsh domjobinfo) - unfortunately block jobs and snapshots are still not supported, thus this feature will stay disabled for now: If you want to give the feature a spin you must use the following XML override: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (16): API: Introduce field for reporting temporary disk space usage of a domain job virsh: Implement VIR_DOMAIN_JOB_DISK_TEMP_(USED|TOTAL) in cmdDomjobinfo API: Add domain job operation for backups qemu: Add infrastructure for statistics of a backup job qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job conf: backup: Add fields for tracking stats of completed sub-jobs doc: Document quirk of getting block job info for a 'backup' blockjob qemu: Implement backup job APIs and qemu handling qemu: backup: Implement stats gathering while the job is running qemu: driver: Allow cancellation of the backup job qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 161 +++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 214 ++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 32 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + po/POTFILES.in | 3 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 499 ++++++++ src/conf/backup_conf.h | 108 ++ src/conf/domain_conf.c | 2 +- src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 12 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 142 +++ src/libvirt_private.syms | 8 + src/libvirt_public.syms | 6 + src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 1017 +++++++++++++++++ src/qemu/qemu_backup.h | 46 + src/qemu/qemu_blockjob.c | 96 +- src/qemu/qemu_blockjob.h | 17 + src/qemu/qemu_domain.c | 140 +++ src/qemu/qemu_domain.h | 14 + src/qemu/qemu_driver.c | 68 +- src/qemu/qemu_migration.c | 2 + src/qemu/qemu_monitor.c | 13 + src/qemu/qemu_monitor.h | 15 + src/qemu/qemu_monitor_json.c | 33 + src/qemu/qemu_monitor_json.h | 8 + src/qemu/qemu_process.c | 25 + src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 33 +- src/remote_protocol-structs | 15 + tests/Makefile.am | 3 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemumonitorjsontest.c | 8 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 144 +++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 26 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 32 + 57 files changed, 3639 insertions(+), 25 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.23.0

3 38

[libvirt] CLZERO support for AMD
by Ani Sinha 03 Dec '19

03 Dec '19

Hi : I see a commit in qemu that adds support for CLZERO but in CPUIDs. : commit e900135dcfb676a4019dc206395e02db5d1de0d6 Author: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> Date: Wed Sep 25 23:49:48 2019 +0200 i386: Add CPUID bit for CLZERO and XSAVEERPTR The CPUID bits CLZERO and XSAVEERPTR are availble on AMD's ZEN platform and could be passed to the guest. Signed-off-by: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> Is there a corresponding change in libvirt adding this capability? Libvirt complains when I add it manually in the vm xml: # virsh create newvm.xml error: Failed to create domain from newvm.xml error: unsupported configuration: unknown CPU feature: clzero # grep clzero newvm.xml <feature policy='require' name='clzero'/> Should we add it? thanks ani

2 2

[libvirt] [PATCH 0/9] qemu: clean up capability handling during startup (incremental backup prequels)
by Peter Krempa 02 Dec '19

02 Dec '19

For libvirt to use incremental backup we will require that blockdev is enabled. Clean up how we handle caps during startup since blockdev may be masked out there and add infrastructure to interlock flags. Peter Krempa (9): qemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque qemu: process: Don't try to redetect missing qemuCaps on reconnect qemu: Move and rename qemuDomainUpdateQEMUCaps qemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps qemu: process: Move handling of qemu capability overrides qemu: caps: Don't check capability before clearing it qemu: capabilities: Lock out incremental backup capability without blockdev qemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps src/qemu/qemu_capabilities.c | 20 +++++++++- src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_domain.c | 26 ------------ src/qemu/qemu_domain.h | 4 -- src/qemu/qemu_process.c | 76 ++++++++++++++++++++++++------------ 5 files changed, 72 insertions(+), 56 deletions(-) -- 2.23.0

4 27

[libvirt] [PATCH 0/2] qemu: blockcopy: fix regression with block devices
by Peter Krempa 02 Dec '19

02 Dec '19

Peter Krempa (2): qemu: blockcopy: Report error on image format detection failure qemu: blockcopy: Fix conditions when virStorageSource should be initialized src/qemu/qemu_driver.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) -- 2.23.0

2 4

[libvirt] [PATCH 0/2] Extend the valid range of scsi unit number
by Han Han 02 Dec '19

02 Dec '19

Han Han (2): conf: Allow virtio scsi to using unit 16383 rng: Extend the valid range of drive unit docs/schemas/domaincommon.rng | 2 +- src/conf/domain_conf.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.23.0

2 5

[libvirt] [PATCH] spec: Disable gluster on i686 on RHEL-8
by Jiri Denemark 29 Nov '19

29 Nov '19

Gluster 6.0 is not built on i686 for RHEL-8, which prevents libvirt from building. Let's just disable gluster there as all we need are client libraries anyway. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- libvirt.spec.in | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index a6219da604..c7fac104d9 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -65,7 +65,15 @@ %else %define with_storage_sheepdog 0 %endif + %define with_storage_gluster 0%{!?_without_storage_gluster:1} +%ifnarch %{qemu_kvm_arches} + # gluster is only built where qemu driver is enabled on RHEL 8 + %if 0%{?rhel} >= 8 + %define with_storage_gluster 0 + %endif +%endif + %define with_numactl 0%{!?_without_numactl:1} # F25+ has zfs-fuse -- 2.24.0

2 1