November 2019 - Devel - libvirt List Archives

[libvirt] [PATCH v2 00/15] docs: some refactoring and new docs about RPM deployment
by Daniel P. Berrangé 04 Dec '19

04 Dec '19

This refactors existing docs related to the remote driver/daemon and URIs. It then also adds a kbase page about RPM package options. This introduces the use of RST for docs as a replacement for HTML. The intent is that all new docs should use RST from this point. The POD man pages will also be converted to RST. I'll post this as a different series. Note this series only touches the main HTML content, not the styling / templating using XSL. Tackling that is a separate job I've not attempted. My guess is that we'd use either Sphinx or Pelican to do the templating. Existing HTML docs are candidates for conversion to RST, however, people should *NOT* attempt todo this manually. In this series I've converted docs/kbase/ files, using the pandoc tool. This does a pretty good job in general but does need some manual cleanups. First I post-processed its output to change the heading highlighting to follow the documented style $ cat convert.pl my @in = <>; my @out; for (my $i ; $i <= $#in; $i++) { my $line = $in[$i]; if ($line =~ /^=+$/) { my @newout = splice @out, 0, $i - 1; push @newout, $line; push @newout, $out[$i-1]; push @newout, $out[$i]; push @newout, $line; push @newout, "\n"; push @newout, ".. contents::\n"; @out = @newout; } elsif ($line =~ /^(-|~|^|')+$/) { $line =~ s/-/=/g; $line =~ s/~/-/g; $line =~ s/\^/~/g; $line =~ s/'/^/g; push @out, $line; } else { push @out, $line; } } print @out; So I'm converting with pandoc -f html -t rst < foo.html.in | perl convert.pl > foo.rst $EDITOR foot.rst git rm -f foot.html.in git add foo.rst After pandoc & the headings convert, there's usually manual fixes needed to deal with a few oddities pandoc gets wrong, or looks ugly. This is still way simpler than converting the doc to rst manually. Changed in v2: - Added a style guide for RST docs - Probe for rst2html's different names - Added make rules for building rst - Fixed permalinks in generated docs - Misc CSS fixes - Auto-converted all of kbase/ directory Daniel P. Berrangé (15): docs: split TLS certificate setup into its own file docs: move docs about remote driver URIs into URI docs docs: introduce rst2html as a mandatory tool for building docs docs: adapt filling of <head> section for rst2html output docs: generate permalinks correctly for rst2html output docs: relax CSS context match for pretty tables docs: add styling for <tt> element docs: add a minimal style guide for writing RST docs docs: convert kbase/domainstatecapture.html.in to RST docs: convert kbase/launch_security_sev.html.in to RST docs: convert kbase/secureusage.html.in to RST docs: convert kbase/locking.html.in to RST docs: convert kbase/locking-lockd.html.in to RST docs: convert kbase/locking-sanlock.html.in to RST docs: add a kbase page about RPM packaging options docs/Makefile.am | 33 +- docs/aclpolkit.html.in | 20 +- docs/docs.html.in | 6 + docs/genaclperms.pl | 2 +- docs/generic.css | 4 +- docs/kbase.html.in | 4 + docs/kbase/domainstatecapture.html.in | 303 ----------- docs/kbase/domainstatecapture.rst | 255 +++++++++ docs/kbase/launch_security_sev.html.in | 533 ------------------- docs/kbase/launch_security_sev.rst | 529 +++++++++++++++++++ docs/kbase/locking-lockd.html.in | 160 ------ docs/kbase/locking-lockd.rst | 121 +++++ docs/kbase/locking-sanlock.html.in | 247 --------- docs/kbase/locking-sanlock.rst | 193 +++++++ docs/kbase/locking.html.in | 48 -- docs/kbase/locking.rst | 33 ++ docs/kbase/rpm-deployment.rst | 410 +++++++++++++++ docs/kbase/secureusage.html.in | 171 ------- docs/kbase/secureusage.rst | 131 +++++ docs/libvirt.css | 76 +-- docs/migration.html.in | 2 +- docs/newapi.xsl | 4 +- docs/page.xsl | 9 +- docs/remote.html.in | 684 +------------------------ docs/styleguide.rst | 66 +++ docs/tlscerts.html.in | 413 +++++++++++++++ docs/uri.html.in | 263 ++++++++-- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 5 + mingw-libvirt.spec.in | 1 + 30 files changed, 2478 insertions(+), 2250 deletions(-) delete mode 100644 docs/kbase/domainstatecapture.html.in create mode 100644 docs/kbase/domainstatecapture.rst delete mode 100644 docs/kbase/launch_security_sev.html.in create mode 100644 docs/kbase/launch_security_sev.rst delete mode 100644 docs/kbase/locking-lockd.html.in create mode 100644 docs/kbase/locking-lockd.rst delete mode 100644 docs/kbase/locking-sanlock.html.in create mode 100644 docs/kbase/locking-sanlock.rst delete mode 100644 docs/kbase/locking.html.in create mode 100644 docs/kbase/locking.rst create mode 100644 docs/kbase/rpm-deployment.rst delete mode 100644 docs/kbase/secureusage.html.in create mode 100644 docs/kbase/secureusage.rst create mode 100644 docs/styleguide.rst create mode 100644 docs/tlscerts.html.in -- 2.23.0

2 17

[libvirt] [PATCH RFC V2] guests: Add support for openSUSE
by Jim Fehlig 03 Dec '19

03 Dec '19

This change adds support for the latest openSUSE Leap and openSUSE Tumbleweed guests. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- This version of the patch overcomes the 'lci build' issues noted in V1 and works well for openSUSE Leap 15.1 and Tumbleweed. I'm still of the opinion of only supporting the latest Leap since there is only a six month support overlap between versions. Version x-1 is only supported six months after x is released. 'lci update ... libvirt' fails in task "Configure hostname": fatal: [libvirt-opensuse-tumbleweed]: FAILED! => {"changed": false, "msg": "hostname module cannot be used on platform Linux (Opensuse-tumbleweed)"} Not sure why that is the case. But more importantly, even though it doesn't fail, the "Update installed packages" task doesn't install any of the packages required to build libvirt. I suppose that agrees with guests/REAMDME, but how then are the packages required to build $project installed? How/when is the list of packages required to build $project conveyed to $package_manager? 'lcitool build libvirt-opensuse-tumbleweed libvirt' fails in the "Gathering Facts" task: fatal: [libvirt-opensuse-tumbleweed]: UNREACHABLE! => {"changed": false, "msg": "Invalid/incorrect password: ", "unreachable": true} Passing --debug to the command didn't help with determining what needed the password. ssh? I can't find where the "Gathering Facts" task is defined. The $root_password_file passed to /usr/bin/ansible-playbook contains the correct password for the root user. /me needs to figure out how to better debug this stuff :-). guests/configs/autoinst.xml | 75 +++++++++++++++++++ .../host_vars/libvirt-opensuse-15/docker.yml | 2 + .../host_vars/libvirt-opensuse-15/install.yml | 2 + guests/host_vars/libvirt-opensuse-15/main.yml | 22 ++++++ .../libvirt-opensuse-tumbleweed/docker.yml | 2 + .../libvirt-opensuse-tumbleweed/install.yml | 2 + .../libvirt-opensuse-tumbleweed/main.yml | 22 ++++++ guests/inventory | 2 + guests/lcitool | 2 + guests/playbooks/update/tasks/base.yml | 15 ++++ guests/vars/mappings.yml | 41 +++++++++- 11 files changed, 185 insertions(+), 2 deletions(-) diff --git a/guests/configs/autoinst.xml b/guests/configs/autoinst.xml new file mode 100644 index 0000000..42c42f8 --- /dev/null +++ b/guests/configs/autoinst.xml @@ -0,0 +1,75 @@ +<?xml version="1.0"?> +<!DOCTYPE profile> +<profile + xmlns="http://www.suse.com/1.0/yast2ns" + xmlns:config="http://www.suse.com/1.0/configns"> + <general> + <mode> + <confirm config:type="boolean">false</confirm> + </mode> + </general> + <partitioning config:type="list"> + <drive> + <device>/dev/vda</device> + <use>all</use> + <partitions config:type="list"> + <partition> + <filesystem config:type="symbol">swap</filesystem> + <size>256M</size> + <mount>swap</mount> + </partition> + <partition> + <filesystem config:type="symbol">ext4</filesystem> + <mount>/</mount> + <size>max</size> + </partition> + </partitions> + </drive> + </partitioning> + <bootloader> + <global> + <terminal>console serial</terminal> + </global> + </bootloader> + <timezone> + <hwclock>UTC</hwclock> + <timezone>UTC</timezone> + </timezone> + <software> + <install_recommended config:type="boolean">false</install_recommended> + <products config:type="list"> + <product>openSUSE</product> + </products> + <patterns config:type="list"> + <pattern>base</pattern> + <pattern>minimal_base</pattern> + <pattern>yast2_basis</pattern> + </patterns> + <packages config:type="list"> + <package>openssh</package> + </packages> + </software> + <networking> + <keep_install_network config:type="boolean">true</keep_install_network> + </networking> + <users config:type="list"> + <user> + <username>root</username> + <user_password>root</user_password> + <encrypted config:type="boolean">false</encrypted> + <uid>0</uid> + <gid>0</gid> + <home>/root</home> + <shell>/bin/bash</shell> + </user> + </users> + <services-manager> + <default_target>multi-user</default_target> + <services config:type="list"> + <service> + <service_name>sshd</service_name> + <service_status>enable</service_status> + </service> + </services> + </services-manager> +</profile> diff --git a/guests/host_vars/libvirt-opensuse-15/docker.yml b/guests/host_vars/libvirt-opensuse-15/docker.yml new file mode 100644 index 0000000..8f32d06 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: opensuse/leap:15 diff --git a/guests/host_vars/libvirt-opensuse-15/install.yml b/guests/host_vars/libvirt-opensuse-15/install.yml new file mode 100644 index 0000000..d0fdbe5 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://download.opensuse.org/distribution/leap/15.1/repo/oss/ diff --git a/guests/host_vars/libvirt-opensuse-15/main.yml b/guests/host_vars/libvirt-opensuse-15/main.yml new file mode 100644 index 0000000..abd83c5 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-15/main.yml @@ -0,0 +1,22 @@ +--- +projects: + - libosinfo + - libvirt + - libvirt-cim + - libvirt-dbus + - libvirt-glib + - libvirt-perl + - libvirt-python + - libvirt-sandbox + - libvirt-tck + - osinfo-db + - osinfo-db-tools + - virt-manager + - virt-viewer + +package_format: 'rpm' +package_manager: 'zypper' +os_name: 'OpenSUSE' +os_version: '15' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml new file mode 100644 index 0000000..cc79a22 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/docker.yml @@ -0,0 +1,2 @@ +--- +docker_base: opensuse/tumbleweed diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml new file mode 100644 index 0000000..36cea68 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/install.yml @@ -0,0 +1,2 @@ +--- +install_url: http://download.opensuse.org/tumbleweed/repo/oss/ diff --git a/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml new file mode 100644 index 0000000..828df81 --- /dev/null +++ b/guests/host_vars/libvirt-opensuse-tumbleweed/main.yml @@ -0,0 +1,22 @@ +--- +projects: + - libosinfo + - libvirt + - libvirt-cim + - libvirt-dbus + - libvirt-glib + - libvirt-perl + - libvirt-python + - libvirt-sandbox + - libvirt-tck + - osinfo-db + - osinfo-db-tools + - virt-manager + - virt-viewer + +package_format: 'rpm' +package_manager: 'zypper' +os_name: 'OpenSUSE' +os_version: 'Tumbleweed' + +ansible_python_interpreter: /usr/bin/python3 diff --git a/guests/inventory b/guests/inventory index 3b15513..71d3c91 100644 --- a/guests/inventory +++ b/guests/inventory @@ -8,5 +8,7 @@ libvirt-fedora-rawhide libvirt-freebsd-11 libvirt-freebsd-12 libvirt-freebsd-current +libvirt-opensuse-15 +libvirt-opensuse-tumbleweed libvirt-ubuntu-16 libvirt-ubuntu-18 diff --git a/guests/lcitool b/guests/lcitool index a630971..7f26731 100755 --- a/guests/lcitool +++ b/guests/lcitool @@ -531,6 +531,8 @@ class Application: install_config = "preseed.cfg" elif facts["os_name"] in ["CentOS", "Fedora"]: install_config = "kickstart.cfg" + elif facts["os_name"] == "OpenSUSE": + install_config = "autoinst.xml" else: raise Exception( "Host {} doesn't support installation".format(host) diff --git a/guests/playbooks/update/tasks/base.yml b/guests/playbooks/update/tasks/base.yml index 3d83e78..b6a2cd2 100644 --- a/guests/playbooks/update/tasks/base.yml +++ b/guests/playbooks/update/tasks/base.yml @@ -65,12 +65,27 @@ when: - package_format == 'pkg' +- name: Update installed packages + command: '{{ package_manager }} update -y -l --force-resolution --no-recommends' + args: + warn: no + when: + - os_name == 'OpenSUSE' + - name: Clean up packages after update shell: '{{ package_manager }} clean packages -y && {{ package_manager }} autoremove -y' args: warn: no when: - package_format == 'rpm' + - not os_name == "OpenSUSE" + +- name: Clean up packages after update + shell: '{{ package_manager }} clean' + args: + warn: no + when: + - os_name == "OpenSUSE" - name: Clean up packages after update apt: diff --git a/guests/vars/mappings.yml b/guests/vars/mappings.yml index bd9b161..8359ce2 100644 --- a/guests/vars/mappings.yml +++ b/guests/vars/mappings.yml @@ -19,10 +19,10 @@ # - deb, pkg, rpm # # Valid OS names are: -# - CentOS, Debian, Fedora, FreeBSD, Ubuntu +# - CentOS, Debian, Fedora, FreeBSD, OpenSUSE, Ubuntu # # Valid OS versions are: -# - CentOS7, Debian9, FedoraRawhide, Ubuntu18 and so on +# - CentOS7, Debian9, FedoraRawhide, OpenSUSE15, Ubuntu18 and so on # # The arch specific rules use a prefix "$ARCH-" where $ARCH # is a libvirt arch name. @@ -70,6 +70,7 @@ mappings: apparmor: deb: libapparmor-dev + OpenSUSE: libapparmor-devel cross-policy-deb: foreign augeas: @@ -117,6 +118,7 @@ mappings: cppi: Fedora: cppi FreeBSD: cppi + OpenSUSE: cppi cyrus-sasl: deb: libsasl2-dev @@ -127,6 +129,7 @@ mappings: dbus-daemon: default: dbus Fedora: dbus-daemon + OpenSUSE: dbus-1 device-mapper: deb: libdevmapper-dev @@ -196,6 +199,7 @@ mappings: deb: libglusterfs-dev rpm: glusterfs-api-devel Debian9: glusterfs-common + OpenSUSE: glusterfs-devel Ubuntu16: glusterfs-common Ubuntu18: glusterfs-common cross-policy-deb: foreign @@ -204,6 +208,7 @@ mappings: deb: libgnutls28-dev pkg: gnutls rpm: gnutls-devel + OpenSUSE: libgnutls-devel cross-policy-deb: foreign go: @@ -228,6 +233,7 @@ mappings: gtk-update-icon-cache: default: gtk-update-icon-cache + OpenSUSE: gtk3-tools Ubuntu16: libgtk2.0-bin gtk-vnc2: @@ -248,14 +254,17 @@ mappings: ip: deb: iproute2 rpm: iproute + OpenSUSE: iproute2 iscsiadm: deb: open-iscsi rpm: iscsi-initiator-utils + OpenSUSE: open-iscsi isoinfo: default: genisoimage FreeBSD: cdrkit + OpenSUSE: mkisofs java: deb: openjdk-11-jre-headless @@ -289,6 +298,7 @@ mappings: libaudit: deb: libaudit-dev rpm: audit-libs-devel + OpenSUSE: audit-devel cross-policy-deb: foreign libblkid: @@ -320,6 +330,7 @@ mappings: deb: libdbus-1-dev pkg: dbus rpm: dbus-devel + OpenSUSE: dbus-1-devel cross-policy-deb: foreign libgovirt: @@ -347,6 +358,7 @@ mappings: rpm: numactl-devel armv6l-deb: armv7l-deb: + OpenSUSE: libnuma-devel cross-policy-deb: foreign libparted: @@ -370,6 +382,7 @@ mappings: deb: librbd-dev Fedora: librbd-devel CentOS7: librbd1-devel + OpenSUSE: librbd-devel cross-policy-deb: foreign libselinux: @@ -436,6 +449,7 @@ mappings: deb: locales Fedora: glibc-langpack-en FreeBSD: + OpenSUSE: glibc-locale lsof: default: lsof @@ -458,6 +472,7 @@ mappings: ninja: default: ninja-build FreeBSD: ninja + OpenSUSE: ninja mingw32-curl: Fedora: mingw32-curl @@ -639,6 +654,7 @@ mappings: netcf: deb: libnetcf-dev rpm: netcf-devel + OpenSUSE: cross-policy-deb: skip net-tools: @@ -678,6 +694,7 @@ mappings: deb: perl pkg: perl5 rpm: perl-Archive-Tar + OpenSUSE: perl-Archive-Tar-Wrapper perl-CPAN-Changes: deb: libcpan-changes-perl @@ -709,6 +726,7 @@ mappings: deb: libio-compress-perl pkg: p5-IO-Compress rpm: perl-IO-Compress-Bzip2 + OpenSUSE: perl-Compress-Bzip2 perl-IO-String: deb: libio-string-perl @@ -747,6 +765,7 @@ mappings: deb: libtest-lwp-useragent-perl pkg: p5-Test-LWP-UserAgent Fedora: perl-Test-LWP-UserAgent + OpenSUSE: perl-Test-LWP-UserAgent perl-Test-Pod: deb: libtest-pod-perl @@ -809,20 +828,24 @@ mappings: deb: python-dev pkg: python2 rpm: python2-devel + OpenSUSE: python-devel cross-policy-deb: foreign python2-lxml: default: python-lxml Fedora: python2-lxml FreeBSD: py27-lxml + OpenSUSE: python2-lxml python2-nose: default: python-nose Fedora: python2-nose FreeBSD: py27-nose + OpenSUSE: python2-nose python2-setuptools: CentOS7: python2-setuptools + OpenSUSE: python2-setuptools python3: default: python3 @@ -831,6 +854,7 @@ mappings: default: python3-dbus FreeBSD: py36-dbus CentOS7: python36-dbus + OpenSUSE: python3-dbus-python python3-devel: deb: python3-dev @@ -838,17 +862,20 @@ mappings: Fedora: python3-devel CentOS7: python36-devel cross-policy-deb: foreign + OpenSUSE: python3-devel python3-gi: deb: python3-gi pkg: py36-gobject3 rpm: python3-gobject CentOS7: python36-gobject + OpenSUSE: python3-gobject python3-libxml2: default: python3-libxml2 FreeBSD: py36-libxml2 CentOS7: + OpenSUSE: python3-libxml2-python Ubuntu16: python3-lxml: @@ -864,6 +891,7 @@ mappings: python3-pip: CentOS7: python3-pip Debian9: python3-pip + OpenSUSE: python3-pip Ubuntu16: python3-pip Ubuntu18: python3-pip @@ -884,6 +912,7 @@ mappings: qemu-img: default: qemu-utils rpm: qemu-img + OpenSUSE: qemu-tools radvd: default: radvd @@ -929,6 +958,7 @@ mappings: deb: libspice-client-gtk-3.0-dev pkg: spice-gtk rpm: spice-gtk3-devel + OpenSUSE: spice-gtk-devel cross-policy-deb: foreign strace: @@ -942,6 +972,7 @@ mappings: deb: iproute2 rpm: iproute-tc CentOS7: iproute + OpenSUSE: iproute2 unzip: default: unzip @@ -964,10 +995,12 @@ mappings: wireshark: deb: wireshark-dev Fedora: wireshark-devel + OpenSUSE: wireshark-devel cross-policy-deb: skip xen: Fedora: xen-devel + OpenSUSE: xen-devel x86_64-deb: libxen-dev armv7l-deb: libxen-dev aarch64-deb: libxen-dev @@ -994,18 +1027,21 @@ mappings: xz-static: deb: liblzma-dev Fedora: xz-static + OpenSUSE: xz-static-devel cross-policy-deb: foreign yajl: deb: libyajl-dev pkg: yajl rpm: yajl-devel + OpenSUSE: libyajl-devel cross-policy-deb: foreign zfs: default: zfs-fuse CentOS: FreeBSD: + OpenSUSE: zlib: deb: zlib1g-dev @@ -1015,6 +1051,7 @@ mappings: zlib-static: deb: zlib1g-dev rpm: zlib-static + OpenSUSE: zlib-devel-static cross-policy-deb: foreign -- 2.24.0

3 13

[libvirt] [PATCH v3 0/4] qemu: block: implement optional removal of committed snapshot images
by Pavel Mores 03 Dec '19

03 Dec '19

v3 aims to incorporate Peter's feedback to v2, mainly: - patch 1 was split into two - the delete flag propagation stays in patch 1, the actual enabling the feature was moved to what's now patch 4 - patch 2 was modified to handle NFS and to use g_strerror() instead of strerror_r() - the former patch 4 was squashed into patch 3. Pavel Mores (4): qemu: block: propagate the delete flag to where it can actually be used qemu: block: use the delete flag to delete snapshot images if requested qemu: block: store the delete flag in libvirtd's status XML qemu: block: enable the snapshot image deletion feature src/qemu/qemu_blockjob.c | 43 ++++++++++++++++++- src/qemu/qemu_blockjob.h | 4 +- src/qemu/qemu_domain.c | 6 ++- src/qemu/qemu_domain.h | 6 +++ src/qemu/qemu_driver.c | 5 ++- .../blockjob-blockdev-in.xml | 1 + 6 files changed, 60 insertions(+), 5 deletions(-) -- 2.21.0

2 9

[libvirt] [PATCH 00/21] Incremental backup support for qemu
by Peter Krempa 03 Dec '19

03 Dec '19

This is a finished version of my RFC posted here: https://www.redhat.com/archives/libvir-list/2019-October/msg01193.html All cleanups of that series are now merged and this series has new cleanups and depending patches that I posted earlier this week. For convenience everything can be fetched at: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-backup This series requires qemu 4.2 to work properly. Please make sure to fetch the current head. Compared to previous versions: - The API is similar to what was posted last time or in Eric's version - the backup job ID was removed, this can't be integrated with qemu async jobs as there is only one async jobs. - the virDomainBackupEnd API was removed as virDomainAbortJob can be used instead - the usage of domain job APIs is fully implemented as it was documented in previous eric's postings - the domain job completion event and completed job statistics are used to notify about the state and successful finishing of the job (see virDomainGetJobStats/virsh domjobinfo) - unfortunately block jobs and snapshots are still not supported, thus this feature will stay disabled for now: If you want to give the feature a spin you must use the following XML override: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (16): API: Introduce field for reporting temporary disk space usage of a domain job virsh: Implement VIR_DOMAIN_JOB_DISK_TEMP_(USED|TOTAL) in cmdDomjobinfo API: Add domain job operation for backups qemu: Add infrastructure for statistics of a backup job qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job conf: backup: Add fields for tracking stats of completed sub-jobs doc: Document quirk of getting block job info for a 'backup' blockjob qemu: Implement backup job APIs and qemu handling qemu: backup: Implement stats gathering while the job is running qemu: driver: Allow cancellation of the backup job qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 161 +++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 214 ++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 32 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + po/POTFILES.in | 3 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 499 ++++++++ src/conf/backup_conf.h | 108 ++ src/conf/domain_conf.c | 2 +- src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 12 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 142 +++ src/libvirt_private.syms | 8 + src/libvirt_public.syms | 6 + src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 1017 +++++++++++++++++ src/qemu/qemu_backup.h | 46 + src/qemu/qemu_blockjob.c | 96 +- src/qemu/qemu_blockjob.h | 17 + src/qemu/qemu_domain.c | 140 +++ src/qemu/qemu_domain.h | 14 + src/qemu/qemu_driver.c | 68 +- src/qemu/qemu_migration.c | 2 + src/qemu/qemu_monitor.c | 13 + src/qemu/qemu_monitor.h | 15 + src/qemu/qemu_monitor_json.c | 33 + src/qemu/qemu_monitor_json.h | 8 + src/qemu/qemu_process.c | 25 + src/remote/remote_driver.c | 2 + src/remote/remote_protocol.x | 33 +- src/remote_protocol-structs | 15 + tests/Makefile.am | 3 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemumonitorjsontest.c | 8 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 144 +++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 26 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 32 + 57 files changed, 3639 insertions(+), 25 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.23.0

3 38

[libvirt] CLZERO support for AMD
by Ani Sinha 03 Dec '19

03 Dec '19

Hi : I see a commit in qemu that adds support for CLZERO but in CPUIDs. : commit e900135dcfb676a4019dc206395e02db5d1de0d6 Author: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> Date: Wed Sep 25 23:49:48 2019 +0200 i386: Add CPUID bit for CLZERO and XSAVEERPTR The CPUID bits CLZERO and XSAVEERPTR are availble on AMD's ZEN platform and could be passed to the guest. Signed-off-by: Sebastian Andrzej Siewior <bigeasy(a)linutronix.de> Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> Is there a corresponding change in libvirt adding this capability? Libvirt complains when I add it manually in the vm xml: # virsh create newvm.xml error: Failed to create domain from newvm.xml error: unsupported configuration: unknown CPU feature: clzero # grep clzero newvm.xml <feature policy='require' name='clzero'/> Should we add it? thanks ani

2 2

[libvirt] [PATCH 0/9] qemu: clean up capability handling during startup (incremental backup prequels)
by Peter Krempa 02 Dec '19

02 Dec '19

For libvirt to use incremental backup we will require that blockdev is enabled. Clean up how we handle caps during startup since blockdev may be masked out there and add infrastructure to interlock flags. Peter Krempa (9): qemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque qemu: process: Don't try to redetect missing qemuCaps on reconnect qemu: Move and rename qemuDomainUpdateQEMUCaps qemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps qemu: process: Move handling of qemu capability overrides qemu: caps: Don't check capability before clearing it qemu: capabilities: Lock out incremental backup capability without blockdev qemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps src/qemu/qemu_capabilities.c | 20 +++++++++- src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_domain.c | 26 ------------ src/qemu/qemu_domain.h | 4 -- src/qemu/qemu_process.c | 76 ++++++++++++++++++++++++------------ 5 files changed, 72 insertions(+), 56 deletions(-) -- 2.23.0

4 27

[libvirt] [PATCH 0/2] qemu: blockcopy: fix regression with block devices
by Peter Krempa 02 Dec '19

02 Dec '19

Peter Krempa (2): qemu: blockcopy: Report error on image format detection failure qemu: blockcopy: Fix conditions when virStorageSource should be initialized src/qemu/qemu_driver.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) -- 2.23.0

2 4

[libvirt] [PATCH 0/2] Extend the valid range of scsi unit number
by Han Han 02 Dec '19

02 Dec '19

Han Han (2): conf: Allow virtio scsi to using unit 16383 rng: Extend the valid range of drive unit docs/schemas/domaincommon.rng | 2 +- src/conf/domain_conf.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.23.0

2 5

[libvirt] [PATCH] spec: Disable gluster on i686 on RHEL-8
by Jiri Denemark 29 Nov '19

29 Nov '19

Gluster 6.0 is not built on i686 for RHEL-8, which prevents libvirt from building. Let's just disable gluster there as all we need are client libraries anyway. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- libvirt.spec.in | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/libvirt.spec.in b/libvirt.spec.in index a6219da604..c7fac104d9 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -65,7 +65,15 @@ %else %define with_storage_sheepdog 0 %endif + %define with_storage_gluster 0%{!?_without_storage_gluster:1} +%ifnarch %{qemu_kvm_arches} + # gluster is only built where qemu driver is enabled on RHEL 8 + %if 0%{?rhel} >= 8 + %define with_storage_gluster 0 + %endif +%endif + %define with_numactl 0%{!?_without_numactl:1} # F25+ has zfs-fuse -- 2.24.0

2 1

[libvirt] [PATCH] remote_daemon_stream: Hold an extra reference to stream in daemonStreamFilter
by Michal Privoznik 29 Nov '19

29 Nov '19

In v5.9.0-273-g8ecab214de I've tried to fix a lock ordering problem, but introduced a crasher. Problem is that because the client lock is unlocked (in order to honour lock ordering) the stream we are currently checking in daemonStreamFilter() might be freed and thus stream->priv might not even exist when the control get to virMutexLock() call. To resolve this, grab an extra reference to the stream and handle its cleanup should the refcounter reach zero after the deref. If that's the case and we are the only ones holding a reference to the stream, we MUST return a positive value to make virNetServerClientDispatchRead() break its loop where it iterates over filters. The problem is, if we did not do so, then "filter = filter->next" line will read from a memory that was just freed (freeing a stream also unregisters its filter). Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- Reproducing this issue is very easy: 1) put sleep(5) right after virObjectUnlock(client) in the fist hunk, 2) virsh console --force $dom and type something so that the stream has some data to process 3) while 2) is still running, run the same command from another terminal 4) observe libvirtd crash src/remote/remote_daemon_stream.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 82cadb67ac..73e4d7befb 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -293,10 +293,25 @@ daemonStreamFilter(virNetServerClientPtr client, daemonClientStream *stream = opaque; int ret = 0; + /* We must honour lock ordering here. Client private data lock must + * be acquired before client lock. Bu we are already called with + * client locked. To avoid stream disappearing while we unlock + * everything, let's increase its refcounter. This has some + * implications though. */ + stream->refs++; virObjectUnlock(client); virMutexLock(&stream->priv->lock); virObjectLock(client); + if (stream->refs == 1) { + /* So we are the only ones holding the reference to the stream. + * Return 1 to signal to the caller that we've processed the + * message. And to "process" means free. */ + virNetMessageFree(msg); + ret = 1; + goto cleanup; + } + if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) goto cleanup; @@ -318,6 +333,10 @@ daemonStreamFilter(virNetServerClientPtr client, cleanup: virMutexUnlock(&stream->priv->lock); + /* Don't pass client here, because client is locked here and this + * function might try to lock it again which would result in a + * deadlock. */ + daemonFreeClientStream(NULL, stream); return ret; } -- 2.23.0

3 8