November 2019 - Devel - libvirt List Archives

[libvirt] [PATCH v2 0/4] PCI hostdev partial assignment support
by Daniel Henrique Barboza 22 Nov '19

22 Nov '19

Changes from previous version [1], all of them result of feedback from Alex Williamson and Abdulla Bubshait: - use <address type='none'/> instead of creating a new subsys attribute; - expand the change to all PCI hostdevs. Former patch 01 was discarded because we don't need the PCI Multifunction helpers for now; - series changed name to reflect what it's being done - new patch 04: add documentation to formatdomain.html.in To avoid a huge wall of text please refer to [1] for context about the road up to here. Commit msgs of the first 3 patches tells the story as well. [1] https://www.redhat.com/archives/libvir-list/2019-October/msg00298.html What I want to discuss here instead is a caveat that I've found while testing this work, since its first version. This test was done in a Power 8 system with a Broadcom BCM5719 PCIe Multifunction card, with 4 virtual functions. This series enables Libvirt to declare PCI hostdevs that will not be visible to the guest using address type='none'. During the tests I faced a scenario that I expected to fail, but it didn't. This is the relevant XML except: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x0'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x2'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x3'/> </source> <address type='none'/> </hostdev> I'm declaring all the BCM5719 functions in the XML, but I am making functions 0, 1 and 3 unassignable by the guest using address type='none'. This test was meant to fail, but it didn't. To my surprise the guest booted and the device is functional: $ lspci 0000:00:01.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon 0000:00:03.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01) 0000:00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device 0001:00:01.2 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) $ I've talked with Michael Roth (QEMU PPC64 developer that worked with the PCI multifunction hotplug/unplug support in the PPC64 machine) about this. He mentioned that this is intended. I'll quote here what he had to say about it: "The current logic is that we only emit the hotplug event when function 0 is attached, but if some other function is attached at boot-time the guest will still see it on the bus, and whether that works or not I think is up to the device/driver" This explains why this test didn't fail as I expected. At least for the PPC64 machine, depending on the device support, this setup is allowed. PPC64 machine uses function 0 hotplug as a signal of 'plug all the queue functions and function 0', but function 0 isn't required at boot time. I would like to hear other opinions in this because I can't say whether this is allowed in x86. I am mentioning all this now because this had a direct impact on the design of this work since the previous version, and I failed to bring it up back then. I am *not* checking for the assignment of function 0 at guest boot time in Libvirt, leaving the user free to decide what to do. I am aware that this will be inconsistent to the logic of the PCI multifunction hotplug/unplug support, where function 0 is required. This also puts a lot of faith in the user, relying that the user is fully aware of the capabilities of the hardware. My question is: should Libvirt force function 0 to be present in boot time as well, regardless of whether the PPC64 guest or some cards are able to boot without it? Thanks, DHB Daniel Henrique Barboza (4): domain_conf: allow address type='none' to unassign PCI hostdevs qemu: handle unassigned PCI hostdevs in command line and alias virhostdev.c: check all IOMMU devs in virHostdevPreparePCIDevices formatdomain.html.in: document <address type='none'/> docs/formatdomain.html.in | 15 +++++ docs/schemas/domaincommon.rng | 5 ++ src/conf/domain_conf.c | 56 ++++++++++++++-- src/conf/domain_conf.h | 3 + src/qemu/qemu_alias.c | 6 ++ src/qemu/qemu_command.c | 4 ++ src/qemu/qemu_domain_address.c | 6 ++ src/util/virhostdev.c | 64 +++++++++++++++++-- .../hostdev-pci-address-none.args | 31 +++++++++ .../hostdev-pci-address-none.xml | 42 ++++++++++++ ...ostdev-pci-multifunction-partial-fail.args | 31 +++++++++ ...hostdev-pci-multifunction-partial-fail.xml | 35 ++++++++++ tests/qemuxml2argvtest.c | 8 +++ .../hostdev-pci-address-none.xml | 58 +++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 15 files changed, 352 insertions(+), 13 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.xml create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-address-none.xml -- 2.21.0

3 8

[libvirt] [PATCH 0/1] Improve travis CI coverage
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

There is currently a serious hardware outage with the host running our Jenkins CI slaves and we're not sure when we'll get it back up and running. Ths leaves us with Travis CI in the meanwhile. This expands the Travis CI coverage to add latest Fedora stable and rawhide, so that it is covering a representative set of distros overall. Daniel P. Berrangé (1): travis: add fedora-31 & fedora-rawhide to the build images .travis.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) -- 2.21.0

2 2

[libvirt] [PATCH v4 00/20] cleanup current build system
by Pavel Hrdina 22 Nov '19

22 Nov '19

As preparation to switch to Meson there are some things that needs be cleaned up to make the conversion easier. The important thing in Meson is that there is a strict separation between source and build directory and the distributed tarball by default contains only files tracked by git with a possibility to write a script which would add some other sources into the tarball. Regardless of the adoption of Meson these patches improve our current build system to fully support VPATH builds. Changes in v2: - some patches from v1 are pushed now - added a patch to mandate build dir != src dir - added a patch to cleanup .gitignore - added patches to fix sc_po_check - improved some patches from v1 Changes in v3: - python-zanata-client is the tool our Makefile uses Pavel Hrdina (20): build: mandate use of a build dir != src dir .gitignore: cleanup old and obsolete ignores syntax-check.mk: fix sc_po_check rule syntax-check.mk: cleanup sc_po_check dependencies syntax-check.mk: cleanup generated_files list for sc_po_check po: generate files into build directory po: rewrite the way how we generate files po: README.md: add a note about which Zanata client is required remote: unify rpc server dispatch generated files src: generate source files into build directory src: access: generate source files into build directory src: admin: generate source files into build directory src: esx: generate source files into build directory src: hyperv: generate source files into build directory src: locking: generate source files into build directory src: logging: generate source files into build directory src: lxc: generate source files into build directory src: remote: generate source files into build directory src: stop distributing generated source files tools: stop distributing generated source files .gitignore | 276 ++------------------- .travis.yml | 3 +- README-hacking | 11 +- README.md | 11 +- bootstrap.conf | 6 + build-aux/syntax-check.mk | 47 ++-- configure.ac | 6 + docs/compiling.html.in | 10 +- docs/windows.html.in | 3 +- libvirt.spec.in | 10 +- po/Makefile.am | 45 ++-- po/POTFILES | 320 ------------------------- po/POTFILES.in | 356 ++++++++++++++++++++++++++++ po/README.md | 3 + src/Makefile.am | 13 +- src/access/Makefile.inc.am | 17 +- src/admin/Makefile.inc.am | 24 +- src/bhyve/Makefile.inc.am | 1 + src/esx/Makefile.inc.am | 9 +- src/esx/esx_vi_generator.py | 11 +- src/hyperv/Makefile.inc.am | 9 +- src/hyperv/hyperv_wmi_generator.py | 11 +- src/interface/Makefile.inc.am | 2 + src/libxl/Makefile.inc.am | 2 + src/locking/Makefile.inc.am | 16 +- src/logging/Makefile.inc.am | 18 +- src/lxc/Makefile.inc.am | 36 ++- src/network/Makefile.inc.am | 2 + src/node_device/Makefile.inc.am | 2 + src/nwfilter/Makefile.inc.am | 2 + src/qemu/Makefile.inc.am | 2 + src/remote/Makefile.inc.am | 45 ++-- src/remote/remote_daemon_dispatch.c | 4 +- src/rpc/Makefile.inc.am | 8 +- src/secret/Makefile.inc.am | 2 + src/storage/Makefile.inc.am | 2 + src/util/Makefile.inc.am | 6 +- src/vbox/Makefile.inc.am | 1 + src/vz/Makefile.inc.am | 1 + tests/Makefile.am | 4 + tools/Makefile.am | 1 - 41 files changed, 636 insertions(+), 722 deletions(-) delete mode 100644 po/POTFILES create mode 100644 po/POTFILES.in -- 2.23.0

3 26

[libvirt] [PATCH 0/2] Simplify hash table data free callbacks
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

2 3

[libvirt] [PATCH 0/8] qemu: Enable -blockdev (blockdev-add saga finale?)
by Peter Krempa 22 Nov '19

22 Nov '19

This patchset depends on: https://www.redhat.com/archives/libvir-list/2019-November/msg00762.html everything together is available at: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-enable Looks like finally all pieces are together. Let's enable it. Peter Krempa (8): conf: domcaps: Add 'backingStoreInput' domain capability docs: Document support for obeying <backingStore> of <disk> on input qemu: caps: Base support of 'backingStoreInput' domain feature on QEMU_CAPS_BLOCKDEV tests: qemucapabilities: Refresh data for unreleased qemu-4.2 on x86_64 qemu: caps: Add capability for dynamic 'auto-read-only' support for files qemu: qapi: Add support for command features qemu: capabilities: Add detection of the 'savevm' fix for -blockdev qemu: enable blockdev support docs/formatdomain.html.in | 29 +- docs/formatdomaincaps.html.in | 7 + docs/schemas/domaincaps.rng | 9 + src/conf/domain_capabilities.c | 1 + src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 17 +- src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_qapi.c | 15 + tests/domaincapsdata/libxl-xenfv.xml | 1 + tests/domaincapsdata/libxl-xenpv.xml | 1 + .../domaincapsdata/qemu_1.5.3-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.5.3-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.5.3.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.1.1.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-tcg.x86_64.xml | 1 + .../qemu_2.10.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.10.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.11.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.11.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml | 1 + .../qemu_2.12.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.12.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.4.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.5.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-tcg.x86_64.xml | 1 + .../qemu_2.6.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.7.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.8.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.8.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.9.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_3.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 1 + .../qemu_4.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_4.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_4.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 1 + .../qemu_4.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../caps_4.2.0.x86_64.replies | 3907 +++++++++++------ .../caps_4.2.0.x86_64.xml | 253 +- ...arch64-os-firmware-efi.aarch64-latest.args | 18 +- .../aarch64-virt-graphics.aarch64-latest.args | 26 +- .../aarch64-virt-headless.aarch64-latest.args | 26 +- .../controller-virtio-scsi.x86_64-latest.args | 35 +- .../disk-aio.x86_64-latest.args | 19 +- ...-backing-chains-noindex.x86_64-latest.args | 145 +- .../disk-cache.x86_64-latest.args | 50 +- ...m-empty-network-invalid.x86_64-latest.args | 3 +- .../disk-cdrom-network.x86_64-latest.args | 32 +- .../disk-cdrom-tray.x86_64-latest.args | 24 +- .../disk-cdrom.x86_64-latest.args | 21 +- .../disk-copy_on_read.x86_64-latest.args | 19 +- .../disk-detect-zeroes.x86_64-latest.args | 17 +- .../disk-error-policy.x86_64-latest.args | 30 +- .../disk-floppy-q35-2_11.x86_64-latest.args | 14 +- .../disk-floppy-q35-2_9.x86_64-latest.args | 14 +- .../disk-floppy.x86_64-latest.args | 21 +- .../disk-network-gluster.x86_64-latest.args | 32 +- .../disk-network-iscsi.x86_64-latest.args | 58 +- .../disk-network-nbd.x86_64-latest.args | 41 +- .../disk-network-rbd.x86_64-latest.args | 67 +- .../disk-network-sheepdog.x86_64-latest.args | 16 +- ...isk-network-source-auth.x86_64-latest.args | 30 +- .../disk-network-tlsx509.x86_64-latest.args | 64 +- .../disk-readonly-disk.x86_64-latest.args | 14 +- .../disk-scsi-device-auto.x86_64-latest.args | 14 +- .../disk-scsi.x86_64-latest.args | 35 +- .../disk-shared.x86_64-latest.args | 36 +- ...irtio-scsi-reservations.x86_64-latest.args | 20 +- .../floppy-drive-fat.x86_64-latest.args | 7 +- ...egl-headless-rendernode.x86_64-latest.args | 7 +- .../graphics-egl-headless.x86_64-latest.args | 7 +- ...threads-virtio-scsi-pci.x86_64-latest.args | 25 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-align.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-label.x86_64-latest.args | 7 +- ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 7 +- ...hotplug-nvdimm-readonly.x86_64-latest.args | 7 +- .../memory-hotplug-nvdimm.x86_64-latest.args | 7 +- ...os-firmware-efi-secboot.x86_64-latest.args | 18 +- .../os-firmware-efi.x86_64-latest.args | 18 +- ...host-user-gpu-secondary.x86_64-latest.args | 7 +- .../vhost-user-vga.x86_64-latest.args | 7 +- ...eo-bochs-display-device.x86_64-latest.args | 10 +- ...eo-ramfb-display-device.x86_64-latest.args | 10 +- ...virtio-non-transitional.x86_64-latest.args | 7 +- .../virtio-transitional.x86_64-latest.args | 7 +- .../x86_64-pc-graphics.x86_64-latest.args | 8 +- .../x86_64-pc-headless.x86_64-latest.args | 8 +- .../x86_64-q35-graphics.x86_64-latest.args | 8 +- .../x86_64-q35-headless.x86_64-latest.args | 8 +- 146 files changed, 3790 insertions(+), 1683 deletions(-) -- 2.23.0

3 22

[libvirt] [PATCH 0/2] Deprecate implicit filters
by Vladimir Sementsov-Ogievskiy 22 Nov '19

22 Nov '19

Hi all! Max's series to fix some problems around filters consists of 42 patches. I'm sure that we didn't find all bugs around filters, and that filters would be a constant source of bugs in future, as during developing new feature nobody will consider all possible cases of dealing with filters (OK, somebody will, but it's hard). So, I'm thinking about starting some deprecations which will help us to simplify all the picture at least in not far future. So, I'd really want to deprecate implicit filters, ->file child based filters (move all filters to use backing child), filename based interfaces (use node-names). Most simple thing is implicit filters, so let's start from them. drive-mirror don't support filter-node-name, so I propose to deprecate it at all, together with drive-backup, instead of adding support of filter-node-name, what do you think? Vladimir Sementsov-Ogievskiy (2): qapi: deprecate drive-mirror and drive-backup qapi: deprecate implicit filters qemu-deprecated.texi | 11 +++++++++++ qapi/block-core.json | 10 ++++++++-- qapi/transaction.json | 2 +- include/block/block_int.h | 10 +++++++++- blockdev.c | 20 ++++++++++++++++++++ 5 files changed, 49 insertions(+), 4 deletions(-) -- 2.18.0

9 43

[libvirt] [PATCH 0/3] Speed up syntax-check a little
by Ján Tomko 22 Nov '19

22 Nov '19

This shaves off 4 % of syntax-check runtime on my 8-core system, which is enough of a reason for me. Surely it saves more on our overloaded CI workers. Ján Tomko (3): scripts: speedup prohibit-duplicate-header scripts: check-aclrules: use in instead of find scripts: check-aclrules: use regular expressions less often scripts/check-aclrules.py | 38 ++++++++++++++++++---------- scripts/prohibit-duplicate-header.py | 4 +++ 2 files changed, 29 insertions(+), 13 deletions(-) -- 2.19.2

3 13

[libvirt] [PATCH RFC 00/40] qemu: Add support for incremental backups
by Peter Krempa 22 Nov '19

22 Nov '19

This series is my attempt to clean up and finish incremental backup as it was last posted in v10 here: https://www.redhat.com/archives/libvir-list/2019-August/msg01018.html Out of that series, patch 1 was dropped for now and I'll revisit it later. The patches adding docs, xml parsing and job plumbing were partially modified and stripped of currently unimplemented functionality. The qemu implementation was completely rewritten. Of the new series first 28 patches are cleanups and can be reviewed for pushing. The main incremental backup functionality is not complete yet and should not be pushed right now not even in experimental (disabled [1]) state due to possible API changes: - figure out whether we want our own event for the backup job or want to deal with the domain job event - if we go with the domain job event everything must be rewritten as a single domain job - otherwise plumbing for the event must be added - documentation must reflect the above after that will be sorted out the following needs to be done until we can finally enable it: - more testing. I merely verified that qemu generates files/exports NBD - blockjob support for bitmaps (for commit/pull/copy) - snapshot support for bitmaps - merging of bitmaps accross backing chain entries - figuring out a more elegant way to do the bitmap for the backup than creating two, one for the 'store' (for NBD server) and one for the disk itself. - ... Until then the following XML can be used to enable the functionality for testing without any guarantee [1]: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='blockdev'/> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (35): util: hash: Add possibility to use simpler data free function in virHash util: hash: Add new constructor 'virHashNew' util: hash: Introduce virHashHasEntry qemu: domain: Split out setup of virStorageSource from qemu driver config qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData qemu: domain: clarify sematics of qemuDomainPrepareDiskSourceData qemu: domain: Tolerate NULL @disk in qemuDomainPrepareDiskSourceData conf: Remove virDomainDiskPathByName conf: Introduce virDomainDiskByTarget qemu: Replace use of virDomainDiskFindByBusAndDst with virDomainDiskByTarget conf: Remove unused virDomainDiskFindByBusAndDst Replace virDomainDiskByName by virDomainDiskByTarget in appropriate cases qemu: monitor: Introduce new interface to query-named-block-nodes qemu: block: Don't query monitor in qemuBlockStorageSourceCreateDetectSize qemu: checkpoint: Fix rollback and access to unlocked 'vm' when deleting checkpoints qemu: monitor: Remove non-transaction based dirty bitmap APIs conf: snapshot: Don't clear current snapshot when redefining an existing one conf: snapshot: Remove 'update_current' parameter from virDomainSnapshotRedefinePrep conf: Don't reuse variable for different object in virDomainCheckpointRedefinePrep conf: checkpoint: Don't clear current checkpoint when redefining an existing one conf: checkpoint: Don't clear current checkpoint when redefining qemu: checkpoint: Enforce that 'bitmap' name must match checkpoint name qemu: checkpoint: Split out checkpoint creation code qemu: checkpoint: Extract finalizing steps of checkpoint creation qemu: monitor: Add helper for generating data for block bitmap merging qemu: checkpoint: Use qemuMonitorTransactionBitmapMergeSourceAddBitmap qemu: blockjob: Refactor qemuBlockJobEventProcessConcludedTransition qemu: blockjob: Use 'g_free' in qemuBlockJobDataDispose Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job qemu: Implement backup job APIs and qemu handling qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 168 ++++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 29 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 518 ++++++++++ src/conf/backup_conf.h | 102 ++ src/conf/checkpoint_conf.c | 17 +- src/conf/domain_addr.c | 4 +- src/conf/domain_conf.c | 50 +- src/conf/domain_conf.h | 8 +- src/conf/snapshot_conf.c | 6 - src/conf/snapshot_conf.h | 1 - src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 20 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 191 ++++ src/libvirt_private.syms | 13 +- src/libvirt_public.syms | 8 + src/libxl/libxl_driver.c | 2 +- src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 895 ++++++++++++++++++ src/qemu/qemu_backup.h | 41 + src/qemu/qemu_block.c | 26 +- src/qemu/qemu_block.h | 5 +- src/qemu/qemu_blockjob.c | 171 ++-- src/qemu/qemu_blockjob.h | 19 + src/qemu/qemu_checkpoint.c | 232 +++-- src/qemu/qemu_checkpoint.h | 15 + src/qemu/qemu_domain.c | 161 +++- src/qemu/qemu_domain.h | 11 +- src/qemu/qemu_driver.c | 116 ++- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_monitor.c | 91 +- src/qemu/qemu_monitor.h | 49 +- src/qemu/qemu_monitor_json.c | 242 ++--- src/qemu/qemu_monitor_json.h | 33 +- src/remote/remote_driver.c | 3 + src/remote/remote_protocol.x | 53 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 2 +- src/util/vircgroup.c | 2 +- src/util/virhash.c | 85 +- src/util/virhash.h | 12 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemublocktest.c | 3 +- tests/qemumonitorjsontest.c | 53 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 209 ++++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 8 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 37 + 71 files changed, 4133 insertions(+), 552 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.21.0

5 83

[libvirt] [PATCH] fix bug libvirt daemon deadlock when another force console break down an existed console client when this deadlock hanppened, libvirtd backtrace as follow, a typical ABBA deadlock circumstance:
by LanceLiu 22 Nov '19

22 Nov '19

(gdb) thread 23 [Switching to thread 23 (Thread 0x7fbb54810700 (LWP 296966))] (gdb) bt at rpc/virnetserverclient.c:1503 client=client@entry=0x5641d20e20a0, msg=0x7fbb08003ab0, rerr=rerr@entry=0x7fbb5480f7b0, procedure=201, type=type@entry=3, serial=9) at rpc/virnetserverprogram.c:173 msg=<optimized out>, rerr=rerr@entry=0x7fbb5480f7b0, procedure=<optimized out>, serial=<optimized out>) at rpc/virnetserverprogram.c:222 opaque=opaque@entry=0x5641d20e20a0) at stream.c:246 force=force@entry=true) at conf/virchrdev.c:386 at qemu/qemu_driver.c:16188 st=st@entry=0x7fbb08008fc0, flags=1) at libvirt-domain.c:9363 rerr=0x7fbb5480fc10, msg=<optimized out>, client=0x5641d20e4e80) at remote_dispatch.h:8540 rerr=0x7fbb5480fc10, args=0x7fbb08005eb0, ret=<optimized out>) at remote_dispatch.h:8505 server=0x5641d20be420, prog=0x5641d20da580) at rpc/virnetserverprogram.c:437 msg=0x5641d20e9a20) at rpc/virnetserverprogram.c:307 srv=0x5641d20be420) at rpc/virnetserver.c:148 ---Type <return> to continue, or q <return> to quit--- (gdb) thread 26 [Switching to thread 26 (Thread 0x7fbb87a378c0 (LWP 295636))] (gdb) bt at stream.c:286 at rpc/virnetserverclient.c:1247 at rpc/virnetserverclient.c:1457 at util/vireventpoll.c:508 Signed-off-by: LanceLiu <liu.lance.89(a)gmail.com> --- src/remote/remote_daemon_stream.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_daemon_stream.c b/src/remote/remote_daemon_stream.c index 1f6e783..b27348f 100644 --- a/src/remote/remote_daemon_stream.c +++ b/src/remote/remote_daemon_stream.c @@ -284,14 +284,16 @@ daemonStreamEvent(virStreamPtr st, int events, void *opaque) * -1 on fatal client error */ static int -daemonStreamFilter(virNetServerClientPtr client ATTRIBUTE_UNUSED, +daemonStreamFilter(virNetServerClientPtr client, virNetMessagePtr msg, void *opaque) { daemonClientStream *stream = opaque; int ret = 0; + virObjectUnlock(client); virMutexLock(&stream->priv->lock); + virObjectLock(client); if (msg->header.type != VIR_NET_STREAM && msg->header.type != VIR_NET_STREAM_HOLE) -- 1.8.3.1

3 3

[libvirt] [PATCH] domcaps: Remove function initializing domain caps as unsupported
by Peter Krempa 21 Nov '19

21 Nov '19

Commit 5751a0b6b1968bb2354b2ac21cc5938b93009590 added a helper function called virDomainCapsFeaturesInitUnsupported which initialized all domain capability features as unsupported. When adding a new feature this would initialize it as unsupported also for hypervisor drivers which the original author possibly didn't intend to modify. To prevent accidental wrong value being reported in such case revert back to initializing individual features in the hypervisor drivers themselves. This is not a straight revert as additonal patches modified how we store the capabilities. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- CCing all people in the original conversation: CC: Cole Robinson <crobinso(a)redhat.com> CC: Ján Tomko <jtomko(a)redhat.com> CC: Michal Prívozník <mprivozn(a)redhat.com> src/bhyve/bhyve_capabilities.c | 4 +++- src/conf/domain_capabilities.c | 15 --------------- src/conf/domain_capabilities.h | 2 -- src/libvirt_private.syms | 1 - src/libxl/libxl_capabilities.c | 5 +++-- src/qemu/qemu_capabilities.c | 3 ++- 6 files changed, 8 insertions(+), 22 deletions(-) diff --git a/src/bhyve/bhyve_capabilities.c b/src/bhyve/bhyve_capabilities.c index f80cf7be62..fb8829d571 100644 --- a/src/bhyve/bhyve_capabilities.c +++ b/src/bhyve/bhyve_capabilities.c @@ -116,7 +116,9 @@ virBhyveDomainCapsFill(virDomainCapsPtr caps, } caps->hostdev.supported = VIR_TRISTATE_BOOL_NO; - virDomainCapsFeaturesInitUnsupported(caps); + caps->features[VIR_DOMAIN_CAPS_FEATURE_IOTHREADS] = VIR_TRISTATE_BOOL_NO; + caps->features[VIR_DOMAIN_CAPS_FEATURE_VMCOREINFO] = VIR_TRISTATE_BOOL_NO; + caps->features[VIR_DOMAIN_CAPS_FEATURE_GENID] = VIR_TRISTATE_BOOL_NO; caps->gic.supported = VIR_TRISTATE_BOOL_NO; return 0; diff --git a/src/conf/domain_capabilities.c b/src/conf/domain_capabilities.c index 0979711fea..fa27e95e1b 100644 --- a/src/conf/domain_capabilities.c +++ b/src/conf/domain_capabilities.c @@ -272,21 +272,6 @@ virDomainCapsEnumClear(virDomainCapsEnumPtr capsEnum) } -/** - * @caps: domain caps - * - * Initializes all features in 'caps' as unsupported. - */ -void -virDomainCapsFeaturesInitUnsupported(virDomainCapsPtr caps) -{ - size_t i; - - for (i = 0; i < VIR_DOMAIN_CAPS_FEATURE_LAST; i++) - caps->features[i] = VIR_TRISTATE_BOOL_NO; -} - - static int virDomainCapsEnumFormat(virBufferPtr buf, const virDomainCapsEnum *capsEnum, diff --git a/src/conf/domain_capabilities.h b/src/conf/domain_capabilities.h index d69bf8d13e..c8eeb035ac 100644 --- a/src/conf/domain_capabilities.h +++ b/src/conf/domain_capabilities.h @@ -227,8 +227,6 @@ int virDomainCapsEnumSet(virDomainCapsEnumPtr capsEnum, unsigned int *values); void virDomainCapsEnumClear(virDomainCapsEnumPtr capsEnum); -void virDomainCapsFeaturesInitUnsupported(virDomainCapsPtr caps); - char * virDomainCapsFormat(const virDomainCaps *caps); int virDomainCapsDeviceDefValidate(const virDomainCaps *caps, diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 3416438841..44201cda14 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -200,7 +200,6 @@ virDomainCapsCPUUsableTypeToString; virDomainCapsDeviceDefValidate; virDomainCapsEnumClear; virDomainCapsEnumSet; -virDomainCapsFeaturesInitUnsupported; virDomainCapsFormat; virDomainCapsNew; virSEVCapabilitiesFree; diff --git a/src/libxl/libxl_capabilities.c b/src/libxl/libxl_capabilities.c index 052fbd4603..c920c2aec5 100644 --- a/src/libxl/libxl_capabilities.c +++ b/src/libxl/libxl_capabilities.c @@ -764,8 +764,9 @@ libxlMakeDomainCapabilities(virDomainCapsPtr domCaps, libxlMakeDomainDeviceHostdevCaps(hostdev) < 0) return -1; - virDomainCapsFeaturesInitUnsupported(domCaps); - + domCaps->features[VIR_DOMAIN_CAPS_FEATURE_IOTHREADS] = VIR_TRISTATE_BOOL_NO; + domCaps->features[VIR_DOMAIN_CAPS_FEATURE_VMCOREINFO] = VIR_TRISTATE_BOOL_NO; + domCaps->features[VIR_DOMAIN_CAPS_FEATURE_GENID] = VIR_TRISTATE_BOOL_NO; domCaps->gic.supported = VIR_TRISTATE_BOOL_NO; return 0; diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 0a406972e7..d74bcd96b0 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -5447,6 +5447,8 @@ virQEMUCapsFillDomainFeaturesFromQEMUCaps(virQEMUCapsPtr qemuCaps, for (i = 0; i < G_N_ELEMENTS(domCapsTuples); i++) { if (virQEMUCapsGet(qemuCaps, domCapsTuples[i].qemucap)) domCaps->features[domCapsTuples[i].domcap] = VIR_TRISTATE_BOOL_YES; + else + domCaps->features[domCapsTuples[i].domcap] = VIR_TRISTATE_BOOL_NO; } } @@ -5735,7 +5737,6 @@ virQEMUCapsFillDomainCaps(virCapsPtr caps, virDomainCapsDeviceVideoPtr video = &domCaps->video; virDomainCapsDeviceRNGPtr rng = &domCaps->rng; - virDomainCapsFeaturesInitUnsupported(domCaps); virQEMUCapsFillDomainFeaturesFromQEMUCaps(qemuCaps, domCaps); domCaps->maxvcpus = virQEMUCapsGetMachineMaxCpus(qemuCaps, -- 2.23.0

3 2