- Devel - libvirt List Archives

[libvirt] [ruby PATCH] Fix default values for node_cpu_stats() and node_memory_stats()
by Stefano Garzarella 23 Nov '19

23 Nov '19

ruby_libvirt_value_to_int() returns 0 if the optional value is not defined, but in node_cpu_stats() and node_memory_stats() the default value of cpuNum and cellNum is -1. Reported-by: Charlie Smurthwaite <charlie(a)atech.media> Signed-off-by: Stefano Garzarella <sgarzare(a)redhat.com> --- ext/libvirt/connect.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/ext/libvirt/connect.c b/ext/libvirt/connect.c index 5932535..b2d041b 100644 --- a/ext/libvirt/connect.c +++ b/ext/libvirt/connect.c @@ -2079,7 +2079,12 @@ static VALUE libvirt_connect_node_cpu_stats(int argc, VALUE *argv, VALUE c) rb_scan_args(argc, argv, "02", &intparam, &flags); - tmp = ruby_libvirt_value_to_int(intparam); + if (NIL_P(intparam)) { + tmp = -1; + } + else { + tmp = ruby_libvirt_value_to_int(intparam); + } return ruby_libvirt_get_parameters(c, ruby_libvirt_value_to_uint(flags), (void *)&tmp, sizeof(virNodeCPUStats), @@ -2139,7 +2144,12 @@ static VALUE libvirt_connect_node_memory_stats(int argc, VALUE *argv, VALUE c) rb_scan_args(argc, argv, "02", &intparam, &flags); - tmp = ruby_libvirt_value_to_int(intparam); + if (NIL_P(intparam)) { + tmp = -1; + } + else { + tmp = ruby_libvirt_value_to_int(intparam); + } return ruby_libvirt_get_parameters(c, ruby_libvirt_value_to_uint(flags), (void *)&tmp, sizeof(virNodeMemoryStats), -- 2.21.0

3 4

Re: [libvirt] [edk2-discuss] [OVMF] resource assignment fails for passthrough PCI GPU
by Eduardo Habkost 22 Nov '19

22 Nov '19

(+Jiri, +libvir-list) On Fri, Nov 22, 2019 at 04:58:25PM +0000, Dr. David Alan Gilbert wrote: > * Laszlo Ersek (lersek(a)redhat.com) wrote: > > (+Dave, +Eduardo) > > > > On 11/22/19 00:00, dann frazier wrote: > > > On Tue, Nov 19, 2019 at 06:06:15AM +0100, Laszlo Ersek wrote: > > >> On 11/19/19 01:54, dann frazier wrote: > > >>> On Fri, Nov 15, 2019 at 11:51:18PM +0100, Laszlo Ersek wrote: > > >>>> On 11/15/19 19:56, dann frazier wrote: > > >>>>> Hi, > > >>>>> I'm trying to passthrough an Nvidia GPU to a q35 KVM guest, but UEFI > > >>>>> is failing to allocate resources for it. I have no issues if I boot w/ > > >>>>> a legacy BIOS, and it works fine if I tell the linux guest to do the > > >>>>> allocation itself - but I'm looking for a way to make this work w/ > > >>>>> OVMF by default. > > >>>>> > > >>>>> I posted a debug log here: > > >>>>> https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1849563/+attachment/530… > > >>>>> > > >>>>> Linux guest lspci output is also available for both seabios/OVMF boots here: > > >>>>> https://bugs.launchpad.net/ubuntu/+source/edk2/+bug/1849563 > > >>>> > > >>>> By default, OVMF exposes such a 64-bit MMIO aperture for PCI MMIO BAR > > >>>> allocation that is 32GB in size. The generic PciBusDxe driver collects, > > >>>> orders, and assigns / allocates the MMIO BARs, but it can work only out > > >>>> of the aperture that platform code advertizes. > > >>>> > > >>>> Your GPU's region 1 is itself 32GB in size. Given that there are further > > >>>> PCI devices in the system with further 64-bit MMIO BARs, the default > > >>>> aperture cannot accommodate everything. In such an event, PciBusDxe > > >>>> avoids assigning the largest BARs (to my knowledge), in order to > > >>>> conserve the most aperture possible, for other devices -- hence break > > >>>> the fewest possible PCI devices. > > >>>> > > >>>> You can control the aperture size from the QEMU command line. You can > > >>>> also do it from the libvirt domain XML, technically speaking. The knob > > >>>> is experimental, so no stability or compatibility guarantees are made. > > >>>> (That's also the reason why it's a bit of a hack in the libvirt domain XML.) > > >>>> > > >>>> The QEMU cmdline options is described in the following edk2 commit message: > > >>>> > > >>>> https://github.com/tianocore/edk2/commit/7e5b1b670c38 > > >>> > > >>> Hi Laszlo, > > >>> > > >>> Thanks for taking the time to describe this in detail! The -fw_cfg > > >>> option did avoid the problem for me. > > >> > > >> Good to hear, thanks. > > >> > > >>> I also noticed that the above > > >>> commit message mentions the existence of a 24GB card as a reasoning > > >>> behind choosing the 32GB default aperture. From what you say below, I > > >>> understand that bumping this above 64GB could break hosts w/ <= 37 > > >>> physical address bits. > > >> > > >> Right. > > >> > > >>> What would be the downside of bumping the > > >>> default aperture to, say, 48GB? > > >> > > >> The placement of the aperture is not trivial (please see the code > > >> comments in the linked commit). The base address of the aperture is > > >> chosen so that the largest BAR that can fit in the aperture may be > > >> naturally aligned. (BARs are whole powers of two.) > > >> > > >> The largest BAR that can fit in a 48 GB aperture is 32 GB. Therefore > > >> such an aperture would be aligned at 32 GB -- the lowest base address > > >> (dependent on guest RAM size) would be 32 GB. Meaning that the aperture > > >> would end at 32 + 48 = 80 GB. That still breaches the 36-bit phys > > >> address width. > > >> > > >> 32 GB is the largest aperture size that can work with 36-bit phys > > >> address width; that's the aperture that ends at 64 GB exactly. > > > > > > Thanks, yeah - now that I read the code comments that is clear (as > > > clear as it can be w/ my low level of base knowledge). In the commit you > > > mention Gerd (CC'd) had suggested a heuristic-based approach for > > > sizing the aperture. When you say "PCPU address width" - is that a > > > function of the available physical bits? > > > > "PCPU address width" is not a "function" of the available physical bits > > -- it *is* the available physical bits. "PCPU" simply stands for > > "physical CPU". > > > > > IOW, would that approach > > > allow OVMF to automatically grow the aperture to the max ^2 supported > > > by the host CPU? > > > > Maybe. > > > > The current logic in OVMF works from the guest-physical address space > > size -- as deduced from multiple factors, such as the 64-bit MMIO > > aperture size, and others -- towards the guest-CPU (aka VCPU) address > > width. The VCPU address width is important for a bunch of other purposes > > in the firmware, so OVMF has to calculate it no matter what. > > > > Again, the current logic is to calculate the highest guest-physical > > address, and then deduce the VCPU address width from that (and then > > expose it to the rest of the firmware). > > > > Your suggestion would require passing the PCPU (physical CPU) address > > width from QEMU/KVM into the guest, and reversing the direction of the > > calculation. The PCPU address width would determine the VCPU address > > width directly, and then the 64-bit PCI MMIO aperture would be > > calculated from that. > > > > However, there are two caveats. > > > > (1) The larger your guest-phys address space (as exposed through the > > VCPU address width to the rest of the firmware), the more guest RAM you > > need for page tables. Because, just before entering the DXE phase, the > > firmware builds 1:1 mapping page tables for the entire guest-phys > > address space. This is necessary e.g. so you can access any PCI MMIO BAR. > > > > Now consider that you have a huge beefy virtualization host with say 46 > > phys address bits, and a wimpy guest with say 1.5GB of guest RAM. Do you > > absolutely want tens of *terabytes* for your 64-bit PCI MMIO aperture? > > Do you really want to pay for the necessary page tables with that meager > > guest RAM? > > > > (Such machines do exist BTW, for example: > > > > http://mid.mail-archive.com/9BD73EA91F8E404F851CF3F519B14AA8036C67B5@DGGEMI… > > ) > > > > In other words, you'd need some kind of knob anyway, because otherwise > > your aperture could grow too *large*. > > > > > > (2) Exposing the PCPU address width to the guest may have nasty > > consequences at the QEMU/KVM level, regardless of guest firmware. For > > example, that kind of "guest enlightenment" could interfere with migration. > > > > If you boot a guest let's say with 16GB of RAM, and tell it "hey friend, > > have 40 bits of phys address width!", then you'll have a difficult time > > migrating that guest to a host with a CPU that only has 36-bits wide > > physical addresses -- even if the destination host has plenty of RAM > > otherwise, such as a full 64GB. > > > > There could be other QEMU/KVM / libvirt issues that I m unaware of > > (hence the CC to Dave and Eduardo). > > host physical address width gets messy. There are differences as well > between upstream qemu behaviour, and some downstreams. > I think the story is that: > > a) Qemu default: 40 bits on any host > b) -cpu blah,host-phys-bits=true to follow the host. > c) RHEL has host-phys-bits=true by default > > As you say, the only real problem with host-phys-bits is migration - > between say an E3 and an E5 xeon with different widths. The magic 40's > is generally wrong as well - I think it came from some ancient AMD, > but it's the default on QEMU TCG as well. Yes, and because it affects live migration ability, we have two constraints: 1) It needs to be exposed in the libvirt domain XML; 2) QEMU and libvirt can't choose a value that works for everybody (because neither QEMU or libvirt know where the VM might be migrated later). Which is why the BZ below is important: > > I don't think there's a way to set it in libvirt; > https://bugzilla.redhat.com/show_bug.cgi?id=1578278 is a bz asking for > that. > > IMHO host-phys-bits is actually pretty safe; and makes most sense in a > lot of cases. Yeah, it is mostly safe and makes sense, but messy if you try to migrate to a host with a different size. > > Dave > > > > Thanks, > > Laszlo > > > > > > > > -dann > > > > > >>>> For example, to set a 64GB aperture, pass: > > >>>> > > >>>> -fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=65536 > > >>>> > > >>>> The libvirt domain XML syntax is a bit tricky (and it might "taint" your > > >>>> domain, as it goes outside of the QEMU features that libvirt directly > > >>>> maps to): > > >>>> > > >>>> <domain > > >>>> type='kvm' > > >>>> xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> > > >>>> <qemu:commandline> > > >>>> <qemu:arg value='-fw_cfg'/> > > >>>> <qemu:arg value='opt/ovmf/X-PciMmio64Mb,string=65536'/> > > >>>> </qemu:commandline> > > >>>> </domain> > > >>>> > > >>>> Some notes: > > >>>> > > >>>> (1) The "xmlns:qemu" namespace definition attribute in the <domain> root > > >>>> element is important. You have to add it manually when you add > > >>>> <qemu:commandline> and <qemu:arg> too. Without the namespace > > >>>> definition, the latter elements will make no sense, and libvirt will > > >>>> delete them immediately. > > >>>> > > >>>> (2) The above change will grow your guest's physical address space to > > >>>> more than 64GB. As a consequence, on your *host*, *if* your physical CPU > > >>>> supports nested paging (called "ept" on Intel and "npt" on AMD), *then* > > >>>> the CPU will have to support at least 37 physical address bits too, for > > >>>> the guest to work. Otherwise, the guest will break, hard. > > >>>> > > >>>> Here's how to verify (on the host): > > >>>> > > >>>> (2a) run "egrep -w 'npt|ept' /proc/cpuinfo" --> if this does not produce > > >>>> output, then stop reading here; things should work. Your CPU does not > > >>>> support nested paging, so KVM will use shadow paging, which is slower, > > >>>> but at least you don't have to care about the CPU's phys address width. > > >>>> > > >>>> (2b) otherwise (i.e. when you do have nested paging), run "grep 'bits > > >>>> physical' /proc/cpuinfo" --> if the physical address width is >=37, > > >>>> you're good. > > >>>> > > >>>> (2c) if you have nested paging but exactly 36 phys address bits, then > > >>>> you'll have to forcibly disable nested paging (assuming you want to run > > >>>> a guest with larger than 64GB guest-phys address space, that is). On > > >>>> Intel, issue: > > >>>> > > >>>> rmmod kvm_intel > > >>>> modprobe kvm_intel ept=N > > >>>> > > >>>> On AMD, go with: > > >>>> > > >>>> rmmod kvm_amd > > >>>> modprobe kvm_amd npt=N > > >>>> > > >>>> Hope this helps, > > >>>> Laszlo > > >>>> > > >>> > > >> > > > > > > -- > Dr. David Alan Gilbert / dgilbert(a)redhat.com / Manchester, UK -- Eduardo

1 0

[libvirt] [PATCH v2 0/4] PCI hostdev partial assignment support
by Daniel Henrique Barboza 22 Nov '19

22 Nov '19

Changes from previous version [1], all of them result of feedback from Alex Williamson and Abdulla Bubshait: - use <address type='none'/> instead of creating a new subsys attribute; - expand the change to all PCI hostdevs. Former patch 01 was discarded because we don't need the PCI Multifunction helpers for now; - series changed name to reflect what it's being done - new patch 04: add documentation to formatdomain.html.in To avoid a huge wall of text please refer to [1] for context about the road up to here. Commit msgs of the first 3 patches tells the story as well. [1] https://www.redhat.com/archives/libvir-list/2019-October/msg00298.html What I want to discuss here instead is a caveat that I've found while testing this work, since its first version. This test was done in a Power 8 system with a Broadcom BCM5719 PCIe Multifunction card, with 4 virtual functions. This series enables Libvirt to declare PCI hostdevs that will not be visible to the guest using address type='none'. During the tests I faced a scenario that I expected to fail, but it didn't. This is the relevant XML except: <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x0'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x1'/> </source> <address type='none'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x2'/> </source> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x2'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0001' bus='0x09' slot='0x00' function='0x3'/> </source> <address type='none'/> </hostdev> I'm declaring all the BCM5719 functions in the XML, but I am making functions 0, 1 and 3 unassignable by the guest using address type='none'. This test was meant to fail, but it didn't. To my surprise the guest booted and the device is functional: $ lspci 0000:00:01.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon 0000:00:03.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01) 0000:00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device 0001:00:01.2 Ethernet controller: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe (rev 01) $ I've talked with Michael Roth (QEMU PPC64 developer that worked with the PCI multifunction hotplug/unplug support in the PPC64 machine) about this. He mentioned that this is intended. I'll quote here what he had to say about it: "The current logic is that we only emit the hotplug event when function 0 is attached, but if some other function is attached at boot-time the guest will still see it on the bus, and whether that works or not I think is up to the device/driver" This explains why this test didn't fail as I expected. At least for the PPC64 machine, depending on the device support, this setup is allowed. PPC64 machine uses function 0 hotplug as a signal of 'plug all the queue functions and function 0', but function 0 isn't required at boot time. I would like to hear other opinions in this because I can't say whether this is allowed in x86. I am mentioning all this now because this had a direct impact on the design of this work since the previous version, and I failed to bring it up back then. I am *not* checking for the assignment of function 0 at guest boot time in Libvirt, leaving the user free to decide what to do. I am aware that this will be inconsistent to the logic of the PCI multifunction hotplug/unplug support, where function 0 is required. This also puts a lot of faith in the user, relying that the user is fully aware of the capabilities of the hardware. My question is: should Libvirt force function 0 to be present in boot time as well, regardless of whether the PPC64 guest or some cards are able to boot without it? Thanks, DHB Daniel Henrique Barboza (4): domain_conf: allow address type='none' to unassign PCI hostdevs qemu: handle unassigned PCI hostdevs in command line and alias virhostdev.c: check all IOMMU devs in virHostdevPreparePCIDevices formatdomain.html.in: document <address type='none'/> docs/formatdomain.html.in | 15 +++++ docs/schemas/domaincommon.rng | 5 ++ src/conf/domain_conf.c | 56 ++++++++++++++-- src/conf/domain_conf.h | 3 + src/qemu/qemu_alias.c | 6 ++ src/qemu/qemu_command.c | 4 ++ src/qemu/qemu_domain_address.c | 6 ++ src/util/virhostdev.c | 64 +++++++++++++++++-- .../hostdev-pci-address-none.args | 31 +++++++++ .../hostdev-pci-address-none.xml | 42 ++++++++++++ ...ostdev-pci-multifunction-partial-fail.args | 31 +++++++++ ...hostdev-pci-multifunction-partial-fail.xml | 35 ++++++++++ tests/qemuxml2argvtest.c | 8 +++ .../hostdev-pci-address-none.xml | 58 +++++++++++++++++ tests/qemuxml2xmltest.c | 1 + 15 files changed, 352 insertions(+), 13 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-none.xml create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-partial-fail.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-address-none.xml -- 2.21.0

3 8

[libvirt] [PATCH 0/1] Improve travis CI coverage
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

There is currently a serious hardware outage with the host running our Jenkins CI slaves and we're not sure when we'll get it back up and running. Ths leaves us with Travis CI in the meanwhile. This expands the Travis CI coverage to add latest Fedora stable and rawhide, so that it is covering a representative set of distros overall. Daniel P. Berrangé (1): travis: add fedora-31 & fedora-rawhide to the build images .travis.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) -- 2.21.0

2 2

[libvirt] [PATCH v4 00/20] cleanup current build system
by Pavel Hrdina 22 Nov '19

22 Nov '19

As preparation to switch to Meson there are some things that needs be cleaned up to make the conversion easier. The important thing in Meson is that there is a strict separation between source and build directory and the distributed tarball by default contains only files tracked by git with a possibility to write a script which would add some other sources into the tarball. Regardless of the adoption of Meson these patches improve our current build system to fully support VPATH builds. Changes in v2: - some patches from v1 are pushed now - added a patch to mandate build dir != src dir - added a patch to cleanup .gitignore - added patches to fix sc_po_check - improved some patches from v1 Changes in v3: - python-zanata-client is the tool our Makefile uses Pavel Hrdina (20): build: mandate use of a build dir != src dir .gitignore: cleanup old and obsolete ignores syntax-check.mk: fix sc_po_check rule syntax-check.mk: cleanup sc_po_check dependencies syntax-check.mk: cleanup generated_files list for sc_po_check po: generate files into build directory po: rewrite the way how we generate files po: README.md: add a note about which Zanata client is required remote: unify rpc server dispatch generated files src: generate source files into build directory src: access: generate source files into build directory src: admin: generate source files into build directory src: esx: generate source files into build directory src: hyperv: generate source files into build directory src: locking: generate source files into build directory src: logging: generate source files into build directory src: lxc: generate source files into build directory src: remote: generate source files into build directory src: stop distributing generated source files tools: stop distributing generated source files .gitignore | 276 ++------------------- .travis.yml | 3 +- README-hacking | 11 +- README.md | 11 +- bootstrap.conf | 6 + build-aux/syntax-check.mk | 47 ++-- configure.ac | 6 + docs/compiling.html.in | 10 +- docs/windows.html.in | 3 +- libvirt.spec.in | 10 +- po/Makefile.am | 45 ++-- po/POTFILES | 320 ------------------------- po/POTFILES.in | 356 ++++++++++++++++++++++++++++ po/README.md | 3 + src/Makefile.am | 13 +- src/access/Makefile.inc.am | 17 +- src/admin/Makefile.inc.am | 24 +- src/bhyve/Makefile.inc.am | 1 + src/esx/Makefile.inc.am | 9 +- src/esx/esx_vi_generator.py | 11 +- src/hyperv/Makefile.inc.am | 9 +- src/hyperv/hyperv_wmi_generator.py | 11 +- src/interface/Makefile.inc.am | 2 + src/libxl/Makefile.inc.am | 2 + src/locking/Makefile.inc.am | 16 +- src/logging/Makefile.inc.am | 18 +- src/lxc/Makefile.inc.am | 36 ++- src/network/Makefile.inc.am | 2 + src/node_device/Makefile.inc.am | 2 + src/nwfilter/Makefile.inc.am | 2 + src/qemu/Makefile.inc.am | 2 + src/remote/Makefile.inc.am | 45 ++-- src/remote/remote_daemon_dispatch.c | 4 +- src/rpc/Makefile.inc.am | 8 +- src/secret/Makefile.inc.am | 2 + src/storage/Makefile.inc.am | 2 + src/util/Makefile.inc.am | 6 +- src/vbox/Makefile.inc.am | 1 + src/vz/Makefile.inc.am | 1 + tests/Makefile.am | 4 + tools/Makefile.am | 1 - 41 files changed, 636 insertions(+), 722 deletions(-) delete mode 100644 po/POTFILES create mode 100644 po/POTFILES.in -- 2.23.0

3 26

[libvirt] [PATCH 0/2] Simplify hash table data free callbacks
by Daniel P. Berrangé 22 Nov '19

22 Nov '19

2 3

[libvirt] [PATCH 0/8] qemu: Enable -blockdev (blockdev-add saga finale?)
by Peter Krempa 22 Nov '19

22 Nov '19

This patchset depends on: https://www.redhat.com/archives/libvir-list/2019-November/msg00762.html everything together is available at: git fetch https://gitlab.com/pipo.sk/libvirt.git blockdev-enable Looks like finally all pieces are together. Let's enable it. Peter Krempa (8): conf: domcaps: Add 'backingStoreInput' domain capability docs: Document support for obeying <backingStore> of <disk> on input qemu: caps: Base support of 'backingStoreInput' domain feature on QEMU_CAPS_BLOCKDEV tests: qemucapabilities: Refresh data for unreleased qemu-4.2 on x86_64 qemu: caps: Add capability for dynamic 'auto-read-only' support for files qemu: qapi: Add support for command features qemu: capabilities: Add detection of the 'savevm' fix for -blockdev qemu: enable blockdev support docs/formatdomain.html.in | 29 +- docs/formatdomaincaps.html.in | 7 + docs/schemas/domaincaps.rng | 9 + src/conf/domain_capabilities.c | 1 + src/conf/domain_capabilities.h | 1 + src/qemu/qemu_capabilities.c | 17 +- src/qemu/qemu_capabilities.h | 2 + src/qemu/qemu_qapi.c | 15 + tests/domaincapsdata/libxl-xenfv.xml | 1 + tests/domaincapsdata/libxl-xenpv.xml | 1 + .../domaincapsdata/qemu_1.5.3-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.5.3-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.5.3.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.6.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_1.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_1.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.1.1-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.1.1.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.10.0-tcg.x86_64.xml | 1 + .../qemu_2.10.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.10.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.10.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.11.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.11.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.11.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.12.0-tcg.x86_64.xml | 1 + .../qemu_2.12.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.12.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.12.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.4.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.4.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.5.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.5.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.6.0-tcg.x86_64.xml | 1 + .../qemu_2.6.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.6.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.7.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.7.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.7.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.8.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.8.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.8.0.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_2.9.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_2.9.0.s390x.xml | 1 + tests/domaincapsdata/qemu_2.9.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_3.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_3.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 1 + .../qemu_4.0.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.0.0.s390x.xml | 1 + tests/domaincapsdata/qemu_4.0.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 1 + tests/domaincapsdata/qemu_4.1.0.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-q35.x86_64.xml | 1 + .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 1 + .../qemu_4.2.0-virt.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.aarch64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.ppc64.xml | 1 + tests/domaincapsdata/qemu_4.2.0.x86_64.xml | 1 + .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../caps_4.2.0.x86_64.replies | 3907 +++++++++++------ .../caps_4.2.0.x86_64.xml | 253 +- ...arch64-os-firmware-efi.aarch64-latest.args | 18 +- .../aarch64-virt-graphics.aarch64-latest.args | 26 +- .../aarch64-virt-headless.aarch64-latest.args | 26 +- .../controller-virtio-scsi.x86_64-latest.args | 35 +- .../disk-aio.x86_64-latest.args | 19 +- ...-backing-chains-noindex.x86_64-latest.args | 145 +- .../disk-cache.x86_64-latest.args | 50 +- ...m-empty-network-invalid.x86_64-latest.args | 3 +- .../disk-cdrom-network.x86_64-latest.args | 32 +- .../disk-cdrom-tray.x86_64-latest.args | 24 +- .../disk-cdrom.x86_64-latest.args | 21 +- .../disk-copy_on_read.x86_64-latest.args | 19 +- .../disk-detect-zeroes.x86_64-latest.args | 17 +- .../disk-error-policy.x86_64-latest.args | 30 +- .../disk-floppy-q35-2_11.x86_64-latest.args | 14 +- .../disk-floppy-q35-2_9.x86_64-latest.args | 14 +- .../disk-floppy.x86_64-latest.args | 21 +- .../disk-network-gluster.x86_64-latest.args | 32 +- .../disk-network-iscsi.x86_64-latest.args | 58 +- .../disk-network-nbd.x86_64-latest.args | 41 +- .../disk-network-rbd.x86_64-latest.args | 67 +- .../disk-network-sheepdog.x86_64-latest.args | 16 +- ...isk-network-source-auth.x86_64-latest.args | 30 +- .../disk-network-tlsx509.x86_64-latest.args | 64 +- .../disk-readonly-disk.x86_64-latest.args | 14 +- .../disk-scsi-device-auto.x86_64-latest.args | 14 +- .../disk-scsi.x86_64-latest.args | 35 +- .../disk-shared.x86_64-latest.args | 36 +- ...irtio-scsi-reservations.x86_64-latest.args | 20 +- .../floppy-drive-fat.x86_64-latest.args | 7 +- ...egl-headless-rendernode.x86_64-latest.args | 7 +- .../graphics-egl-headless.x86_64-latest.args | 7 +- ...threads-virtio-scsi-pci.x86_64-latest.args | 25 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-align.x86_64-latest.args | 7 +- ...ry-hotplug-nvdimm-label.x86_64-latest.args | 7 +- ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 7 +- ...hotplug-nvdimm-readonly.x86_64-latest.args | 7 +- .../memory-hotplug-nvdimm.x86_64-latest.args | 7 +- ...os-firmware-efi-secboot.x86_64-latest.args | 18 +- .../os-firmware-efi.x86_64-latest.args | 18 +- ...host-user-gpu-secondary.x86_64-latest.args | 7 +- .../vhost-user-vga.x86_64-latest.args | 7 +- ...eo-bochs-display-device.x86_64-latest.args | 10 +- ...eo-ramfb-display-device.x86_64-latest.args | 10 +- ...virtio-non-transitional.x86_64-latest.args | 7 +- .../virtio-transitional.x86_64-latest.args | 7 +- .../x86_64-pc-graphics.x86_64-latest.args | 8 +- .../x86_64-pc-headless.x86_64-latest.args | 8 +- .../x86_64-q35-graphics.x86_64-latest.args | 8 +- .../x86_64-q35-headless.x86_64-latest.args | 8 +- 146 files changed, 3790 insertions(+), 1683 deletions(-) -- 2.23.0

3 22

[libvirt] [PATCH 0/2] Deprecate implicit filters
by Vladimir Sementsov-Ogievskiy 22 Nov '19

22 Nov '19

Hi all! Max's series to fix some problems around filters consists of 42 patches. I'm sure that we didn't find all bugs around filters, and that filters would be a constant source of bugs in future, as during developing new feature nobody will consider all possible cases of dealing with filters (OK, somebody will, but it's hard). So, I'm thinking about starting some deprecations which will help us to simplify all the picture at least in not far future. So, I'd really want to deprecate implicit filters, ->file child based filters (move all filters to use backing child), filename based interfaces (use node-names). Most simple thing is implicit filters, so let's start from them. drive-mirror don't support filter-node-name, so I propose to deprecate it at all, together with drive-backup, instead of adding support of filter-node-name, what do you think? Vladimir Sementsov-Ogievskiy (2): qapi: deprecate drive-mirror and drive-backup qapi: deprecate implicit filters qemu-deprecated.texi | 11 +++++++++++ qapi/block-core.json | 10 ++++++++-- qapi/transaction.json | 2 +- include/block/block_int.h | 10 +++++++++- blockdev.c | 20 ++++++++++++++++++++ 5 files changed, 49 insertions(+), 4 deletions(-) -- 2.18.0

9 43

[libvirt] [PATCH 0/3] Speed up syntax-check a little
by Ján Tomko 22 Nov '19

22 Nov '19

This shaves off 4 % of syntax-check runtime on my 8-core system, which is enough of a reason for me. Surely it saves more on our overloaded CI workers. Ján Tomko (3): scripts: speedup prohibit-duplicate-header scripts: check-aclrules: use in instead of find scripts: check-aclrules: use regular expressions less often scripts/check-aclrules.py | 38 ++++++++++++++++++---------- scripts/prohibit-duplicate-header.py | 4 +++ 2 files changed, 29 insertions(+), 13 deletions(-) -- 2.19.2

3 13

[libvirt] [PATCH RFC 00/40] qemu: Add support for incremental backups
by Peter Krempa 22 Nov '19

22 Nov '19

This series is my attempt to clean up and finish incremental backup as it was last posted in v10 here: https://www.redhat.com/archives/libvir-list/2019-August/msg01018.html Out of that series, patch 1 was dropped for now and I'll revisit it later. The patches adding docs, xml parsing and job plumbing were partially modified and stripped of currently unimplemented functionality. The qemu implementation was completely rewritten. Of the new series first 28 patches are cleanups and can be reviewed for pushing. The main incremental backup functionality is not complete yet and should not be pushed right now not even in experimental (disabled [1]) state due to possible API changes: - figure out whether we want our own event for the backup job or want to deal with the domain job event - if we go with the domain job event everything must be rewritten as a single domain job - otherwise plumbing for the event must be added - documentation must reflect the above after that will be sorted out the following needs to be done until we can finally enable it: - more testing. I merely verified that qemu generates files/exports NBD - blockjob support for bitmaps (for commit/pull/copy) - snapshot support for bitmaps - merging of bitmaps accross backing chain entries - figuring out a more elegant way to do the bitmap for the backup than creating two, one for the 'store' (for NBD server) and one for the disk itself. - ... Until then the following XML can be used to enable the functionality for testing without any guarantee [1]: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='blockdev'/> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (35): util: hash: Add possibility to use simpler data free function in virHash util: hash: Add new constructor 'virHashNew' util: hash: Introduce virHashHasEntry qemu: domain: Split out setup of virStorageSource from qemu driver config qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData qemu: domain: clarify sematics of qemuDomainPrepareDiskSourceData qemu: domain: Tolerate NULL @disk in qemuDomainPrepareDiskSourceData conf: Remove virDomainDiskPathByName conf: Introduce virDomainDiskByTarget qemu: Replace use of virDomainDiskFindByBusAndDst with virDomainDiskByTarget conf: Remove unused virDomainDiskFindByBusAndDst Replace virDomainDiskByName by virDomainDiskByTarget in appropriate cases qemu: monitor: Introduce new interface to query-named-block-nodes qemu: block: Don't query monitor in qemuBlockStorageSourceCreateDetectSize qemu: checkpoint: Fix rollback and access to unlocked 'vm' when deleting checkpoints qemu: monitor: Remove non-transaction based dirty bitmap APIs conf: snapshot: Don't clear current snapshot when redefining an existing one conf: snapshot: Remove 'update_current' parameter from virDomainSnapshotRedefinePrep conf: Don't reuse variable for different object in virDomainCheckpointRedefinePrep conf: checkpoint: Don't clear current checkpoint when redefining an existing one conf: checkpoint: Don't clear current checkpoint when redefining qemu: checkpoint: Enforce that 'bitmap' name must match checkpoint name qemu: checkpoint: Split out checkpoint creation code qemu: checkpoint: Extract finalizing steps of checkpoint creation qemu: monitor: Add helper for generating data for block bitmap merging qemu: checkpoint: Use qemuMonitorTransactionBitmapMergeSourceAddBitmap qemu: blockjob: Refactor qemuBlockJobEventProcessConcludedTransition qemu: blockjob: Use 'g_free' in qemuBlockJobDataDispose Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job qemu: Implement backup job APIs and qemu handling qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 168 ++++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 29 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 518 ++++++++++ src/conf/backup_conf.h | 102 ++ src/conf/checkpoint_conf.c | 17 +- src/conf/domain_addr.c | 4 +- src/conf/domain_conf.c | 50 +- src/conf/domain_conf.h | 8 +- src/conf/snapshot_conf.c | 6 - src/conf/snapshot_conf.h | 1 - src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 20 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 191 ++++ src/libvirt_private.syms | 13 +- src/libvirt_public.syms | 8 + src/libxl/libxl_driver.c | 2 +- src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 895 ++++++++++++++++++ src/qemu/qemu_backup.h | 41 + src/qemu/qemu_block.c | 26 +- src/qemu/qemu_block.h | 5 +- src/qemu/qemu_blockjob.c | 171 ++-- src/qemu/qemu_blockjob.h | 19 + src/qemu/qemu_checkpoint.c | 232 +++-- src/qemu/qemu_checkpoint.h | 15 + src/qemu/qemu_domain.c | 161 +++- src/qemu/qemu_domain.h | 11 +- src/qemu/qemu_driver.c | 116 ++- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_monitor.c | 91 +- src/qemu/qemu_monitor.h | 49 +- src/qemu/qemu_monitor_json.c | 242 ++--- src/qemu/qemu_monitor_json.h | 33 +- src/remote/remote_driver.c | 3 + src/remote/remote_protocol.x | 53 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 2 +- src/util/vircgroup.c | 2 +- src/util/virhash.c | 85 +- src/util/virhash.h | 12 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemublocktest.c | 3 +- tests/qemumonitorjsontest.c | 53 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 209 ++++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 8 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 37 + 71 files changed, 4133 insertions(+), 552 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.21.0

5 83