- Devel - libvirt List Archives

[libvirt] Call for Google Summer of Code 2020 project ideas
by Michal Privoznik 10 Jan '20

10 Jan '20

Dear list, GSoC 2020 has been announced and I plan to apply again [1]. Please, update the list of ideas [2] for students to work on. If you have an idea but don't feel like mentoring, that is okay too, just put it onto the wiki and I will find a mentor myself. Thanks, Michal 1: https://summerofcode.withgoogle.com/ 2: https://wiki.libvirt.org/page/Google_Summer_of_Code_Ideas

1 0

[libvirt] [PATCH v2 0/5] introduce support for CPU dies in host/guest topology
by Daniel P. Berrangé 10 Jan '20

10 Jan '20

Latest generation CPUs (CascadeLake-AP) support a new topology level known as a 'die', sitting between a socket and a core. QEMU supports this with -smp arg since 4.1.0 Linux can report this via /sys/devices/system/cpu/cpuNNN/topology via 'die_id' and 'die_cpus' and 'die_cpus_list' files since 5.2 This series adds support for <topology> in guest XML to have a new 'dies' parameter, passed to QEMU, which defaults to '1' if omitted. It extends host capabilities so that NUMA topology reports a new 'die_id' attribute We can't expand 'virNodeInfoPtr' struct to have a die field, so this will remain forever reporting 'cores' as being 'dies * cores'. The <topology> in host capabilities XML is an interesting question. If we are strict with our API semantics we would *not* add a 'dies' parameter with any value other than '1' to <topology> in the host capabilities. If we reported a value other than 1, then any existing apps which multiple sockets*cores*threads will get the wrong total CPU count. We already know <topology> is broken by design for asymetric hardware, so we could simply document that it will forever be broken wrt to CPU dies too. In this case we might be better to not even report 'dies=1', just leave out the attr entirely. Interestingly though, <topology> is already more broken than it should be. For a VM with -smp 12,sockets=2,dies=3,cores=2,threads=1 it is reporting <topology sockets=1 dies=1 cores=12 threads=1>. It should at least do <topology sockets=2 dies=1 cores=6 threads=1>. I suspect the presence of dies is confusing the really incredibly horrible logic in virHostCPUGetInfoLinux. This will also impact virNodeInfoPtr data. So even if we don't report dies, I think we still have a bug that needs fixing here for the coarse node topology. I'm also confused about what I see with EPYC. IIUC, it was supposed to use the 'dies' concept, but in machines I've tested, Linux never reports die count other than 1. Perhaps only certain EPYC CPU models or generations use 'dies', or perhaps Linux isn't reporting correctly for EPYC, or perhaps I'm mislead into believeing it uses dies. Anyway, the upshot is I've not found any real hardware to test this series on. I've tested it only inside a QEMU guest with the suitable -smp arg to fake dies. Changed in v2: - Fixed docs for dies attribute - Improve error checking when parsing - Misc typos Daniel P. Berrangé (5): conf: add support for specifying CPU "dies" parameter conf: remove unused virCapabilitiesSetHostCPU method qemu: add support for specifying CPU "dies" topology parameter hostcpu: add support for reporting die_id in NUMA topology tests: add host CPU data files for validating die_id docs/formatcaps.html.in | 2 +- docs/formatdomain.html.in | 22 +- docs/schemas/capability.rng | 3 + docs/schemas/cputypes.rng | 5 + src/bhyve/bhyve_command.c | 5 + src/conf/capabilities.c | 26 +- src/conf/capabilities.h | 7 +- src/conf/cpu_conf.c | 23 +- src/conf/cpu_conf.h | 1 + src/conf/domain_conf.c | 3 +- src/cpu/cpu.c | 1 + src/libvirt_linux.syms | 1 + src/libvirt_private.syms | 1 - src/libxl/libxl_capabilities.c | 1 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 12 +- src/util/virhostcpu.c | 16 + src/util/virhostcpu.h | 1 + src/vmx/vmx.c | 7 + .../x86_64-host+guest,model486-result.xml | 2 +- .../x86_64-host+guest,models-result.xml | 2 +- .../cputestdata/x86_64-host+guest-result.xml | 2 +- tests/cputestdata/x86_64-host+guest.xml | 2 +- .../x86_64-host+host-model-nofallback.xml | 2 +- ...t-Haswell-noTSX+Haswell,haswell-result.xml | 2 +- ...ell-noTSX+Haswell-noTSX,haswell-result.xml | 2 +- ...ost-Haswell-noTSX+Haswell-noTSX-result.xml | 2 +- .../x86_64-host-worse+guest-result.xml | 2 +- .../caps_4.1.0.x86_64.xml | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 1 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../ppc64-modern-bulk-result-conf.xml | 2 +- .../ppc64-modern-bulk-result-live.xml | 2 +- .../ppc64-modern-individual-result-conf.xml | 2 +- .../ppc64-modern-individual-result-live.xml | 2 +- .../x86-modern-bulk-result-conf.xml | 2 +- .../x86-modern-bulk-result-live.xml | 2 +- .../x86-modern-individual-add-result-conf.xml | 2 +- .../x86-modern-individual-add-result-live.xml | 2 +- .../x86-old-bulk-result-conf.xml | 2 +- .../x86-old-bulk-result-live.xml | 2 +- .../cpu-hotplug-granularity.xml | 2 +- .../qemuxml2argvdata/cpu-hotplug-startup.xml | 2 +- tests/qemuxml2argvdata/cpu-numa-disjoint.xml | 2 +- .../qemuxml2argvdata/cpu-numa-disordered.xml | 2 +- tests/qemuxml2argvdata/cpu-numa-memshared.xml | 2 +- .../cpu-numa-no-memory-element.xml | 2 +- tests/qemuxml2argvdata/cpu-numa1.xml | 2 +- tests/qemuxml2argvdata/cpu-numa2.xml | 2 +- tests/qemuxml2argvdata/cpu-numa3.xml | 2 +- tests/qemuxml2argvdata/cpu-topology1.xml | 2 +- tests/qemuxml2argvdata/cpu-topology2.xml | 2 +- tests/qemuxml2argvdata/cpu-topology3.xml | 2 +- .../fd-memory-no-numa-topology.xml | 2 +- .../fd-memory-numa-topology.xml | 2 +- .../fd-memory-numa-topology2.xml | 2 +- .../fd-memory-numa-topology3.xml | 2 +- .../graphics-spice-timeout.xml | 2 +- .../hugepages-nvdimm.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/hugepages-nvdimm.xml | 2 +- ...memory-default-hugepage.x86_64-latest.args | 2 +- .../memfd-memory-default-hugepage.xml | 2 +- .../memfd-memory-numa.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/memfd-memory-numa.xml | 2 +- tests/qemuxml2argvdata/memory-align-fail.xml | 2 +- .../memory-hotplug-dimm-addr.xml | 2 +- .../qemuxml2argvdata/memory-hotplug-dimm.xml | 2 +- ...y-hotplug-nvdimm-access.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm-access.xml | 2 +- ...ry-hotplug-nvdimm-align.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm-align.xml | 2 +- ...ry-hotplug-nvdimm-label.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm-label.xml | 2 +- ...ory-hotplug-nvdimm-pmem.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm-pmem.xml | 2 +- ...hotplug-nvdimm-readonly.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm-readonly.xml | 2 +- .../memory-hotplug-nvdimm.x86_64-latest.args | 2 +- .../memory-hotplug-nvdimm.xml | 2 +- tests/qemuxml2argvdata/memory-hotplug.xml | 2 +- .../numad-auto-memory-vcpu-cpuset.xml | 2 +- ...to-memory-vcpu-no-cpuset-and-placement.xml | 2 +- .../numad-auto-vcpu-no-numatune.xml | 2 +- ...d-auto-vcpu-static-numatune-no-nodeset.xml | 2 +- .../numad-auto-vcpu-static-numatune.xml | 2 +- .../numad-static-memory-auto-vcpu.xml | 2 +- .../numad-static-vcpu-no-numatune.xml | 2 +- tests/qemuxml2argvdata/numad.xml | 2 +- .../numatune-auto-nodeset-invalid.xml | 2 +- .../numatune-memory-invalid-nodeset.xml | 2 +- tests/qemuxml2argvdata/numatune-memory.xml | 2 +- .../pci-expander-bus-bad-machine.xml | 2 +- tests/qemuxml2argvdata/pci-expander-bus.xml | 2 +- .../pcie-expander-bus-bad-bus.xml | 2 +- .../pcie-expander-bus-bad-machine.xml | 2 +- tests/qemuxml2argvdata/pcie-expander-bus.xml | 2 +- .../pseries-default-phb-numa-node.xml | 2 +- .../pseries-phb-numa-node.xml | 2 +- tests/qemuxml2argvdata/smp-dies.args | 29 ++ tests/qemuxml2argvdata/smp-dies.xml | 33 ++ tests/qemuxml2argvdata/smp.xml | 2 +- tests/qemuxml2argvtest.c | 1 + .../qemuxml2xmloutdata/cpu-numa-disjoint.xml | 2 +- .../cpu-numa-disordered.xml | 2 +- .../qemuxml2xmloutdata/cpu-numa-memshared.xml | 2 +- .../cpu-numa-no-memory-element.xml | 2 +- tests/qemuxml2xmloutdata/cpu-numa1.xml | 2 +- tests/qemuxml2xmloutdata/cpu-numa2.xml | 2 +- .../graphics-spice-timeout.xml | 2 +- .../memory-hotplug-dimm.xml | 2 +- tests/qemuxml2xmloutdata/memory-hotplug.xml | 2 +- .../numad-auto-memory-vcpu-cpuset.xml | 2 +- ...to-memory-vcpu-no-cpuset-and-placement.xml | 2 +- .../numad-auto-vcpu-no-numatune.xml | 2 +- .../numad-static-vcpu-no-numatune.xml | 2 +- tests/qemuxml2xmloutdata/pci-expander-bus.xml | 2 +- .../qemuxml2xmloutdata/pcie-expander-bus.xml | 2 +- .../pseries-phb-numa-node.xml | 2 +- tests/qemuxml2xmloutdata/smp.xml | 2 +- .../linux-basic-dies/system/cpu | 1 + .../linux-basic-dies/system/node | 1 + .../vircaps2xmldata/vircaps-aarch64-basic.xml | 32 +- .../vircaps-x86_64-basic-dies.xml | 35 ++ .../vircaps2xmldata/vircaps-x86_64-basic.xml | 32 +- .../vircaps2xmldata/vircaps-x86_64-caches.xml | 16 +- .../vircaps-x86_64-resctrl-cdp.xml | 24 +- .../vircaps-x86_64-resctrl-cmt.xml | 24 +- .../vircaps-x86_64-resctrl-fake-feature.xml | 24 +- .../vircaps-x86_64-resctrl-skx-twocaches.xml | 2 +- .../vircaps-x86_64-resctrl-skx.xml | 2 +- .../vircaps-x86_64-resctrl.xml | 24 +- tests/vircaps2xmltest.c | 1 + .../cpu/cpu0/topology/core_cpus | 1 + .../cpu/cpu0/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu0/topology/core_id | 1 + .../cpu/cpu0/topology/core_siblings | 1 + .../cpu/cpu0/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu0/topology/die_cpus | 1 + .../cpu/cpu0/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu0/topology/die_id | 1 + .../cpu/cpu0/topology/package_cpus | 1 + .../cpu/cpu0/topology/package_cpus_list | 1 + .../cpu/cpu0/topology/physical_package_id | 1 + .../cpu/cpu0/topology/thread_siblings | 1 + .../cpu/cpu0/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu1/online | 1 + .../cpu/cpu1/topology/core_cpus | 1 + .../cpu/cpu1/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu1/topology/core_id | 1 + .../cpu/cpu1/topology/core_siblings | 1 + .../cpu/cpu1/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu1/topology/die_cpus | 1 + .../cpu/cpu1/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu1/topology/die_id | 1 + .../cpu/cpu1/topology/package_cpus | 1 + .../cpu/cpu1/topology/package_cpus_list | 1 + .../cpu/cpu1/topology/physical_package_id | 1 + .../cpu/cpu1/topology/thread_siblings | 1 + .../cpu/cpu1/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu10/online | 1 + .../cpu/cpu10/topology/core_cpus | 1 + .../cpu/cpu10/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu10/topology/core_id | 1 + .../cpu/cpu10/topology/core_siblings | 1 + .../cpu/cpu10/topology/core_siblings_list | 1 + .../cpu/cpu10/topology/die_cpus | 1 + .../cpu/cpu10/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu10/topology/die_id | 1 + .../cpu/cpu10/topology/package_cpus | 1 + .../cpu/cpu10/topology/package_cpus_list | 1 + .../cpu/cpu10/topology/physical_package_id | 1 + .../cpu/cpu10/topology/thread_siblings | 1 + .../cpu/cpu10/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu11/online | 1 + .../cpu/cpu11/topology/core_cpus | 1 + .../cpu/cpu11/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu11/topology/core_id | 1 + .../cpu/cpu11/topology/core_siblings | 1 + .../cpu/cpu11/topology/core_siblings_list | 1 + .../cpu/cpu11/topology/die_cpus | 1 + .../cpu/cpu11/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu11/topology/die_id | 1 + .../cpu/cpu11/topology/package_cpus | 1 + .../cpu/cpu11/topology/package_cpus_list | 1 + .../cpu/cpu11/topology/physical_package_id | 1 + .../cpu/cpu11/topology/thread_siblings | 1 + .../cpu/cpu11/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu2/online | 1 + .../cpu/cpu2/topology/core_cpus | 1 + .../cpu/cpu2/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu2/topology/core_id | 1 + .../cpu/cpu2/topology/core_siblings | 1 + .../cpu/cpu2/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu2/topology/die_cpus | 1 + .../cpu/cpu2/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu2/topology/die_id | 1 + .../cpu/cpu2/topology/package_cpus | 1 + .../cpu/cpu2/topology/package_cpus_list | 1 + .../cpu/cpu2/topology/physical_package_id | 1 + .../cpu/cpu2/topology/thread_siblings | 1 + .../cpu/cpu2/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu3/online | 1 + .../cpu/cpu3/topology/core_cpus | 1 + .../cpu/cpu3/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu3/topology/core_id | 1 + .../cpu/cpu3/topology/core_siblings | 1 + .../cpu/cpu3/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu3/topology/die_cpus | 1 + .../cpu/cpu3/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu3/topology/die_id | 1 + .../cpu/cpu3/topology/package_cpus | 1 + .../cpu/cpu3/topology/package_cpus_list | 1 + .../cpu/cpu3/topology/physical_package_id | 1 + .../cpu/cpu3/topology/thread_siblings | 1 + .../cpu/cpu3/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu4/online | 1 + .../cpu/cpu4/topology/core_cpus | 1 + .../cpu/cpu4/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu4/topology/core_id | 1 + .../cpu/cpu4/topology/core_siblings | 1 + .../cpu/cpu4/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu4/topology/die_cpus | 1 + .../cpu/cpu4/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu4/topology/die_id | 1 + .../cpu/cpu4/topology/package_cpus | 1 + .../cpu/cpu4/topology/package_cpus_list | 1 + .../cpu/cpu4/topology/physical_package_id | 1 + .../cpu/cpu4/topology/thread_siblings | 1 + .../cpu/cpu4/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu5/online | 1 + .../cpu/cpu5/topology/core_cpus | 1 + .../cpu/cpu5/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu5/topology/core_id | 1 + .../cpu/cpu5/topology/core_siblings | 1 + .../cpu/cpu5/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu5/topology/die_cpus | 1 + .../cpu/cpu5/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu5/topology/die_id | 1 + .../cpu/cpu5/topology/package_cpus | 1 + .../cpu/cpu5/topology/package_cpus_list | 1 + .../cpu/cpu5/topology/physical_package_id | 1 + .../cpu/cpu5/topology/thread_siblings | 1 + .../cpu/cpu5/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu6/online | 1 + .../cpu/cpu6/topology/core_cpus | 1 + .../cpu/cpu6/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu6/topology/core_id | 1 + .../cpu/cpu6/topology/core_siblings | 1 + .../cpu/cpu6/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu6/topology/die_cpus | 1 + .../cpu/cpu6/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu6/topology/die_id | 1 + .../cpu/cpu6/topology/package_cpus | 1 + .../cpu/cpu6/topology/package_cpus_list | 1 + .../cpu/cpu6/topology/physical_package_id | 1 + .../cpu/cpu6/topology/thread_siblings | 1 + .../cpu/cpu6/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu7/online | 1 + .../cpu/cpu7/topology/core_cpus | 1 + .../cpu/cpu7/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu7/topology/core_id | 1 + .../cpu/cpu7/topology/core_siblings | 1 + .../cpu/cpu7/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu7/topology/die_cpus | 1 + .../cpu/cpu7/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu7/topology/die_id | 1 + .../cpu/cpu7/topology/package_cpus | 1 + .../cpu/cpu7/topology/package_cpus_list | 1 + .../cpu/cpu7/topology/physical_package_id | 1 + .../cpu/cpu7/topology/thread_siblings | 1 + .../cpu/cpu7/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu8/online | 1 + .../cpu/cpu8/topology/core_cpus | 1 + .../cpu/cpu8/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu8/topology/core_id | 1 + .../cpu/cpu8/topology/core_siblings | 1 + .../cpu/cpu8/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu8/topology/die_cpus | 1 + .../cpu/cpu8/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu8/topology/die_id | 1 + .../cpu/cpu8/topology/package_cpus | 1 + .../cpu/cpu8/topology/package_cpus_list | 1 + .../cpu/cpu8/topology/physical_package_id | 1 + .../cpu/cpu8/topology/thread_siblings | 1 + .../cpu/cpu8/topology/thread_siblings_list | 1 + .../linux-with-die/cpu/cpu9/online | 1 + .../cpu/cpu9/topology/core_cpus | 1 + .../cpu/cpu9/topology/core_cpus_list | 1 + .../linux-with-die/cpu/cpu9/topology/core_id | 1 + .../cpu/cpu9/topology/core_siblings | 1 + .../cpu/cpu9/topology/core_siblings_list | 1 + .../linux-with-die/cpu/cpu9/topology/die_cpus | 1 + .../cpu/cpu9/topology/die_cpus_list | 1 + .../linux-with-die/cpu/cpu9/topology/die_id | 1 + .../cpu/cpu9/topology/package_cpus | 1 + .../cpu/cpu9/topology/package_cpus_list | 1 + .../cpu/cpu9/topology/physical_package_id | 1 + .../cpu/cpu9/topology/thread_siblings | 1 + .../cpu/cpu9/topology/thread_siblings_list | 1 + .../virhostcpudata/linux-with-die/cpu/online | 1 + .../virhostcpudata/linux-with-die/cpu/present | 1 + .../linux-with-die/node/node0/cpu0 | 1 + .../linux-with-die/node/node0/cpu1 | 1 + .../linux-with-die/node/node0/cpu10 | 1 + .../linux-with-die/node/node0/cpu11 | 1 + .../linux-with-die/node/node0/cpu2 | 1 + .../linux-with-die/node/node0/cpu3 | 1 + .../linux-with-die/node/node0/cpu4 | 1 + .../linux-with-die/node/node0/cpu5 | 1 + .../linux-with-die/node/node0/cpu6 | 1 + .../linux-with-die/node/node0/cpu7 | 1 + .../linux-with-die/node/node0/cpu8 | 1 + .../linux-with-die/node/node0/cpu9 | 1 + .../linux-with-die/node/node0/cpulist | 1 + .../virhostcpudata/linux-with-die/node/online | 1 + .../linux-with-die/node/possible | 1 + .../linux-x86_64-with-die.cpuinfo | 323 ++++++++++++++++++ .../linux-x86_64-with-die.expected | 1 + tests/virhostcputest.c | 1 + .../vmx2xmldata/vmx2xml-esx-in-the-wild-8.xml | 2 +- .../vmx2xmldata/vmx2xml-esx-in-the-wild-9.xml | 2 +- 324 files changed, 896 insertions(+), 231 deletions(-) create mode 100644 tests/qemuxml2argvdata/smp-dies.args create mode 100644 tests/qemuxml2argvdata/smp-dies.xml create mode 120000 tests/vircaps2xmldata/linux-basic-dies/system/cpu create mode 120000 tests/vircaps2xmldata/linux-basic-dies/system/node create mode 100644 tests/vircaps2xmldata/vircaps-x86_64-basic-dies.xml create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu0/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu1/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu10/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu11/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu2/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu3/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu4/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu5/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu6/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu7/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu8/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/core_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/core_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/core_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/core_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/core_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/die_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/die_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/die_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/package_cpus create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/package_cpus_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/physical_package_id create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/thread_siblings create mode 100644 tests/virhostcpudata/linux-with-die/cpu/cpu9/topology/thread_siblings_list create mode 100644 tests/virhostcpudata/linux-with-die/cpu/online create mode 100644 tests/virhostcpudata/linux-with-die/cpu/present create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu0 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu1 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu10 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu11 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu2 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu3 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu4 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu5 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu6 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu7 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu8 create mode 120000 tests/virhostcpudata/linux-with-die/node/node0/cpu9 create mode 100644 tests/virhostcpudata/linux-with-die/node/node0/cpulist create mode 100644 tests/virhostcpudata/linux-with-die/node/online create mode 100644 tests/virhostcpudata/linux-with-die/node/possible create mode 100644 tests/virhostcpudata/linux-x86_64-with-die.cpuinfo create mode 100644 tests/virhostcpudata/linux-x86_64-with-die.expected -- 2.23.0

2 7

[libvirt] [dockerfiles PATCH] Refresh after dropping librbd from Debian sid on mipsel
by Andrea Bolognani 10 Jan '20

10 Jan '20

The corresponding libvirt-jenkins-ci commit is d08ad842e12d. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Pushed under the Dockerfile refresh rule. buildenv-libvirt-debian-sid-cross-mipsel.zip | Bin 1016 -> 1013 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/buildenv-libvirt-debian-sid-cross-mipsel.zip b/buildenv-libvirt-debian-sid-cross-mipsel.zip index 19594f2..815772b 100644 --- a/buildenv-libvirt-debian-sid-cross-mipsel.zip +++ b/buildenv-libvirt-debian-sid-cross-mipsel.zip @@ -89,7 +89,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libparted-dev:mipsel \ libpcap0.8-dev:mipsel \ libpciaccess-dev:mipsel \ - librbd-dev:mipsel \ libreadline-dev:mipsel \ libsanlock-dev:mipsel \ libsasl2-dev:mipsel \ -- 2.24.1

1 0

[libvirt] [jenkins-ci PATCH 0/2] Fixes necessary for buildenv-libvirt-debian-sid-cross-mipsel
by Andrea Bolognani 10 Jan '20

10 Jan '20

The latest container build failed; this series addresses the underlying issue. Andrea Bolognani (2): mappings: Refactor librbd mapping mappings: Remove librbd from Debian on mipsel going forward guests/vars/mappings.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) -- 2.24.1

2 6

[libvirt] [PATCH] docs: add pages to support Go module package resolution
by Daniel P. Berrangé 10 Jan '20

10 Jan '20

Currently the libvirt Go modules are accessed by applications using their github repository URLs. This is undesirable as we don't want applications to have a direct dependancy on a specific source repo location. We want to enable applications to use the Go packages via the libvirt.org namespace. When you do "go get libvirt.org/libvirt-go", the Go client will do an HTTPS request to that URL, and parse the HTML content to look for a <meta> tag which tells it where to the find the GIT repository. https://golang.org/cmd/go/#hdr-Remote_import_paths This adds two pages to support this Go module resolution. They are not linked from anywhere as we don't expect users to actually look at them. If someone does happen upon them, there's some boilerplate text to send them off to godoc.org for API documentation. Since the pages we're adding have a .html extension, we will also use a small apache config tweak on the server RewriteEngine on RewriteRule ^/libvirt-go$ /libvirt-go.html [L] RewriteRule ^/libvirt-go-xml$ /libvirt-go-xml.html [L] Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/libvirt-go-xml.rst | 10 ++++++++++ docs/libvirt-go.rst | 13 +++++++++++++ docs/page.xsl | 6 ++++++ 3 files changed, 29 insertions(+) create mode 100644 docs/libvirt-go-xml.rst create mode 100644 docs/libvirt-go.rst diff --git a/docs/libvirt-go-xml.rst b/docs/libvirt-go-xml.rst new file mode 100644 index 0000000000..ee3b878068 --- /dev/null +++ b/docs/libvirt-go-xml.rst @@ -0,0 +1,10 @@ +========================== +Libvirt Go XML parsing API +========================== + +The `Go <https://golang.org/>`__ package ``libvirt.org/libvirt-go-xml`` provides +annotated Go struct definitions for parsing (and formatting) XML documents used +with libvirt APIs. + +For details of Go specific behaviour consult the +`Go package documentation <https://godoc.org/libvirt.org/libvirt-go-xml>`__ diff --git a/docs/libvirt-go.rst b/docs/libvirt-go.rst new file mode 100644 index 0000000000..6ec5c0c123 --- /dev/null +++ b/docs/libvirt-go.rst @@ -0,0 +1,13 @@ +======================= +Libvirt Go Language API +======================= + +The `Go <https://golang.org/>`__ package ``libvirt.org/libvirt-go`` provides +`CGo <https://golang.org/cmd/cgo/>`__ binding from the OS native Libvirt API. + +In general the Go representation is a direct 1-1 mapping from native API +concepts to Go, so the native API documentation should serve as a reference +for most behaviour. + +For details of Go specific behaviour consult the +`Go package documentation <https://godoc.org/libvirt.org/libvirt-go>`__ diff --git a/docs/page.xsl b/docs/page.xsl index f8f7ff8cf9..ddae5ab508 100644 --- a/docs/page.xsl +++ b/docs/page.xsl @@ -99,6 +99,12 @@ <meta name="theme-color" content="#ffffff"/> <title>libvirt: <xsl:value-of select="html:html/html:body//html:h1"/></title> <meta name="description" content="libvirt, virtualization, virtualization API"/> + <xsl:if test="$pagename = 'libvirt-go.html'"> + <meta name="go-import" content="libvirt.org/libvirt-go git https://libvirt.org/git/libvirt-go.git"/> + </xsl:if> + <xsl:if test="$pagename = 'libvirt-go-xml.html'"> + <meta name="go-import" content="libvirt.org/libvirt-go-xml git https://libvirt.org/git/libvirt-go-xml.git"/> + </xsl:if> <xsl:apply-templates select="/html:html/html:head/html:script" mode="content"/> <script type="text/javascript" src="{$href_base}js/main.js"> -- 2.23.0

2 3

[libvirt] [PATCH 0/2] Improve error message for missing backing image format
by Peter Krempa 10 Jan '20

10 Jan '20

Improve the error message and mention it in the news. Peter Krempa (2): util: storage: Link to knowledge base when reporting missing image backing format news: Mention problems with backing image format probing docs/news.xml | 13 +++++++++++++ src/util/virstoragefile.c | 3 ++- 2 files changed, 15 insertions(+), 1 deletion(-) -- 2.24.1

2 4

[libvirt] [PATCH v3 0/5] introduce support for an embedded driver mode
by Daniel P. Berrangé 10 Jan '20

10 Jan '20

This is a followup to: https://www.redhat.com/archives/libvir-list/2019-May/msg00467.html https://www.redhat.com/archives/libvir-list/2019-December/msg00050.html This series implements support for an embedded driver mode for libvirt, with initial support in the QEMU and secrets drivers. In this mode of operation, the driver stores all its config and state under a private directory tree. See the individual patches for the illustrated directory hierarchy used. The intent of this embedded mode is to suit cases where the application is using virtualization as a building block for some functionality, as opposed to running traditional "full OS" builds. The long time posterchild example would be libguestfs, while a more recent example could be Kata containers. The general principal in enabling this embedded mode is that the functionality available should be identical to that seen when the driver is running inside libvirtd. This is achieved by loading the exact same driver .so module as libvirtd would load, and simply configuring it with a different directory layout. The result of this is that when running in embedded mode, the driver can still talk to other secondary drivers running inside libvirtd if desired. This is useful, for example, to connect a VM to the default virtual network. The secondary drivers can be made to operate in embedded mode as well, however, this will require some careful consideration for each driver to ensure they don't clash with each other. Thus in this series only the secret driver is enabled for embedded mode. This is required to enable use of VMs with encrypted disks, or authenticated network block storage. In this series we introduce a new command line tool 'virt-qemu-run' which is a really simple tool for launching a VM in embedded mode. I'm not entirely sure whether we should provide this as an official supported tool in this way, or merely put it into the 'examples' directory as demo-ware. With testing of the virt-qemu-run tool we can immediately see what the next important thing to tackle is: performance. We have not really cared too much about the startup performance of libvirtd as this is a one time cost when the mgmt application connects. We did none the less cache capabilities because probing caps for 30 QEMU binaries takes a long time. Even with this caching it takes an unacceptably long time to start a VM in embedded mode. About 100 ms to open the embedded QEMU driver, assuming pre-cached capabilies - ~2 seconds if not cached and all 30 QEMU targets are present. Then about 300 ms to actually start the QEMU guest. IOW, about 400 ms to get QEMU running. NB this is measuring time from launching the virt-run-qemu program, to the point at which the API call 'virDomainCreate' returns control. This has both libvirt & QEMU overhead in & I don't have clear figures to distinguish, but I can see a 40 ms delay between issuing the 'qmp_capabilities' call and getting a reply, which is QEMU startup overead. This is a i440fx based QEMU with a general purpose virtio-pci config (disk, net, etc) tyupical for running a full OS. I've not tried any kind of optimized QEMU config with microvm. I've already started on measuring & optimizing & identified several key areas that can be addressed, but it is all ultimately about not doing work before we need the answers from that work (which often means we will never do the work at all). For example, we shouldn't probe all 30 QEMU's upfront. If the app is only going to create an x86_64 KVM guest we should only care about that 1 QEMU. This is painful because parsing any guest XML requires a virCapsPtr which in turn causes probing of every QEMU binary. I've got in progress patches to eliminate virCapsPtr almost entirely and work directly with the virQEMUCapsPtr instead. It is possible we'll want to use a different file format for storing the cached QEMU capabilities, and the CPU feature/model info. Parsing this XML is a non-negligible time sink. A binary format is likely way quicker, especially if its designed to be just mmap'able for direct read. To be investigated... We shouldn't probe for whether host PM suspend is possible unless someone wants that info, or tries to issue that API call. After starting QEMU we spend 150-200 ms issuing a massive number of qom-get calls to check whether QEMU enabled each individual CPU feature flag. We only need this info if someone asks for the live XML or we intend to live migrate etc. So we shouldn't issue these qom-get calls in the "hot path" of QEMU startup. It can be done later in a non-time critical point. Also the QEMU API for this is horribly inefficient to require so many qom-get calls. There's more but I won't talk about it now. Suffice to say that I think we can get libvirt overhead down to less than 100 ms fairly easily and probably even down to less than 50 ms without much effort. The exact figure will depend on what libvirt features you want enabled, and how much work we want/need to put into optimization. We'll want to fix the really gross mistakes & slow downs, but we'll want guidance from likely users as to their VM startup targets to decide how much work needs investing. This optimization will ultimately help non-embedded QEMU mode too, making it faster to respond & start. Changed in v3: - Rebased to master - Merge some simple acked patches Changed in v2: - Use a simplified directory layout for embedded mode. Previously we just put a dir prefix onto the normal paths. This has the downside that the embedded drivers paths are needlessly different for privileged vs unprivileged user. It also results in very long paths which can be a problem for the UNIX socket name length limits. - Also ported the secret driver to support embedded mode - Check to validate that the event loop is registered. - Add virt-qemu-run tool for embedded usage. - Added docs for the qemu & secret driver explaining embedded mode Daniel P. Berrangé (5): libvirt: pass a directory path into drivers for embedded usage libvirt: support an "embed" URI path selector for opening drivers qemu: add support for running QEMU driver in embedded mode secrets: add support for running secret driver in embedded mode qemu: introduce a new "virt-qemu-run" program build-aux/syntax-check.mk | 2 +- docs/Makefile.am | 5 + docs/drivers.html.in | 1 + docs/drvqemu.html.in | 84 +++++++ docs/drvsecret.html.in | 82 ++++++ docs/manpages/index.rst | 1 + docs/manpages/virt-qemu-run.rst | 114 +++++++++ libvirt.spec.in | 2 + src/Makefile.am | 1 + src/driver-state.h | 2 + src/driver.h | 2 + src/interface/interface_backend_netcf.c | 7 + src/interface/interface_backend_udev.c | 7 + src/libvirt.c | 93 ++++++- src/libvirt_internal.h | 4 +- src/libxl/libxl_driver.c | 7 + src/lxc/lxc_driver.c | 8 + src/network/bridge_driver.c | 7 + src/node_device/node_device_hal.c | 7 + src/node_device/node_device_udev.c | 7 + src/nwfilter/nwfilter_driver.c | 7 + src/qemu/Makefile.inc.am | 13 + src/qemu/qemu_conf.c | 38 ++- src/qemu/qemu_conf.h | 6 +- src/qemu/qemu_driver.c | 21 +- src/qemu/qemu_process.c | 15 +- src/qemu/qemu_shim.c | 322 ++++++++++++++++++++++++ src/remote/remote_daemon.c | 1 + src/remote/remote_driver.c | 1 + src/secret/secret_driver.c | 41 ++- src/storage/storage_driver.c | 7 + src/vz/vz_driver.c | 7 + tests/domaincapstest.c | 2 +- tests/testutilsqemu.c | 3 +- 34 files changed, 901 insertions(+), 26 deletions(-) create mode 100644 docs/drvsecret.html.in create mode 100644 docs/manpages/virt-qemu-run.rst create mode 100644 src/qemu/qemu_shim.c -- 2.23.0

2 9

[libvirt] [PATCH v3 0/3] Mark and document restore with managed save as risky
by Michael Weiser 10 Jan '20

10 Jan '20

This series marks restore of an inactive qemu snapshot while there is managed saved state as risky due to the reasons explained in patch 1 and 3. Patch 2 is a simple reformatting of the documentation with no other changes in preparation of addition of more reasons why reverts might need to be forced. Changes from v2: - fix manpage typo exising -> existing - change manpage qemu hypervisor name case to QEMU Changes from v1: - reword error message to "error: revert requires force: snapshot without memory state, removal of existing managed saved state strongly recommended to avoid corruption" - add documentation of the new behaviour Michael Weiser (3): qemu: Warn of restore with managed save being risky docs: Reformat snapshot-revert force reasons docs: Add snapshot-revert qemu managedsave force docs/manpages/virsh.rst | 39 ++++++++++++++++++++++++--------------- src/qemu/qemu_driver.c | 9 +++++++++ 2 files changed, 33 insertions(+), 15 deletions(-) -- 2.24.1

3 6

[libvirt] [PATCH] tests: avoid re-execing test once for each mock
by Daniel P. Berrangé 10 Jan '20

10 Jan '20

When debugging tests under GDB there is a significant time delay each time an execve is done as GDB scans shared libraries once again. For tests which use many mock libraries, we have been invoking execve many times which makes the GDB experience horrible. This changes our framework to activate the full set of mock libraries in one single execve. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- tests/qemucapsprobe.c | 8 +++++++- tests/testutils.c | 29 ++++++++++++++++++++++++----- tests/testutils.h | 10 +++------- 3 files changed, 34 insertions(+), 13 deletions(-) diff --git a/tests/qemucapsprobe.c b/tests/qemucapsprobe.c index 6837f2a0f6..c7e8f3309d 100644 --- a/tests/qemucapsprobe.c +++ b/tests/qemucapsprobe.c @@ -46,8 +46,14 @@ main(int argc, char **argv) { virThread thread; virQEMUCapsPtr caps; + const char *mock = VIR_TEST_MOCK("qemucapsprobe"); - VIR_TEST_PRELOAD(VIR_TEST_MOCK("qemucapsprobe")); + if (!virFileIsExecutable(mock)) { + perror(mock); + return EXIT_FAILURE; + } + + VIR_TEST_PRELOAD(mock); if (argc != 2) { fprintf(stderr, "%s QEMU_binary\n", argv[0]); diff --git a/tests/testutils.c b/tests/testutils.c index 6f0ef2b2f1..b490609e36 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -856,15 +856,34 @@ int virTestMain(int argc, virLogOutputPtr *outputs = NULL; g_autofree char *baseprogname = NULL; const char *progname; - - if (getenv("VIR_TEST_FILE_ACCESS")) - VIR_TEST_PRELOAD(VIR_TEST_MOCK("virtest")); + g_autofree const char **preloads = NULL; + size_t npreloads = 0; + g_autofree char *mock = NULL; + + if (getenv("VIR_TEST_FILE_ACCESS")) { + preloads = g_renew(const char *, preloads, npreloads + 2); + preloads[npreloads++] = VIR_TEST_MOCK("virtest"); + preloads[npreloads] = NULL; + } va_start(ap, func); - while ((lib = va_arg(ap, const char *))) - VIR_TEST_PRELOAD(lib); + while ((lib = va_arg(ap, const char *))) { + if (!virFileIsExecutable(lib)) { + perror(lib); + return EXIT_FAILURE; + } + + preloads = g_renew(const char *, preloads, npreloads + 2); + preloads[npreloads++] = lib; + preloads[npreloads] = NULL; + } va_end(ap); + if (preloads) { + mock = g_strjoinv(":", (char **)preloads); + VIR_TEST_PRELOAD(mock); + } + progname = baseprogname = g_path_get_basename(argv[0]); if (STRPREFIX(progname, "lt-")) progname += 3; diff --git a/tests/testutils.h b/tests/testutils.h index 8a7ea38f43..2944c25e8b 100644 --- a/tests/testutils.h +++ b/tests/testutils.h @@ -127,19 +127,15 @@ int virTestMain(int argc, # define MOCK_EXT ".so" #endif -#define VIR_TEST_PRELOAD(lib) \ +#define VIR_TEST_PRELOAD(libs) \ do { \ const char *preload = getenv(PRELOAD_VAR); \ if (preload == NULL || strstr(preload, lib) == NULL) { \ char *newenv; \ - if (!virFileIsExecutable(lib)) { \ - perror(lib); \ - return EXIT_FAILURE; \ - } \ if (!preload) { \ - newenv = (char *) lib; \ + newenv = (char *) libs; \ } else { \ - newenv = g_strdup_printf("%s:%s", lib, preload); \ + newenv = g_strdup_printf("%s:%s", libs, preload); \ } \ g_setenv(PRELOAD_VAR, newenv, TRUE); \ FORCE_FLAT_NAMESPACE \ -- 2.24.1

2 1

[libvirt] [PATCH 00/19] qemu: Add support for backups in combination with snapshots
by Peter Krempa 10 Jan '20

10 Jan '20

IMPORTANT: This does NOT add support for merging the bitmaps during block jobs, so manual deletion of snapshots will corrupt the incremental metadata!. Allow combining snapshots and backups. During a snapshot we create a bitmap for any active persistent bitmap to continue tracking the bitmaps properly (as qemu doesn't support creating a dirty bitmap from the allocation map). This also means that it works only for an active VM since there also isn't a way to do it via qemu-img. Thus a workaround is to start a VM as paused if this is required for offline VMs. We also now can merge bitmaps accross the backing chains. You can fetch the changes at: git fetch https://gitlab.com/pipo.sk/libvirt.git backup-snapshots Peter Krempa (19): virsh: Add QMP command wrapping for 'qemu-monitor-command' virsh: Allow extracting 'return' section of QMP command in 'qemu-monitor-command' qemu: monitor: Extract data about dirty-bimaps in qemuMonitorBlockGetNamedNodeData qemu: monitor: Extract internals of qemuMonitorJSONBlockGetNamedNodeData tests: qemublock: Add test for bitmap detection tests: qemublocktest: Add a syntetic test case for bitmap detection qemu: Check for explicit failure of qemuBlockSnapshotAddBlockdev qemu: snapshot: Fold formatting of snapshot transaction into prepare func qemu: monitor: Add 'granularity' parameter for block-dirty-bitmap-add qemu: snapshot: Propagate active bitmaps through external snapshots tests: qemublock: Add test case for detecting bitmaps as we create snapshots qemu: backup: Return 'def' instead of 'obj' from qemuBackupBeginCollectIncrementalCheckpoints qemu: backup: Extract calculations of bitmaps to merge for incremental backup qemu: backup: Propagate bitmap metadata into qemuBackupDiskPrepareOneBitmapsChain qemu: backup: Export qemuBackupDiskPrepareOneBitmapsChain for tests tests: qemublock: Add testing of bitmap merging for incremental backups qemu: block: Introduce qemuBlockNamedNodeDataGetBitmapByName qemu: backup: Merge bitmaps accross the backing chain tests: qemublock: Add tests for cross-snapshot incremental backups docs/manpages/virsh.rst | 27 +- src/qemu/qemu_backup.c | 145 ++- src/qemu/qemu_backup.h | 7 + src/qemu/qemu_block.c | 32 + src/qemu/qemu_block.h | 5 + src/qemu/qemu_checkpoint.c | 2 +- src/qemu/qemu_driver.c | 84 +- src/qemu/qemu_monitor.c | 6 +- src/qemu/qemu_monitor.h | 18 +- src/qemu/qemu_monitor_json.c | 96 +- src/qemu/qemu_monitor_json.h | 6 +- tests/qemublocktest.c | 241 +++++ .../backupmerge/basic-deep-out.json | 22 + .../backupmerge/basic-flat-out.json | 6 + .../backupmerge/basic-intermediate-out.json | 10 + .../backupmerge/snapshot-deep-out.json | 38 + .../backupmerge/snapshot-flat-out.json | 6 + .../snapshot-intermediate-out.json | 14 + tests/qemublocktestdata/bitmap/basic.json | 117 +++ tests/qemublocktestdata/bitmap/basic.out | 6 + tests/qemublocktestdata/bitmap/snapshots.json | 836 ++++++++++++++++++ tests/qemublocktestdata/bitmap/snapshots.out | 14 + tests/qemublocktestdata/bitmap/synthetic.json | 118 +++ tests/qemublocktestdata/bitmap/synthetic.out | 6 + tests/qemumonitorjsontest.c | 2 +- tools/virsh-domain.c | 76 +- 26 files changed, 1845 insertions(+), 95 deletions(-) create mode 100644 tests/qemublocktestdata/backupmerge/basic-deep-out.json create mode 100644 tests/qemublocktestdata/backupmerge/basic-flat-out.json create mode 100644 tests/qemublocktestdata/backupmerge/basic-intermediate-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshot-deep-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshot-flat-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json create mode 100644 tests/qemublocktestdata/bitmap/basic.json create mode 100644 tests/qemublocktestdata/bitmap/basic.out create mode 100644 tests/qemublocktestdata/bitmap/snapshots.json create mode 100644 tests/qemublocktestdata/bitmap/snapshots.out create mode 100644 tests/qemublocktestdata/bitmap/synthetic.json create mode 100644 tests/qemublocktestdata/bitmap/synthetic.out -- 2.23.0

4 46