- Devel - libvirt List Archives

[PATCH v2] numa: forbid '-numa node, mem' for 5.1 and newer machine types
by Igor Mammedov 23 Jun '20

23 Jun '20

Deprecation period is run out and it's a time to flip the switch introduced by cd5ff8333a. Disable legacy option for new machine types (since 5.1) and amend documentation. '-numa node,memdev' shall be used instead of disabled option with new machine types. Signed-off-by: Igor Mammedov <imammedo(a)redhat.com> Reviewed-by: Michal Privoznik <mprivozn(a)redhat.com> --- v1: - rebased on top of current master - move compat mode from 4.2 to 5.0 v2: - move deprection text to recently removed section - pick up reviewed-bys CC: peter.maydell(a)linaro.org CC: ehabkost(a)redhat.com CC: marcel.apfelbaum(a)gmail.com CC: mst(a)redhat.com CC: pbonzini(a)redhat.com CC: rth(a)twiddle.net CC: david(a)gibson.dropbear.id.au CC: libvir-list(a)redhat.com CC: qemu-arm(a)nongnu.org CC: qemu-ppc(a)nongnu.org CC: eblake(a)redhat.com --- docs/system/deprecated.rst | 37 ++++++++++++++++++++----------------- hw/arm/virt.c | 2 +- hw/core/numa.c | 6 ++++++ hw/i386/pc.c | 1 - hw/i386/pc_piix.c | 1 + hw/i386/pc_q35.c | 1 + hw/ppc/spapr.c | 2 +- qemu-options.hx | 9 +++++---- 8 files changed, 35 insertions(+), 24 deletions(-) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index f0061f94aa..6f717e4a1d 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -101,23 +101,6 @@ error in the future. The ``-realtime mlock=on|off`` argument has been replaced by the ``-overcommit mem-lock=on|off`` argument. -``-numa node,mem=``\ *size* (since 4.1) -''''''''''''''''''''''''''''''''''''''' - -The parameter ``mem`` of ``-numa node`` is used to assign a part of -guest RAM to a NUMA node. But when using it, it's impossible to manage specified -RAM chunk on the host side (like bind it to a host node, setting bind policy, ...), -so guest end-ups with the fake NUMA configuration with suboptiomal performance. -However since 2014 there is an alternative way to assign RAM to a NUMA node -using parameter ``memdev``, which does the same as ``mem`` and adds -means to actualy manage node RAM on the host side. Use parameter ``memdev`` -with *memory-backend-ram* backend as an replacement for parameter ``mem`` -to achieve the same fake NUMA effect or a properly configured -*memory-backend-file* backend to actually benefit from NUMA configuration. -In future new machine versions will not accept the option but it will still -work with old machine types. User can check QAPI schema to see if the legacy -option is supported by looking at MachineInfo::numa-mem-supported property. - ``-numa`` node (without memory specified) (since 4.1) ''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -512,3 +495,23 @@ long starting at 1MiB, the old command:: can be rewritten as:: qemu-nbd -t --image-opts driver=raw,offset=1M,size=100M,file.driver=qcow2,file.file.driver=file,file.file.filename=file.qcow2 + +Command line options +-------------------- + +``-numa node,mem=``\ *size* (removed in 5.1) +''''''''''''''''''''''''''''''''''''''' + +The parameter ``mem`` of ``-numa node`` is used to assign a part of +guest RAM to a NUMA node. But when using it, it's impossible to manage specified +RAM chunk on the host side (like bind it to a host node, setting bind policy, ...), +so guest end-ups with the fake NUMA configuration with suboptiomal performance. +However since 2014 there is an alternative way to assign RAM to a NUMA node +using parameter ``memdev``, which does the same as ``mem`` and adds +means to actualy manage node RAM on the host side. Use parameter ``memdev`` +with *memory-backend-ram* backend as an replacement for parameter ``mem`` +to achieve the same fake NUMA effect or a properly configured +*memory-backend-file* backend to actually benefit from NUMA configuration. +In future new machine versions will not accept the option but it will still +work with old machine types. User can check QAPI schema to see if the legacy +option is supported by looking at MachineInfo::numa-mem-supported property. diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 37462a6f78..063d4703f7 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -2262,7 +2262,6 @@ static void virt_machine_class_init(ObjectClass *oc, void *data) hc->pre_plug = virt_machine_device_pre_plug_cb; hc->plug = virt_machine_device_plug_cb; hc->unplug_request = virt_machine_device_unplug_request_cb; - mc->numa_mem_supported = true; mc->nvdimm_supported = true; mc->auto_enable_numa_with_memhp = true; mc->default_ram_id = "mach-virt.ram"; @@ -2375,6 +2374,7 @@ DEFINE_VIRT_MACHINE_AS_LATEST(5, 1) static void virt_machine_5_0_options(MachineClass *mc) { virt_machine_5_1_options(mc); + mc->numa_mem_supported = true; } DEFINE_VIRT_MACHINE(5, 0) diff --git a/hw/core/numa.c b/hw/core/numa.c index 316bc50d75..05be412e59 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -117,6 +117,12 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node, } if (node->has_mem) { + if (!mc->numa_mem_supported) { + error_setg(errp, "Parameter -numa node,mem is not supported by this" + " machine type. Use -numa node,memdev instead"); + return; + } + numa_info[nodenr].node_mem = node->mem; if (!qtest_enabled()) { warn_report("Parameter -numa node,mem is deprecated," diff --git a/hw/i386/pc.c b/hw/i386/pc.c index 2128f3d6fe..a86136069c 100644 --- a/hw/i386/pc.c +++ b/hw/i386/pc.c @@ -1960,7 +1960,6 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) hc->unplug = pc_machine_device_unplug_cb; mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE; mc->nvdimm_supported = true; - mc->numa_mem_supported = true; mc->default_ram_id = "pc.ram"; object_class_property_add(oc, PC_MACHINE_DEVMEM_REGION_SIZE, "int", diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index f66e1d73ce..8955436efa 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -438,6 +438,7 @@ static void pc_i440fx_5_0_machine_options(MachineClass *m) pc_i440fx_5_1_machine_options(m); m->alias = NULL; m->is_default = false; + m->numa_mem_supported = true; compat_props_add(m->compat_props, hw_compat_5_0, hw_compat_5_0_len); compat_props_add(m->compat_props, pc_compat_5_0, pc_compat_5_0_len); } diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index 4ba8ac8774..af6b32adb7 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -366,6 +366,7 @@ static void pc_q35_5_0_machine_options(MachineClass *m) { pc_q35_5_1_machine_options(m); m->alias = NULL; + m->numa_mem_supported = true; compat_props_add(m->compat_props, hw_compat_5_0, hw_compat_5_0_len); compat_props_add(m->compat_props, pc_compat_5_0, pc_compat_5_0_len); } diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 3b1a5ed865..210d61d1b3 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -4511,7 +4511,6 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) * in which LMBs are represented and hot-added */ mc->numa_mem_align_shift = 28; - mc->numa_mem_supported = true; mc->auto_enable_numa = true; smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; @@ -4599,6 +4598,7 @@ static void spapr_machine_5_0_class_options(MachineClass *mc) { spapr_machine_5_1_class_options(mc); compat_props_add(mc->compat_props, hw_compat_5_0, hw_compat_5_0_len); + mc->numa_mem_supported = true; } DEFINE_SPAPR_MACHINE(5_0, "5.0", false); diff --git a/qemu-options.hx b/qemu-options.hx index 93bde2bbc8..196f468786 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -239,10 +239,11 @@ SRST -numa node,nodeid=0 -numa node,nodeid=1 \ -numa cpu,node-id=0,socket-id=0 -numa cpu,node-id=1,socket-id=1 - '\ ``mem``\ ' assigns a given RAM amount to a node. '\ ``memdev``\ ' - assigns RAM from a given memory backend device to a node. If - '\ ``mem``\ ' and '\ ``memdev``\ ' are omitted in all nodes, RAM is - split equally between them. + Legacy '\ ``mem``\ ' assigns a given RAM amount to a node (not supported + for 5.1 and newer machine types). '\ ``memdev``\ ' assigns RAM from + a given memory backend device to a node. If '\ ``mem``\ ' and + '\ ``memdev``\ ' are omitted in all nodes, RAM is split equally between them. + '\ ``mem``\ ' and '\ ``memdev``\ ' are mutually exclusive. Furthermore, if one node uses '\ ``memdev``\ ', all of them have to -- 2.18.4

3 3

[PATCH 0/5] qemu: backup: Memory handling fixes
by Peter Krempa 23 Jun '20

23 Jun '20

See individual patches. Peter Krempa (5): qemuBackupBegin: Don't leak 'def' on early failures qemu: backup: Initialize 'store' source properly and just once qemuBackupDiskStarted: Fix improper dereference of array qemuBackupDiskDataCleanupOne: Don't exit early when the job has started qemuBackupDiskDataCleanupOne: Free 'incrementalBitmap' src/qemu/qemu_backup.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) -- 2.26.2

2 6

[libvirt PATCH] qemu: format 'ramfb' attribute for mediated devices
by Jonathon Jongsma 23 Jun '20

23 Jun '20

It's possible to use ramfb as the boot display of an assigned vgpu device. This was introduced in 4b95738c, but unfortunately the attribute was not formatted into the xml output for such a device. This patch fixes that oversight and adds a xml2xml test to verify proper behavior. https://bugzilla.redhat.com/show_bug.cgi?id=1847791 Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- src/conf/domain_conf.c | 3 ++ ...stdev-mdev-display-ramfb.x86_64-latest.xml | 44 +++++++++++++++++++ .../hostdev-mdev-display-ramfb.xml | 33 ++++++++++++++ tests/qemuxml2xmltest.c | 1 + 4 files changed, 81 insertions(+) create mode 100644 tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.x86_64-latest.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e9336fd72d..e942dc41ae 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -28312,6 +28312,9 @@ virDomainHostdevDefFormat(virBufferPtr buf, if (mdevsrc->display != VIR_TRISTATE_SWITCH_ABSENT) virBufferAsprintf(buf, " display='%s'", virTristateSwitchTypeToString(mdevsrc->display)); + if (mdevsrc->ramfb != VIR_TRISTATE_SWITCH_ABSENT) + virBufferAsprintf(buf, " ramfb='%s'", + virTristateSwitchTypeToString(mdevsrc->display)); } } diff --git a/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.x86_64-latest.xml b/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.x86_64-latest.xml new file mode 100644 index 0000000000..c134400486 --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.x86_64-latest.xml @@ -0,0 +1,44 @@ +<domain type='qemu'> + <name>QEMUGuest2</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <cpu mode='custom' match='exact' check='none'> + <model fallback='forbid'>qemu64</model> + </cpu> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0' model='piix3-uhci'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <graphics type='vnc' port='-1' autoport='yes'> + <listen type='address'/> + </graphics> + <video> + <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </video> + <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='on' ramfb='on'> + <source> + <address uuid='53764d0e-85a0-42b4-af5c-2046b460b1dc'/> + </source> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </hostdev> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.xml b/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.xml new file mode 100644 index 0000000000..6fdc477ca2 --- /dev/null +++ b/tests/qemuxml2xmloutdata/hostdev-mdev-display-ramfb.xml @@ -0,0 +1,33 @@ +<domain type='qemu'> + <name>QEMUGuest2</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <currentMemory unit='KiB'>219136</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <controller type='usb' index='0'> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <controller type='ide' index='0'> + </controller> + <graphics type='vnc'/> + <hostdev mode='subsystem' type='mdev' model='vfio-pci' display='on' ramfb='on'> + <source> + <address uuid='53764d0e-85a0-42b4-af5c-2046b460b1dc'/> + </source> + </hostdev> + <video> + <model type='qxl' heads='1'/> + </video> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 022f6786b9..e3919821af 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -529,6 +529,7 @@ mymain(void) QEMU_CAPS_VFIO_PCI_DISPLAY, QEMU_CAPS_DEVICE_VFIO_PCI, QEMU_CAPS_VNC); + DO_TEST_CAPS_LATEST("hostdev-mdev-display-ramfb"); DO_TEST("pci-rom", NONE); DO_TEST("pci-rom-disabled", NONE); DO_TEST("pci-rom-disabled-invalid", NONE); -- 2.21.3

2 1

[libvirt PATCH 00/23] remove use of terms 'whitelist'/'blacklist' and 'master'/'slave'
by Daniel P. Berrangé 23 Jun '20

23 Jun '20

The terms "whitelist" / "blacklist" perpetuate the notion that white is good and black is bad[1]. Their usage is trivially eliminated from libvirt with a variety of alternative terms. The replacements are more applicable to the usage context in most cases. The only exceptions are • Libvirt needs to keep "seccomp-blacklist" back compat for running guest capabilities on upgrade • kmod uses "blacklist" for modprobe config file key The terms "master" / "slave" have inescapable historical context that makes them a bad choice for metaphors in software[1]. Again there are a variety of alternatives that can be used, many of which are more applicable to the usage context. Eliminating all usage though is not practical, due to libvirt's need to interface with external systems, where the terminology is part of the formal API. The unfixable exceptions are: • Libvirt uses slave=NNN master=NNN in XML schema for FreeBSD nmdm devs • Linux kernel sysfs uses "slave_$NIC" for bonding NIC members • Linux kernel mount options MS_SLAVE for mount propagation • QEMU uses "slave" in many device/property names • Jenkins website calls the libvirt plugin "libvirt-slave" • VirtualBox uses 'Slave' in many config file attributes Note, this doesn't attempt to remove cases of the word "master" which are used in isolation, only those paired with usage of the word "slave". Remaining usage of "master" needs evaluating, as some of these contexts are none the less implicitly associated with the "master/slave" concept. Nothing in the po/ directory is updated. This will be updated when we refresh translations at time of freeze. There should be no functional change in any of these patches with the exception of the patch tweaking matching for NICs in the interface driver. [1] There are many docs on the web covering this in detail, with one fairly clear description being: https://tools.ietf.org/id/draft-knodel-terminology-00.html Daniel P. Berrangé (23): scripts: remove use of the term 'whitelist' from build helpers rpc: remove use of the term 'whitelist' from RPC code cgroup: remove use of the term 'whitelist' from cgroup code qemu: remove use of the terms 'whitelist' and 'blacklist' from CPU code qemu: remove use of the term 'blacklist' in seccomp capability util: use short form -g arg to scsi_id docs: remove use of the term 'whitelist' from documentation util: rename method to virKModIsProhibited nodedev: remove use of the term 'blacklist' from enumeration code build: remove use of the term 'blacklist' from helper files src: remove use of the term 'whitelist' from remaining code interface: use a constant for the sysfs bond device file prefix interface: remove most use of the term 'slave' from bonding code docs: remove use of the term 'enslaved' wrt tap & bridge devices tools: remove use of the term 'slave' in code dealing with bridges util: remove use of the terms 'master' and 'slave' in PTY code qemu: remove use of the terms 'master' and 'slave' when iterating CPUs lxc: remove use of the terms 'master' and 'slave' in PTY setup docs: update link to the libvirt jenkins plugin docs: remove use of the term 'slave' in Jenkins agent docs conf: remove use of the terms 'master' and 'slave' in mndm config lxc: replace use of term 'slave' filesystem mount setup build: add syntax-check rules for undesirable terms build-aux/syntax-check.mk | 18 ++++- docs/apps.html.in | 6 +- docs/drvqemu.html.in | 12 ++-- docs/firewall.html.in | 6 +- docs/formatdomain.html.in | 4 +- docs/internals/rpc.html.in | 9 +-- docs/kbase/qemu-passthrough-security.rst | 3 +- docs/schemas/interface.rng | 2 +- m4/virt-compile-warnings.m4 | 2 +- scripts/check-aclrules.py | 8 +-- scripts/check-file-access.py | 16 ++--- scripts/mock-noinline.py | 1 - src/bhyve/bhyve_driver.c | 2 +- src/bhyve/bhyve_parse_command.c | 20 +++--- src/conf/domain_conf.c | 24 +++---- src/conf/domain_conf.h | 4 +- src/interface/interface_backend_udev.c | 65 +++++++++---------- src/libvirt.c | 2 +- src/libvirt_private.syms | 2 +- src/lxc/lxc_cgroup.c | 2 +- src/lxc/lxc_controller.c | 20 +++--- src/node_device/node_device_udev.c | 6 +- src/qemu/qemu.conf | 4 +- src/qemu/qemu_capabilities.c | 22 +++---- src/qemu/qemu_capabilities.h | 6 +- src/qemu/qemu_cgroup.c | 2 +- src/qemu/qemu_command.c | 6 +- src/qemu/qemu_conf.c | 4 +- src/qemu/qemu_domain.c | 10 ++- src/qemu/qemu_monitor.c | 46 ++++++------- src/remote/libvirtd.conf.in | 6 +- src/remote/remote_daemon_dispatch.c | 4 +- src/rpc/gendispatch.pl | 2 +- src/rpc/virnetsaslcontext.c | 10 +-- src/rpc/virnetsaslcontext.h | 2 +- src/rpc/virnettlscontext.c | 32 ++++----- src/rpc/virnettlscontext.h | 4 +- src/util/vircgroup.c | 2 +- src/util/virfile.c | 42 ++++++------ src/util/virkmod.c | 24 +++---- src/util/virkmod.h | 2 +- src/util/virnetdevtap.c | 2 +- src/util/virpci.c | 4 +- src/util/virprocess.c | 2 +- src/util/virstoragefile.c | 4 +- src/vbox/vbox_common.c | 4 +- tests/Makefile.am | 4 +- ...hitelist.txt => permitted_file_access.txt} | 6 +- .../caps_2.11.0.s390x.xml | 2 +- .../caps_2.11.0.x86_64.xml | 2 +- .../caps_2.12.0.aarch64.xml | 2 +- .../caps_2.12.0.ppc64.xml | 2 +- .../caps_2.12.0.s390x.xml | 2 +- .../caps_2.12.0.x86_64.xml | 2 +- .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml | 2 +- .../caps_3.0.0.riscv32.xml | 2 +- .../caps_3.0.0.riscv64.xml | 2 +- .../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 2 +- .../caps_3.0.0.x86_64.xml | 2 +- .../qemucapabilitiesdata/caps_3.1.0.ppc64.xml | 2 +- .../caps_3.1.0.x86_64.xml | 2 +- .../caps_4.0.0.aarch64.xml | 2 +- .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 2 +- .../caps_4.0.0.riscv32.xml | 2 +- .../caps_4.0.0.riscv64.xml | 2 +- .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 2 +- .../caps_4.0.0.x86_64.xml | 2 +- .../caps_4.1.0.x86_64.xml | 2 +- .../caps_4.2.0.aarch64.xml | 2 +- .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 2 +- .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 2 +- .../caps_4.2.0.x86_64.xml | 2 +- .../caps_5.0.0.aarch64.xml | 2 +- .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 2 +- .../caps_5.0.0.riscv64.xml | 2 +- .../caps_5.0.0.x86_64.xml | 2 +- .../caps_5.1.0.x86_64.xml | 2 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 2 +- .../blockjob-blockdev-in.xml | 2 +- tests/qemuxml2argvtest.c | 2 +- tests/virconfdata/libvirtd.conf | 6 +- tests/virconfdata/libvirtd.out | 6 +- tools/virsh-interface.c | 16 ++--- 83 files changed, 301 insertions(+), 281 deletions(-) rename tests/{file_access_whitelist.txt => permitted_file_access.txt} (82%) -- 2.24.1

5 67

[libvirt PATCH] qemu: don't continue loading caps if outdated
by Daniel P. Berrangé 22 Jun '20

22 Jun '20

The XML format used for QEMU capabilities is not required to be stable across releases, as we invalidate the cache whenever the libvirt binary changes. We none the less always try to parse te entire XML file before we do any validity checks. Thus if we change the format of any part of the data, or change permitted values for enums, then libvirtd logs will be spammed with errors. These are not in fact errors, but an expected scenario. This change makes the loading code validate the cache timestamp against the libvirtd timestamp immediately. If they don't match then we stop loading the rest of the XML file. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/qemu/qemu_capabilities.c | 48 ++++++++++++++++++++++++++---------- src/util/virfilecache.c | 11 ++++++--- src/util/virfilecache.h | 11 ++++++--- tests/virfilecachetest.c | 3 ++- 4 files changed, 52 insertions(+), 21 deletions(-) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 310be800e2..8694db2463 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -4205,6 +4205,8 @@ virQEMUCapsParseSEVInfo(virQEMUCapsPtr qemuCaps, xmlXPathContextPtr ctxt) * <machine name='pc-1.0' alias='pc' hotplugCpus='yes' maxCpus='4' default='yes' numaMemSupported='yes'/> * ... * </qemuCaps> + * + * Returns 0 on success, 1 if outdated, -1 on error */ int virQEMUCapsLoadCache(virArch hostArch, @@ -4237,6 +4239,30 @@ virQEMUCapsLoadCache(virArch hostArch, goto cleanup; } + if (virXPathLongLong("string(./selfctime)", ctxt, &l) < 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("missing selfctime in QEMU capabilities XML")); + goto cleanup; + } + qemuCaps->libvirtCtime = (time_t)l; + + qemuCaps->libvirtVersion = 0; + if (virXPathULong("string(./selfvers)", ctxt, &lu) == 0) + qemuCaps->libvirtVersion = lu; + + if (qemuCaps->libvirtCtime != virGetSelfLastChanged() || + qemuCaps->libvirtVersion != LIBVIR_VERSION_NUMBER) { + VIR_DEBUG("Outdated capabilities in %s: libvirt changed " + "(%lld vs %lld, %lu vs %lu), stopping load", + qemuCaps->binary, + (long long)qemuCaps->libvirtCtime, + (long long)virGetSelfLastChanged(), + (unsigned long)qemuCaps->libvirtVersion, + (unsigned long)LIBVIR_VERSION_NUMBER); + ret = 1; + goto cleanup; + } + if (!(str = virXPathString("string(./emulator)", ctxt))) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("missing emulator in QEMU capabilities cache")); @@ -4256,17 +4282,6 @@ virQEMUCapsLoadCache(virArch hostArch, } qemuCaps->ctime = (time_t)l; - if (virXPathLongLong("string(./selfctime)", ctxt, &l) < 0) { - virReportError(VIR_ERR_XML_ERROR, "%s", - _("missing selfctime in QEMU capabilities XML")); - goto cleanup; - } - qemuCaps->libvirtCtime = (time_t)l; - - qemuCaps->libvirtVersion = 0; - if (virXPathULong("string(./selfvers)", ctxt, &lu) == 0) - qemuCaps->libvirtVersion = lu; - if ((n = virXPathNodeSet("./flag", ctxt, &nodes)) < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("failed to parse qemu capabilities flags")); @@ -5489,16 +5504,23 @@ virQEMUCapsNewData(const char *binary, static void * virQEMUCapsLoadFile(const char *filename, const char *binary, - void *privData) + void *privData, + bool *outdated) { virQEMUCapsPtr qemuCaps = virQEMUCapsNewBinary(binary); virQEMUCapsCachePrivPtr priv = privData; + int ret; if (!qemuCaps) return NULL; - if (virQEMUCapsLoadCache(priv->hostArch, qemuCaps, filename) < 0) + ret = virQEMUCapsLoadCache(priv->hostArch, qemuCaps, filename); + if (ret < 0) goto error; + if (ret == 1) { + *outdated = true; + goto error; + } return qemuCaps; diff --git a/src/util/virfilecache.c b/src/util/virfilecache.c index aecabf173d..2162917b11 100644 --- a/src/util/virfilecache.c +++ b/src/util/virfilecache.c @@ -130,6 +130,7 @@ virFileCacheLoad(virFileCachePtr cache, g_autofree char *file = NULL; int ret = -1; void *loadData = NULL; + bool outdated = false; *data = NULL; @@ -148,10 +149,12 @@ virFileCacheLoad(virFileCachePtr cache, goto cleanup; } - if (!(loadData = cache->handlers.loadFile(file, name, cache->priv))) { - VIR_WARN("Failed to load cached data from '%s' for '%s': %s", - file, name, virGetLastErrorMessage()); - virResetLastError(); + if (!(loadData = cache->handlers.loadFile(file, name, cache->priv, &outdated))) { + if (!outdated) { + VIR_WARN("Failed to load cached data from '%s' for '%s': %s", + file, name, virGetLastErrorMessage()); + virResetLastError(); + } ret = 0; goto cleanup; } diff --git a/src/util/virfilecache.h b/src/util/virfilecache.h index 006a9717cb..9a7edf07e6 100644 --- a/src/util/virfilecache.h +++ b/src/util/virfilecache.h @@ -62,15 +62,20 @@ typedef void * * @filename: name of a file with cached data * @name: name of the cached data * @priv: private data created together with cache + * @outdated: set to true if data was outdated * - * Loads the cached data from a file @filename. + * Loads the cached data from a file @filename. If + * NULL is returned, then @oudated indicates whether + * this was due to the data being outdated, or an + * error loading the cache. * - * Returns cached data object or NULL on error. + * Returns cached data object or NULL on outdated data or error. */ typedef void * (*virFileCacheLoadFilePtr)(const char *filename, const char *name, - void *priv); + void *priv, + bool *outdated); /** * virFileCacheSaveFilePtr: diff --git a/tests/virfilecachetest.c b/tests/virfilecachetest.c index 6d280b3bec..34e0d0ab2f 100644 --- a/tests/virfilecachetest.c +++ b/tests/virfilecachetest.c @@ -110,7 +110,8 @@ testFileCacheNewData(const char *name G_GNUC_UNUSED, static void * testFileCacheLoadFile(const char *filename, const char *name G_GNUC_UNUSED, - void *priv G_GNUC_UNUSED) + void *priv G_GNUC_UNUSED, + bool *outdated G_GNUC_UNUSED) { testFileCacheObjPtr obj; char *data; -- 2.24.1

2 2

[libvirt PATCH 0/2] util: simplify virKMod code
by Daniel P. Berrangé 22 Jun '20

22 Jun '20

Daniel P. Berrangé (2): src: remove redundant arg to virKModLoad util: remove unused virKModConfig method src/libvirt_private.syms | 1 - src/util/virfile.c | 2 +- src/util/virkmod.c | 27 ++------------- src/util/virkmod.h | 3 +- src/util/virpci.c | 2 +- tests/virkmodtest.c | 71 ++++++---------------------------------- 6 files changed, 16 insertions(+), 90 deletions(-) -- 2.24.1

2 4

[PATCH v3 00/19] audio: deprecate -soundhw
by Gerd Hoffmann 22 Jun '20

22 Jun '20

v2: - use g_assert_not_reached() for stubs. - add deprecation notice. v3: - rebase to latest master. - adapt to armbru's device initialization changes. Gerd Hoffmann (19): stubs: add isa_create_simple stubs: add pci_create_simple audio: add deprecated_register_soundhw audio: deprecate -soundhw ac97 audio: deprecate -soundhw es1370 audio: deprecate -soundhw adlib audio: deprecate -soundhw cs4231a audio: deprecate -soundhw gus audio: deprecate -soundhw sb16 audio: deprecate -soundhw hda audio: deprecate -soundhw pcspk audio: add soundhw deprecation notice pc_basic_device_init: pass PCMachineState pc_basic_device_init: drop has_pit arg pc_basic_device_init: drop no_vmport arg softmmu: initialize spice and audio earlier audio: rework pcspk_init() audio: create pcspk device early, add audiodev alias. audio: set default value for pcspk.iobase property include/hw/audio/pcspk.h | 12 ++---------- include/hw/audio/soundhw.h | 2 ++ include/hw/i386/pc.h | 6 +++--- hw/audio/ac97.c | 9 ++------- hw/audio/adlib.c | 8 +------- hw/audio/cs4231a.c | 8 +------- hw/audio/es1370.c | 9 ++------- hw/audio/gus.c | 8 +------- hw/audio/intel-hda.c | 3 +++ hw/audio/pcspk.c | 26 ++++++++++++++++++++++---- hw/audio/sb16.c | 9 ++------- hw/audio/soundhw.c | 24 +++++++++++++++++++++++- hw/i386/pc.c | 14 ++++++++------ hw/i386/pc_piix.c | 3 +-- hw/i386/pc_q35.c | 3 +-- hw/isa/i82378.c | 2 +- qdev-monitor.c | 2 ++ softmmu/vl.c | 14 ++++++++------ stubs/isa-bus.c | 7 +++++++ stubs/pci-bus.c | 7 +++++++ docs/system/deprecated.rst | 9 +++++++++ stubs/Makefile.objs | 2 ++ 22 files changed, 110 insertions(+), 77 deletions(-) create mode 100644 stubs/isa-bus.c create mode 100644 stubs/pci-bus.c -- 2.18.4

5 28

CFP: KVM Forum 2020
by Paolo Bonzini 22 Jun '20

22 Jun '20

================================================================ KVM Forum 2020: Call For Participation October 28-30, 2020 - Convention Centre Dublin - Dublin, Ireland (All submissions must be received before June 15, 2020 at 23:59 PST) ================================================================= KVM Forum is an annual event that presents a rare opportunity for developers and users to meet, discuss the state of Linux virtualization technology, and plan for the challenges ahead. This highly technical conference unites the developers who drive KVM development and the users who depend on KVM as part of their offerings, or to power their data centers and clouds. Sessions include updates on the state of the KVM virtualization stack, planning for the future, and many opportunities for attendees to collaborate. After more than ten years in the mainline kernel, KVM continues to be a critical part of the FOSS cloud infrastructure. Come join us in continuing to improve the KVM ecosystem. We understand that these are uncertain times and we are continuously monitoring the COVID-19/Novel Coronavirus situation. Our attendees' safety is of the utmost importance. If we feel it is not safe to meet in person, we will turn the event into a virtual experience. We hope to announce this at the same time as the speaker notification. Speakers will still be expected to attend if a physical event goes ahead, but we understand that exceptional circumstances might arise after speakers are accepted and we will do our best to accommodate such circumstances. Based on these factors, we encourage you to submit and reach out to us should you have any questions. The program committee may be contacted as a group via email: kvm-forum-2020-pc(a)redhat.com. SUGGESTED TOPICS ---------------- * Scaling, latency optimizations, performance tuning * Hardening and security * New features * Testing KVM and the Linux Kernel: * Nested virtualization * Resource management (CPU, I/O, memory) and scheduling * VFIO: IOMMU, SR-IOV, virtual GPU, etc. * Networking: Open vSwitch, XDP, etc. * virtio and vhost * Architecture ports and new processor features QEMU: * Management interfaces: QOM and QMP * New devices, new boards, new architectures * New storage features * High availability, live migration and fault tolerance * Emulation and TCG * Firmware: ACPI, UEFI, coreboot, U-Boot, etc. Management & Infrastructure * Managing KVM: Libvirt, OpenStack, oVirt, KubeVirt, etc. * Storage: Ceph, Gluster, SPDK, etc. * Network Function Virtualization: DPDK, OPNFV, OVN, etc. * Provisioning SUBMITTING YOUR PROPOSAL ------------------------ Abstracts due: June 15, 2020 Please submit a short abstract (~150 words) describing your presentation proposal. Slots vary in length up to 45 minutes. Submit your proposal here: https://events.linuxfoundation.org/kvm-forum/program/cfp/ Please only use the categories "presentation" and "panel discussion" You will receive a notification whether or not your presentation proposal was accepted by August 17, 2020. Speakers will receive a complimentary pass for the event. In case your submission has multiple presenters, only the primary speaker for a proposal will receive a complimentary event pass. For panel discussions, all panelists will receive a complimentary event pass. TECHNICAL TALKS A good technical talk should not just report on what has happened over the last year; it should present a concrete problem and how it impacts the user and/or developer community. Whenever applicable, focus on work that needs to be done, difficulties that haven't yet been solved, and on decisions that other developers should be aware of. Summarizing recent developments is okay but it should not be more than a small portion of the overall talk. END-USER TALKS One of the big challenges as developers is to know what, where and how people actually use our software. We will reserve a few slots for end users talking about their deployment challenges and achievements. If you are using KVM in production you are encouraged submit a speaking proposal. Simply mark it as an end-user talk. As an end user, this is a unique opportunity to get your input to developers. PANEL DISCUSSIONS If you are proposing a panel discussion, please make sure that you list all of your potential panelists in your the abstract. We will request full biographies if a panel is accepted. HANDS-ON / BOF SESSIONS We will reserve some time for people to get together and discuss strategic decisions as well as other topics that are best solved within smaller groups. These sessions will be announced during the event. If you are interested in organizing such a session, please add it to the list at http://www.linux-kvm.org/page/KVM_Forum_2020_BOF Let people you think who might be interested know about your BOF, and encourage them to add their names to the wiki page as well. Please add your ideas to the list before KVM Forum starts. HOTEL / TRAVEL -------------- This year's event will take place at the Conference Center Dublin. For information on hotels close to the conference, please visit https://events.linuxfoundation.org/kvm-forum/attend/venue-travel/. Information on conference hotel blocks will be available later in Spring 2020. DATES TO REMEMBER ----------------- * CFP Opens: Monday, April 27, 2020 * CFP Closes: Monday, June 15 at 11:59 PM PST * CFP Notifications: Monday, August 17 * Schedule Announcement: Thursday, August 20 * Slide Due Date: Monday, October 26 * Event Dates: Wednesday, October 28 – Friday, October 30 Thank you for your interest in KVM. We're looking forward to your submissions and, if the conditions will permit it, to seeing you at the KVM Forum 2020 in October! -your KVM Forum 2020 Program Committee Please contact us with any questions or comments at kvm-forum-2020-pc(a)redhat.com

2 3

[PATCH 00/32] qemu: backup: Rewrite/simplify internals and document semantics
by Peter Krempa 22 Jun '20

22 Jun '20

Simplify how we approach bitmap handling to increase robustness and understanding of the code. Additionally document intricacies of how we approach handling of bitmaps so that in cases when users decide to do snapshots outside of libvirt they can re-create the bitmaps so that they will still work with backups (requires new qemu-img). I'll also follow up documenting implicationsof blockjobs on bitmaps later. To fetch this including a commit which allows testing: git fetch https://gitlab.com/pipo.sk/libvirt.git checkpoint-rewrite-and-document This doesn't require any special qemu patches. Note that this shares many patches from my previous posting which tried to make use of 'block-dirty-bitmap-populate', but keeps explicit bitmap tracking for now. We'll be able to revisit use of the blockjob if the qemu work will continue somehow. Peter Krempa (32): qemu: backup: Split up code traversing checkpoint list looking for bitmaps qemu: backup: Fix backup of disk skipped in an intermediate checkpoint conf: backup: Store incremental backup checkpoint name per-disk qemu: backup: Move fetching of checkpoint list for incremental backup qemublocktest: Add 'empty' test case for bitmaps qemublocktest: Add 'empty' case for incremental backup test qemublocktest: Add 'empty' case for checkpoint deletion qemublocktest: Add 'empty' case for blockcopy bitmap handling test qemublocktest: Add 'empty' case for checkpoint bitmap handling qemublocktest: Disable testcases for all bitmap handling qemublocktest: Delete 'synthetic' bitmap test cases qemublocktest: Extract printing of nodename list qemu: checkpoint: Don't chain bitmaps for checkpoints qemublocktest: Replace 'basic' bitmap detection test case data qemublocktest: Replace 'snapshots' bitmap detection test case data qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases qemublocktest: Add new 'synthetic' bitmap detection and validation test case qemu: checkpoint: Don't merge checkpoints during deletion qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE qemublocktest: Re-introduce testing of checkpoint deletion qemu: block: Add universal helper for merging dirty bitmaps for all scenarios qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics qemublocktest: Add 'basic' tests for backup bitmap handling qemublocktest: Add 'snapshots' tests for backup bitmap handling qemu: Rewrite bitmap handling for block commit qemublocktest: Add 'basic' tests for commit bitmap handling qemublocktest: Add 'snapshots' tests for block commit bitmap handling qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data qemu: Rewrite bitmap handling for block copy qemublocktest: Add test cases for handling bitmaps during block-copy kbase: Add document outlining internals of incremental backup in qemu docs/kbase.html.in | 3 + docs/kbase/incrementalbackupinternals.rst | 210 +++ src/conf/backup_conf.c | 8 + src/conf/backup_conf.h | 1 + src/qemu/qemu_backup.c | 199 +-- src/qemu/qemu_backup.h | 12 +- src/qemu/qemu_block.c | 500 +++--- src/qemu/qemu_block.h | 21 +- src/qemu/qemu_blockjob.c | 109 +- src/qemu/qemu_blockjob.h | 3 - src/qemu/qemu_checkpoint.c | 174 +-- src/qemu/qemu_checkpoint.h | 1 - src/qemu/qemu_domain.c | 22 - src/qemu/qemu_driver.c | 70 +- tests/qemublocktest.c | 254 ++- .../backupmerge/basic-deep-out.json | 35 +- .../backupmerge/basic-flat-out.json | 23 +- .../backupmerge/basic-intermediate-out.json | 23 +- .../backupmerge/empty-out.json | 3 + .../backupmerge/snapshot-deep-out.json | 38 - .../backupmerge/snapshot-flat-out.json | 6 - .../snapshot-intermediate-out.json | 14 - .../backupmerge/snapshots-deep-out.json | 41 + .../backupmerge/snapshots-flat-out.json | 25 + .../snapshots-intermediate-out.json | 29 + tests/qemublocktestdata/bitmap/basic.json | 229 +-- tests/qemublocktestdata/bitmap/basic.out | 8 +- tests/qemublocktestdata/bitmap/empty.json | 70 + tests/qemublocktestdata/bitmap/empty.out | 1 + .../bitmap/snapshots-synthetic-broken.json | 837 ---------- .../bitmap/snapshots-synthetic-broken.out | 14 - .../snapshots-synthetic-checkpoint.json | 827 ---------- .../bitmap/snapshots-synthetic-checkpoint.out | 13 - tests/qemublocktestdata/bitmap/snapshots.json | 1382 +++++++---------- tests/qemublocktestdata/bitmap/snapshots.out | 13 +- tests/qemublocktestdata/bitmap/synthetic.json | 606 ++++++-- tests/qemublocktestdata/bitmap/synthetic.out | 19 +- .../bitmapblockcommit/basic-1-2 | 54 +- .../bitmapblockcommit/basic-1-3 | 54 +- .../bitmapblockcommit/basic-2-3 | 1 - .../qemublocktestdata/bitmapblockcommit/empty | 1 + .../bitmapblockcommit/snapshots-1-2 | 76 +- .../bitmapblockcommit/snapshots-1-3 | 77 +- .../bitmapblockcommit/snapshots-1-4 | 99 +- .../bitmapblockcommit/snapshots-1-5 | 103 +- .../bitmapblockcommit/snapshots-2-3 | 48 +- .../bitmapblockcommit/snapshots-2-4 | 58 +- .../bitmapblockcommit/snapshots-2-5 | 62 +- .../bitmapblockcommit/snapshots-3-4 | 34 +- .../bitmapblockcommit/snapshots-3-5 | 42 +- .../bitmapblockcommit/snapshots-4-5 | 17 - .../snapshots-synthetic-broken-1-2 | 57 - .../snapshots-synthetic-broken-1-3 | 112 -- .../snapshots-synthetic-broken-1-4 | 119 -- .../snapshots-synthetic-broken-1-5 | 119 -- .../snapshots-synthetic-broken-2-3 | 89 -- .../snapshots-synthetic-broken-2-4 | 96 -- .../snapshots-synthetic-broken-2-5 | 96 -- .../snapshots-synthetic-broken-3-4 | 27 - .../snapshots-synthetic-broken-3-5 | 27 - .../snapshots-synthetic-broken-4-5 | 20 - .../bitmapblockcopy/basic-deep-out.json | 53 +- .../bitmapblockcopy/basic-shallow-out.json | 53 +- .../bitmapblockcopy/empty-deep-out.json | 0 .../bitmapblockcopy/empty-shallow-out.json | 0 .../bitmapblockcopy/snapshots-deep-out.json | 95 +- .../snapshots-shallow-out.json | 98 +- .../checkpointdelete/basic-current-out.json | 20 - .../basic-intermediate1-out.json | 13 - .../basic-intermediate2-out.json | 13 - .../basic-intermediate3-out.json | 13 - .../checkpointdelete/empty-out.json | 1 + .../snapshots-current-out.json | 20 - .../snapshots-intermediate1-out.json | 20 +- .../snapshots-intermediate2-out.json | 42 +- .../snapshots-intermediate3-out.json | 43 - .../snapshots-noparent-out.json | 15 + ...hots-synthetic-checkpoint-current-out.json | 29 - ...ynthetic-checkpoint-intermediate1-out.json | 31 - ...ynthetic-checkpoint-intermediate2-out.json | 34 - ...ynthetic-checkpoint-intermediate3-out.json | 61 - ...ots-synthetic-checkpoint-noparent-out.json | 27 - .../synthetic-current-out.json | 9 + .../synthetic-intermediate1-out.json | 11 + .../synthetic-intermediate2-out.json | 11 + .../synthetic-intermediate3-out.json | 19 + .../synthetic-noparent-out.json | 11 + .../blockjob-blockdev-in.xml | 4 - 88 files changed, 2927 insertions(+), 5150 deletions(-) create mode 100644 docs/kbase/incrementalbackupinternals.rst create mode 100644 tests/qemublocktestdata/backupmerge/empty-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-deep-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-flat-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-deep-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-flat-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json create mode 100644 tests/qemublocktestdata/bitmap/empty.json create mode 100644 tests/qemublocktestdata/bitmap/empty.out delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out create mode 100644 tests/qemublocktestdata/bitmapblockcommit/empty delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5 create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/empty-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json -- 2.26.2

3 71

[libvirt PATCH] ci: Use fedora image for code style checking
by Jonathon Jongsma 22 Jun '20

22 Jun '20

Centos does not have the cppi package, so some code style checks are skipped. Switch to a fedora image to do the code style checks. Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index bfb66a652d..b8d0645e99 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -496,7 +496,7 @@ website: codestyle: stage: builds - image: $CI_REGISTRY_IMAGE/ci-centos-8:latest + image: $CI_REGISTRY_IMAGE/ci-fedora-32:latest before_script: - *script_variables script: -- 2.21.3

3 5

[RFC PATCH 00/41] qemu: Rewrite checkpoint code for 'block-dirty-bitmap-populate'
by Peter Krempa 22 Jun '20

22 Jun '20

Use 'block-dirty-bitmap-populate' and change how we create bitmaps corresponding to checkpoints to simplify the code and also properly integrate with backing chain images created outside of libvirt. This patchset changes how we approach checkpoints by keeping one bitmap per checkpoint and disk and not propagating the bitmaps into overlays on snapshots. This massively simplifies the code handling all the operations during blockjobs and backups. While the change isn't compatible with checkpoints created previously, we didn't yet enable the support for checkpoints/backups. Note that 'block-dirty-bitmap-populate' is _not_ in finished state in qemu yet, so I'm posting this as RFC and as reference for qemu developers of the usefulnes of it. The changes can be fetched by: git fetch https://gitlab.com/pipo.sk/libvirt.git checkpoint-bitmap-populate Note that the above branch also contains a commit enabling incremental backup for simpler testing. I've pushed the appropriate qemu patches for convenience here: git fetch https://gitlab.com/pipo.sk/qemu.git block-dirty-bitmap-populate Peter Krempa (41): virErrorPreserveLast: Return the saved error object in addition to storing it util: error: Introduce VIR_ERROR_AUTOPRESERVE_LAST macro qemuBackupDiskDataCleanup: Use VIR_ERROR_AUTO_PRESERVE_LAST qemu: backup: Split up code traversing checkpoint list looking for bitmaps qemu: backup: Fix backup of disk skipped in an intermediate checkpoint conf: backup: Store incremental backup checkpoint name per-disk qemu: backup: Move fetching of checkpoint list for incremental backup qemublocktest: Add 'empty' test case for bitmaps qemublocktest: Add 'empty' case for incremental backup test qemublocktest: Add 'empty' case for checkpoint deletion qemublocktest: Add 'empty' case for blockcopy bitmap handling test qemublocktest: Add 'empty' case for checkpoint bitmap handling qemublocktest: Disable testcases for all bitmap handling qemublocktest: Delete 'synthetic' bitmap test cases qemublocktest: Extract printing of nodename list qemu: checkpoint: Don't chain bitmaps for checkpoints qemublocktest: Replace 'basic' bitmap detection test case data qemu: snapshot: Don't propagate bitmaps to upper layers qemublocktest: Replace 'snapshots' bitmap detection test case data qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases qemublocktest: Add new 'synthetic' bitmap detection and validation test case qemu: checkpoint: Don't merge checkpoints during deletion qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE qemublocktest: Re-introduce testing of checkpoint deletion qemucapabilitiestest: Bump qemu-5.1 caps on x86_64 for 'dirty-bitmap-populate' qemu: monitor: Add support for 'block-dirty-bitmap-populate' transaction member qemuDomainStorageSourcePrivate: Add per-source private blockjob qemu: blockjob: Introduce 'populate' blockjob qemu: domain: Introduce helper for always fetching virStorageSource private data qemu: block: Add helper to add temporary block bitmaps from allocation maps qemu: block: Add universal helper for merging dirty bitmaps for all scenarios qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics qemublocktest: Add 'basic' tests for backup bitmap handling qemublocktest: Add 'snapshots' tests for backup bitmap handling qemu: Rewrite bitmap handling for block commit qemublocktest: Add 'basic' tests for commit bitmap handling qemublocktest: Add 'snapshots' tests for block commit bitmap handling qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data qemu: Rewrite bitmap handling for block copy qemublocktest: Add test cases for handling bitmaps during block-copy src/conf/backup_conf.c | 8 + src/conf/backup_conf.h | 1 + src/qemu/qemu_backup.c | 221 +- src/qemu/qemu_backup.h | 13 +- src/qemu/qemu_block.c | 674 ++-- src/qemu/qemu_block.h | 35 +- src/qemu/qemu_blockjob.c | 182 +- src/qemu/qemu_blockjob.h | 16 +- src/qemu/qemu_checkpoint.c | 174 +- src/qemu/qemu_checkpoint.h | 1 - src/qemu/qemu_domain.c | 46 +- src/qemu/qemu_domain.h | 6 + src/qemu/qemu_driver.c | 112 +- src/qemu/qemu_monitor.c | 11 + src/qemu/qemu_monitor.h | 7 + src/qemu/qemu_monitor_json.c | 18 + src/qemu/qemu_monitor_json.h | 6 + src/util/virerror.c | 18 +- src/util/virerror.h | 12 +- tests/qemublocktest.c | 267 +- .../backupmerge/basic-deep-out.json | 35 +- .../backupmerge/basic-flat-out.json | 23 +- .../backupmerge/basic-intermediate-out.json | 23 +- .../backupmerge/empty-out.json | 3 + .../backupmerge/snapshot-deep-out.json | 38 - .../backupmerge/snapshot-flat-out.json | 6 - .../snapshot-intermediate-out.json | 14 - .../backupmerge/snapshots-deep-out.json | 46 + .../backupmerge/snapshots-flat-out.json | 25 + .../snapshots-intermediate-out.json | 31 + tests/qemublocktestdata/bitmap/basic.json | 229 +- tests/qemublocktestdata/bitmap/basic.out | 8 +- tests/qemublocktestdata/bitmap/empty.json | 70 + tests/qemublocktestdata/bitmap/empty.out | 1 + .../bitmap/snapshots-synthetic-broken.json | 837 ----- .../bitmap/snapshots-synthetic-broken.out | 14 - .../snapshots-synthetic-checkpoint.json | 827 ----- .../bitmap/snapshots-synthetic-checkpoint.out | 13 - tests/qemublocktestdata/bitmap/snapshots.json | 1254 +++----- tests/qemublocktestdata/bitmap/snapshots.out | 6 +- tests/qemublocktestdata/bitmap/synthetic.json | 606 +++- tests/qemublocktestdata/bitmap/synthetic.out | 19 +- .../bitmapblockcommit/basic-1-2 | 66 +- .../bitmapblockcommit/basic-1-3 | 66 +- .../bitmapblockcommit/basic-2-3 | 1 - .../qemublocktestdata/bitmapblockcommit/empty | 1 + .../bitmapblockcommit/snapshots-1-2 | 24 +- .../bitmapblockcommit/snapshots-1-3 | 42 +- .../bitmapblockcommit/snapshots-1-4 | 97 +- .../bitmapblockcommit/snapshots-1-5 | 99 +- .../bitmapblockcommit/snapshots-2-3 | 23 - .../bitmapblockcommit/snapshots-2-4 | 55 +- .../bitmapblockcommit/snapshots-2-5 | 59 +- .../bitmapblockcommit/snapshots-3-4 | 25 +- .../bitmapblockcommit/snapshots-3-5 | 29 +- .../bitmapblockcommit/snapshots-4-5 | 32 - .../snapshots-synthetic-broken-1-2 | 57 - .../snapshots-synthetic-broken-1-3 | 112 - .../snapshots-synthetic-broken-1-4 | 119 - .../snapshots-synthetic-broken-1-5 | 119 - .../snapshots-synthetic-broken-2-3 | 89 - .../snapshots-synthetic-broken-2-4 | 96 - .../snapshots-synthetic-broken-2-5 | 96 - .../snapshots-synthetic-broken-3-4 | 27 - .../snapshots-synthetic-broken-3-5 | 27 - .../snapshots-synthetic-broken-4-5 | 20 - .../bitmapblockcopy/basic-deep-out.json | 65 +- .../bitmapblockcopy/basic-shallow-out.json | 65 +- .../bitmapblockcopy/empty-deep-out.json | 0 .../bitmapblockcopy/empty-shallow-out.json | 0 .../bitmapblockcopy/snapshots-deep-out.json | 104 +- .../snapshots-shallow-out.json | 24 +- .../checkpointdelete/basic-current-out.json | 20 - .../basic-intermediate1-out.json | 13 - .../basic-intermediate2-out.json | 13 - .../basic-intermediate3-out.json | 13 - .../checkpointdelete/empty-out.json | 1 + .../snapshots-current-out.json | 20 - .../snapshots-intermediate1-out.json | 13 - .../snapshots-intermediate2-out.json | 51 - .../snapshots-intermediate3-out.json | 50 - .../snapshots-noparent-out.json | 16 - ...hots-synthetic-checkpoint-current-out.json | 29 - ...ynthetic-checkpoint-intermediate1-out.json | 31 - ...ynthetic-checkpoint-intermediate2-out.json | 34 - ...ynthetic-checkpoint-intermediate3-out.json | 61 - ...ots-synthetic-checkpoint-noparent-out.json | 27 - .../synthetic-current-out.json | 9 + .../synthetic-intermediate1-out.json | 11 + .../synthetic-intermediate2-out.json | 11 + .../synthetic-intermediate3-out.json | 19 + .../synthetic-noparent-out.json | 11 + .../caps_5.1.0.x86_64.replies | 2737 +++++++++-------- .../caps_5.1.0.x86_64.xml | 2 +- tests/qemumonitorjsontest.c | 3 +- .../blockjob-blockdev-in.xml | 4 - 96 files changed, 4077 insertions(+), 6717 deletions(-) create mode 100644 tests/qemublocktestdata/backupmerge/empty-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-deep-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-flat-out.json delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-deep-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-flat-out.json create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json create mode 100644 tests/qemublocktestdata/bitmap/empty.json create mode 100644 tests/qemublocktestdata/bitmap/empty.out delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out create mode 100644 tests/qemublocktestdata/bitmapblockcommit/empty delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5 delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5 create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/empty-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json -- 2.26.2

2 46

[PATCH] virsh: Fix typo of metadata
by Han Han 22 Jun '20

22 Jun '20

Signed-off-by: Han Han <hhan(a)redhat.com> --- tools/virsh-domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 5222949566..b7dea5e8f7 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -13559,7 +13559,7 @@ virshEventMetadataChangePrint(virConnectPtr conn G_GNUC_UNUSED, { virBuffer buf = VIR_BUFFER_INITIALIZER; - virBufferAsprintf(&buf, _("event 'metdata-change' for domain %s: %s %s\n"), + virBufferAsprintf(&buf, _("event 'metadata-change' for domain %s: %s %s\n"), virDomainGetName(dom), UNKNOWNSTR(virshEventMetadataChangeTypeTypeToString(type)), NULLSTR(nsuri)); -- 2.24.0.rc1

2 1

[PATCH v5 0/4] introduction of migration_version attribute for VFIO live migration
by Yan Zhao 22 Jun '20

22 Jun '20

This patchset introduces a migration_version attribute under sysfs of VFIO Mediated devices. This migration_version attribute is used to check migration compatibility between two mdev devices. Currently, it has two locations: (1) under mdev_type node, which can be used even before device creation, but only for mdev devices of the same mdev type. (2) under mdev device node, which can only be used after the mdev devices are created, but the src and target mdev devices are not necessarily be of the same mdev type (The second location is newly added in v5, in order to keep consistent with the migration_version node for migratable pass-though devices) Patch 1 defines migration_version attribute for the first location in Documentation/vfio-mediated-device.txt Patch 2 uses GVT as an example for patch 1 to show how to expose migration_version attribute and check migration compatibility in vendor driver. Patch 3 defines migration_version attribute for the second location in Documentation/vfio-mediated-device.txt Patch 4 uses GVT as an example for patch 3 to show how to expose migration_version attribute and check migration compatibility in vendor driver. (The previous "Reviewed-by" and "Acked-by" for patch 1 and patch 2 are kept in v5, as there are only small changes to commit messages of the two patches.) v5: added patch 2 and 4 for mdev device part of migration_version attribute. v4: 1. fixed indentation/spell errors, reworded several error messages 2. added a missing memory free for error handling in patch 2 v3: 1. renamed version to migration_version 2. let errno to be freely defined by vendor driver 3. let checking mdev_type be prerequisite of migration compatibility check 4. reworded most part of patch 1 5. print detailed error log in patch 2 and generate migration_version string at init time v2: 1. renamed patched 1 2. made definition of device version string completely private to vendor driver 3. reverted changes to sample mdev drivers 4. described intent and usage of version attribute more clearly. Yan Zhao (4): vfio/mdev: add migration_version attribute for mdev (under mdev_type node) drm/i915/gvt: export migration_version to mdev sysfs (under mdev_type node) vfio/mdev: add migration_version attribute for mdev (under mdev device node) drm/i915/gvt: export migration_version to mdev sysfs (under mdev device node) .../driver-api/vfio-mediated-device.rst | 183 ++++++++++++++++++ drivers/gpu/drm/i915/gvt/Makefile | 2 +- drivers/gpu/drm/i915/gvt/gvt.c | 39 ++++ drivers/gpu/drm/i915/gvt/gvt.h | 7 + drivers/gpu/drm/i915/gvt/kvmgt.c | 55 ++++++ drivers/gpu/drm/i915/gvt/migration_version.c | 170 ++++++++++++++++ drivers/gpu/drm/i915/gvt/vgpu.c | 13 +- 7 files changed, 466 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/drm/i915/gvt/migration_version.c -- 2.17.1

7 39

[PATCH 0/1] configure: prefer python's sphinx module
by John Snow 19 Jun '20

19 Jun '20

Hi, this is a followup to my patch which attempted to prefer 'sphinx-build' to 'sphinx-build-3' which intended to prefer a VENV installation of sphinx over Fedora's system installation. That got confusing, though, so here's something that's simpler and better: use python to invoke sphinx so we don't have to worry about python/sphinx version mismatches. This should essentially always DTRT. In cases where a specific sphinx is preferred, you would use a virtual environment with sphinx installed and configure QEMU with e.g. --python /path/to/my/venv/python35 John Snow (1): configure: prefer python's sphinx module docs/system/deprecated.rst | 19 +++++++++++++++++++ configure | 20 +++++++++----------- 2 files changed, 28 insertions(+), 11 deletions(-) -- 2.21.3

3 6

[libvirt-sandbox PATCH] gitlab: introduce CI jobs testing git master & distro libvirt
by Daniel P. Berrangé 19 Jun '20

19 Jun '20

The sandbox build needs to validate one axis - A variety of libvirt versions We test a variety of libvirt versions by running a build against the distro provided libvirt packages. All that is then missing is a build against the latest libvirt git master, which only needs to be run on a single distro, for which Ubuntu 20.04 is picked as a stable long life base. For mingw, we only test against libvirt git master, using Fedora rawhide. It is expected that any issues against older libvirt will have already been caught by the native builds. Several distros are not included because they don't provide the package pre-requisites. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 163 ++++++++++++++++++ ci/containers/README.rst | 14 ++ ci/containers/libvirt-debian-10.Dockerfile | 64 +++++++ ci/containers/libvirt-debian-9.Dockerfile | 67 +++++++ ci/containers/libvirt-debian-sid.Dockerfile | 64 +++++++ ci/containers/libvirt-fedora-31.Dockerfile | 64 +++++++ ci/containers/libvirt-fedora-32.Dockerfile | 64 +++++++ .../libvirt-fedora-rawhide.Dockerfile | 65 +++++++ ci/containers/libvirt-ubuntu-1804.Dockerfile | 67 +++++++ ci/containers/libvirt-ubuntu-2004.Dockerfile | 69 ++++++++ ci/containers/refresh | 27 +++ 11 files changed, 728 insertions(+) create mode 100644 ci/containers/README.rst create mode 100644 ci/containers/libvirt-debian-10.Dockerfile create mode 100644 ci/containers/libvirt-debian-9.Dockerfile create mode 100644 ci/containers/libvirt-debian-sid.Dockerfile create mode 100644 ci/containers/libvirt-fedora-31.Dockerfile create mode 100644 ci/containers/libvirt-fedora-32.Dockerfile create mode 100644 ci/containers/libvirt-fedora-rawhide.Dockerfile create mode 100644 ci/containers/libvirt-ubuntu-1804.Dockerfile create mode 100644 ci/containers/libvirt-ubuntu-2004.Dockerfile create mode 100755 ci/containers/refresh diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 50dae92..c12a262 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,87 @@ stages: - prebuild + - containers + - builds + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt-sandbox/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/libvirt-$NAME.Dockerfile" ci + - docker push "$TAG" + after_script: + - docker logout + +.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export SCRATCH_DIR="/tmp/scratch" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_DIR="$PWD/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig" + export XDG_DATA_DIRS="$VROOT/share:/usr/share" + export GI_TYPELIB_PATH="$VROOT/lib/girepository-1.0" + +.git_native_build_job_template: &git_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + - export LD_LIBRARY_PATH="$VROOT/lib" + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - git clone --depth 1 https://gitlab.com/libvirt/libvirt-glib.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh --prefix="$VROOT" --without-libvirtd + - $MAKE install + - cd ../.. + - mkdir libvirt-glib/build + - cd libvirt-glib/build + - ../autogen.sh --prefix="$VROOT" --without-libvirtd + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" --enable-gtk-doc + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild --nodeps -ta libvirt-sandbox*.tar.gz ; fi + +.dist_native_build_job_template: &dist_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" --enable-gtk-doc + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild -ta libvirt-sandbox*.tar.gz ; fi + # Check that all commits are signed-off for the DCO. # Skip on "libvirt" namespace, since we only need to run @@ -14,3 +95,85 @@ check-dco: except: variables: - $CI_PROJECT_NAMESPACE == 'libvirt' + +x64-debian-9-container: + <<: *container_job_definition + variables: + NAME: debian-9 + +x64-debian-10-container: + <<: *container_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-container: + <<: *container_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-container: + <<: *container_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-container: + <<: *container_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide + +x64-ubuntu-1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu-2004 + + + +x64-debian-9-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-9 + +x64-debian-10-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-rawhide + +x64-ubuntu-1804-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-git-build: + <<: *git_native_build_job_definition + variables: + NAME: ubuntu-2004 diff --git a/ci/containers/README.rst b/ci/containers/README.rst new file mode 100644 index 0000000..530897e --- /dev/null +++ b/ci/containers/README.rst @@ -0,0 +1,14 @@ +CI job assets +============= + +This directory contains assets used in the automated CI jobs, most +notably the Dockerfiles used to build container images in which the +CI jobs then run. + +The ``refresh`` script is used to re-create the Dockerfiles using the +``lcitool`` command that is provided by repo +https://gitlab.com/libvirt/libvirt-ci + +The containers are built during the CI process and cached in the GitLab +container registry of the project doing the build. The cached containers +can be deleted at any time and will be correctly rebuilt. diff --git a/ci/containers/libvirt-debian-10.Dockerfile b/ci/containers/libvirt-debian-10.Dockerfile new file mode 100644 index 0000000..51f79e2 --- /dev/null +++ b/ci/containers/libvirt-debian-10.Dockerfile @@ -0,0 +1,64 @@ +FROM debian:10 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc-dev-bin \ + libc6-dev \ + libcap-ng-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + liblzma-dev \ + libselinux1-dev \ + libtirpc-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libvirt-glib-1.0-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + vim \ + zlib1g-dev && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-debian-9.Dockerfile b/ci/containers/libvirt-debian-9.Dockerfile new file mode 100644 index 0000000..cf94a67 --- /dev/null +++ b/ci/containers/libvirt-debian-9.Dockerfile @@ -0,0 +1,67 @@ +FROM debian:9 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc-dev-bin \ + libc6-dev \ + libcap-ng-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + liblzma-dev \ + libselinux1-dev \ + libtirpc-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libvirt-glib-1.0-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + vim \ + zlib1g-dev && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-debian-sid.Dockerfile b/ci/containers/libvirt-debian-sid.Dockerfile new file mode 100644 index 0000000..6a34306 --- /dev/null +++ b/ci/containers/libvirt-debian-sid.Dockerfile @@ -0,0 +1,64 @@ +FROM debian:sid + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc-dev-bin \ + libc6-dev \ + libcap-ng-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + liblzma-dev \ + libselinux1-dev \ + libtirpc-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libvirt-glib-1.0-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + vim \ + zlib1g-dev && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-31.Dockerfile b/ci/containers/libvirt-fedora-31.Dockerfile new file mode 100644 index 0000000..6e0da18 --- /dev/null +++ b/ci/containers/libvirt-fedora-31.Dockerfile @@ -0,0 +1,64 @@ +FROM fedora:31 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glibc-static \ + gobject-introspection-devel \ + gtk-doc \ + libcap-ng-devel \ + libselinux-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libvirt-gobject-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vim \ + xz-devel \ + xz-static \ + zlib-devel \ + zlib-static && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-32.Dockerfile b/ci/containers/libvirt-fedora-32.Dockerfile new file mode 100644 index 0000000..5d86af2 --- /dev/null +++ b/ci/containers/libvirt-fedora-32.Dockerfile @@ -0,0 +1,64 @@ +FROM fedora:32 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glibc-static \ + gobject-introspection-devel \ + gtk-doc \ + libcap-ng-devel \ + libselinux-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libvirt-gobject-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vim \ + xz-devel \ + xz-static \ + zlib-devel \ + zlib-static && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-rawhide.Dockerfile b/ci/containers/libvirt-fedora-rawhide.Dockerfile new file mode 100644 index 0000000..8a299fd --- /dev/null +++ b/ci/containers/libvirt-fedora-rawhide.Dockerfile @@ -0,0 +1,65 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glibc-static \ + gobject-introspection-devel \ + gtk-doc \ + libcap-ng-devel \ + libselinux-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libvirt-gobject-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vim \ + xz-devel \ + xz-static \ + zlib-devel \ + zlib-static && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-ubuntu-1804.Dockerfile b/ci/containers/libvirt-ubuntu-1804.Dockerfile new file mode 100644 index 0000000..bea1fb4 --- /dev/null +++ b/ci/containers/libvirt-ubuntu-1804.Dockerfile @@ -0,0 +1,67 @@ +FROM ubuntu:18.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc-dev-bin \ + libc6-dev \ + libcap-ng-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + liblzma-dev \ + libselinux1-dev \ + libtirpc-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libvirt-glib-1.0-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + vim \ + zlib1g-dev && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-ubuntu-2004.Dockerfile b/ci/containers/libvirt-ubuntu-2004.Dockerfile new file mode 100644 index 0000000..0c2c4d6 --- /dev/null +++ b/ci/containers/libvirt-ubuntu-2004.Dockerfile @@ -0,0 +1,69 @@ +FROM ubuntu:20.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc-dev-bin \ + libc6-dev \ + libcap-ng-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libgnutls28-dev \ + liblzma-dev \ + libnl-3-dev \ + libnl-route-3-dev \ + libselinux1-dev \ + libtirpc-dev \ + libtool \ + libtool-bin \ + libxml2-dev \ + libxml2-utils \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-docutils \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim \ + xsltproc \ + zlib1g-dev && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/refresh b/ci/containers/refresh new file mode 100755 index 0000000..ce65700 --- /dev/null +++ b/ci/containers/refresh @@ -0,0 +1,27 @@ +#!/bin/sh + +if test -z "$1" +then + echo "syntax: $0 PATH-TO-LCITOOL" + exit 1 +fi + +LCITOOL=$1 + +if ! test -x "$LCITOOL" +then + echo "$LCITOOL is not executable" + exit 1 +fi + +HOSTS=$($LCITOOL hosts | grep -v -E '(freebsd|centos|opensuse)') + +for host in $HOSTS +do + if test "$host" = "libvirt-ubuntu-2004" + then + $LCITOOL dockerfile $host libvirt+minimal,libvirt-glib,libvirt-sandbox > $host.Dockerfile + else + $LCITOOL dockerfile $host libvirt+dist,libvirt-glib+dist,libvirt-sandbox > $host.Dockerfile + fi +done -- 2.26.2

2 3

[libvirt PATCH v4 00/12] Add ability to create mediated devices in libvirt
by Jonathon Jongsma 19 Jun '20

19 Jun '20

This is the first portion of an effort to support persistent mediated devices with libvirt. This first series simply enables creating and destroying non-persistent mediated devices via the virNodeDeviceCreateXML() and virNodeDeviceDestroy() functions. The 'mdevctl' utility[1] provides the backend implementation. Hopefully these are the last changes and version can simply be pushed upstream. Changes in v4: - coding style / spacing fixes - remove 'persist' arg from start command - fixed distcheck failure by including test data dir in EXTRA_DIST - Add an item to NEWS.rst [1] https://github.com/mdevctl/mdevctl Jonathon Jongsma (12): nodedev: make iommuGroup optional for mdevs nodedev: factor out nodeDeviceHasCapability() nodedev: add support for mdev attributes nodedev: refactor nodeDeviceFindNewDevice() nodedev: store mdev UUID in mdev caps nodedev: add mdev support to virNodeDeviceCreateXML() nodedev: Build a non-loadable driver lib nodedev: Add testing for 'mdevctl start' nodedev: add mdev support to virNodeDeviceDestroy() nodedev: Add testing for 'mdevctl stop' docs: note node device fields that are read-only news: mediated devices can be created NEWS.rst | 7 + build-aux/syntax-check.mk | 2 +- docs/formatnode.html.in | 21 +- docs/schemas/nodedev.rng | 18 +- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 3 + src/conf/node_device_conf.c | 68 +++- src/conf/node_device_conf.h | 3 + src/conf/virnodedeviceobj.c | 34 ++ src/conf/virnodedeviceobj.h | 3 + src/libvirt_private.syms | 3 + src/node_device/Makefile.inc.am | 23 +- src/node_device/node_device_driver.c | 360 ++++++++++++++++-- src/node_device/node_device_driver.h | 7 + src/node_device/node_device_udev.c | 5 +- src/util/virmdev.c | 12 + src/util/virmdev.h | 11 + tests/Makefile.am | 15 + ...019_36ea_4111_8f0a_8c9a70e21366-start.argv | 1 + ...019_36ea_4111_8f0a_8c9a70e21366-start.json | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.argv | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.json | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.argv | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.json | 1 + tests/nodedevmdevctldata/mdevctl-stop.argv | 1 + tests/nodedevmdevctltest.c | 303 +++++++++++++++ ...v_d069d019_36ea_4111_8f0a_8c9a70e21366.xml | 7 + ...v_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml | 9 + ...v_fedc4916_1ca8_49ac_b176_871d16c13076.xml | 8 + 29 files changed, 863 insertions(+), 68 deletions(-) create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.json create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.json create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.json create mode 100644 tests/nodedevmdevctldata/mdevctl-stop.argv create mode 100644 tests/nodedevmdevctltest.c create mode 100644 tests/nodedevschemadata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366.xml create mode 100644 tests/nodedevschemadata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml create mode 100644 tests/nodedevschemadata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076.xml -- 2.21.3

4 18

[libvirt PATCH] nodedev: Makefile: Use AM_LDFLAGS_MOD_NOUNDEF with the installed library
by Erik Skultety 19 Jun '20

19 Jun '20

Rather than using the AM_LDFLAGS_MOD_NOUNDEF options with the noinstall library that will come out of libtool from libvirt_driver_nodedev_impl_la, use it with the installed version libvirt_driver_nodedev_la. Broken-by-commit: c44bffb9 Fixes: https://ci.centos.org/job/libvirt-rpm/systems=libvirt-fedora-32/1155/ Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- Technically a build fix, but I'd like one more pair of eyes to look at this, especially if they understand libtool. src/node_device/Makefile.inc.am | 1 + 1 file changed, 1 insertion(+) diff --git a/src/node_device/Makefile.inc.am b/src/node_device/Makefile.inc.am index 5993165b56..abe26b23bf 100644 --- a/src/node_device/Makefile.inc.am +++ b/src/node_device/Makefile.inc.am @@ -37,6 +37,7 @@ if WITH_NODE_DEVICES noinst_LTLIBRARIES += libvirt_driver_nodedev_impl.la libvirt_driver_nodedev_la_SOURCES = libvirt_driver_nodedev_la_LIBADD = libvirt_driver_nodedev_impl.la +libvirt_driver_nodedev_la_LDFLAGS = $(AM_LDFLAGS_MOD_NOUNDEF) mod_LTLIBRARIES += libvirt_driver_nodedev.la libvirt_driver_nodedev_impl_la_SOURCES = $(NODE_DEVICE_DRIVER_SOURCES) -- 2.26.2

2 3

[PATCH] qemu: auto-assign hostdev interface devices to PCIe
by Laine Stump 18 Jun '20

18 Jun '20

Until recently, an <interface type='network'> would automatically be assigned model "rtl8139" (the default on x86 machinetypes), which in turn would lead to the device being assigned a PCI address on a conventional PCI controller (i.e. a pcie-to-pci-bridge). If the network was a typical Linux host bridge-based network that used an emulated device, this would be appropriate, since the guest actually would get an emulated rtl8139 NIC, and that device is a conventional PCI device. However, if the network being used was a pool of hostdev devices, the guest would get an actual PCIe network device assigned from the host via VFIO; while the interface model in that case is irrelevant for the QEMU commandline to assign the device, the PCI address would have already been assigned prior to runtime, so the address assignment would be done based on the model='rtl8139' - a conventional PCI device. VFIO assignment of a PCIe device to a conventional PCI slot works, but we would rather have these devices in a PCIe slot. Since commit bdb8f2e4186, if <interface type='network'> points to a etwork that is a pool of hostdev devices, the interface model will be _unset_ by default. This patch uses that information when deciding what type of slot to assign to the device: since all hostdev network interfaces are SR-IOV VFs, and *all* SR-IOV network cards are PCIe, it is safe to assume that the VFs are PCIe and we should assign then to a PCIe slot in the guest. Signed-off-by: Laine Stump <laine(a)redhat.com> --- (NB: all talk of PCIe vs. conventional PCI is irrelevant for machinetypes that don't support PCIe - any requirement for PCIe is "squashed" down to just "PCI" in those cases) (Note to Paulo - I realize this doesn't describe exactly what happens on s390, since the default interface model in that case is "virtio" rather than "rtl8139", and this patch should actually cause no behavior change on S390. I'm Cc'ing you since you're the author of the other patch I mention in the commit message, and also so you can try running your same tests with this patch added, and verify that it really doesn't break anything for S390.) src/qemu/qemu_domain_address.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c index 07431343ed..05cf251cd5 100644 --- a/src/qemu/qemu_domain_address.c +++ b/src/qemu/qemu_domain_address.c @@ -734,6 +734,14 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDefPtr dev, if (net->model == VIR_DOMAIN_NET_MODEL_E1000E) return pcieFlags; + /* the only time model can be "unknown" is for type='hostdev' + * or for type='network' where the network is a pool of + * hostdev devices. These will always be pcie on the host, and + * should be pcie in the guest if it supports pcie. + */ + if (net->model == VIR_DOMAIN_NET_MODEL_UNKNOWN) + return pcieFlags; + return pciFlags; } -- 2.25.4

3 2

[libvirt PATCH 0/4] cpu_map: Add some missing x86 features
by Jiri Denemark 18 Jun '20

18 Jun '20

The features were added to QEMU long ago. Jiri Denemark (4): cpu_map: Request test files update when adding x86 features cpu_map: Add missing x86 features in 0x7 CPUID leaf cpu_map: Add missing x86 features in 0x80000008 CPUID leaf cpu_map: Add missing AMD SVM features src/cpu_map/x86_features.xml | 57 +++++++++++++++++++ .../x86_64-cpuid-A10-5800K-disabled.xml | 1 + .../x86_64-cpuid-A10-5800K-enabled.xml | 1 + .../x86_64-cpuid-A10-5800K-guest.xml | 10 ++++ .../x86_64-cpuid-A10-5800K-host.xml | 10 ++++ .../x86_64-cpuid-A10-5800K-json.xml | 2 + .../x86_64-cpuid-Cooperlake-enabled.xml | 2 +- .../x86_64-cpuid-Cooperlake-json.xml | 1 + .../x86_64-cpuid-Core-i5-6600-disabled.xml | 1 + ...6_64-cpuid-Core-i7-5600U-arat-disabled.xml | 1 + .../x86_64-cpuid-Core-i7-5600U-disabled.xml | 1 + ...6_64-cpuid-Core-i7-5600U-ibrs-disabled.xml | 2 +- .../x86_64-cpuid-Core-i7-7700-disabled.xml | 1 + ...86_64-cpuid-EPYC-7601-32-Core-disabled.xml | 2 + ...x86_64-cpuid-EPYC-7601-32-Core-enabled.xml | 1 + .../x86_64-cpuid-EPYC-7601-32-Core-guest.xml | 11 ++++ .../x86_64-cpuid-EPYC-7601-32-Core-host.xml | 11 ++++ ...-cpuid-EPYC-7601-32-Core-ibpb-disabled.xml | 2 + ...4-cpuid-EPYC-7601-32-Core-ibpb-enabled.xml | 1 + ..._64-cpuid-EPYC-7601-32-Core-ibpb-guest.xml | 11 ++++ ...6_64-cpuid-EPYC-7601-32-Core-ibpb-host.xml | 11 ++++ ...6_64-cpuid-EPYC-7601-32-Core-ibpb-json.xml | 2 + .../x86_64-cpuid-EPYC-7601-32-Core-json.xml | 2 + .../x86_64-cpuid-FX-8150-guest.xml | 10 ++++ .../cputestdata/x86_64-cpuid-FX-8150-host.xml | 10 ++++ ...-cpuid-Hygon-C86-7185-32-core-disabled.xml | 2 + ...4-cpuid-Hygon-C86-7185-32-core-enabled.xml | 1 + ..._64-cpuid-Hygon-C86-7185-32-core-guest.xml | 11 ++++ ...6_64-cpuid-Hygon-C86-7185-32-core-host.xml | 11 ++++ ...6_64-cpuid-Hygon-C86-7185-32-core-json.xml | 2 + .../x86_64-cpuid-Ice-Lake-Server-disabled.xml | 2 +- .../x86_64-cpuid-Ice-Lake-Server-guest.xml | 1 + .../x86_64-cpuid-Ice-Lake-Server-host.xml | 1 + .../x86_64-cpuid-Opteron-1352-guest.xml | 3 + .../x86_64-cpuid-Opteron-1352-host.xml | 3 + .../x86_64-cpuid-Opteron-2350-disabled.xml | 1 + .../x86_64-cpuid-Opteron-2350-enabled.xml | 1 + .../x86_64-cpuid-Opteron-2350-guest.xml | 3 + .../x86_64-cpuid-Opteron-2350-host.xml | 3 + .../x86_64-cpuid-Opteron-2350-json.xml | 1 + .../x86_64-cpuid-Opteron-6234-disabled.xml | 1 + .../x86_64-cpuid-Opteron-6234-enabled.xml | 1 + .../x86_64-cpuid-Opteron-6234-guest.xml | 10 ++++ .../x86_64-cpuid-Opteron-6234-host.xml | 10 ++++ .../x86_64-cpuid-Opteron-6234-json.xml | 2 + .../x86_64-cpuid-Opteron-6282-guest.xml | 10 ++++ .../x86_64-cpuid-Opteron-6282-host.xml | 10 ++++ .../x86_64-cpuid-Phenom-B95-disabled.xml | 1 + .../x86_64-cpuid-Phenom-B95-enabled.xml | 1 + .../x86_64-cpuid-Phenom-B95-guest.xml | 4 ++ .../x86_64-cpuid-Phenom-B95-host.xml | 4 ++ .../x86_64-cpuid-Phenom-B95-json.xml | 2 + ...puid-Ryzen-7-1800X-Eight-Core-disabled.xml | 2 + ...cpuid-Ryzen-7-1800X-Eight-Core-enabled.xml | 1 + ...4-cpuid-Ryzen-7-1800X-Eight-Core-guest.xml | 11 ++++ ...64-cpuid-Ryzen-7-1800X-Eight-Core-host.xml | 11 ++++ ...64-cpuid-Ryzen-7-1800X-Eight-Core-json.xml | 2 + ...4-cpuid-Ryzen-9-3900X-12-Core-disabled.xml | 2 + ...64-cpuid-Ryzen-9-3900X-12-Core-enabled.xml | 5 +- ...6_64-cpuid-Ryzen-9-3900X-12-Core-guest.xml | 13 +++++ ...86_64-cpuid-Ryzen-9-3900X-12-Core-host.xml | 13 +++++ ...86_64-cpuid-Ryzen-9-3900X-12-Core-json.xml | 4 ++ .../x86_64-cpuid-Xeon-E3-1245-v5-disabled.xml | 1 + .../x86_64-cpuid-Xeon-E5-2623-v4-disabled.xml | 2 +- .../x86_64-cpuid-Xeon-E5-2650-v4-disabled.xml | 2 +- .../x86_64-cpuid-Xeon-Gold-5115-disabled.xml | 2 +- .../x86_64-cpuid-Xeon-Gold-6148-disabled.xml | 2 +- ...6_64-cpuid-Xeon-Platinum-8268-disabled.xml | 3 +- .../domaincapsdata/qemu_3.0.0-tcg.x86_64.xml | 1 + .../domaincapsdata/qemu_3.1.0-tcg.x86_64.xml | 1 + .../domaincapsdata/qemu_4.0.0-tcg.x86_64.xml | 2 + .../domaincapsdata/qemu_4.1.0-tcg.x86_64.xml | 2 + .../domaincapsdata/qemu_4.2.0-tcg.x86_64.xml | 2 + .../domaincapsdata/qemu_5.0.0-tcg.x86_64.xml | 2 + .../domaincapsdata/qemu_5.1.0-q35.x86_64.xml | 5 ++ .../domaincapsdata/qemu_5.1.0-tcg.x86_64.xml | 2 + tests/domaincapsdata/qemu_5.1.0.x86_64.xml | 5 ++ .../cpu-host-model-cmt.x86_64-4.0.0.args | 6 +- 78 files changed, 357 insertions(+), 13 deletions(-) -- 2.27.0

2 6

[libvirt PATCH] Fix allocation of virDomainFSDef
by Ján Tomko 18 Jun '20

18 Jun '20

Some less commonly used drivers were omitted when we switched the allocator from a plain VIR_ALLOC to virDomainFSDefNew. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Fixes: da665fbd4858890fbb3bbf5da2a7b6ca37bb3220 https://bugzilla.redhat.com/show_bug.cgi?id=1846450 --- src/openvz/openvz_conf.c | 2 +- src/vbox/vbox_common.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/openvz/openvz_conf.c b/src/openvz/openvz_conf.c index 190c57b622..db3df5ed5a 100644 --- a/src/openvz/openvz_conf.c +++ b/src/openvz/openvz_conf.c @@ -357,7 +357,7 @@ openvzReadFSConf(virDomainDefPtr def, goto error; } - if (VIR_ALLOC(fs) < 0) + if (!(fs = virDomainFSDefNew(NULL))) goto error; veid_str = g_strdup_printf("%d", veid); diff --git a/src/vbox/vbox_common.c b/src/vbox/vbox_common.c index 85935ba731..15f8eb074a 100644 --- a/src/vbox/vbox_common.c +++ b/src/vbox/vbox_common.c @@ -3626,7 +3626,7 @@ vboxDumpSharedFolders(virDomainDefPtr def, vboxDriverPtr data, IMachine *machine char *hostPath = NULL; PRBool writable = PR_FALSE; - if (VIR_ALLOC(def->fss[i]) < 0) + if (!(def->fss[i] = virDomainFSDefNew(data->xmlopt))) goto cleanup; def->fss[i]->type = VIR_DOMAIN_FS_TYPE_MOUNT; -- 2.25.4

2 3

[libvirt PATCH] util: remove outdated comment from virLogFormatString
by Ján Tomko 18 Jun '20

18 Jun '20

Introduced by commit 72ab0b6dc8129a388aff260a6e444d48495b0e93 which added some code depending on libvirt's log format string into qemuProcessReadLogOutput. This function was deleted by commit 932534e85f34a479c7eac174e997bfd9c85bd22d later. Drop the comment. Signed-off-by: Ján Tomko <jtomko(a)redhat.com> Closes: https://gitlab.com/libvirt/libvirt/-/issues/35 --- Pushed as trivial. src/util/virlog.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/src/util/virlog.c b/src/util/virlog.c index e4944d1258..2c18eeed03 100644 --- a/src/util/virlog.c +++ b/src/util/virlog.c @@ -421,13 +421,6 @@ virLogFormatString(char **msg, virLogPriority priority, const char *str) { - /* - * Be careful when changing the following log message formatting, we rely - * on it when stripping libvirt debug messages from qemu log files. So when - * changing this, you might also need to change the code there. - * virLogFormatString() function name is mentioned there so it's sufficient - * to just grep for it to find the right place. - */ if ((funcname != NULL)) { *msg = g_strdup_printf("%llu: %s : %s:%d : %s\n", virThreadSelfID(), virLogPriorityString(priority), -- 2.25.4

1 0

[PATCH 1/1] NEWS.rst: document the 'auto-fill' feature
by Daniel Henrique Barboza 18 Jun '20

18 Jun '20

Although this can be considered a new feature, from the user standpoint is more of a QoL improvement. Suggested-by: Michal Privoznik <mprivozn(a)redhat.com> Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- NEWS.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index a16903c7c3..20964b94d7 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -50,6 +50,14 @@ v6.4.0 (2020-06-02) allowed only when the format is 'raw' and no other block layer features are requested. + * qemu: auto-fill of incomplete NUMA topologies + + Domains with incomplete NUMA topologies, where the sum of vCPUs in all NUMA + cells is less than the total of vCPUs, will get their first NUMA cell to + be auto-filled with the remaining vCPUs. This behavior reproduces what QEMU + already does in these cases. Users are encouraged to provide complete NUMA + topologies to avoid unexpected changes in the domain XML. + * **Bug fixes** * qemu: fixed regression in network device hotplug with new qemu versions -- 2.26.2

2 1

[PATCH v2 0/4] NUMA CPUs 'auto-fill' for incomplete topologies
by Daniel Henrique Barboza 18 Jun '20

18 Jun '20

changes in v2: - removed patch 5/5 Gitlab link: https://gitlab.com/danielhb/libvirt/-/tree/vcpus_numa_v2 v1 link: https://www.redhat.com/archives/libvir-list/2020-June/msg00016.html Daniel Henrique Barboza (4): numa_conf.c: add helper functions for cpumap operations qemu_domain.c: NUMA CPUs auto-fill for incomplete topologies qemuxml2xmltest.c: add NUMA vcpus auto fill tests formatdomain.html.in: document the NUMA cpus auto fill feature docs/formatdomain.html.in | 11 ++++- src/conf/numa_conf.c | 46 ++++++++++++++++++ src/conf/numa_conf.h | 3 ++ src/libvirt_private.syms | 1 + src/qemu/qemu_domain.c | 47 +++++++++++++++++++ src/qemu/qemu_domain.h | 4 ++ src/qemu/qemu_driver.c | 9 ++++ .../numavcpus-topology-mismatch.xml | 37 +++++++++++++++ ...avcpus-topology-mismatch.x86_64-latest.xml | 38 +++++++++++++++ tests/qemuxml2xmltest.c | 1 + 10 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/numavcpus-topology-mismatch.xml create mode 100644 tests/qemuxml2xmloutdata/numavcpus-topology-mismatch.x86_64-latest.xml -- 2.26.2

2 12

[PATCH 0/6] Restore seclabels on the correct path on 'virsh restore'
by Michal Privoznik 18 Jun '20

18 Jun '20

When restoring a domain from a block device a harmless error is reported. See 6/6 for explanation. But anyway, still worth fixing. Michal Prívozník (6): qemu: Use qemuSecurityDomainSetPathLabel() to set seclabes on not saved state files qemu: Drop unused qemuSecuritySetSavedStateLabel() security: Drop unused virSecurityManagerSetSavedStateLabel() security: Rename virSecurityManagerRestoreSavedStateLabel() qemu: Rename qemuSecurityRestoreSavedStateLabel() qemuSecurityDomainRestorePathLabel: Introduce @ignoreNS argument src/libvirt_private.syms | 3 +- src/qemu/qemu_driver.c | 6 +-- src/qemu/qemu_security.c | 43 ++++--------------- src/qemu/qemu_security.h | 9 ++-- src/security/security_apparmor.c | 18 ++------ src/security/security_dac.c | 46 +++++--------------- src/security/security_driver.h | 13 ++---- src/security/security_manager.c | 63 +++++++++++++--------------- src/security/security_manager.h | 11 +++-- src/security/security_nop.c | 19 --------- src/security/security_selinux.c | 49 +++++++--------------- src/security/security_stack.c | 72 +++++++++----------------------- 12 files changed, 101 insertions(+), 251 deletions(-) -- 2.26.2

2 12

[PATCH] qemu: do not add model when actual iface type is hostdev
by Paulo de Rezende Pinatti 18 Jun '20

18 Jun '20

No default model should be added to the interface entry at post parse when its actual network type is hostdev as doing so might cause a mismatch between the interface definition and its actual device type. --- src/qemu/qemu_domain.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 2dad823a86..33ce0ad992 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -5831,6 +5831,7 @@ qemuDomainDeviceNetDefPostParse(virDomainNetDefPtr net, virQEMUCapsPtr qemuCaps) { if (net->type != VIR_DOMAIN_NET_TYPE_HOSTDEV && + virDomainNetResolveActualType(net) != VIR_DOMAIN_NET_TYPE_HOSTDEV && !virDomainNetGetModelString(net)) net->model = qemuDomainDefaultNetModel(def, qemuCaps); -- 2.26.2

3 7

[PATCHv2] network: Fix a race condition when shutdown & start vm at the same time
by Bingsong Si 18 Jun '20

18 Jun '20

When shutdown vm, the qemuProcessStop cleanup virtual interface in two steps: 1. qemuProcessKill kill qemu process, and vif disappeared 2. ovs-vsctl del-port from the ovs bridge If start a vm in the middle of the two steps, the new vm will reused the vif, but removed from ovs bridge by step 2 https://www.redhat.com/archives/libvir-list/2020-June/msg00681.html Signed-off-by: Bingsong Si <owen.si(a)ucloud.cn> --- src/libvirt_private.syms | 1 + src/qemu/qemu_process.c | 9 ++++++--- src/util/virnetdevtap.c | 12 ++++++++++-- src/util/virnetdevtap.h | 2 ++ 4 files changed, 19 insertions(+), 5 deletions(-) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index fc7406f2b7..5258c07368 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -2655,6 +2655,7 @@ virNetDevTapDelete; virNetDevTapGetName; virNetDevTapGetRealDeviceName; virNetDevTapInterfaceStats; +virNetDevTapMutex; virNetDevTapReattachBridge; diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index d36088ba98..2493b092b1 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7483,9 +7483,12 @@ void qemuProcessStop(virQEMUDriverPtr driver, if (vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_MIDONET) { ignore_value(virNetDevMidonetUnbindPort(vport)); } else if (vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_OPENVSWITCH) { - ignore_value(virNetDevOpenvswitchRemovePort( - virDomainNetGetActualBridgeName(net), - net->ifname)); + virMutexLock(&virNetDevTapMutex); + if (!virNetDevExists(net->ifname)) + ignore_value(virNetDevOpenvswitchRemovePort( + virDomainNetGetActualBridgeName(net), + net->ifname)); + virMutexUnlock(&virNetDevTapMutex); } } diff --git a/src/util/virnetdevtap.c b/src/util/virnetdevtap.c index 7bd30ea0f9..d685e30a50 100644 --- a/src/util/virnetdevtap.c +++ b/src/util/virnetdevtap.c @@ -54,6 +54,8 @@ VIR_LOG_INIT("util.netdevtap"); +virMutex virNetDevTapMutex = VIR_MUTEX_INITIALIZER; + /** * virNetDevTapGetName: * @tapfd: a tun/tap file descriptor @@ -238,6 +240,7 @@ int virNetDevTapCreate(char **ifname, if (!tunpath) tunpath = "/dev/net/tun"; + virMutexLock(&virNetDevTapMutex); memset(&ifr, 0, sizeof(ifr)); for (i = 0; i < tapfdSize; i++) { if ((fd = open(tunpath, O_RDWR)) < 0) { @@ -302,6 +305,7 @@ int virNetDevTapCreate(char **ifname, ret = 0; cleanup: + virMutexUnlock(&virNetDevTapMutex); if (ret < 0) { VIR_FORCE_CLOSE(fd); while (i--) @@ -369,6 +373,7 @@ int virNetDevTapCreate(char **ifname, int ret = -1; char *newifname = NULL; + virMutexLock(&virNetDevTapMutex); if (tapfdSize > 1) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Multiqueue devices are not supported on this system")); @@ -379,8 +384,10 @@ int virNetDevTapCreate(char **ifname, * we have to create 'tap' interface first and * then rename it to 'vnet' */ - if ((s = virNetDevSetupControl("tap", &ifr)) < 0) + if ((s = virNetDevSetupControl("tap", &ifr)) < 0) { + virMutexUnlock(&virNetDevTapMutex); return -1; + } if (ioctl(s, SIOCIFCREATE2, &ifr) < 0) { virReportSystemError(errno, "%s", @@ -434,6 +441,7 @@ int virNetDevTapCreate(char **ifname, ret = 0; cleanup: + virMutexUnlock(&virNetDevTapMutex); VIR_FORCE_CLOSE(s); return ret; @@ -710,7 +718,7 @@ int virNetDevTapCreateInBridgePort(const char *brname, if (virNetDevSetMAC(*ifname, &tapmac) < 0) goto error; - if (virNetDevTapAttachBridge(*ifname, brname, macaddr, vmuuid, + if (virNetDevTapReattachBridge(*ifname, brname, macaddr, vmuuid, virtPortProfile, virtVlan, isolatedPort, mtu, actualMTU) < 0) { goto error; diff --git a/src/util/virnetdevtap.h b/src/util/virnetdevtap.h index c6bd9285ba..f2d332e5aa 100644 --- a/src/util/virnetdevtap.h +++ b/src/util/virnetdevtap.h @@ -29,6 +29,8 @@ # define VIR_NETDEV_TAP_REQUIRE_MANUAL_CLEANUP 1 #endif +extern virMutex virNetDevTapMutex; + int virNetDevTapCreate(char **ifname, const char *tunpath, int *tapfd, -- 2.18.4

1 0

[libvirt-glib PATCH v2] gitlab: introduce CI jobs testing git master & distro libvirt
by Daniel P. Berrangé 17 Jun '20

17 Jun '20

The glib build needs to validate two axis - A variety of libvirt versions - Native and mingw We test a variety of libvirt versions by running a build against the distro provided libvirt packages. All that is then missing is a build against the latest libvirt git master, which only needs to be run on a single distro, for which CentOS 8 is picked as a stable long life base. For mingw, we only test against libvirt git master, using Fedora rawhide. It is expected that any issues against older libvirt will have already been caught by the native builds. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 244 ++++++++++++++++++ ci/containers/README.rst | 14 + ci/containers/libvirt-centos-7.Dockerfile | 88 +++++++ ci/containers/libvirt-centos-8.Dockerfile | 64 +++++ .../libvirt-centos-stream.Dockerfile | 58 +++++ ci/containers/libvirt-debian-10.Dockerfile | 58 +++++ ci/containers/libvirt-debian-9.Dockerfile | 61 +++++ ci/containers/libvirt-debian-sid.Dockerfile | 58 +++++ ci/containers/libvirt-fedora-31.Dockerfile | 55 ++++ ci/containers/libvirt-fedora-32.Dockerfile | 55 ++++ ...rt-fedora-rawhide-cross-mingw32.Dockerfile | 133 ++++++++++ ...rt-fedora-rawhide-cross-mingw64.Dockerfile | 133 ++++++++++ .../libvirt-fedora-rawhide.Dockerfile | 56 ++++ ci/containers/libvirt-opensuse-151.Dockerfile | 57 ++++ ci/containers/libvirt-ubuntu-1804.Dockerfile | 61 +++++ ci/containers/libvirt-ubuntu-2004.Dockerfile | 58 +++++ ci/containers/refresh | 36 +++ 17 files changed, 1289 insertions(+) create mode 100644 ci/containers/README.rst create mode 100644 ci/containers/libvirt-centos-7.Dockerfile create mode 100644 ci/containers/libvirt-centos-8.Dockerfile create mode 100644 ci/containers/libvirt-centos-stream.Dockerfile create mode 100644 ci/containers/libvirt-debian-10.Dockerfile create mode 100644 ci/containers/libvirt-debian-9.Dockerfile create mode 100644 ci/containers/libvirt-debian-sid.Dockerfile create mode 100644 ci/containers/libvirt-fedora-31.Dockerfile create mode 100644 ci/containers/libvirt-fedora-32.Dockerfile create mode 100644 ci/containers/libvirt-fedora-rawhide-cross-mingw32.Dockerfile create mode 100644 ci/containers/libvirt-fedora-rawhide-cross-mingw64.Dockerfile create mode 100644 ci/containers/libvirt-fedora-rawhide.Dockerfile create mode 100644 ci/containers/libvirt-opensuse-151.Dockerfile create mode 100644 ci/containers/libvirt-ubuntu-1804.Dockerfile create mode 100644 ci/containers/libvirt-ubuntu-2004.Dockerfile create mode 100755 ci/containers/refresh diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 50dae92..8f662b1 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,103 @@ stages: - prebuild + - containers + - builds + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt-glib/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/containers/libvirt-$NAME.Dockerfile" ci + - docker push "$TAG" + after_script: + - docker logout + +.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export SCRATCH_DIR="/tmp/scratch" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_DIR="$PWD/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig" + export LD_LIBRARY_PATH="$VROOT/lib" + +.git_native_build_job_template: &git_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh --prefix="$VROOT" --without-libvirtd + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild --nodeps -ta libvirt-glib*.tar.gz ; fi + +.dist_native_build_job_template: &dist_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild -ta libvirt-glib*.tar.gz ; fi + +# Default cross build jobs that are always run +.git_cross_build_default_job_template: &git_cross_build_default_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh $CONFIGURE_OPTS --prefix="$VROOT" + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1) + - $MAKE # Check that all commits are signed-off for the DCO. # Skip on "libvirt" namespace, since we only need to run @@ -14,3 +111,150 @@ check-dco: except: variables: - $CI_PROJECT_NAMESPACE == 'libvirt' + +x64-centos-7-container: + <<: *container_job_definition + variables: + NAME: centos-7 + +x64-centos-8-container: + <<: *container_job_definition + variables: + NAME: centos-8 + +x64-centos-stream-container: + <<: *container_job_definition + variables: + NAME: centos-stream + +x64-debian-9-container: + <<: *container_job_definition + variables: + NAME: debian-9 + +x64-debian-10-container: + <<: *container_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-container: + <<: *container_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-container: + <<: *container_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-container: + <<: *container_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-container: + <<: *container_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 + + + +x64-centos-8-git-build: + <<: *git_native_build_job_definition + variables: + NAME: centos-8 + +x64-centos-7-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-7 + +x64-centos-8-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-8 + +x64-centos-stream-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-stream + +x64-debian-9-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-9 + +x64-debian-10-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 diff --git a/ci/containers/README.rst b/ci/containers/README.rst new file mode 100644 index 0000000..530897e --- /dev/null +++ b/ci/containers/README.rst @@ -0,0 +1,14 @@ +CI job assets +============= + +This directory contains assets used in the automated CI jobs, most +notably the Dockerfiles used to build container images in which the +CI jobs then run. + +The ``refresh`` script is used to re-create the Dockerfiles using the +``lcitool`` command that is provided by repo +https://gitlab.com/libvirt/libvirt-ci + +The containers are built during the CI process and cached in the GitLab +container registry of the project doing the build. The cached containers +can be deleted at any time and will be correctly rebuilt. diff --git a/ci/containers/libvirt-centos-7.Dockerfile b/ci/containers/libvirt-centos-7.Dockerfile new file mode 100644 index 0000000..ad6b262 --- /dev/null +++ b/ci/containers/libvirt-centos-7.Dockerfile @@ -0,0 +1,88 @@ +FROM centos:7 + +RUN echo -e '[openvz]\n\ +name=OpenVZ addons\n\ +baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/\n\ +enabled=1\n\ +gpgcheck=1\n\ +skip_if_unavailable=0\n\ +metadata_expire=6h\n\ +priority=90\n\ +includepkgs=libprl*' > /etc/yum.repos.d/openvz.repo && \ + echo -e '-----BEGIN PGP PUBLIC KEY BLOCK-----\n\ +Version: GnuPG v2.0.22 (GNU/Linux)\n\ +\n\ +mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk\n\ +YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil\n\ +BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0\n\ +SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp\n\ +IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD\n\ +AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv\n\ +oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt\n\ +EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y\n\ +zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj\n\ +yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM\n\ +t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC\n\ +oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q\n\ +bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V\n\ +igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS\n\ +WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy\n\ +=fRMg\n\ +-----END PGP PUBLIC KEY BLOCK-----' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + yum install -y epel-release && \ + yum update -y && \ + yum install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-common \ + glibc-devel \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + yum autoremove -y && \ + yum clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja-build" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-centos-8.Dockerfile b/ci/containers/libvirt-centos-8.Dockerfile new file mode 100644 index 0000000..360cf08 --- /dev/null +++ b/ci/containers/libvirt-centos-8.Dockerfile @@ -0,0 +1,64 @@ +FROM centos:8 + +RUN dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + libnl3-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-docutils \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-centos-stream.Dockerfile b/ci/containers/libvirt-centos-stream.Dockerfile new file mode 100644 index 0000000..6f6c830 --- /dev/null +++ b/ci/containers/libvirt-centos-stream.Dockerfile @@ -0,0 +1,58 @@ +FROM centos:8 + +RUN dnf install -y centos-release-stream && \ + dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y Stream-PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-debian-10.Dockerfile b/ci/containers/libvirt-debian-10.Dockerfile new file mode 100644 index 0000000..f55fe06 --- /dev/null +++ b/ci/containers/libvirt-debian-10.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:10 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-debian-9.Dockerfile b/ci/containers/libvirt-debian-9.Dockerfile new file mode 100644 index 0000000..182b3a6 --- /dev/null +++ b/ci/containers/libvirt-debian-9.Dockerfile @@ -0,0 +1,61 @@ +FROM debian:9 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-debian-sid.Dockerfile b/ci/containers/libvirt-debian-sid.Dockerfile new file mode 100644 index 0000000..fbb1cb6 --- /dev/null +++ b/ci/containers/libvirt-debian-sid.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:sid + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-31.Dockerfile b/ci/containers/libvirt-fedora-31.Dockerfile new file mode 100644 index 0000000..e23f6d5 --- /dev/null +++ b/ci/containers/libvirt-fedora-31.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:31 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-32.Dockerfile b/ci/containers/libvirt-fedora-32.Dockerfile new file mode 100644 index 0000000..1da98c3 --- /dev/null +++ b/ci/containers/libvirt-fedora-32.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:32 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-fedora-rawhide-cross-mingw32.Dockerfile b/ci/containers/libvirt-fedora-rawhide-cross-mingw32.Dockerfile new file mode 100644 index 0000000..c366c5f --- /dev/null +++ b/ci/containers/libvirt-fedora-rawhide-cross-mingw32.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw32-curl \ + mingw32-dbus \ + mingw32-dlfcn \ + mingw32-gcc \ + mingw32-gettext \ + mingw32-glib2 \ + mingw32-gnutls \ + mingw32-libssh2 \ + mingw32-libxml2 \ + mingw32-openssl \ + mingw32-pkg-config \ + mingw32-portablexdr \ + mingw32-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "i686-w64-mingw32" +ENV CONFIGURE_OPTS "--host=i686-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw32.meson" diff --git a/ci/containers/libvirt-fedora-rawhide-cross-mingw64.Dockerfile b/ci/containers/libvirt-fedora-rawhide-cross-mingw64.Dockerfile new file mode 100644 index 0000000..1c5c07c --- /dev/null +++ b/ci/containers/libvirt-fedora-rawhide-cross-mingw64.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw64-curl \ + mingw64-dbus \ + mingw64-dlfcn \ + mingw64-gcc \ + mingw64-gettext \ + mingw64-glib2 \ + mingw64-gnutls \ + mingw64-libssh2 \ + mingw64-libxml2 \ + mingw64-openssl \ + mingw64-pkg-config \ + mingw64-portablexdr \ + mingw64-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "x86_64-w64-mingw32" +ENV CONFIGURE_OPTS "--host=x86_64-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw64.meson" diff --git a/ci/containers/libvirt-fedora-rawhide.Dockerfile b/ci/containers/libvirt-fedora-rawhide.Dockerfile new file mode 100644 index 0000000..7017282 --- /dev/null +++ b/ci/containers/libvirt-fedora-rawhide.Dockerfile @@ -0,0 +1,56 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-opensuse-151.Dockerfile b/ci/containers/libvirt-opensuse-151.Dockerfile new file mode 100644 index 0000000..87237d4 --- /dev/null +++ b/ci/containers/libvirt-opensuse-151.Dockerfile @@ -0,0 +1,57 @@ +FROM opensuse/leap:15.1 + +RUN zypper update -y && \ + zypper install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-locale \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + zypper clean --all && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-ubuntu-1804.Dockerfile b/ci/containers/libvirt-ubuntu-1804.Dockerfile new file mode 100644 index 0000000..2b8bd41 --- /dev/null +++ b/ci/containers/libvirt-ubuntu-1804.Dockerfile @@ -0,0 +1,61 @@ +FROM ubuntu:18.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/libvirt-ubuntu-2004.Dockerfile b/ci/containers/libvirt-ubuntu-2004.Dockerfile new file mode 100644 index 0000000..38920f0 --- /dev/null +++ b/ci/containers/libvirt-ubuntu-2004.Dockerfile @@ -0,0 +1,58 @@ +FROM ubuntu:20.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/containers/refresh b/ci/containers/refresh new file mode 100755 index 0000000..dfbfa87 --- /dev/null +++ b/ci/containers/refresh @@ -0,0 +1,36 @@ +#!/bin/sh + +if test -z "$1" +then + echo "syntax: $0 PATH-TO-LCITOOL" + exit 1 +fi + +LCITOOL=$1 + +if ! test -x "$LCITOOL" +then + echo "$LCITOOL is not executable" + exit 1 +fi + +HOSTS=$($LCITOOL hosts | grep -v freebsd) + +for host in $HOSTS +do + + if test "$host" = "libvirt-fedora-rawhide" + then + for cross in mingw32 mingw64 + do + $LCITOOL dockerfile $host libvirt,libvirt-glib --cross $cross >$host-cross-$cross.Dockerfile + done + fi + + if test "$host" = "libvirt-centos-8" + then + $LCITOOL dockerfile $host libvirt+minimal,libvirt+dist,libvirt-glib > $host.Dockerfile + else + $LCITOOL dockerfile $host libvirt+dist,libvirt-glib > $host.Dockerfile + fi +done -- 2.26.2

2 1

[libvirt PATCH] ci: Makefile: Mention CI_MAKE_ARGS and CI_CONFIGURE_ARGS in ci-help
by Erik Skultety 17 Jun '20

17 Jun '20

Document the CI_MAKE_ARGS and CI_CONFIGURE_ARGS so that users don't have to skim through the Makefile to be able to pass arbitrary recognized make targets to the build system. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- ci/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ci/Makefile b/ci/Makefile index 7840cc8d89..df60e6bf3c 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -266,4 +266,6 @@ ci-help: @echo " CI_CLEAN=0 - do not delete '$(CI_SCRATCHDIR)' after completion" @echo " CI_REUSE=1 - re-use existing '$(CI_SCRATCHDIR)' content" @echo " CI_ENGINE=auto - container engine to use (podman, docker)" + @echo " CI_CONFIGURE_ARGS= - extra arguments passed to configure" + @echo " CI_MAKE_ARGS= - extra arguments passed to make, e.g. space delimited list of make targets" @echo -- 2.26.2

2 3

Re: ovmf / PCI passthrough impaired due to very limiting PCI64 aperture
by Eduardo Habkost 17 Jun '20

17 Jun '20

+libvir-list On Wed, Jun 17, 2020 at 05:04:12PM +0100, Dr. David Alan Gilbert wrote: > * Eduardo Habkost (ehabkost(a)redhat.com) wrote: > > On Wed, Jun 17, 2020 at 02:46:52PM +0100, Dr. David Alan Gilbert wrote: > > > * Laszlo Ersek (lersek(a)redhat.com) wrote: > > > > On 06/16/20 19:14, Guilherme Piccoli wrote: > > > > > Thanks Gerd, Dave and Eduardo for the prompt responses! > > > > > > > > > > So, I understand that when we use "-host-physical-bits", we are > > > > > passing the *real* number for the guest, correct? So, in this case we > > > > > can trust that the guest physbits matches the true host physbits. > > > > > > > > > > What if then we have OVMF relying in the physbits *iff* > > > > > "-host-phys-bits" is used (which is the default in RH and a possible > > > > > machine configuration on libvirt XML in Ubuntu), and we have OVMF > > > > > fallbacks to 36-bit otherwise? > > > > > > > > I've now read the commit message on QEMU commit 258fe08bd341d, and the > > > > complexity is simply stunning. > > > > > > > > Right now, OVMF calculates the guest physical address space size from > > > > various range sizes (such as hotplug memory area end, default or > > > > user-configured PCI64 MMIO aperture), and derives the minimum suitable > > > > guest-phys address width from that address space size. This width is > > > > then exposed to the rest of the firmware with the CPU HOB (hand-off > > > > block), which in turn controls how the GCD (global coherency domain) > > > > memory space map is sized. Etc. > > > > > > > > If QEMU can provide a *reliable* GPA width, in some info channel (CPUID > > > > or even fw_cfg), then the above calculation could be reversed in OVMF. > > > > We could take the width as a given (-> produce the CPU HOB directly), > > > > plus calculate the *remaining* address space between the GPA space size > > > > given by the width, and the end of the memory hotplug area end. If the > > > > "remaining size" were negative, then obviously QEMU would have been > > > > misconfigured, so we'd halt the boot. Otherwise, the remaining area > > > > could be used as PCI64 MMIO aperture (PEI memory footprint of DXE page > > > > tables be darned). > > > > > > > > > Now, regarding the problem "to trust or not" in the guests' physbits, > > > > > I think it's an orthogonal discussion to some extent. It'd be nice to > > > > > have that check, and as Eduardo said, prevent migration in such cases. > > > > > But it's not really preventing OVMF big PCI64 aperture if we only > > > > > increase the aperture _when "-host-physical-bits" is used_. > > > > > > > > I don't know what exactly those flags do, but I doubt they are clearly > > > > visible to OVMF in any particular way. > > > > > > The firmware should trust whatever it reads from the cpuid and thus gets > > > told from qemu; if qemu is doing the wrong thing there then that's our > > > problem and we need to fix it in qemu. > > > > It is impossible to provide a MAXPHYADDR that the guest can trust > > unconditionally and allow live migration to hosts with different > > sizes at the same time. > > It would be nice to get to a point where we could say that the reported > size is no bigger than the physical hardware. > The gotcha here is that (upstream) qemu is still reporting 40 by default > when even modern Intel desktop chips are 39. I agree it would be nice. We just need a method to tell guest software that we are really making this additional guarantee. > > > Unless we want to drop support live migration to hosts with > > different sizes entirely, we need additional bits to tell the > > guest how much it can trust MAXPHYADDR. > > Could we go with host-phys-bits=true by default, that at least means the > normal behaviour is correct; if people want to migrate between different > hosts with different sizes they should set phys-bits (or > host-phys-limit) to the lowest in their set of hardware. Host-dependent defaults may be convenient for end users running QEMU directly, but not a good idea if a stable guest ABI is expected from your VM configuration (which is the case for software using the libvirt API). -- Eduardo

1 0

Re: ovmf / PCI passthrough impaired due to very limiting PCI64 aperture
by Eduardo Habkost 17 Jun '20

17 Jun '20

+libvir-list On Wed, Jun 17, 2020 at 05:04:12PM +0100, Dr. David Alan Gilbert wrote: > * Eduardo Habkost (ehabkost(a)redhat.com) wrote: > > On Wed, Jun 17, 2020 at 02:46:52PM +0100, Dr. David Alan Gilbert wrote: > > > * Laszlo Ersek (lersek(a)redhat.com) wrote: > > > > On 06/16/20 19:14, Guilherme Piccoli wrote: > > > > > Thanks Gerd, Dave and Eduardo for the prompt responses! > > > > > > > > > > So, I understand that when we use "-host-physical-bits", we are > > > > > passing the *real* number for the guest, correct? So, in this case we > > > > > can trust that the guest physbits matches the true host physbits. > > > > > > > > > > What if then we have OVMF relying in the physbits *iff* > > > > > "-host-phys-bits" is used (which is the default in RH and a possible > > > > > machine configuration on libvirt XML in Ubuntu), and we have OVMF > > > > > fallbacks to 36-bit otherwise? > > > > > > > > I've now read the commit message on QEMU commit 258fe08bd341d, and the > > > > complexity is simply stunning. > > > > > > > > Right now, OVMF calculates the guest physical address space size from > > > > various range sizes (such as hotplug memory area end, default or > > > > user-configured PCI64 MMIO aperture), and derives the minimum suitable > > > > guest-phys address width from that address space size. This width is > > > > then exposed to the rest of the firmware with the CPU HOB (hand-off > > > > block), which in turn controls how the GCD (global coherency domain) > > > > memory space map is sized. Etc. > > > > > > > > If QEMU can provide a *reliable* GPA width, in some info channel (CPUID > > > > or even fw_cfg), then the above calculation could be reversed in OVMF. > > > > We could take the width as a given (-> produce the CPU HOB directly), > > > > plus calculate the *remaining* address space between the GPA space size > > > > given by the width, and the end of the memory hotplug area end. If the > > > > "remaining size" were negative, then obviously QEMU would have been > > > > misconfigured, so we'd halt the boot. Otherwise, the remaining area > > > > could be used as PCI64 MMIO aperture (PEI memory footprint of DXE page > > > > tables be darned). > > > > > > > > > Now, regarding the problem "to trust or not" in the guests' physbits, > > > > > I think it's an orthogonal discussion to some extent. It'd be nice to > > > > > have that check, and as Eduardo said, prevent migration in such cases. > > > > > But it's not really preventing OVMF big PCI64 aperture if we only > > > > > increase the aperture _when "-host-physical-bits" is used_. > > > > > > > > I don't know what exactly those flags do, but I doubt they are clearly > > > > visible to OVMF in any particular way. > > > > > > The firmware should trust whatever it reads from the cpuid and thus gets > > > told from qemu; if qemu is doing the wrong thing there then that's our > > > problem and we need to fix it in qemu. > > > > It is impossible to provide a MAXPHYADDR that the guest can trust > > unconditionally and allow live migration to hosts with different > > sizes at the same time. > > It would be nice to get to a point where we could say that the reported > size is no bigger than the physical hardware. > The gotcha here is that (upstream) qemu is still reporting 40 by default > when even modern Intel desktop chips are 39. I agree it would be nice. We just need a method to tell guest software that we are really making this additional guarantee. > > > Unless we want to drop support live migration to hosts with > > different sizes entirely, we need additional bits to tell the > > guest how much it can trust MAXPHYADDR. > > Could we go with host-phys-bits=true by default, that at least means the > normal behaviour is correct; if people want to migrate between different > hosts with different sizes they should set phys-bits (or > host-phys-limit) to the lowest in their set of hardware. Host-dependent defaults may be convenient for end users running QEMU directly, but not a good idea if a stable guest ABI is expected from your VM configuration (which is the case for software using the libvirt API). -- Eduardo

1 0

[libvirt PATCH 0/3] Remove obsolete conditionals
by Daniel P. Berrangé 17 Jun '20

17 Jun '20

Given our platform support matrix we can cull many conditionally defined constants and rely on system headers Daniel P. Berrangé (3): lxc: drop compat code for mount constants lxc: drop compat code for clone constants lxc: drop compat code for capability constants src/lxc/lxc_container.c | 151 --------------------------------------- src/lxc/lxc_controller.c | 8 --- 2 files changed, 159 deletions(-) -- 2.24.1

2 4

[libvirt PATCH] ci: Makefile: Add distcheck target
by Erik Skultety 17 Jun '20

17 Jun '20

>From time to time we see failures in our gitlab pipelines that we as developers don't see locally, simply because we don't run "make distcheck" that often which is what the gitlab containers run and which can indeed reveal mistakes like not shipping the necessary test data. Since most of us prefer local testing over waiting for the gitlab pipeline to finish, introduce the distcheck target to the CI which we have inside the repo under ci/. Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- ci/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ci/Makefile b/ci/Makefile index 7840cc8d89..e1c626ed34 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -239,6 +239,9 @@ ci-build@%: ci-check@%: $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_MAKE_ARGS="check" +ci-distcheck@%: + $(MAKE) -C $(CI_ROOTDIR) ci-build@$* CI_MAKE_ARGS="distcheck" + ci-list-images: @echo @echo "Available x86 container images:" -- 2.26.2

3 5

[PATCH] virtlogd: solve some memory leaks
by wangjian 17 Jun '20

17 Jun '20

We used asan to find some memory leaks in virtlogd. In the virThreadPoolFree function, When job->data is of type virNetServerJobPtr, the following memory leak problem exists. 1. job->data is not released Direct leak of 24 byte(s) in 1 object(s) allocated from: #0 0x7f14ab932560 in calloc (/usr/local/gcc-6-4/lib64/libasan.so.3+0xc7560) ??:? #1 0x55ab07088853 in virAlloc (/usr/sbin/virtlogd+0x31853) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/viralloc.c:144 #2 0x55ab0707a515 (/usr/sbin/virtlogd+0x23515) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserver.c:209 #3 0x55ab07076d87 (/usr/sbin/virtlogd+0x1fd87) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:1374 #4 0x55ab070770e2 (/usr/sbin/virtlogd+0x200e2) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:1563 #5 0x55ab0709c67f in virEventPollRunOnce (/usr/sbin/virtlogd+0x4567f) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/vireventpoll.c:508 #6 0x55ab0709ad30 in virEventRunDefaultImpl (/usr/sbin/virtlogd+0x43d30) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/virevent.c:314 #7 0x55ab07079f6c in virNetDaemonRun (/usr/sbin/virtlogd+0x22f6c) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetdaemon.c:847 #8 0x55ab070714db in main (/usr/sbin/virtlogd+0x1a4db) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/logging/log_daemon.c:1162 #9 0x7f14aa3c2c56 in __libc_start_main (/usr/lib64/libc.so.6+0x25c56) ??:? #10 0x55ab07072639 in _start (/usr/sbin/virtlogd+0x1b639) ??:? 2. job->data->msg->buffer was not released Indirect leak of 152 byte(s) in 1 object(s) allocated from: #0 0x7f14ab932750 in realloc (/usr/local/gcc-6-4/lib64/libasan.so.3+0xc7750) ??:? #1 0x55ab0708894d in virReallocN (/usr/sbin/virtlogd+0x3194d) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/viralloc.c:245 #2 0x55ab0707d830 in virNetMessageDecodeLength (/usr/sbin/virtlogd+0x26830) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetmessage.c:157 #3 0x55ab07076b36 (/usr/sbin/virtlogd+0x1fb36) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:1269 #4 0x55ab070770e2 (/usr/sbin/virtlogd+0x200e2) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:1563 #5 0x55ab0709c67f in virEventPollRunOnce (/usr/sbin/virtlogd+0x4567f) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/vireventpoll.c:508 #6 0x55ab0709ad30 in virEventRunDefaultImpl (/usr/sbin/virtlogd+0x43d30) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/virevent.c:314 #7 0x55ab07079f6c in virNetDaemonRun (/usr/sbin/virtlogd+0x22f6c) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetdaemon.c:847 #8 0x55ab070714db in main (/usr/sbin/virtlogd+0x1a4db) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/logging/log_daemon.c:1162 #9 0x7f14aa3c2c56 in __libc_start_main (/usr/lib64/libc.so.6+0x25c56) ??:? #10 0x55ab07072639 in _start (/usr/sbin/virtlogd+0x1b639) ??:? 3. job->data->msg was not released Indirect leak of 104 byte(s) in 1 object(s) allocated from: #0 0x7f14ab932560 in calloc (/usr/local/gcc-6-4/lib64/libasan.so.3+0xc7560) ??:? #1 0x55ab07088853 in virAlloc (/usr/sbin/virtlogd+0x31853) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/viralloc.c:144 #2 0x55ab0707d541 in virNetMessageNew (/usr/sbin/virtlogd+0x26541) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetmessage.c:42 #3 0x55ab07076618 (/usr/sbin/virtlogd+0x1f618) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:416 #4 0x55ab0707755c in virNetServerClientNew (/usr/sbin/virtlogd+0x2055c) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverclient.c:467 #5 0x55ab0707a9df (/usr/sbin/virtlogd+0x239df) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserver.c:319 #6 0x55ab07075ad9 (/usr/sbin/virtlogd+0x1ead9) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverservice.c:88 #7 0x55ab0709c67f in virEventPollRunOnce (/usr/sbin/virtlogd+0x4567f) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/vireventpoll.c:508 #8 0x55ab0709ad30 in virEventRunDefaultImpl (/usr/sbin/virtlogd+0x43d30) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/virevent.c:314 #9 0x55ab07079f6c in virNetDaemonRun (/usr/sbin/virtlogd+0x22f6c) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetdaemon.c:847 #10 0x55ab070714db in main (/usr/sbin/virtlogd+0x1a4db) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/logging/log_daemon.c:1162 #11 0x7f14aa3c2c56 in __libc_start_main (/usr/lib64/libc.so.6+0x25c56) ??:? #12 0x55ab07072639 in _start (/usr/sbin/virtlogd+0x1b639) ??:? 4. job->data->prog was not released Indirect leak of 64 byte(s) in 1 object(s) allocated from: #0 0x7f14ab932560 in calloc (/usr/local/gcc-6-4/lib64/libasan.so.3+0xc7560) ??:? #1 0x55ab07088d34 in virAllocVar (/usr/sbin/virtlogd+0x31d34) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/viralloc.c:560 #2 0x55ab070ac45c in virObjectNew (/usr/sbin/virtlogd+0x5545c) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/util/virobject.c:200 #3 0x55ab07074f06 in virNetServerProgramNew (/usr/sbin/virtlogd+0x1df06) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/rpc/virnetserverprogram.c:80 #4 0x55ab07071432 in main (/usr/sbin/virtlogd+0x1a432) /usr/src/debug/libvirt-3.2.0-529.x86_64/src/logging/log_daemon.c:1131 #5 0x7f14aa3c2c56 in __libc_start_main (/usr/lib64/libc.so.6+0x25c56) ??:? #6 0x55ab07072639 in _start (/usr/sbin/virtlogd+0x1b639) ??:? Signed-off-by: Jian wang <wangjian161(a)huawei.com> --- src/libvirt_private.syms | 1 + src/rpc/virnetserver.c | 12 ++++++++++++ src/util/virthreadpool.c | 9 +++++++++ src/util/virthreadpool.h | 2 ++ 4 files changed, 24 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index a6af44f..b23fc33 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -3316,6 +3316,7 @@ virThreadPoolGetMaxWorkers; virThreadPoolGetMinWorkers; virThreadPoolGetPriorityWorkers; virThreadPoolNewFull; +virThreadPoolSetFreeFunc; virThreadPoolSendJob; virThreadPoolSetParameters; diff --git a/src/rpc/virnetserver.c b/src/rpc/virnetserver.c index e0a2386..66f0fb6 100644 --- a/src/rpc/virnetserver.c +++ b/src/rpc/virnetserver.c @@ -188,6 +188,15 @@ virNetServerGetProgramLocked(virNetServerPtr srv, return NULL; } +static void virNetServerHandleFree(void *jobOpaque) +{ + virNetServerJobPtr job = jobOpaque; + + virObjectUnref(job->prog); + virNetMessageFree(job->msg); + VIR_FREE(job); +} + static void virNetServerDispatchNewMessage(virNetServerClientPtr client, virNetMessagePtr msg, @@ -376,6 +385,9 @@ virNetServerPtr virNetServerNew(const char *name, srv))) goto error; + if (srv->workers) + virThreadPoolSetFreeFunc(srv->workers, virNetServerHandleFree); + srv->name = g_strdup(name); srv->next_client_id = next_client_id; diff --git a/src/util/virthreadpool.c b/src/util/virthreadpool.c index 379d236..f885c04 100644 --- a/src/util/virthreadpool.c +++ b/src/util/virthreadpool.c @@ -54,6 +54,7 @@ struct _virThreadPool { bool quit; virThreadPoolJobFunc jobFunc; + virThreadPoolFreeFunc freeFunc; const char *jobName; void *jobOpaque; virThreadPoolJobList jobList; @@ -271,6 +272,12 @@ virThreadPoolNewFull(size_t minWorkers, } +void virThreadPoolSetFreeFunc(virThreadPoolPtr pool, + virThreadPoolFreeFunc func) +{ + pool->freeFunc = func; +} + void virThreadPoolFree(virThreadPoolPtr pool) { virThreadPoolJobPtr job; @@ -293,6 +300,8 @@ void virThreadPoolFree(virThreadPoolPtr pool) while ((job = pool->jobList.head)) { pool->jobList.head = pool->jobList.head->next; + if (pool->freeFunc) + pool->freeFunc(job->data); VIR_FREE(job); } diff --git a/src/util/virthreadpool.h b/src/util/virthreadpool.h index c97d9b3..d2e24f7 100644 --- a/src/util/virthreadpool.h +++ b/src/util/virthreadpool.h @@ -27,6 +27,7 @@ typedef struct _virThreadPool virThreadPool; typedef virThreadPool *virThreadPoolPtr; typedef void (*virThreadPoolJobFunc)(void *jobdata, void *opaque); +typedef void (*virThreadPoolFreeFunc)(void *jobdata); #define virThreadPoolNew(min, max, prio, func, opaque) \ virThreadPoolNewFull(min, max, prio, func, #func, opaque) @@ -46,6 +47,7 @@ size_t virThreadPoolGetFreeWorkers(virThreadPoolPtr pool); size_t virThreadPoolGetJobQueueDepth(virThreadPoolPtr pool); void virThreadPoolFree(virThreadPoolPtr pool); +void virThreadPoolSetFreeFunc(virThreadPoolPtr pool, virThreadPoolFreeFunc func); int virThreadPoolSendJob(virThreadPoolPtr pool, unsigned int priority, -- 2.23.0

5 5

[libvirt-glib PATCH] gitlab: introduce CI jobs testing git master & distro libvirt
by Daniel P. Berrangé 17 Jun '20

17 Jun '20

The glib build needs to validate two axis - A variety of libvirt versions - Native and mingw We test a variety of libvirt versions by running a build against the distro provided libvirt packages. All that is then missing is a build against the latest libvirt git master, which only needs to be run on a single distro, for which CentOS 8 is picked as a stable long life base. For mingw, we only test against libvirt git master, using Fedora rawhide. It is expected that any issues against older libvirt will have already been caught by the native builds. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- .gitlab-ci.yml | 228 ++++++++++++++++++ ci/libvirt-centos-7.Dockerfile | 88 +++++++ ci/libvirt-centos-8.Dockerfile | 64 +++++ ci/libvirt-centos-stream.Dockerfile | 58 +++++ ci/libvirt-debian-10.Dockerfile | 58 +++++ ci/libvirt-debian-9.Dockerfile | 61 +++++ ci/libvirt-debian-sid.Dockerfile | 58 +++++ ci/libvirt-fedora-31.Dockerfile | 55 +++++ ci/libvirt-fedora-32.Dockerfile | 55 +++++ ...rt-fedora-rawhide-cross-mingw32.Dockerfile | 133 ++++++++++ ...rt-fedora-rawhide-cross-mingw64.Dockerfile | 133 ++++++++++ ci/libvirt-fedora-rawhide.Dockerfile | 56 +++++ ci/libvirt-opensuse-151.Dockerfile | 57 +++++ ci/libvirt-ubuntu-1804.Dockerfile | 61 +++++ ci/libvirt-ubuntu-2004.Dockerfile | 58 +++++ ci/refresh | 36 +++ 16 files changed, 1259 insertions(+) create mode 100644 ci/libvirt-centos-7.Dockerfile create mode 100644 ci/libvirt-centos-8.Dockerfile create mode 100644 ci/libvirt-centos-stream.Dockerfile create mode 100644 ci/libvirt-debian-10.Dockerfile create mode 100644 ci/libvirt-debian-9.Dockerfile create mode 100644 ci/libvirt-debian-sid.Dockerfile create mode 100644 ci/libvirt-fedora-31.Dockerfile create mode 100644 ci/libvirt-fedora-32.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile create mode 100644 ci/libvirt-fedora-rawhide.Dockerfile create mode 100644 ci/libvirt-opensuse-151.Dockerfile create mode 100644 ci/libvirt-ubuntu-1804.Dockerfile create mode 100644 ci/libvirt-ubuntu-2004.Dockerfile create mode 100755 ci/refresh diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 50dae92..da1e588 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,6 +1,97 @@ stages: - prebuild + - containers + - builds + - docs + +.container_job_template: &container_job_definition + image: docker:stable + stage: containers + services: + - docker:dind + before_script: + - export TAG="$CI_REGISTRY_IMAGE/ci-$NAME:latest" + - export COMMON_TAG="$CI_REGISTRY/libvirt/libvirt-glib/ci-$NAME:latest" + - docker info + - docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" + script: + - docker pull "$TAG" || docker pull "$COMMON_TAG" || true + - docker build --cache-from "$TAG" --cache-from "$COMMON_TAG" --tag "$TAG" -f "ci/libvirt-$NAME.Dockerfile" ci + - docker push "$TAG" + after_script: + - docker logout + +.script_variables: &script_variables | + export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)" + export VROOT="$SCRATCH_DIR/vroot" + export CCACHE_BASEDIR="$(pwd)" + export CCACHE_DIR="$CCACHE_BASEDIR/ccache" + export CCACHE_MAXSIZE="500M" + export PATH="$CCACHE_WRAPPERSDIR:$VROOT/bin:$PATH" + export SCRATCH_DIR="/tmp/scratch" + export PKG_CONFIG_PATH="$VROOT/lib/pkgconfig" + +.git_native_build_job_template: &git_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables + - export LD_LIBRARY_PATH="$VROOT/lib" + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh --prefix="$VROOT" --without-libvirtd + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild --nodeps -ta libvirt-glib*.tar.gz ; fi + +.dist_native_build_job_template: &dist_native_build_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + before_script: + - *script_variables + script: + - mkdir build + - cd build + - ../autogen.sh --prefix="$VROOT" + - $MAKE install + - $MAKE dist + - if test -x /usr/bin/rpmbuild && test "$RPM" != "skip" ; then rpmbuild -ta libvirt-glib*.tar.gz ; fi + +# Default cross build jobs that are always run +.git_cross_build_default_job_template: &git_cross_build_default_job_definition + image: $CI_REGISTRY_IMAGE/ci-$NAME:latest + stage: builds + cache: + paths: + - ccache/ + key: "$CI_JOB_NAME" + before_script: + - *script_variables + script: + - pushd "$PWD" + - mkdir -p "$SCRATCH_DIR" + - cd "$SCRATCH_DIR" + - git clone --depth 1 https://gitlab.com/libvirt/libvirt.git + - mkdir libvirt/build + - cd libvirt/build + - ../autogen.sh $CONFIGURE_OPTS --prefix="$VROOT" + - $MAKE install + - popd + - mkdir build + - cd build + - ../autogen.sh $CONFIGURE_OPTS || (cat config.log && exit 1) + - $MAKE # Check that all commits are signed-off for the DCO. # Skip on "libvirt" namespace, since we only need to run @@ -14,3 +105,140 @@ check-dco: except: variables: - $CI_PROJECT_NAMESPACE == 'libvirt' + +x64-centos-7-container: + <<: *container_job_definition + variables: + NAME: centos-7 + +x64-centos-8-container: + <<: *container_job_definition + variables: + NAME: centos-8 + +x64-debian-9-container: + <<: *container_job_definition + variables: + NAME: debian-9 + +x64-debian-10-container: + <<: *container_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-container: + <<: *container_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-container: + <<: *container_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-container: + <<: *container_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-container: + <<: *container_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-container: + <<: *container_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-container: + <<: *container_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-container: + <<: *container_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 + + + +x64-centos-8-git-build: + <<: *git_native_build_job_definition + variables: + NAME: centos-8 + +x64-centos-7-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-7 + +x64-centos-8-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: centos-8 + +x64-debian-9-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-9 + +x64-debian-10-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-10 + +x64-debian-sid-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: debian-sid + +x64-fedora-31-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-31 + +x64-fedora-32-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-32 + +x64-fedora-rawhide-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: fedora-rawhide + +x64-opensuse-151-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: opensuse-151 + +x64-ubuntu-1804-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-1804 + +x64-ubuntu-2004-dist-build: + <<: *dist_native_build_job_definition + variables: + NAME: ubuntu-2004 + +mingw32-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw32 + +mingw64-fedora-rawhide-git-build: + <<: *git_cross_build_default_job_definition + variables: + NAME: fedora-rawhide-cross-mingw64 diff --git a/ci/libvirt-centos-7.Dockerfile b/ci/libvirt-centos-7.Dockerfile new file mode 100644 index 0000000..ad6b262 --- /dev/null +++ b/ci/libvirt-centos-7.Dockerfile @@ -0,0 +1,88 @@ +FROM centos:7 + +RUN echo -e '[openvz]\n\ +name=OpenVZ addons\n\ +baseurl=https://download.openvz.org/virtuozzo/releases/openvz-7.0.11-235/x86_64/os/\n\ +enabled=1\n\ +gpgcheck=1\n\ +skip_if_unavailable=0\n\ +metadata_expire=6h\n\ +priority=90\n\ +includepkgs=libprl*' > /etc/yum.repos.d/openvz.repo && \ + echo -e '-----BEGIN PGP PUBLIC KEY BLOCK-----\n\ +Version: GnuPG v2.0.22 (GNU/Linux)\n\ +\n\ +mI0EVl80nQEEAKrEeyeTCwrzS9kYedZ/sAc/GUqlb81C7pA9SaR3fyck5mVw1Ogk\n\ +YdmNBPM2kY7QDxR9F0EpSpnxSCAXZXugsQ8KzZ0DRLVeBDQyGs9IGK5hI0zzxIil\n\ +BzfvIexLiQQhLy7YlIi8Jt/uUqKkW0pIMNMGcduY97VATtczpncpkmSzABEBAAG0\n\ +SFZpcnR1b3p6byBUZWFtIChHUEcga2V5IHNpZ25hdHVyZSBmb3IgcGFja2FnZXMp\n\ +IDxzZWN1cml0eUB2aXJ0dW96em8uY29tPoi5BBMBAgAjBQJWXzSdAhsDBwsJCAcD\n\ +AgEGFQgCCQoLBBYCAwECHgECF4AACgkQygt9GUTNrSruIgP/er70Eyo73A1gfrjv\n\ +oPUkyo4rslVRZu3qqCwoMFtJc/Z/UxWgEka1buorlcGLa6eO/EZ49c0n+KGa4Kvt\n\ +EUboIq0yEu5i0FyAj92ifm+hNhoAbGfm0cZ4/fD0oGr3l8OsQo4+iHX4xAPwFe7Y\n\ +zABuB8I1ZDZ4OIp5tDfTTuF2LT24jQRWXzSdAQQAog2Aqb+Ptl68O7cQhWLjVGkj\n\ +yyigZrdeReLx3HloKJPBeQ/kA6uvMJc/IYS3uppMWXv9v+QenS6uhP1TUJ2k9FvM\n\ +t94MQZfALN7Vpf8AF+UeWu4Ru+y4BNzcFhrPhIFNFChOR2QqW6FkgE57D9I177NC\n\ +oJMyrlNe8wcGa178An8AEQEAAYifBBgBAgAJBQJWXzSdAhsMAAoJEMoLfRlEza0q\n\ +bKwD/3+OFVIEXnIv5XgdGRNX5fHggsUN1bb8gva7HANRlKdd4LD8foDM3F/yv/3V\n\ +igG14D5EjKz56SaBDNgiI4++hOzb2M8jhAsR86jxkXFrrP1U3ZNRKg6av9DPFAPS\n\ +WEiJKtQrZDJloqtyi/mmRa1VsV7RYR0VPJjhK/R8EQ7Ysshy\n\ +=fRMg\n\ +-----END PGP PUBLIC KEY BLOCK-----' > /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-OpenVZ && \ + yum install -y epel-release && \ + yum update -y && \ + yum install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-common \ + glibc-devel \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + yum autoremove -y && \ + yum clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja-build" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-centos-8.Dockerfile b/ci/libvirt-centos-8.Dockerfile new file mode 100644 index 0000000..360cf08 --- /dev/null +++ b/ci/libvirt-centos-8.Dockerfile @@ -0,0 +1,64 @@ +FROM centos:8 + +RUN dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + libnl3-devel \ + libtirpc-devel \ + libtool \ + libvirt-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-docutils \ + python3-setuptools \ + python3-wheel \ + rpcgen \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-centos-stream.Dockerfile b/ci/libvirt-centos-stream.Dockerfile new file mode 100644 index 0000000..6f6c830 --- /dev/null +++ b/ci/libvirt-centos-stream.Dockerfile @@ -0,0 +1,58 @@ +FROM centos:8 + +RUN dnf install -y centos-release-stream && \ + dnf install 'dnf-command(config-manager)' -y && \ + dnf config-manager --set-enabled -y Stream-PowerTools && \ + dnf install -y epel-release && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-10.Dockerfile b/ci/libvirt-debian-10.Dockerfile new file mode 100644 index 0000000..f55fe06 --- /dev/null +++ b/ci/libvirt-debian-10.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:10 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-9.Dockerfile b/ci/libvirt-debian-9.Dockerfile new file mode 100644 index 0000000..182b3a6 --- /dev/null +++ b/ci/libvirt-debian-9.Dockerfile @@ -0,0 +1,61 @@ +FROM debian:9 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-debian-sid.Dockerfile b/ci/libvirt-debian-sid.Dockerfile new file mode 100644 index 0000000..fbb1cb6 --- /dev/null +++ b/ci/libvirt-debian-sid.Dockerfile @@ -0,0 +1,58 @@ +FROM debian:sid + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-31.Dockerfile b/ci/libvirt-fedora-31.Dockerfile new file mode 100644 index 0000000..e23f6d5 --- /dev/null +++ b/ci/libvirt-fedora-31.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:31 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-32.Dockerfile b/ci/libvirt-fedora-32.Dockerfile new file mode 100644 index 0000000..1da98c3 --- /dev/null +++ b/ci/libvirt-fedora-32.Dockerfile @@ -0,0 +1,55 @@ +FROM fedora:32 + +RUN dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile b/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile new file mode 100644 index 0000000..c366c5f --- /dev/null +++ b/ci/libvirt-fedora-rawhide-cross-mingw32.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/i686-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw32-curl \ + mingw32-dbus \ + mingw32-dlfcn \ + mingw32-gcc \ + mingw32-gettext \ + mingw32-glib2 \ + mingw32-gnutls \ + mingw32-libssh2 \ + mingw32-libxml2 \ + mingw32-openssl \ + mingw32-pkg-config \ + mingw32-portablexdr \ + mingw32-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "i686-w64-mingw32" +ENV CONFIGURE_OPTS "--host=i686-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw32.meson" diff --git a/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile b/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile new file mode 100644 index 0000000..1c5c07c --- /dev/null +++ b/ci/libvirt-fedora-rawhide-cross-mingw64.Dockerfile @@ -0,0 +1,133 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + audit-libs-devel \ + augeas \ + autoconf \ + automake \ + avahi-devel \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + cyrus-sasl-devel \ + dbus-devel \ + device-mapper-devel \ + dnsmasq \ + dwarves \ + ebtables \ + fuse-devel \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + glusterfs-api-devel \ + gnutls-devel \ + gobject-introspection-devel \ + gtk-doc \ + iproute \ + iproute-tc \ + iscsi-initiator-utils \ + kmod \ + libacl-devel \ + libattr-devel \ + libblkid-devel \ + libcap-ng-devel \ + libcurl-devel \ + libiscsi-devel \ + libnl3-devel \ + libpcap-devel \ + libpciaccess-devel \ + librbd-devel \ + libselinux-devel \ + libssh-devel \ + libssh2-devel \ + libtirpc-devel \ + libtool \ + libudev-devel \ + libwsman-devel \ + libxml2 \ + libxml2-devel \ + libxslt \ + lsof \ + lvm2 \ + make \ + meson \ + ncurses-devel \ + net-tools \ + netcf-devel \ + nfs-utils \ + ninja-build \ + numactl-devel \ + numad \ + parted \ + parted-devel \ + patch \ + perl \ + pkgconfig \ + polkit \ + python3 \ + python3-docutils \ + python3-flake8 \ + python3-setuptools \ + python3-wheel \ + qemu-img \ + radvd \ + readline-devel \ + rpcgen \ + rpm-build \ + sanlock-devel \ + screen \ + scrub \ + sheepdog \ + strace \ + sudo \ + systemtap-sdt-devel \ + vala \ + vim \ + wireshark-devel \ + xen-devel \ + xfsprogs-devel \ + yajl-devel \ + zfs-fuse && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/x86_64-w64-mingw32-$(basename /usr/bin/gcc) + +RUN dnf install -y \ + mingw64-curl \ + mingw64-dbus \ + mingw64-dlfcn \ + mingw64-gcc \ + mingw64-gettext \ + mingw64-glib2 \ + mingw64-gnutls \ + mingw64-libssh2 \ + mingw64-libxml2 \ + mingw64-openssl \ + mingw64-pkg-config \ + mingw64-portablexdr \ + mingw64-readline && \ + dnf clean all -y + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" + +ENV ABI "x86_64-w64-mingw32" +ENV CONFIGURE_OPTS "--host=x86_64-w64-mingw32" +ENV MESON_OPTS "--cross-file=/usr/share/mingw/toolchain-mingw64.meson" diff --git a/ci/libvirt-fedora-rawhide.Dockerfile b/ci/libvirt-fedora-rawhide.Dockerfile new file mode 100644 index 0000000..7017282 --- /dev/null +++ b/ci/libvirt-fedora-rawhide.Dockerfile @@ -0,0 +1,56 @@ +FROM fedora:rawhide + +RUN dnf update -y --nogpgcheck fedora-gpg-keys && \ + dnf update -y && \ + dnf install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-langpack-en \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + dnf autoremove -y && \ + dnf clean all -y && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-opensuse-151.Dockerfile b/ci/libvirt-opensuse-151.Dockerfile new file mode 100644 index 0000000..87237d4 --- /dev/null +++ b/ci/libvirt-opensuse-151.Dockerfile @@ -0,0 +1,57 @@ +FROM opensuse/leap:15.1 + +RUN zypper update -y && \ + zypper install -y \ + autoconf \ + automake \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + cppi \ + gcc \ + gdb \ + gettext \ + gettext-devel \ + git \ + glib2-devel \ + glibc-devel \ + glibc-locale \ + gobject-introspection-devel \ + gtk-doc \ + libtool \ + libvirt-devel \ + libxml2-devel \ + lsof \ + make \ + net-tools \ + ninja \ + patch \ + perl \ + pkgconfig \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + rpm-build \ + screen \ + strace \ + sudo \ + vala \ + vim && \ + zypper clean --all && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-ubuntu-1804.Dockerfile b/ci/libvirt-ubuntu-1804.Dockerfile new file mode 100644 index 0000000..2b8bd41 --- /dev/null +++ b/ci/libvirt-ubuntu-1804.Dockerfile @@ -0,0 +1,61 @@ +FROM ubuntu:18.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-pip \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +RUN pip3 install \ + meson==0.49.0 + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/libvirt-ubuntu-2004.Dockerfile b/ci/libvirt-ubuntu-2004.Dockerfile new file mode 100644 index 0000000..38920f0 --- /dev/null +++ b/ci/libvirt-ubuntu-2004.Dockerfile @@ -0,0 +1,58 @@ +FROM ubuntu:20.04 + +RUN export DEBIAN_FRONTEND=noninteractive && \ + apt-get update && \ + apt-get dist-upgrade -y && \ + apt-get install --no-install-recommends -y \ + autoconf \ + automake \ + autopoint \ + bash \ + bash-completion \ + ca-certificates \ + ccache \ + chrony \ + gcc \ + gdb \ + gettext \ + git \ + gtk-doc-tools \ + libc6-dev \ + libgirepository1.0-dev \ + libglib2.0-dev \ + libtool \ + libtool-bin \ + libvirt-dev \ + libxml2-dev \ + locales \ + lsof \ + make \ + meson \ + net-tools \ + ninja-build \ + patch \ + perl \ + pkgconf \ + python3 \ + python3-setuptools \ + python3-wheel \ + screen \ + strace \ + sudo \ + valac \ + vim && \ + apt-get autoremove -y && \ + apt-get autoclean -y && \ + sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \ + dpkg-reconfigure locales && \ + mkdir -p /usr/libexec/ccache-wrappers && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \ + ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/$(basename /usr/bin/gcc) + +ENV LANG "en_US.UTF-8" + +ENV MAKE "/usr/bin/make" +ENV NINJA "/usr/bin/ninja" +ENV PYTHON "/usr/bin/python3" + +ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" diff --git a/ci/refresh b/ci/refresh new file mode 100755 index 0000000..dfbfa87 --- /dev/null +++ b/ci/refresh @@ -0,0 +1,36 @@ +#!/bin/sh + +if test -z "$1" +then + echo "syntax: $0 PATH-TO-LCITOOL" + exit 1 +fi + +LCITOOL=$1 + +if ! test -x "$LCITOOL" +then + echo "$LCITOOL is not executable" + exit 1 +fi + +HOSTS=$($LCITOOL hosts | grep -v freebsd) + +for host in $HOSTS +do + + if test "$host" = "libvirt-fedora-rawhide" + then + for cross in mingw32 mingw64 + do + $LCITOOL dockerfile $host libvirt,libvirt-glib --cross $cross >$host-cross-$cross.Dockerfile + done + fi + + if test "$host" = "libvirt-centos-8" + then + $LCITOOL dockerfile $host libvirt+minimal,libvirt+dist,libvirt-glib > $host.Dockerfile + else + $LCITOOL dockerfile $host libvirt+dist,libvirt-glib > $host.Dockerfile + fi +done -- 2.26.2

2 3

[PATCH] virsh: cmdBlockcopy: Remove 'error:' prefix for an empty line
by Peter Krempa 17 Jun '20

17 Jun '20

When a block copy job fails prior to reaching the synchronized phase while we are waiting for the job to finish virsh would print the following: $ virsh blockcopy backup-test vda /tmp/dst.qcow2 --wait --reuse-external --transient-job error: Copy failed The above message looks like we've forgot to print the error message itself as the line ends after 'error:'. Unfortunately with the current API design clients have no way of actually getting the error message as the VIR_DOMAIN_EVENT_ID_BLOCK_JOB(_2) event only reports the status but not an error and the job then vanishes. Fix the expectations by using vshPrintExtra instead of vshError: $ virsh blockcopy backup-test vda /tmp/dst.qcow2 --wait --reuse-external --transient-job Copy failed Note that the newline is required to avoid printing the 'Copy failed' message on the same line when printing the job progress percentage. Inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1847867 Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- tools/virsh-domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 5222949566..3597fb6272 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -2457,7 +2457,7 @@ cmdBlockcopy(vshControl *ctl, const vshCmd *cmd) break; case VIR_DOMAIN_BLOCK_JOB_FAILED: - vshError(ctl, "\n%s", _("Copy failed")); + vshPrintExtra(ctl, "\n%s", _("Copy failed")); goto cleanup; break; -- 2.26.2

3 4

[PATCH] utils: check flags and errno before reporting errno
by Bihong Yu 17 Jun '20

17 Jun '20

>From 099707463944faeae75da640b0d1d780cb675406 Mon Sep 17 00:00:00 2001 From: Bihong Yu <yubihong(a)huawei.com> Date: Tue, 16 Jun 2020 22:08:55 +0800 Subject: [PATCH] utils: check flags and errno before reporting errno There are races condiction to make '/run/libvirt/qemu/dbus' directory in virDirCreateNoFork() while concurrent start VMs, and get "failed to create directory '/run/libvirt/qemu/dbus': File exists" error message. Check flags and errno before reporting errno after mkdir(). Signed-off-by:Bihong Yu <yubihong(a)huawei.com> Reviewed-by:Chuan Zheng <zhengchuan(a)huawei.com> Reviewed-by:alexchen <alex.chen(a)huawei.com> --- src/util/virfile.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index 20260a2..96ac4e5 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -2614,12 +2614,15 @@ virDirCreateNoFork(const char *path, if (!((flags & VIR_DIR_CREATE_ALLOW_EXIST) && virFileExists(path))) { if (mkdir(path, mode) < 0) { - ret = -errno; - virReportSystemError(errno, _("failed to create directory '%s'"), - path); - goto error; + if (!((flags & VIR_DIR_CREATE_ALLOW_EXIST) && errno == EEXIST)) { + ret = -errno; + virReportSystemError(errno, _("failed to create directory '%s'"), + path); + goto error; + } + } else { + created = true; } - created = true; } if (stat(path, &st) == -1) { -- 1.8.3.1

2 1

[libvirt PATCH] spec: Require Fedora 31
by Andrea Bolognani 17 Jun '20

17 Jun '20

Fedora 30 has been EOL for almost a month now. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- libvirt.spec.in | 2 +- mingw-libvirt.spec.in | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/libvirt.spec.in b/libvirt.spec.in index 262e66f3cc..450c97b46d 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -4,7 +4,7 @@ # that's still supported by the vendor. It may work on other distros # or versions, but no effort will be made to ensure that going forward. %define min_rhel 7 -%define min_fedora 30 +%define min_fedora 31 %if (0%{?fedora} && 0%{?fedora} >= %{min_fedora}) || (0%{?rhel} && 0%{?rhel} >= %{min_rhel}) %define supported_platform 1 diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index 94d88e2edc..96c6624ca4 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -3,7 +3,7 @@ # This spec file assumes you are building on a Fedora version # that's still supported by the vendor. It may work on other distros # or versions, but no effort will be made to ensure that going forward. -%define min_fedora 30 +%define min_fedora 31 %if 0%{?fedora} && 0%{?fedora} >= %{min_fedora} %define supported_platform 1 -- 2.25.4

2 1

[libvirt PATCH] ci: Makefile: Fix 'overridde' typo
by Erik Skultety 17 Jun '20

17 Jun '20

Signed-off-by: Erik Skultety <eskultet(a)redhat.com> --- Pushed as trivial. ci/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/Makefile b/ci/Makefile index a6a65bdc0f..7840cc8d89 100644 --- a/ci/Makefile +++ b/ci/Makefile @@ -48,7 +48,7 @@ CI_PREPARE_SCRIPT = $(CI_ROOTDIR)/prepare.sh CI_BUILD_SCRIPT = $(CI_ROOTDIR)/build.sh # Location of the container images we're going to pull -# Can be useful to overridde to use a locally built +# Can be useful to override to use a locally built # image instead CI_IMAGE_PREFIX = registry.gitlab.com/libvirt/libvirt/ci- -- 2.26.2

1 0

[libvirt] Request for a new release of libvirt-java in Maven
by Slavka Peleva 17 Jun '20

17 Jun '20

Hello, The Apache CloudStack project is using Maven libvirt java bindings in KVM code. In libvirt-java project were added qemu methods that will prevent the usage of bash scripts or Process class methods. It will be great to use those features in CloudStack. >From what I see, the previous release was requested in this list, and there doesn’t seem to be a formal procedure for this, so how should I proceed? Best regards and thank you in advance, Slavka

1 1

[libvirt PATCH] utils: add mutex to avoid races in virfile
by Bihong Yu 17 Jun '20

17 Jun '20

>From d9f7ed2af581222804392f9b93dc6aaf7d8c8995 Mon Sep 17 00:00:00 2001 From: Bihong Yu <yubihong(a)huawei.com> Date: Tue, 16 Jun 2020 22:08:55 +0800 Subject: [PATCH] utils: add mutex to avoid races in virfile There are races condiction to make '/run/libvirt/qemu/dbus' directory in virDirCreateNoFork() while concurrent start VMs, and get "failed to create directory '/run/libvirt/qemu/dbus': File exists" error message. Add an mutex to avoid races. Signed-off-by:Bihong Yu <yubihong(a)huawei.com> Reviewed-by:Chuan Zheng <zhengchuan(a)huawei.com> Reviewed-by:alexchen <alex.chen(a)huawei.com> --- src/util/virfile.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/util/virfile.c b/src/util/virfile.c index 20260a2..ae02a6e 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -88,6 +88,7 @@ #include "virstring.h" #include "virutil.h" #include "virsocket.h" +#include "virthread.h" #define VIR_FROM_THIS VIR_FROM_NONE @@ -108,6 +109,9 @@ VIR_LOG_INIT("util.file"); # define O_DIRECT 0 #endif +/* Global mutex to avoid races */ +virMutex fileLock = VIR_MUTEX_INITIALIZER; + int virFileClose(int *fdptr, virFileCloseFlags flags) { int saved_errno = 0; @@ -2612,15 +2616,18 @@ virDirCreateNoFork(const char *path, struct stat st; bool created = false; + virMutexLock(&fileLock); if (!((flags & VIR_DIR_CREATE_ALLOW_EXIST) && virFileExists(path))) { if (mkdir(path, mode) < 0) { ret = -errno; virReportSystemError(errno, _("failed to create directory '%s'"), path); + virMutexUnlock(&fileLock); goto error; } created = true; } + virMutexUnlock(&fileLock); if (stat(path, &st) == -1) { ret = -errno; -- 1.8.3.1

2 2

[PATCH] Fixed missing VM vport when batch start or migration partially failed
by gongwei＠smartx.com 17 Jun '20

17 Jun '20

From: gongwei <gongwei(a)smartx.com> start to failed will not remove the openvswitch port, the port recycling in this case lets openvswitch handle it by itself Signed-off-by: gongwei <gongwei(a)smartx.com> --- src/qemu/qemu_process.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index d36088ba98..439bd5b396 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -7482,7 +7482,8 @@ void qemuProcessStop(virQEMUDriverPtr driver, if (vport) { if (vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_MIDONET) { ignore_value(virNetDevMidonetUnbindPort(vport)); - } else if (vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_OPENVSWITCH) { + } else if (vport->virtPortType == VIR_NETDEV_VPORT_PROFILE_OPENVSWITCH && + reason != VIR_DOMAIN_SHUTOFF_FAILED) { ignore_value(virNetDevOpenvswitchRemovePort( virDomainNetGetActualBridgeName(net), net->ifname)); -- 2.18.2

5 5

[libvirt PATCH v3 00/11] Add ability to create mediated devices in libvirt
by Jonathon Jongsma 16 Jun '20

16 Jun '20

This is the first portion of an effort to support persistent mediated devices with libvirt. This first series simply enables creating and destroying non-persistent mediated devices via the virNodeDeviceCreateXML() and virNodeDeviceDestroy() functions. The 'mdevctl' utility[1] provides the backend implementation. Changes in v3: - Includes all of the changes mentioned by Michal - makes iommuGroup element optional for the nodedev schema (per Erik) - substitutes a placeholder for the path to the mdevctl binary before comparing to the expected output [1] https://github.com/mdevctl/mdevctl Jonathon Jongsma (11): nodedev: make iommuGroup optional for mdevs nodedev: factor out nodeDeviceHasCapability() nodedev: add support for mdev attributes nodedev: refactor nodeDeviceFindNewDevice() nodedev: store mdev UUID in mdev caps nodedev: add mdev support to virNodeDeviceCreateXML() nodedev: Build a non-loadable driver lib nodedev: Add testing for 'mdevctl start' nodedev: add mdev support to virNodeDeviceDestroy() nodedev: Add testing for 'mdevctl stop' docs: note node device fields that are read-only build-aux/syntax-check.mk | 2 +- docs/formatnode.html.in | 21 +- docs/schemas/nodedev.rng | 18 +- libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 3 + src/conf/node_device_conf.c | 64 +++- src/conf/node_device_conf.h | 3 + src/conf/virnodedeviceobj.c | 34 ++ src/conf/virnodedeviceobj.h | 3 + src/libvirt_private.syms | 3 + src/node_device/Makefile.inc.am | 23 +- src/node_device/node_device_driver.c | 360 ++++++++++++++++-- src/node_device/node_device_driver.h | 8 + src/node_device/node_device_udev.c | 5 +- src/util/virmdev.c | 12 + src/util/virmdev.h | 11 + tests/Makefile.am | 14 + ...019_36ea_4111_8f0a_8c9a70e21366-start.argv | 1 + ...019_36ea_4111_8f0a_8c9a70e21366-start.json | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.argv | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.json | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.argv | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.json | 1 + tests/nodedevmdevctldata/mdevctl-stop.argv | 1 + tests/nodedevmdevctltest.c | 304 +++++++++++++++ ...v_d069d019_36ea_4111_8f0a_8c9a70e21366.xml | 7 + ...v_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml | 9 + ...v_fedc4916_1ca8_49ac_b176_871d16c13076.xml | 8 + 28 files changed, 853 insertions(+), 68 deletions(-) create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.json create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.json create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.json create mode 100644 tests/nodedevmdevctldata/mdevctl-stop.argv create mode 100644 tests/nodedevmdevctltest.c create mode 100644 tests/nodedevschemadata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366.xml create mode 100644 tests/nodedevschemadata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml create mode 100644 tests/nodedevschemadata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076.xml -- 2.21.3

3 29

[RFC] Expanding hooks API for running several scripts by one call virHookCall
by Dmitry Nesterenko 16 Jun '20

16 Jun '20

Hi, all. I have the proposal about subject. Libvirt hook calls only one script (/etc/libvirt/hooks/<driver>) now. This is not convenient if scripts for hook are provided by many vendors. Script one vendor can replace previously installed script other vendor. Let's use usual linux scheme - running all scripts from directory /etc/libvirt/hooks/<driver>.d in alphabetical order. If we find script in old place we will execute it before others in subdirectory for backward compatibility. I want to implement patch for this scheme. What is your opinion? Dmitry

3 2

[PATCH 1/1] manpages/virsh.rst: clarify numatune memory migration on Linux
by Daniel Henrique Barboza 16 Jun '20

16 Jun '20

On Linux, changing the nodeset on 'numatune' does not imply that the guest memory will be migrated on the spot to the new nodeset. The memory migration is tied on guest usage of the memory pages, and an idle guest will take longer to have its memory migrated to the new nodeset. This is a behavior explained in detail in the Linux kernel documentation in Documentation/admin-guide/cgroup-v1/cpusets.rst. The user doesn't need this level of detail though - just needs his/her expectations under check. Running 'numastat' and hoping for instant memory migration from the previous nodeset to the new one is not viable. There's also parts of the memory that are locked by QEMU in the same place, e.g. when VFIO devices are present. Let's also mention it as another factor that impacts the results the user might expect from NUMA memory migration with numatune. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- docs/manpages/virsh.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst index 969a4d5543..ff32338f43 100644 --- a/docs/manpages/virsh.rst +++ b/docs/manpages/virsh.rst @@ -3395,6 +3395,13 @@ If *--live* is specified, set scheduler information of a running guest. If *--config* is specified, affect the next boot of a persistent guest. If *--current* is specified, affect the current guest state. +For running guests in Linux hosts, the changes made in the domain's +numa parameters does not imply that the guest memory will be moved to a +different nodeset immediately. The memory migration depends on the +guest activity, and the memory of an idle guest will remain in its +previous nodeset for longer. The presence of VFIO devices will also +lock parts of the guest memory in the same nodeset used to start the +guest, regardless of nodeset changes. perf ---- -- 2.26.2

2 3

[PATCH 1/1] qemuDomainSetNumaParamsLive: set nodeset for root cgroup
by Daniel Henrique Barboza 16 Jun '20

16 Jun '20

This function handles the change of NUMA nodeset for a given guest, setting CpusetMems for the emulator, vcpus and IOThread sub-groups. It doesn't set the same nodeset to the root cgroup though. This means that cpuset.mems of the root cgroup ends up holding the new nodeset and the old nodeset as well. For a guest with placement=strict, nodeset='0', doing virsh numatune <vm> 0 8 --live Will make cpuset.mems of emulator, vcpus and iothread to be "8", but cpuset.mems of the root cgroup will be "0,8". This means that any new tasks that ends up landing in the root cgroup, aside from the emulator/vcpus/iothread sub-groups, will be split between the old nodeset and the new nodeset, which is not what we want. Signed-off-by: Daniel Henrique Barboza <danielhb413(a)gmail.com> --- src/qemu/qemu_driver.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 88517ba6a7..59d322c8f3 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -9702,6 +9702,10 @@ qemuDomainSetNumaParamsLive(virDomainObjPtr vm, virCgroupFree(&cgroup_temp); } + /* set nodeset for root cgroup */ + if (virCgroupSetCpusetMems(priv->cgroup, nodeset_str) < 0) + goto cleanup; + ret = 0; cleanup: virCgroupFree(&cgroup_temp); -- 2.26.2

2 1

[PATCH] virsysinfo: Don't leak fw_cfg
by Michal Privoznik 16 Jun '20

16 Jun '20

In v6.4.0-72-g3dda889a44 I've introduced parsing and formatting of new sysinfo type 'fwcfg'. However, I've forgot to introduce code that would free parsed data. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/util/virsysinfo.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/util/virsysinfo.c b/src/util/virsysinfo.c index 52e576f129..db84339bda 100644 --- a/src/util/virsysinfo.c +++ b/src/util/virsysinfo.c @@ -131,6 +131,19 @@ void virSysinfoOEMStringsDefFree(virSysinfoOEMStringsDefPtr def) VIR_FREE(def); } + +static void +virSysinfoFWCfgDefClear(virSysinfoFWCfgDefPtr def) +{ + if (!def) + return; + + VIR_FREE(def->name); + VIR_FREE(def->value); + VIR_FREE(def->file); +} + + /** * virSysinfoDefFree: * @def: a sysinfo structure @@ -184,6 +197,10 @@ void virSysinfoDefFree(virSysinfoDefPtr def) virSysinfoOEMStringsDefFree(def->oemStrings); + for (i = 0; i < def->nfw_cfgs; i++) + virSysinfoFWCfgDefClear(&def->fw_cfgs[i]); + VIR_FREE(def->fw_cfgs); + VIR_FREE(def); } -- 2.26.2

2 1

[PATCH 0/9] Some Coverity memory leak patches
by John Ferlan 16 Jun '20

16 Jun '20

Even though I changed roles, the cron job still runs. I recently updated to a newer coverity version and it found some new stuff and even more false positives. It's still very unhappy with the usage of GLIB_DEPRECATED_ENUMERATOR_IN_* macros for *glib/gspawn.h and *gobject/gparam.h, but I can work around that. I had to disable the use_after_free type checking because the virObject to GObject change just caused a large number of possible false positives for virObjectUnref usages (coverity cannot keep track of all the counts). Anyway, I had a few spare cycles, so I figured I'd pass along at least the memory leak and usage things that were found. I haven't posted in a while so hopefully I haven't missed some subtlety. FWIW: I no longer have commit access so I'm at the mercy of someone else pushing my patches now. John Ferlan (9): util: Fix memory leak in virAuthGetCredential util: Fix memory leak in virAuthConfigLookup lxc: Fix memory leak in virLXCControllerPopulateDevices util: Fix memory leak in virPCIProbeStubDriver test: Fix memory leak in testParseXMLDocFromFile conf: Fix memory leak in openvzWriteConfigParam conf: Fix memory leak in openvzReadFSConf conf: Fix memory leak in virCPUDefParseXML util: Avoid using wrong free function src/conf/cpu_conf.c | 2 +- src/lxc/lxc_controller.c | 3 ++- src/openvz/openvz_conf.c | 7 ++----- src/remote/remote_driver.c | 2 +- src/test/test_driver.c | 2 +- src/util/iohelper.c | 7 +++++++ src/util/virauth.c | 5 +---- src/util/virauthconfig.c | 4 ++-- src/util/virauthconfig.h | 2 +- src/util/virpci.c | 1 + tests/virauthconfigtest.c | 2 +- 11 files changed, 20 insertions(+), 17 deletions(-) -- 2.25.4

3 16

[libvirt PATCH 0/5] Avoid deprecated commands for migration parameters
by Jiri Denemark 16 Jun '20

16 Jun '20

Jiri Denemark (5): qemu: Probe for a few params supported by migrate-set-parameters qemu: Avoid deprecated migrate_set_speed QMP command qemu: Avoid deprecated migrate_set_downtime QMP command qemu: Avoid deprecated query-migrate-cache-size QMP command qemu: Avoid deprecated migrate-set-cache-size QMP command src/qemu/qemu_capabilities.c | 8 ++ src/qemu/qemu_capabilities.h | 5 + src/qemu/qemu_driver.c | 92 +++++++++++++++---- src/qemu/qemu_migration.c | 53 +++++++++-- src/qemu/qemu_migration_params.c | 3 +- .../caps_2.10.0.aarch64.xml | 2 + .../caps_2.10.0.ppc64.xml | 2 + .../caps_2.10.0.s390x.xml | 2 + .../caps_2.10.0.x86_64.xml | 2 + .../caps_2.11.0.s390x.xml | 3 + .../caps_2.11.0.x86_64.xml | 3 + .../caps_2.12.0.aarch64.xml | 3 + .../caps_2.12.0.ppc64.xml | 3 + .../caps_2.12.0.s390x.xml | 3 + .../caps_2.12.0.x86_64.xml | 3 + .../qemucapabilitiesdata/caps_2.8.0.s390x.xml | 2 + .../caps_2.8.0.x86_64.xml | 2 + .../qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 2 + .../qemucapabilitiesdata/caps_2.9.0.s390x.xml | 2 + .../caps_2.9.0.x86_64.xml | 2 + .../qemucapabilitiesdata/caps_3.0.0.ppc64.xml | 3 + .../caps_3.0.0.riscv32.xml | 3 + .../caps_3.0.0.riscv64.xml | 3 + .../qemucapabilitiesdata/caps_3.0.0.s390x.xml | 3 + .../caps_3.0.0.x86_64.xml | 3 + .../qemucapabilitiesdata/caps_3.1.0.ppc64.xml | 3 + .../caps_3.1.0.x86_64.xml | 3 + .../caps_4.0.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 3 + .../caps_4.0.0.riscv32.xml | 3 + .../caps_4.0.0.riscv64.xml | 3 + .../qemucapabilitiesdata/caps_4.0.0.s390x.xml | 3 + .../caps_4.0.0.x86_64.xml | 3 + .../caps_4.1.0.x86_64.xml | 3 + .../caps_4.2.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_4.2.0.ppc64.xml | 3 + .../qemucapabilitiesdata/caps_4.2.0.s390x.xml | 3 + .../caps_4.2.0.x86_64.xml | 3 + .../caps_5.0.0.aarch64.xml | 3 + .../qemucapabilitiesdata/caps_5.0.0.ppc64.xml | 3 + .../caps_5.0.0.riscv64.xml | 3 + .../caps_5.0.0.x86_64.xml | 3 + .../caps_5.1.0.x86_64.xml | 3 + tests/qemumonitorjsontest.c | 6 +- 44 files changed, 240 insertions(+), 32 deletions(-) -- 2.27.0

2 7

[libvirt PATCH v2 00/10] Add ability to create mediated devices in libvirt
by Jonathon Jongsma 16 Jun '20

16 Jun '20

Apologies for the delay in posting the second version of this series. This is the first portion of an effort to support persistent mediated devices with libvirt. This first series simply enables creating and destroying non-persistent mediated devices via the virNodeDeviceCreateXML() and virNodeDeviceDestroy() functions. The 'mdevctl' utility[1] provides the backend implementation. Changes in v2: - review findings fixed - Added Unit testing - passed JSON config via stdin instead of a temporary file (see portability note in patch 5) [1] https://github.com/mdevctl/mdevctl Jonathon Jongsma (10): nodedev: factor out nodeDeviceHasCapability() nodedev: add support for mdev attributes nodedev: refactor nodeDeviceFindNewDevice() nodedev: store mdev UUID in mdev caps nodedev: add mdev support to virNodeDeviceCreateXML() nodedev: Build a non-loadable driver lib nodedev: Add testing for 'mdevctl start' nodedev: add mdev support to virNodeDeviceDestroy() nodedev: Add testing for 'mdevctl stop' docs: note node device fields that are read-only build-aux/syntax-check.mk | 2 +- docs/formatnode.html.in | 16 +- docs/schemas/nodedev.rng | 6 + libvirt.spec.in | 2 + m4/virt-external-programs.m4 | 3 + src/conf/node_device_conf.c | 60 ++- src/conf/node_device_conf.h | 3 + src/conf/virnodedeviceobj.c | 34 ++ src/conf/virnodedeviceobj.h | 3 + src/libvirt_private.syms | 3 + src/node_device/Makefile.inc.am | 23 +- src/node_device/node_device_driver.c | 377 ++++++++++++++++-- src/node_device/node_device_driver.h | 9 + src/node_device/node_device_udev.c | 5 +- src/util/virmdev.c | 12 + src/util/virmdev.h | 11 + tests/Makefile.am | 14 + ...019_36ea_4111_8f0a_8c9a70e21366-start.argv | 1 + ...019_36ea_4111_8f0a_8c9a70e21366-start.json | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.argv | 1 + ...d39_495e_4243_ad9f_beb3f14c23d9-start.json | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.argv | 1 + ...916_1ca8_49ac_b176_871d16c13076-start.json | 1 + tests/nodedevmdevctldata/mdevctl-stop.argv | 1 + tests/nodedevmdevctltest.c | 300 ++++++++++++++ ...v_d069d019_36ea_4111_8f0a_8c9a70e21366.xml | 8 + ...v_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml | 10 + ...v_fedc4916_1ca8_49ac_b176_871d16c13076.xml | 9 + 28 files changed, 862 insertions(+), 55 deletions(-) create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366-start.json create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9-start.json create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.argv create mode 100644 tests/nodedevmdevctldata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076-start.json create mode 100644 tests/nodedevmdevctldata/mdevctl-stop.argv create mode 100644 tests/nodedevmdevctltest.c create mode 100644 tests/nodedevschemadata/mdev_d069d019_36ea_4111_8f0a_8c9a70e21366.xml create mode 100644 tests/nodedevschemadata/mdev_d2441d39_495e_4243_ad9f_beb3f14c23d9.xml create mode 100644 tests/nodedevschemadata/mdev_fedc4916_1ca8_49ac_b176_871d16c13076.xml -- 2.21.3

3 33

[PATCH libvirt v1 0/6] Fix ZPCI address auto-generation on s390
by Shalini Chellathurai Saroja 16 Jun '20

16 Jun '20

The ZPCI address validation or autogeneration does not work as expected in the following scenarios 1. uid = 0 and fid = 0 2. uid = 0 and fid > 0 3. uid = 0 and fid not specified 4. uid not specified and fid > 0 5. 2 ZPCI devices with uid > 0 and fid not specified. This is because of the following reasons 1. If uid = 0 or fid = 0 the code assumes that user has not specified the corresponding address 2. If either uid or fid is provided, the code assumes that both uid and fid addresses are specified by the user. This patch fixes these issues. Shalini Chellathurai Saroja (6): conf: fix ZPCI address validation on s390 tests: qemu: add tests for ZPCI on s390 conf: fix ZPCI address auto-generation on s390 qemu: move ZPCI uid validation into device validation tests: qemu: add more tests for ZPCI on S390 tests: add test with PCI and CCW device src/conf/device_conf.c | 45 ++++++-------- src/conf/domain_addr.c | 59 +++++++++++++------ src/conf/domain_conf.c | 2 +- src/libvirt_private.syms | 4 +- src/qemu/qemu_validate.c | 26 +++++++- src/util/virpci.c | 23 ++------ src/util/virpci.h | 6 +- .../hostdev-vfio-zpci-autogenerate-fids.args | 31 ++++++++++ .../hostdev-vfio-zpci-autogenerate-fids.xml | 29 +++++++++ .../hostdev-vfio-zpci-autogenerate-uids.args | 31 ++++++++++ .../hostdev-vfio-zpci-autogenerate-uids.xml | 29 +++++++++ .../hostdev-vfio-zpci-ccw-memballoon.args | 28 +++++++++ .../hostdev-vfio-zpci-ccw-memballoon.xml | 17 ++++++ .../hostdev-vfio-zpci-duplicate.xml | 30 ++++++++++ ...ostdev-vfio-zpci-invalid-uid-valid-fid.xml | 21 +++++++ .../hostdev-vfio-zpci-multidomain-many.args | 6 +- .../hostdev-vfio-zpci-set-zero.xml | 21 +++++++ .../hostdev-vfio-zpci-uid-set-zero.xml | 20 +++++++ tests/qemuxml2argvtest.c | 25 ++++++++ .../hostdev-vfio-zpci-multidomain-many.xml | 6 +- 20 files changed, 387 insertions(+), 72 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-fids.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-fids.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-uids.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-autogenerate-uids.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-ccw-memballoon.args create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-ccw-memballoon.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-duplicate.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-invalid-uid-valid-fid.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-set-zero.xml create mode 100644 tests/qemuxml2argvdata/hostdev-vfio-zpci-uid-set-zero.xml -- 2.21.1

5 35