May 2021 - Devel - libvirt List Archives

Re: Let's remove some deprecated stuff
by Eric Blake 04 May '21

04 May '21

On 4/29/21 4:59 AM, Markus Armbruster wrote: > If you're cc'ed, you added a section to docs/system/deprecated.rst that > is old enough to permit removal. This is *not* a demand to remove, it's > a polite request to consider whether the time for removal has come. > Extra points for telling us in a reply. "We should remove, but I can't > do it myself right now" is a valid answer. Let's review the file: > [adjusting cc for this response] > > Eric Blake: > > qemu-img amend to adjust backing file (since 5.1) > ''''''''''''''''''''''''''''''''''''''''''''''''' > > The use of ``qemu-img amend`` to modify the name or format of a qcow2 > backing image is deprecated; this functionality was never fully > documented or tested, and interferes with other amend operations that > need access to the original backing image (such as deciding whether a > v3 zero cluster may be left unallocated when converting to a v2 > image). Rather, any changes to the backing chain should be performed > with ``qemu-img rebase -u`` either before or after the remaining > changes being performed by amend, as appropriate. > > qemu-img backing file without format (since 5.1) > '''''''''''''''''''''''''''''''''''''''''''''''' > > The use of ``qemu-img create``, ``qemu-img rebase``, or ``qemu-img > convert`` to create or modify an image that depends on a backing file > now recommends that an explicit backing format be provided. This is > for safety: if QEMU probes a different format than what you thought, > the data presented to the guest will be corrupt; similarly, presenting > a raw image to a guest allows a potential security exploit if a future > probe sees a non-raw image based on guest writes. > > To avoid the warning message, or even future refusal to create an > unsafe image, you must pass ``-o backing_fmt=`` (or the shorthand > ``-F`` during create) to specify the intended backing format. You may > use ``qemu-img rebase -u`` to retroactively add a backing format to an > existing image. However, be aware that there are already potential > security risks to blindly using ``qemu-img info`` to probe the format > of an untrusted backing image, when deciding what format to add into > an existing image. I'm not sure how widely used these were, but I'm game for writing a patch to drop them. I'm fairly certain libvirt is not using them. > > Kevin Wolf: > > ``nbd-server-add`` and ``nbd-server-remove`` (since 5.2) > '''''''''''''''''''''''''''''''''''''''''''''''''''''''' > > Use the more generic commands ``block-export-add`` and ``block-export-del`` > instead. As part of this deprecation, where ``nbd-server-add`` used a > single ``bitmap``, the new ``block-export-add`` uses a list of ``bitmaps``. Peter, is libvirt good for this one to go? -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

2 1

[libvirt PATCH 0/6] Enable sanitizers
by Tim Wiederhake 04 May '21

04 May '21

This series enables and adds AddressSanitizer and UndefinedBehaviorSanitizer builds to the CI. See: https://clang.llvm.org/docs/AddressSanitizer.html and https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html These sanitizers already found some issues in libvirt, e.g. 4eb7c621985dad4de911ec394ac628bd1a5b29ab, 1294de209cee6643511265c7e2d4283c047cf652, 8b8c91f487592c6c067847ca59dde405ca17573f, or 1c34211c22de28127a509edbf2cf2f44cb0d891e. There exist two more relevant sanitizers, ThreadSanitizer and MemorySanitizer. Unfortunately, those two require an instrumented build of all dependencies, including libc, to work correctly. Note that clang and gcc have different implementations of these sanitizers, hence the introduction of two new jobs to the CI. The latter one issues a warning about the use of LD_PRELOAD in `virTestMain`, which in this particular case can be safely ignored by setting `ASAN_OPTIONS` to verify_asan_link_order=0` for the gcc build. Cheers, Tim Tim Wiederhake (6): meson: Allow larger stack frames when instrumenting meson: Allow undefined symbols when sanitizers are enabled tests: virfilemock: realpath: Allow non-null second parameter tests: openvzutilstest: Remove duplicate linking with libvirt_openvz.a virt-aa-helper: Remove duplicate linking with src/datatypes.o ci: Enable address and undefined behavior sanitizers .gitlab-ci.yml | 35 +++++++++++++++++++++++++++++++++++ build-aux/syntax-check.mk | 2 +- meson.build | 8 +++++++- src/libvirt_openvz.syms | 2 ++ src/security/meson.build | 1 - tests/meson.build | 2 +- tests/virfilemock.c | 12 ++++++------ 7 files changed, 52 insertions(+), 10 deletions(-) -- 2.26.3

3 14

[PATCH] Add page_per_vq flag to the 'driver' element of virtio devices
by Gavi Teitz 04 May '21

04 May '21

https://bugzilla.redhat.com/show_bug.cgi?id=1925363 Add support for setting the page-per-vq flag, which is important for vdpa with vhost-user performance. Signed-off-by: Gavi Teitz <gavi(a)nvidia.com> --- docs/formatdomain.rst | 11 +++++- docs/schemas/domaincommon.rng | 5 +++ src/conf/domain_conf.c | 15 ++++++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 5 +++ src/qemu/qemu_hotplug.c | 1 + tests/qemuxml2argvdata/net-virtio-page-per-vq.args | 31 ++++++++++++++++ tests/qemuxml2argvdata/net-virtio-page-per-vq.xml | 29 +++++++++++++++ tests/qemuxml2argvtest.c | 2 + .../qemuxml2xmloutdata/net-virtio-page-per-vq.xml | 43 ++++++++++++++++++++++ tests/qemuxml2xmltest.c | 2 + 13 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.args create mode 100644 tests/qemuxml2argvdata/net-virtio-page-per-vq.xml create mode 100644 tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 282176c..f5a9c70 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -5120,7 +5120,7 @@ Setting NIC driver-specific options <source network='default'/> <target dev='vnet1'/> <model type='virtio'/> - <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'> + <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256' page_per_vq='on'> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver> @@ -5215,6 +5215,15 @@ following attributes are available for the ``"virtio"`` NIC driver: only for ``vhostuser`` type. :since:`Since 3.7.0 (QEMU and KVM only)` **In general you should leave this option alone, unless you are very certain you know what you are doing.** +``page_per_vq`` + This optional attribute controls the layout of the notification capabilities + exposed to the guest. When enabled, each virtio queue will have a dedicated + page on the device BAR exposed to the guest. It is recommended to be used when + vDPA is enabled on the hypervisor, as it enables mapping the notification area + to the physical device, which is only supported in page granularity. The + default is determined by QEMU; as off. :since:`Since 2.8.0 (QEMU only)` + **In general you should leave this option alone, unless you are very certain + you know what you are doing.** virtio options For virtio interfaces, `Virtio-specific options <#elementsVirtio>`__ can also be set. ( :since:`Since 3.5.0` ) diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index a2e5c50..e61ad67 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -3463,6 +3463,11 @@ </attribute> </optional> <optional> + <attribute name="page_per_vq"> + <ref name="virOnOff"/> + </attribute> + </optional> + <optional> <attribute name="txmode"> <choice> <value>iothread</value> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9d98f48..0350fde 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -10446,6 +10446,7 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, g_autofree char *queues = NULL; g_autofree char *rx_queue_size = NULL; g_autofree char *tx_queue_size = NULL; + g_autofree char *page_per_vq = NULL; g_autofree char *filter = NULL; g_autofree char *internal = NULL; g_autofree char *mode = NULL; @@ -10615,6 +10616,7 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, queues = virXMLPropString(cur, "queues"); rx_queue_size = virXMLPropString(cur, "rx_queue_size"); tx_queue_size = virXMLPropString(cur, "tx_queue_size"); + page_per_vq = virXMLPropString(cur, "page_per_vq"); if (virDomainVirtioOptionsParseXML(cur, &def->virtio) < 0) goto error; @@ -11041,6 +11043,15 @@ virDomainNetDefParseXML(virDomainXMLOption *xmlopt, } def->driver.virtio.tx_queue_size = q; } + if (page_per_vq) { + if ((val = virTristateSwitchTypeFromString(page_per_vq)) <= 0) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("unknown page_per_vq mode '%s'"), + page_per_vq); + goto error; + } + def->driver.virtio.page_per_vq = val; + } if ((tmpNode = virXPathNode("./driver/host", ctxt))) { if (virXMLPropTristateSwitch(tmpNode, "csum", VIR_XML_PROP_NONE, @@ -25453,6 +25464,10 @@ virDomainVirtioNetDriverFormat(virBuffer *buf, if (def->driver.virtio.tx_queue_size) virBufferAsprintf(buf, " tx_queue_size='%u'", def->driver.virtio.tx_queue_size); + if (def->driver.virtio.page_per_vq) { + virBufferAsprintf(buf, " page_per_vq='%s'", + virTristateSwitchTypeToString(def->driver.virtio.page_per_vq)); + } virDomainVirtioOptionsFormat(buf, def->virtio); } diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 85c318d..7aab565 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1027,6 +1027,7 @@ struct _virDomainNetDef { virDomainNetVirtioTxModeType txmode; virTristateSwitch ioeventfd; virTristateSwitch event_idx; + virTristateSwitch page_per_vq; unsigned int queues; /* Multiqueue virtio-net */ unsigned int rx_queue_size; unsigned int tx_queue_size; diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 7971a9c..e5646cb 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -629,6 +629,7 @@ VIR_ENUM_IMPL(virQEMUCaps, /* 400 */ "compat-deprecated", "acpi-index", + "virtio-net.page_per_vq", ); @@ -1405,6 +1406,7 @@ static struct virQEMUCapsDevicePropsFlags virQEMUCapsDevicePropsVirtioNet[] = { { "event_idx", QEMU_CAPS_VIRTIO_NET_EVENT_IDX, NULL }, { "rx_queue_size", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE, NULL }, { "tx_queue_size", QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE, NULL }, + { "page_per_vq", QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ, NULL }, { "host_mtu", QEMU_CAPS_VIRTIO_NET_HOST_MTU, NULL }, { "disable-legacy", QEMU_CAPS_VIRTIO_PCI_DISABLE_LEGACY, NULL }, { "iommu_platform", QEMU_CAPS_VIRTIO_PCI_IOMMU_PLATFORM, NULL }, diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index f54aad5..75feed4 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -609,6 +609,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ /* 400 */ QEMU_CAPS_COMPAT_DEPRECATED, /* -compat deprecated-(input|output) is supported */ QEMU_CAPS_ACPI_INDEX, /* PCI device 'acpi-index' property */ + QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ, /* virtio-net-*.page_per_vq */ QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index d7f1c71..7b5834f 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -3630,6 +3630,11 @@ qemuBuildNicDevStr(virDomainDef *def, if (net->driver.virtio.tx_queue_size) virBufferAsprintf(&buf, ",tx_queue_size=%u", net->driver.virtio.tx_queue_size); + if (net->driver.virtio.page_per_vq) { + virBufferAsprintf(&buf, ",page-per-vq=%s", + virTristateSwitchTypeToString(net->driver.virtio.page_per_vq)); + } + if (net->mtu) virBufferAsprintf(&buf, ",host_mtu=%u", net->mtu); diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 4344edc..f79c3d8 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -3578,6 +3578,7 @@ qemuDomainChangeNet(virQEMUDriver *driver, olddev->driver.virtio.queues != newdev->driver.virtio.queues || olddev->driver.virtio.rx_queue_size != newdev->driver.virtio.rx_queue_size || olddev->driver.virtio.tx_queue_size != newdev->driver.virtio.tx_queue_size || + olddev->driver.virtio.page_per_vq != newdev->driver.virtio.page_per_vq || olddev->driver.virtio.host.csum != newdev->driver.virtio.host.csum || olddev->driver.virtio.host.gso != newdev->driver.virtio.host.gso || olddev->driver.virtio.host.tso4 != newdev->driver.virtio.host.tso4 || diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.args b/tests/qemuxml2argvdata/net-virtio-page-per-vq.args new file mode 100644 index 0000000..d7db71f --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.args @@ -0,0 +1,31 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/tmp/lib/domain--1-QEMUGuest1 \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-i386 \ +-name QEMUGuest1 \ +-S \ +-machine pc,accel=tcg,usb=off,dump-guest-core=off \ +-m 214 \ +-realtime mlock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-no-acpi \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \ +-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ +-netdev user,id=hostnet0 \ +-device virtio-net-pci,page-per-vq=on,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 diff --git a/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml b/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml new file mode 100644 index 0000000..e22ecd6 --- /dev/null +++ b/tests/qemuxml2argvdata/net-virtio-page-per-vq.xml @@ -0,0 +1,29 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + </disk> + <controller type='usb' index='0'/> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <driver page_per_vq='on'/> + </interface> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index f0efe98..f6b8f34 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1654,6 +1654,8 @@ mymain(void) QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE); DO_TEST_PARSE_ERROR("net-virtio-rxqueuesize-invalid-size", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE); + DO_TEST("net-virtio-page-per-vq", + QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ); DO_TEST("net-virtio-teaming", QEMU_CAPS_VIRTIO_NET_FAILOVER, QEMU_CAPS_DEVICE_VFIO_PCI); diff --git a/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml b/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml new file mode 100644 index 0000000..a35ed8a --- /dev/null +++ b/tests/qemuxml2xmloutdata/net-virtio-page-per-vq.xml @@ -0,0 +1,43 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219100</memory> + <currentMemory unit='KiB'>219100</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-i386</emulator> + <disk type='block' device='disk'> + <driver name='qemu' type='raw'/> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pci-root'/> + <controller type='ide' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> + </controller> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <driver page_per_vq='on'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <audio id='1' type='none'/> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </memballoon> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index c37de0c..2bd6eea 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -450,6 +450,8 @@ mymain(void) DO_TEST("net-virtio-rxtxqueuesize", QEMU_CAPS_VIRTIO_NET_RX_QUEUE_SIZE, QEMU_CAPS_VIRTIO_NET_TX_QUEUE_SIZE); + DO_TEST("net-virtio-page-per-vq", + QEMU_CAPS_VIRTIO_NET_PAGE_PER_VQ); DO_TEST("net-virtio-teaming", QEMU_CAPS_VIRTIO_NET_FAILOVER, QEMU_CAPS_DEVICE_VFIO_PCI); -- 1.8.3.1

3 4

[PATCH v3 RESEND for v7.4.0 00/14] Introduce virtio-mem <memory/> model
by Michal Privoznik 04 May '21

04 May '21

This is a rebased version of v3 I've sent about a month ago: https://listman.redhat.com/archives/libvir-list/2021-March/msg00823.html v2 here: https://listman.redhat.com/archives/libvir-list/2021-February/msg00961.html My intent is to merge these only after the upcoming release so that we have the biggest test window possible. diff to v2: - Dropped code that forbade use of virtio-mem and memballoon at the same time; - This meant that I had to adjust memory accounting, qemuDomainSetMemoryFlags() - see patches 11/15 and 12/15 which are new. - Fixed small nits raised by Peter in his review of v2 Michal Prívozník (14): virhostmem: Introduce virHostMemGetTHPSize() qemu_process: Deduplicate code in qemuProcessNeedHugepagesPath() qemu_process: Drop needless check in qemuProcessNeedMemoryBackingPath() qemu_capabilities: Introduce QEMU_CAPS_DEVICE_VIRTIO_MEM_PCI conf: Introduce virtio-mem <memory/> model qemu: Build command line for virtio-mem qemu: Wire up <memory/> live update qemu: Wire up MEMORY_DEVICE_SIZE_CHANGE event qemu: Refresh the actual size of virtio-mem on monitor reconnect qemu: Account for both memballoon and virtio-mem qemuDomainSetMemoryFlags: Take virtio-mem into consideration virsh: Introduce update-memory-device command news: document recent virtio memory addition kbase: Document virtio-mem NEWS.rst | 16 ++ docs/formatdomain.rst | 45 +++- docs/kbase/index.rst | 4 + docs/kbase/memorydevices.rst | 150 +++++++++++ docs/kbase/meson.build | 1 + docs/manpages/virsh.rst | 30 +++ docs/schemas/domaincommon.rng | 16 ++ examples/c/misc/event-test.c | 17 ++ include/libvirt/libvirt-domain.h | 23 ++ src/conf/domain_conf.c | 115 ++++++++- src/conf/domain_conf.h | 15 ++ src/conf/domain_event.c | 84 +++++++ src/conf/domain_event.h | 10 + src/conf/domain_validate.c | 39 +++ src/libvirt_private.syms | 5 + src/qemu/qemu_alias.c | 10 +- src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 13 +- src/qemu/qemu_domain.c | 50 +++- src/qemu/qemu_domain.h | 1 + src/qemu/qemu_domain_address.c | 38 ++- src/qemu/qemu_driver.c | 233 +++++++++++++++++- src/qemu/qemu_hotplug.c | 18 ++ src/qemu/qemu_hotplug.h | 5 + src/qemu/qemu_monitor.c | 37 +++ src/qemu/qemu_monitor.h | 28 +++ src/qemu/qemu_monitor_json.c | 97 ++++++-- src/qemu/qemu_monitor_json.h | 5 + src/qemu/qemu_process.c | 118 ++++++++- src/qemu/qemu_validate.c | 8 + src/remote/remote_daemon_dispatch.c | 30 +++ src/remote/remote_driver.c | 32 +++ src/remote/remote_protocol.x | 14 +- src/remote_protocol-structs | 7 + src/security/security_apparmor.c | 1 + src/security/security_dac.c | 2 + src/security/security_selinux.c | 2 + src/util/virhostmem.c | 63 +++++ src/util/virhostmem.h | 3 + tests/domaincapsmock.c | 9 + .../caps_5.1.0.x86_64.xml | 1 + .../caps_5.2.0.x86_64.xml | 1 + .../caps_6.0.0.x86_64.xml | 1 + ...mory-hotplug-virtio-mem.x86_64-latest.args | 41 +++ .../memory-hotplug-virtio-mem.xml | 67 +++++ tests/qemuxml2argvtest.c | 1 + ...emory-hotplug-virtio-mem.x86_64-latest.xml | 1 + tests/qemuxml2xmltest.c | 1 + tools/virsh-domain.c | 169 +++++++++++++ 50 files changed, 1613 insertions(+), 67 deletions(-) create mode 100644 docs/kbase/memorydevices.rst create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/memory-hotplug-virtio-mem.xml create mode 120000 tests/qemuxml2xmloutdata/memory-hotplug-virtio-mem.x86_64-latest.xml -- 2.26.3

4 32

[PATCH 1/2] qcow2: Prohibit backing file changes in 'qemu-img amend'
by Eric Blake 03 May '21

03 May '21

This was deprecated back in bc5ee6da7 (qcow2: Deprecate use of qemu-img amend to change backing file), and no one in the meantime has given any reasons why it should be supported. Time to make change attempts a hard error (but for convenience, specifying the _same_ backing chain is not forbidden). Update a couple of iotests to match. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- docs/system/deprecated.rst | 12 ------------ docs/system/removed-features.rst | 12 ++++++++++++ block/qcow2.c | 13 ++++--------- tests/qemu-iotests/061 | 3 +++ tests/qemu-iotests/061.out | 3 ++- tests/qemu-iotests/082.out | 6 ++++-- 6 files changed, 25 insertions(+), 24 deletions(-) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index 80cae862528a..9ec1a9d0e03e 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -315,18 +315,6 @@ this CPU is also deprecated. Related binaries ---------------- -qemu-img amend to adjust backing file (since 5.1) -''''''''''''''''''''''''''''''''''''''''''''''''' - -The use of ``qemu-img amend`` to modify the name or format of a qcow2 -backing image is deprecated; this functionality was never fully -documented or tested, and interferes with other amend operations that -need access to the original backing image (such as deciding whether a -v3 zero cluster may be left unallocated when converting to a v2 -image). Rather, any changes to the backing chain should be performed -with ``qemu-img rebase -u`` either before or after the remaining -changes being performed by amend, as appropriate. - qemu-img backing file without format (since 5.1) '''''''''''''''''''''''''''''''''''''''''''''''' diff --git a/docs/system/removed-features.rst b/docs/system/removed-features.rst index 29e90601a51a..28b5df757d35 100644 --- a/docs/system/removed-features.rst +++ b/docs/system/removed-features.rst @@ -454,6 +454,18 @@ topologies described with -smp include all possible cpus, i.e. The ``enforce-config-section`` property was replaced by the ``-global migration.send-configuration={on|off}`` option. +qemu-img amend to adjust backing file (removed in 6.1) +'''''''''''''''''''''''''''''''''''''''''''''''''''''' + +The use of ``qemu-img amend`` to modify the name or format of a qcow2 +backing image was never fully documented or tested, and interferes +with other amend operations that need access to the original backing +image (such as deciding whether a v3 zero cluster may be left +unallocated when converting to a v2 image). Any changes to the +backing chain should be performed with ``qemu-img rebase -u`` either +before or after the remaining changes being performed by amend, as +appropriate. + Block devices ------------- diff --git a/block/qcow2.c b/block/qcow2.c index 9727ae8fe341..898c4fc4464c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -5620,15 +5620,10 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, if (backing_file || backing_format) { if (g_strcmp0(backing_file, s->image_backing_file) || g_strcmp0(backing_format, s->image_backing_format)) { - warn_report("Deprecated use of amend to alter the backing file; " - "use qemu-img rebase instead"); - } - ret = qcow2_change_backing_file(bs, - backing_file ?: s->image_backing_file, - backing_format ?: s->image_backing_format); - if (ret < 0) { - error_setg_errno(errp, -ret, "Failed to change the backing file"); - return ret; + error_setg(errp, "Cannot amend the backing file"); + error_append_hint(errp, + "You can use 'qemu-img rebase' instead.\n"); + return -EINVAL; } } diff --git a/tests/qemu-iotests/061 b/tests/qemu-iotests/061 index e26d94a0df31..9507c223bda4 100755 --- a/tests/qemu-iotests/061 +++ b/tests/qemu-iotests/061 @@ -167,6 +167,9 @@ _make_test_img -o "compat=1.1" 64M TEST_IMG="$TEST_IMG.base" _make_test_img -o "compat=1.1" 64M $QEMU_IO -c "write -P 0x2a 0 128k" "$TEST_IMG.base" | _filter_qemu_io $QEMU_IO -c "read -P 0 0 128k" "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG amend -o "backing_file=$TEST_IMG.base,backing_fmt=qcow2" \ + "$TEST_IMG" && echo "unexpected pass" +$QEMU_IMG rebase -u -b "$TEST_IMG.base" -F qcow2 "$TEST_IMG" $QEMU_IMG amend -o "backing_file=$TEST_IMG.base,backing_fmt=qcow2" "$TEST_IMG" $QEMU_IO -c "read -P 0x2a 0 128k" "$TEST_IMG" | _filter_qemu_io _check_test_img diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index ee30da266514..7ecbd4dea875 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -370,7 +370,8 @@ wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead +qemu-img: Cannot amend the backing file +You can use 'qemu-img rebase' instead. read 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) No errors were found on the image. diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index b70c12c1393d..077ed0f2c7d9 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -808,12 +808,14 @@ Amend options for 'qcow2': size=<size> - Virtual disk size Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 -qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead +qemu-img: Cannot amend the backing file +You can use 'qemu-img rebase' instead. Testing: rebase -u -b -f qcow2 TEST_DIR/t.qcow2 Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,? TEST_DIR/t.qcow2 -qemu-img: warning: Deprecated use of amend to alter the backing file; use qemu-img rebase instead +qemu-img: Cannot amend the backing file +You can use 'qemu-img rebase' instead. Testing: rebase -u -b -f qcow2 TEST_DIR/t.qcow2 -- 2.31.1

1 0

[PATCH] Deprecate pmem=on with non-DAX capable backend file
by Igor Mammedov 03 May '21

03 May '21

It is not safe to pretend that emulated NVDIMM supports persistence while backend actually failed to enable it and used non-persistent mapping as fall back. Instead of falling-back, QEMU should be more strict and error out with clear message that it's not supported. So if user asks for persistence (pmem=on), they should store backing file on NVDIMM. Signed-off-by: Igor Mammedov <imammedo(a)redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd(a)redhat.com> --- v2: rephrase deprecation comment andwarning message (Philippe Mathieu-Daudé <philmd(a)redhat.com>) --- docs/system/deprecated.rst | 17 +++++++++++++++++ util/mmap-alloc.c | 3 +++ 2 files changed, 20 insertions(+) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index bacd76d7a5..e79fb02b3a 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -327,6 +327,23 @@ The Raspberry Pi machines come in various models (A, A+, B, B+). To be able to distinguish which model QEMU is implementing, the ``raspi2`` and ``raspi3`` machines have been renamed ``raspi2b`` and ``raspi3b``. +Backend options +--------------- + +Using non-persistent backing file with pmem=on (since 6.0) +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +This option is used when ``memory-backend-file`` is consumed by emulated NVDIMM +device. However enabling ``memory-backend-file.pmem`` option, when backing file +is (a) not DAX capable or (b) not on a filesystem that support direct mapping +of persistent memory, is not safe and may lead to data loss or corruption in case +of host crash. +Options are: + - modify VM configuration to set ``pmem=off`` to continue using fake NVDIMM + (without persistence guaranties) with backing file on non DAX storage + - move backing file to NVDIMM storage and keep ``pmem=on`` + (to have NVDIMM with persistence guaranties). + Device options -------------- diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 27dcccd8ec..0388cc3be2 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -20,6 +20,7 @@ #include "qemu/osdep.h" #include "qemu/mmap-alloc.h" #include "qemu/host-utils.h" +#include "qemu/error-report.h" #define HUGETLBFS_MAGIC 0x958458f6 @@ -166,6 +167,8 @@ void *qemu_ram_mmap(int fd, "crash.\n", file_name); g_free(proc_link); g_free(file_name); + warn_report("Using non DAX backing file with 'pmem=on' option" + " is deprecated"); } /* * if map failed with MAP_SHARED_VALIDATE | MAP_SYNC, -- 2.27.0

3 5

[PATCH v2] meson: Declare GLIB_VERSION_* macros at configure
by Michal Privoznik 03 May '21

03 May '21

So far we have three places where glib version is recorded: meson.build and then in config.h. The latter is so well hidden that it's easy to miss when bumping minimal glib version in the former. With a bit of python^Wmeson string magic GLIB_VERSION_MIN_REQUIRED and GLIB_VERSION_MAX_ALLOWED macros can be defined to match glib_version from meson.build. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- v2 of: https://listman.redhat.com/archives/libvir-list/2021-April/msg01300.html diff to v1: - Introduced new glib_version_str variable per Pavel's suggestion config.h | 10 ---------- meson.build | 8 ++++++++ 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/config.h b/config.h index ca6720f37d..0eacfd139d 100644 --- a/config.h +++ b/config.h @@ -51,13 +51,3 @@ #else # error You either need at least GCC 4.8 or Clang 3.4 or XCode Clang 5.1 to compile libvirt #endif - -/* Ask for warnings for anything that was marked deprecated in - * the defined version, or before. It is a candidate for rewrite. - */ -#define GLIB_VERSION_MIN_REQUIRED GLIB_VERSION_2_48 - -/* Ask for warnings if code tries to use function that did not - * exist in the defined version. These risk breaking builds - */ -#define GLIB_VERSION_MAX_ALLOWED GLIB_VERSION_2_48 diff --git a/meson.build b/meson.build index 951da67896..ecbfdb1208 100644 --- a/meson.build +++ b/meson.build @@ -949,6 +949,14 @@ endif glib_dep = declare_dependency( dependencies: [ glib_dep, gobject_dep, gio_dep ], ) +glib_version_arr = glib_version.split('.') +glib_version_str = 'GLIB_VERSION_@0@_@1@'.format(glib_version_arr[0], glib_version_arr[1]) +# Ask for warnings for anything that was marked deprecated in +# the defined version, or before. It is a candidate for rewrite. +conf.set('GLIB_VERSION_MIN_REQUIRED', glib_version_str) +# Ask for warnings if code tries to use function that did not +# exist in the defined version. These risk breaking builds +conf.set('GLIB_VERSION_MAX_ALLOWED', glib_version_str) glusterfs_version = '3.4.1' glusterfs_dep = dependency('glusterfs-api', version: '>=' + glusterfs_version, required: get_option('glusterfs')) -- 2.26.3

2 1

[libvirt PATCH v2 0/6] Refactor more XML parsing boilerplate code, part VI
by Tim Wiederhake 03 May '21

03 May '21

For background, see https://listman.redhat.com/archives/libvir-list/2021-April/msg00668.html V1: https://listman.redhat.com/archives/libvir-list/2021-April/msg01184.html Changes since V1: * Split up "virDomainFeaturesDefParse" * Added missing VIR_XML_PROP_NONZERO * Some line break changes * Mentioned printf %s NULL fix in commit message * Improved justification for parser strictening Tim Wiederhake (6): virDomainGraphicsDefParseXMLSpice: Use virXMLProp* virDomainSoundDefParseXML: Use virXMLProp* virDomainFeaturesDefParse: Use virXMLPropUInt virDomainFeaturesDefParse: Use virXMLPropTristateSwitch virDomainFeaturesDefParse: Use virXMLPropEnum virDomainAudioSDLParse: Use virXMLProp* src/conf/domain_conf.c | 609 +++++------------- .../qemuxml2argvdata/aarch64-gic-invalid.err | 2 +- 2 files changed, 170 insertions(+), 441 deletions(-) -- 2.26.3

2 13

[libvirt PATCH v2 00/11] Refactor more XML parsing boilerplate code, part V
by Tim Wiederhake 03 May '21

03 May '21

For background, see https://listman.redhat.com/archives/libvir-list/2021-April/msg00668.html Changes since V1: * Split up patch for virDomainControllerDefParseXML * Code style fixes Tim Wiederhake (11): virXMLPropEnum: Fix return value virDomainControllerDef: Change type of ioeventfd to virTristateSwitch virDomainPCIControllerOpts: Change type of modelName to virDomainControllerPCIModelName virDomainControllerDefParseXML: Use virXMLProp* virDomainControllerDefParseXML: Cosmetic changes virDomainActualNetDef: Change type of type to virDomainNetType virDomainActualNetDefParseXML: Use virXMLProp* virDomainNetDefParseXML: Use virXMLProp* virDomainGraphicsListenDefParseXML: Use virXMLProp* virDomainGraphicsDef: Change type of sharePolicy to virDomainGraphicsVNCSharePolicy virDomainGraphicsDefParseXMLVNC: Use virXMLProp* src/conf/domain_conf.c | 643 +++++++++++---------------------- src/conf/domain_conf.h | 8 +- src/qemu/qemu_domain_address.c | 2 +- src/util/virxml.c | 2 +- 4 files changed, 227 insertions(+), 428 deletions(-) -- 2.26.3

2 13

Release of libvirt-7.3.0
by Jiri Denemark 03 May '21

03 May '21

The 7.3.0 release of both libvirt and libvirt-python is tagged and signed tarballs and source RPMs are available at https://libvirt.org/sources/ https://libvirt.org/sources/python/ Thanks everybody who helped with this release by sending patches, reviewing, testing, or providing any other feedback. Your work is greatly appreciated. * New features * xen: Support domains with more than 4TB The xen driver now supports domains with more than 4TB of memory with xen >= 4.13. * qemu: add socket for virtiofs filesystems Libvirt now supports ``filesystem`` devices that connect to a ``virtiofsd`` daemon launched outside of libvirtd, via the ``socket`` attribute of the ``source`` element. * nodedev: Add ability to manage persistent mediated devices Persistent mediated devices can now be managed with libvirt. ``virNodeDeviceDefineXML()`` defines a new device, ``virNodeDeviceUndefine()`` removes an existing definition, and ``virNodeDeviceCreate()`` starts a device definition that is currently inactive. Corresponding virsh commands ``nodedev-define``, ``nodedev-undefine``, and ``nodedev-start`` were also added. ``nodedev-list`` only lists active devices by default. Inactive device definitions can be shown with the new ``--inactive`` and ``--all`` flags. * qemu: Allow use of qemu's ``-compat`` option Curious developers or testers now can enable certain ``-compat`` modes which allow to notice use of deprecated commands and options as qemu will use the selected method to notify the user. The new behaviour can be requested using either the ``deprecation_behavior`` option in ``qemu.conf`` for all VMs or using ``<qemu:deprecation behavior='...'/>`` in the VM XML. * Improvements * virsh: Improve errors with ``virsh snapshot-create-as`` The XML document constructed by virsh was forced through XML schema validation which yielded unintelligible error messages in cases such as when the path to the new image did not start with a slash. XML documents are no longer validated as the XML parser actually has better error messages which allow users to figure the problem out quickly. * qemu: Terminate backing store when doing a full-chain block pull When pulling everything into the overlay image the chain can be terminated since we know that it won't depend on any backing image and thus can prevent attempts to probe the backing chain. * qemu: Expose disk serial in virDomainGetGuestInfo() The ``virDomainGetGuestInfo()`` reports disk serial number among with other disk information. * Bug fixes * qemu: Fix crash of libvirt on full block pull of a disk When the persistent definition contains a compatible disk (meaning the definition of the running and persistent config match) a block pull job would leave a dangling pointer in the config definition which resulted in a crash. * qemu: Use proper job cancelling command Libvirt's API contract for aborting a block copy job in 'ready' state declares that the destination image of the copy will contain a consistent image of the disk from the time when the block job was aborted. This requires that libvirt uses the proper cancelling qemu command to ensure that the data is consistent which was not the case. * qemu: Don't attempt storage migration when there are no migratable disks Due to a logic bug introduced in the previous release libvirt would attempt to migrate disks in case when no disks are selected/eligible for migration. * qemu: Fix very rare race when two block job 'ready' events are delivered In certain high-load scenarios, qemu might deliver the 'ready' event twice and if it's delivered when pivoting to the destination during a block copy job, libvirt would get confused and execute the code as if the job were aborted. * lxc: Fix container destroy with CGroupsV2 When an LXC container was started and the host used CGroupsV2 it might have had created nested controllers under the container's scope. Libvirt was unaware and thus destroying the container failed with a cryptic error: ``failed to get cgroup backend for 'pathOfController'``. The CGroup removal code was reworked and is now capable of dealing with such scenario. * bash-completion: Fix argument passing to $1 Due to a bug in bash completion script, the auto completion did not work properly when a connection URI or read only flag were specified on ``virsh`` or ``virt-admin`` command line. Enjoy. Jirka

1 0