- Devel - libvirt List Archives

[libvirt PATCH] Fix spelling
by Tim Wiederhake 30 Mar '22

30 Mar '22

Signed-off-by: Tim Wiederhake <twiederh(a)redhat.com> --- ci/integration.yml | 8 ++++---- docs/drvqemu.rst | 6 +++--- src/qemu/qemu_capabilities.h | 2 +- src/qemu/qemu_domain.c | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/ci/integration.yml b/ci/integration.yml index 2808e829ef..17f20b3945 100644 --- a/ci/integration.yml +++ b/ci/integration.yml @@ -62,7 +62,7 @@ centos-stream-8-tests: variables: # needed by libvirt-gitlab-executor DISTRO: centos-stream-8 - # can be overriden in forks to set a different runner tag + # can be overridden in forks to set a different runner tag LIBVIRT_CI_INTEGRATION_RUNNER_TAG: redhat-vm-host tags: - $LIBVIRT_CI_INTEGRATION_RUNNER_TAG @@ -78,7 +78,7 @@ centos-stream-9-tests: variables: # needed by libvirt-gitlab-executor DISTRO: centos-stream-9 - # can be overriden in forks to set a different runner tag + # can be overridden in forks to set a different runner tag LIBVIRT_CI_INTEGRATION_RUNNER_TAG: redhat-vm-host tags: - $LIBVIRT_CI_INTEGRATION_RUNNER_TAG @@ -94,7 +94,7 @@ fedora-34-tests: variables: # needed by libvirt-gitlab-executor DISTRO: fedora-34 - # can be overriden in forks to set a different runner tag + # can be overridden in forks to set a different runner tag LIBVIRT_CI_INTEGRATION_RUNNER_TAG: redhat-vm-host tags: - $LIBVIRT_CI_INTEGRATION_RUNNER_TAG @@ -110,7 +110,7 @@ fedora-35-tests: variables: # needed by libvirt-gitlab-executor DISTRO: fedora-35 - # can be overriden in forks to set a different runner tag + # can be overridden in forks to set a different runner tag LIBVIRT_CI_INTEGRATION_RUNNER_TAG: redhat-vm-host tags: - $LIBVIRT_CI_INTEGRATION_RUNNER_TAG diff --git a/docs/drvqemu.rst b/docs/drvqemu.rst index afc499ad50..cfe4473951 100644 --- a/docs/drvqemu.rst +++ b/docs/drvqemu.rst @@ -589,13 +589,13 @@ specific properties of devices instantiated by libvirt. The ``<qemu:device>`` sub-element groups overrides for a device identified via the ``alias`` attribute. The alias corresponds to the ``<alias name=''>`` property of a device. It's strongly recommended to use user-specified aliases -for devices with overriden properties. +for devices with overridden properties. Sub element ``<qemu:frontend>`` encapsulates all overrides of properties for the device frontend and overrides what libvirt formats via ``-device``. :since:`Since 8.2.0`. -The individual properties are overriden by a ``<qemu:property>`` element. The +The individual properties are overridden by a ``<qemu:property>`` element. The ``name`` specifies the name of the property to override. In case when libvirt doesn't configure the property a property with the name is added to the commandline. The ``type`` attribute specifies a type of the argument used. The @@ -612,7 +612,7 @@ remove a device property which is not formatted by libvirt will cause failure to startup the VM. *Note:* The libvirt project doesn't guarantee any form of compatibility and -stability of devices with overriden properties. The domain is tainted when +stability of devices with overridden properties. The domain is tainted when such configuration is used. Example: diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 948029d60d..b2426a6d25 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -642,7 +642,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ QEMU_CAPS_DIRTYRATE_MODE , /* calc-dirty-rate accepts mode parameter */ /* 425 */ - QEMU_CAPS_BLOCKDEV_NBD_TLS_HOSTNAME, /* tls hostname can be overriden for NBD clients */ + QEMU_CAPS_BLOCKDEV_NBD_TLS_HOSTNAME, /* tls hostname can be overridden for NBD clients */ QEMU_CAPS_MEMORY_BACKEND_PREALLOC_THREADS, /* -object memory-backend-*.prealloc-threads */ QEMU_CAPS_LAST /* this must always be the last item */ diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 18d403e099..1bfea0d470 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3368,7 +3368,7 @@ qemuDomainDefNamespaceParseOverrideProperties(qemuDomainXmlNsOverrideProperty *p if (STREQ(prop->name, "id")) { virReportError(VIR_ERR_XML_ERROR, "%s", - _("property with name 'id' can't be overriden")); + _("property with name 'id' can't be overridden")); return -1; } -- 2.31.1

2 1

[libvirt PATCH] fix documentation for sockets topology
by John Levon 30 Mar '22

30 Mar '22

From: John Levon <john.levon(a)nutanix.com> In 0895a0e, it was noted that the "sockets" value in the topology section of capabilities reflects not the number of sockets per NUMA node, not the total number. Unfortunately, the fix was applied to the wrong place: the domain XML format documentation, not that for the capabilities output. And, in fact, the domain XML interprets "sockets" as the total number, not a per-node value. Back out this change in favour of a note in the capabilities documentation instead. Fixes: 0895a0e75d13874254218e16dc66dcad673671d3 Suggested-by: Daniel P. Berrangé <berrange(a)redhat.com> Signed-off-by: John Levon <john.levon(a)nutanix.com> --- docs/formatcaps.html.in | 5 ++++- docs/formatdomain.rst | 12 ++++++------ 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/formatcaps.html.in b/docs/formatcaps.html.in index 09662f78c8..a4abad5d20 100644 --- a/docs/formatcaps.html.in +++ b/docs/formatcaps.html.in @@ -51,7 +51,10 @@ <dt><code>topology</code></dt> <dd>This element embodies the host internal topology. Management applications may want to learn this information when orchestrating new - guests - e.g. due to reduce inter-NUMA node transfers.</dd> + guests - e.g. due to reduce inter-NUMA node transfers. Note that the + <code>sockets</code> value reported here is per-NUMA-node; this is in + contrast to the value given in domain definitions, which is interpreted + as a total number of sockets for the domain.</dd> <dt><code>secmodel</code></dt> <dd>To find out default security labels for different security models you diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index e492532004..9652e97eec 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -1489,12 +1489,12 @@ In case no restrictions need to be put on CPU model and its features, a simpler The ``topology`` element specifies requested topology of virtual CPU provided to the guest. Four attributes, ``sockets``, ``dies``, ``cores``, and ``threads``, accept non-zero positive integer values. They refer to the - number of CPU sockets per NUMA node, number of dies per socket, number of - cores per die, and number of threads per core, respectively. The ``dies`` - attribute is optional and will default to 1 if omitted, while the other - attributes are all mandatory. Hypervisors may require that the maximum number - of vCPUs specified by the ``cpus`` element equals to the number of vcpus - resulting from the topology. + total number of CPU sockets, number of dies per socket, number of cores per + die, and number of threads per core, respectively. The ``dies`` attribute is + optional and will default to 1 if omitted, while the other attributes are all + mandatory. Hypervisors may require that the maximum number of vCPUs specified + by the ``cpus`` element equals to the number of vcpus resulting from the + topology. ``feature`` The ``cpu`` element can contain zero or more ``feature`` elements used to fine-tune features provided by the selected CPU model. The list of known -- 2.32.0

2 1

[PATCH 0/3] Cleanups based on 'nvram' refactor patch
by Peter Krempa 30 Mar '22

30 Mar '22

Few cleanups that I've suggested in the review for adding remote store of nvram. Peter Krempa (3): virXMLFormatElement: Introduce virXMLFormatElementInternal virDomainLoaderDefFormat: Use modern XML formatting approach vboxSetBootDeviceOrder: Remove whitespace alignment in VIR_DEBUG statements src/conf/domain_conf.c | 40 +++++++++++++-------------- src/libvirt_private.syms | 1 + src/util/virxml.c | 58 +++++++++++++++++++++++++++++++--------- src/util/virxml.h | 7 +++++ src/vbox/vbox_common.c | 36 ++++++++++++------------- 5 files changed, 90 insertions(+), 52 deletions(-) -- 2.35.1

2 4

[libvirt PATCH] qemu: Rename virDomainAsyncJobPhase{From,To}String
by Jiri Denemark 30 Mar '22

30 Mar '22

Recent refactor (v8.1.0-217-ga193f4bef6) generalized job related enums and functions by changing "qemu" prefix to "vir" and moving them to src/hypervisor/domain_job.[ch]. This was in most cases a good thing, but async job phases are driver specific and the corresponding functions remained in src/qemu/qemu_domainjob.[ch], but still their prefix was changed to "vir". Let's change it back to "qemu". Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_domainjob.c | 10 +++++----- src/qemu/qemu_domainjob.h | 4 ++-- src/qemu/qemu_migration.c | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_domainjob.c b/src/qemu/qemu_domainjob.c index 56a3f6cb19..1cddc7f2f0 100644 --- a/src/qemu/qemu_domainjob.c +++ b/src/qemu/qemu_domainjob.c @@ -74,7 +74,7 @@ qemuDomainJobSetStatsType(virDomainJobData *jobData, const char * -virDomainAsyncJobPhaseToString(virDomainAsyncJob job, +qemuDomainAsyncJobPhaseToString(virDomainAsyncJob job, int phase G_GNUC_UNUSED) { switch (job) { @@ -97,7 +97,7 @@ virDomainAsyncJobPhaseToString(virDomainAsyncJob job, } int -virDomainAsyncJobPhaseFromString(virDomainAsyncJob job, +qemuDomainAsyncJobPhaseFromString(virDomainAsyncJob job, const char *phase) { if (!phase) @@ -682,7 +682,7 @@ qemuDomainObjSetJobPhase(virDomainObj *obj, VIR_DEBUG("Setting '%s' phase to '%s'", virDomainAsyncJobTypeToString(priv->job.asyncJob), - virDomainAsyncJobPhaseToString(priv->job.asyncJob, phase)); + qemuDomainAsyncJobPhaseToString(priv->job.asyncJob, phase)); if (priv->job.asyncOwner == 0) { priv->job.asyncOwnerAPI = g_strdup(virThreadJobGet()); @@ -1191,7 +1191,7 @@ qemuDomainObjPrivateXMLFormatJob(virBuffer *buf, if (priv->job.phase) { virBufferAsprintf(&attrBuf, " phase='%s'", - virDomainAsyncJobPhaseToString(priv->job.asyncJob, + qemuDomainAsyncJobPhaseToString(priv->job.asyncJob, priv->job.phase)); } @@ -1244,7 +1244,7 @@ qemuDomainObjPrivateXMLParseJob(virDomainObj *vm, priv->job.asyncJob = async; if ((tmp = virXPathString("string(@phase)", ctxt))) { - priv->job.phase = virDomainAsyncJobPhaseFromString(async, tmp); + priv->job.phase = qemuDomainAsyncJobPhaseFromString(async, tmp); if (priv->job.phase < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unknown job phase %s"), tmp); diff --git a/src/qemu/qemu_domainjob.h b/src/qemu/qemu_domainjob.h index faf47105a3..f67eace36c 100644 --- a/src/qemu/qemu_domainjob.h +++ b/src/qemu/qemu_domainjob.h @@ -119,9 +119,9 @@ struct _qemuDomainJobObj { void qemuDomainJobSetStatsType(virDomainJobData *jobData, qemuDomainJobStatsType type); -const char *virDomainAsyncJobPhaseToString(virDomainAsyncJob job, +const char *qemuDomainAsyncJobPhaseToString(virDomainAsyncJob job, int phase); -int virDomainAsyncJobPhaseFromString(virDomainAsyncJob job, +int qemuDomainAsyncJobPhaseFromString(virDomainAsyncJob job, const char *phase); void qemuDomainEventEmitJobCompleted(virQEMUDriver *driver, diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 3e653543c6..98f82abe7c 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -2190,7 +2190,7 @@ qemuMigrationSrcCleanup(virDomainObj *vm, VIR_DEBUG("vm=%s, conn=%p, asyncJob=%s, phase=%s", vm->def->name, conn, virDomainAsyncJobTypeToString(priv->job.asyncJob), - virDomainAsyncJobPhaseToString(priv->job.asyncJob, + qemuDomainAsyncJobPhaseToString(priv->job.asyncJob, priv->job.phase)); if (!qemuMigrationJobIsActive(vm, VIR_ASYNC_JOB_MIGRATION_OUT)) -- 2.35.1

2 1

[libvirt PATCH] qemu: fix hotplug for multiqueue vdpa net device
by Jonathon Jongsma 30 Mar '22

30 Mar '22

While commit a5e659f0 removed the restriction against multiple queues for the vdpa net device, there were some missing pieces. Configuring a device statically and then starting the domain worked as expected, but hotplugging a device didn't have the expected multiqueue support enabled. Add the missing bits. Consider the following device xml: <interface type="vdpa"> <mac address="00:11:22:33:44:03" /> <source dev="/dev/vhost-vdpa-0" /> <model type="virtio" /> <driver queues='2' /> </interface> Without this patch, hotplugging the above XML description resulted in the following: {"execute":"netdev_add","arguments":{"type":"vhost-vdpa","vhostdev":"/dev/fdset/0","id":"hostnet1"},"id":"libvirt-392"} {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:03","bus":"pci.5","addr":"0x0"},"id":"libvirt-393"} With the patch, hotplugging results in the following: {"execute":"netdev_add","arguments":{"type":"vhost-vdpa","vhostdev":"/dev/fdset/0","queues":2,"id":"hostnet1"},"id":"libvirt-392"} {"execute":"device_add","arguments":{"driver":"virtio-net-pci","mq":true,"vectors":6,"netdev":"hostnet1","id":"net1","mac":"00:11:22:33:44:03","bus":"pci.5","addr":"0x0"},"id":"libvirt-393"} Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2024406 Signed-off-by: Jonathon Jongsma <jjongsma(a)redhat.com> --- src/qemu/qemu_command.c | 4 ++++ src/qemu/qemu_hotplug.c | 3 +++ tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args | 2 +- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 453891a725..f7f55b20ef 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -4346,6 +4346,10 @@ qemuBuildHostNetProps(virDomainNetDef *net, if (virJSONValueObjectAdd(&netprops, "s:type", "vhost-vdpa", NULL) < 0 || virJSONValueObjectAppendString(netprops, "vhostdev", vdpadev) < 0) return NULL; + + if (net->driver.virtio.queues > 1 && + virJSONValueObjectAppendNumberUlong(netprops, "queues", net->driver.virtio.queues) < 0) + return NULL; break; case VIR_DOMAIN_NET_TYPE_HOSTDEV: diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c index 3d1bb1be2a..d9ba0b7abe 100644 --- a/src/qemu/qemu_hotplug.c +++ b/src/qemu/qemu_hotplug.c @@ -1386,6 +1386,9 @@ qemuDomainAttachNetDevice(virQEMUDriver *driver, break; case VIR_DOMAIN_NET_TYPE_VDPA: + queueSize = net->driver.virtio.queues; + if (!queueSize) + queueSize = 1; if (qemuDomainAdjustMaxMemLock(vm, false) < 0) goto cleanup; adjustmemlock = true; diff --git a/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args index 26ef666036..4cb805451a 100644 --- a/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args +++ b/tests/qemuxml2argvdata/net-vdpa-multiqueue.x86_64-latest.args @@ -29,7 +29,7 @@ XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \ -boot strict=on \ -device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0x2"}' \ -add-fd set=0,fd=1732,opaque=net0-vdpa \ --netdev vhost-vdpa,vhostdev=/dev/fdset/0,id=hostnet0 \ +-netdev vhost-vdpa,vhostdev=/dev/fdset/0,queues=2,id=hostnet0 \ -device '{"driver":"virtio-net-pci","mq":true,"vectors":6,"netdev":"hostnet0","id":"net0","mac":"52:54:00:95:db:c0","bus":"pci.0","addr":"0x2"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -- 2.35.1

2 1

[libvirt PATCH] util: Improve macOS workaround
by Andrea Bolognani 29 Mar '22

29 Mar '22

Since the workaround is specific to macOS, only disable compiler warnings when building on that platform. While at it, update the comment to reflect the fact that the workaround is needed for all versions of the OS, including the modern ones that we currently target. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/502990271 src/util/virutil.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/util/virutil.c b/src/util/virutil.c index fe5500726e..176e2f8f59 100644 --- a/src/util/virutil.c +++ b/src/util/virutil.c @@ -869,9 +869,11 @@ virDoesGroupExist(const char *name) -/* Work around an incompatibility of OS X 10.11: getgrouplist +/* Work around an incompatibility of macOS: getgrouplist accepts int *, not gid_t *, and int and gid_t differ in sign. */ +# ifdef __APPLE__ VIR_WARNINGS_NO_POINTER_SIGN +# endif /* Compute the list of primary and supplementary groups associated * with @uid, and including @gid in the list (unless it is -1), @@ -934,7 +936,9 @@ virGetGroupList(uid_t uid, gid_t gid, gid_t **list) return ret; } +# ifdef __APPLE__ VIR_WARNINGS_RESET +# endif /* Set the real and effective uid and gid to the given values, as well -- 2.35.1

2 1

[libvirt PATCH] qemu: Use real defaults for user and group in qemu.conf
by Andrea Bolognani 29 Mar '22

29 Mar '22

The default values used by the library are determined at configure time based on a number of factors, and we should reflect them in the installed configuration file to make the comments it contains more useful. Resolves: https://gitlab.com/libvirt/libvirt/-/issues/263 Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- src/qemu/meson.build | 30 +++++++++++++++++++++++++--- src/qemu/{qemu.conf => qemu.conf.in} | 4 ++-- src/qemu/test_libvirtd_qemu.aug.in | 4 ++-- 3 files changed, 31 insertions(+), 7 deletions(-) rename src/qemu/{qemu.conf => qemu.conf.in} (99%) diff --git a/src/qemu/meson.build b/src/qemu/meson.build index 39f0f615cc..96952cc52d 100644 --- a/src/qemu/meson.build +++ b/src/qemu/meson.build @@ -130,12 +130,36 @@ if conf.has('WITH_QEMU') ], } - virt_conf_files += files('qemu.conf') + qemu_user_group_conf = configuration_data({ + 'QEMU_USER': qemu_user, + 'QEMU_GROUP': qemu_group, + }) + qemu_conf = configure_file( + input: 'qemu.conf.in', + output: 'qemu.conf', + configuration: qemu_user_group_conf, + ) + + qemu_user_group_hack_conf = configuration_data({ + 'QEMU_USER': qemu_user, + 'QEMU_GROUP': qemu_group, + # This hack is necessary because the output file is going to be + # used as input for another configure_file() call later, which + # will take care of substituting @CONFIG@ with useful data + 'CONFIG': '@CONFIG@', + }) + test_libvirtd_qemu_aug_tmp = configure_file( + input: 'test_libvirtd_qemu.aug.in', + output: 'test_libvirtd_qemu.aug.tmp', + configuration: qemu_user_group_hack_conf, + ) + + virt_conf_files += qemu_conf virt_aug_files += files('libvirtd_qemu.aug') virt_test_aug_files += { 'name': 'test_libvirtd_qemu.aug', - 'aug': files('test_libvirtd_qemu.aug.in'), - 'conf': files('qemu.conf'), + 'aug': test_libvirtd_qemu_aug_tmp, + 'conf': qemu_conf, 'test_name': 'libvirtd_qemu', 'test_srcdir': meson.current_source_dir(), 'test_builddir': meson.current_build_dir(), diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf.in similarity index 99% rename from src/qemu/qemu.conf rename to src/qemu/qemu.conf.in index 71fd125699..04b7740136 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf.in @@ -516,11 +516,11 @@ # user = "+0" # Super user (uid=0) # user = "100" # A user named "100" or a user with uid=100 # -#user = "root" +#user = "@QEMU_USER@" # The group for QEMU processes run by the system instance. It can be # specified in a similar way to user. -#group = "root" +#group = "@QEMU_GROUP@" # Whether libvirt should dynamically change file ownership # to match the configured user/group above. Defaults to 1. diff --git a/src/qemu/test_libvirtd_qemu.aug.in b/src/qemu/test_libvirtd_qemu.aug.in index 20a89ade32..757d21c33f 100644 --- a/src/qemu/test_libvirtd_qemu.aug.in +++ b/src/qemu/test_libvirtd_qemu.aug.in @@ -47,8 +47,8 @@ module Test_libvirtd_qemu = { "security_driver" = "selinux" } { "security_default_confined" = "1" } { "security_require_confined" = "1" } -{ "user" = "root" } -{ "group" = "root" } +{ "user" = "@QEMU_USER@" } +{ "group" = "@QEMU_GROUP@" } { "dynamic_ownership" = "1" } { "remember_owner" = "1" } { "cgroup_controllers" -- 2.35.1

2 1

[PATCH] Introduce network-backed NVRAM
by Rohit Kumar 28 Mar '22

28 Mar '22

Libvirt domain XML allows only local filepaths to specify a NVRAM element. Since VMs can move across different hosts, it should be possibe to allocate NVRAM disks on network storage for uninturrupted access. This Patch extends the NVRAM disk elements to be described as virStorageSource* elements. Sample XML with new annotation: <nvram> <source protocol='iscsi' name='iqn.2013-07.com.example:iscsi-nopool/0'> <host name='example.com' port='6000'/> </source> </nvram> or <nvram> <source file='/var/lib/libvirt/nvram/guest_VARS.fd'/> </nvram> Signed-off-by: Prerna Saxena <prerna.saxena(a)nutanix.com> Signed-off-by: Florian Schmidt <flosch(a)nutanix.com> Signed-off-by: Rohit Kumar <rohit.kumar3(a)nutanix.com> --- docs/schemas/domaincommon.rng | 72 +++++++++------ src/conf/domain_conf.c | 90 +++++++++++++++---- src/conf/domain_conf.h | 2 +- src/qemu/qemu_cgroup.c | 3 +- src/qemu/qemu_command.c | 5 +- src/qemu/qemu_domain.c | 23 +++-- src/qemu/qemu_driver.c | 5 +- src/qemu/qemu_firmware.c | 17 ++-- src/qemu/qemu_namespace.c | 5 +- src/qemu/qemu_process.c | 5 +- src/security/security_dac.c | 6 +- src/security/security_selinux.c | 6 +- src/security/virt-aa-helper.c | 5 +- src/vbox/vbox_common.c | 8 +- .../qemuxml2argvdata/bios-nvram-network.args | 35 ++++++++ tests/qemuxml2argvdata/bios-nvram-network.xml | 40 +++++++++ tests/qemuxml2argvtest.c | 1 + 17 files changed, 256 insertions(+), 72 deletions(-) create mode 100644 tests/qemuxml2argvdata/bios-nvram-network.args create mode 100644 tests/qemuxml2argvdata/bios-nvram-network.xml diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 9c1b64a644..a25c84a0b7 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -330,7 +330,17 @@ </attribute> </optional> <optional> - <ref name="absFilePath"/> + <choice> + <group> + <ref name="absFilePath"/> + </group> + <group> + <ref name="diskSourceFileElement"/> + </group> + <group> + <ref name="diskSourceNetworkElement"/> + </group> + </choice> </optional> </element> </optional> @@ -1714,6 +1724,31 @@ </choice> </define> + <define name="diskSourceFileElement"> + <element name="source"> + <interleave> + <optional> + <attribute name="file"> + <choice> + <ref name="absFilePath"/> + <ref name="vmwarePath"/> + </choice> + </attribute> + </optional> + <ref name="diskSourceCommon"/> + <optional> + <ref name="storageStartupPolicy"/> + </optional> + <optional> + <ref name="encryption"/> + </optional> + <zeroOrMore> + <ref name="devSeclabel"/> + </zeroOrMore> + </interleave> + </element> + </define> + <define name="diskSourceFile"> <optional> <attribute name="type"> @@ -1721,28 +1756,7 @@ </attribute> </optional> <optional> - <element name="source"> - <interleave> - <optional> - <attribute name="file"> - <choice> - <ref name="absFilePath"/> - <ref name="vmwarePath"/> - </choice> - </attribute> - </optional> - <ref name="diskSourceCommon"/> - <optional> - <ref name="storageStartupPolicy"/> - </optional> - <optional> - <ref name="encryption"/> - </optional> - <zeroOrMore> - <ref name="devSeclabel"/> - </zeroOrMore> - </interleave> - </element> + <ref name="diskSourceFileElement"/> </optional> </define> @@ -2137,10 +2151,7 @@ </element> </define> - <define name="diskSourceNetwork"> - <attribute name="type"> - <value>network</value> - </attribute> + <define name="diskSourceNetworkElement"> <choice> <ref name="diskSourceNetworkProtocolNBD"/> <ref name="diskSourceNetworkProtocolGluster"/> @@ -2155,6 +2166,13 @@ </choice> </define> + <define name="diskSourceNetwork"> + <attribute name="type"> + <value>network</value> + </attribute> + <ref name="diskSourceNetworkElement"/> + </define> + <define name="diskSourceVolume"> <attribute name="type"> <value>volume</value> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index e0dfc9e45f..5fcc09db05 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -3535,7 +3535,7 @@ virDomainLoaderDefFree(virDomainLoaderDef *loader) return; g_free(loader->path); - g_free(loader->nvram); + virObjectUnref(loader->nvram); g_free(loader->nvramTemplate); g_free(loader); } @@ -17834,6 +17834,37 @@ virDomainLoaderDefParseXML(xmlNodePtr node, } +static int +virDomainNvramDefParseXML(virStorageSource *nvram, + xmlXPathContextPtr ctxt, + virDomainXMLOption *opt, + unsigned int flags) +{ + g_autofree char *srcTypeFile = NULL; + g_autofree char *srcTypeNetwork = NULL; + xmlNodePtr source; + + srcTypeFile = virXPathString("string(./os/nvram/source/@file)", ctxt); + srcTypeNetwork = virXPathString("string(./os/nvram/source/@protocol)", ctxt); + + if (!srcTypeFile && !srcTypeNetwork) { + nvram->type = VIR_STORAGE_TYPE_FILE; + nvram->path = virXPathString("string(./os/nvram[1])", ctxt); + return 0; + } else { + if (srcTypeFile) { + nvram->type = VIR_STORAGE_TYPE_FILE; + } else { + nvram->type = VIR_STORAGE_TYPE_NETWORK; + } + source = virXPathNode("./os/nvram/source[1]", ctxt); + if (!source) + return -1; + return virDomainStorageSourceParse(source, ctxt, nvram, flags, opt); + } + +} + static int virDomainSchedulerParseCommonAttrs(xmlNodePtr node, virProcessSchedPolicy *policy, @@ -18219,7 +18250,9 @@ virDomainDefParseBootFirmwareOptions(virDomainDef *def, static int virDomainDefParseBootLoaderOptions(virDomainDef *def, - xmlXPathContextPtr ctxt) + xmlXPathContextPtr ctxt, + virDomainXMLOption *xmlopt, + unsigned int flags) { xmlNodePtr loader_node = virXPathNode("./os/loader[1]", ctxt); const bool fwAutoSelect = def->os.firmware != VIR_DOMAIN_OS_DEF_FIRMWARE_NONE; @@ -18234,7 +18267,14 @@ virDomainDefParseBootLoaderOptions(virDomainDef *def, fwAutoSelect) < 0) return -1; - def->os.loader->nvram = virXPathString("string(./os/nvram[1])", ctxt); + if (virXPathNode("./os/nvram[1]", ctxt)) { + def->os.loader->nvram = g_new0(virStorageSource, 1); + + if (virDomainNvramDefParseXML(def->os.loader->nvram, + ctxt, xmlopt, flags) < 0) + return -1; + } + if (!fwAutoSelect) def->os.loader->nvramTemplate = virXPathString("string(./os/nvram[1]/@template)", ctxt); @@ -18288,7 +18328,9 @@ virDomainDefParseBootAcpiOptions(virDomainDef *def, static int virDomainDefParseBootOptions(virDomainDef *def, - xmlXPathContextPtr ctxt) + xmlXPathContextPtr ctxt, + virDomainXMLOption *xmlopt, + unsigned int flags) { /* * Booting options for different OS types.... @@ -18306,7 +18348,7 @@ virDomainDefParseBootOptions(virDomainDef *def, if (virDomainDefParseBootFirmwareOptions(def, ctxt) < 0) return -1; - if (virDomainDefParseBootLoaderOptions(def, ctxt) < 0) + if (virDomainDefParseBootLoaderOptions(def, ctxt, xmlopt, flags) < 0) return -1; if (virDomainDefParseBootAcpiOptions(def, ctxt) < 0) @@ -18322,7 +18364,7 @@ virDomainDefParseBootOptions(virDomainDef *def, case VIR_DOMAIN_OSTYPE_UML: virDomainDefParseBootKernelOptions(def, ctxt); - if (virDomainDefParseBootLoaderOptions(def, ctxt) < 0) + if (virDomainDefParseBootLoaderOptions(def, ctxt, xmlopt, flags) < 0) return -1; break; @@ -19606,7 +19648,7 @@ virDomainDefParseXML(xmlXPathContextPtr ctxt, if (virDomainDefClockParse(def, ctxt) < 0) return NULL; - if (virDomainDefParseBootOptions(def, ctxt) < 0) + if (virDomainDefParseBootOptions(def, ctxt, xmlopt, flags) < 0) return NULL; /* analysis of the disk devices */ @@ -26899,7 +26941,9 @@ virDomainHugepagesFormat(virBuffer *buf, static void virDomainLoaderDefFormat(virBuffer *buf, - virDomainLoaderDef *loader) + virDomainLoaderDef *loader, + virDomainXMLOption *xmlopt, + unsigned int flags) { const char *readonly = virTristateBoolTypeToString(loader->readonly); const char *secure = virTristateBoolTypeToString(loader->secure); @@ -26921,13 +26965,27 @@ virDomainLoaderDefFormat(virBuffer *buf, else virBufferAddLit(buf, "/>\n"); - if (loader->nvram || loader->nvramTemplate) { - virBufferAddLit(buf, "<nvram"); - virBufferEscapeString(buf, " template='%s'", loader->nvramTemplate); - if (loader->nvram) - virBufferEscapeString(buf, ">%s</nvram>\n", loader->nvram); - else - virBufferAddLit(buf, "/>\n"); + if (loader->nvram) { + if (loader->nvram->type == VIR_STORAGE_TYPE_FILE) { + virBufferAddLit(buf, "<nvram"); + virBufferEscapeString(buf, " template='%s'", loader->nvramTemplate); + if (loader->nvram->path) + virBufferEscapeString(buf, ">%s</nvram>\n", loader->nvram->path); + else + virBufferAddLit(buf, "/>\n"); + } else { + virBufferAddLit(buf, "<nvram"); + virBufferEscapeString(buf, " template='%s'", loader->nvramTemplate); + virBufferAdjustIndent(buf, 2); + if (virDomainDiskSourceFormat(buf, loader->nvram, "source", 0, + 0, false, flags, true, xmlopt) < 0) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Cannot format NVRAM source")); + return; + } + virBufferAdjustIndent(buf, -2); + virBufferAddLit(buf, "</nvram>\n"); + } } } @@ -28122,7 +28180,7 @@ virDomainDefFormatInternalSetRootName(virDomainDef *def, virBufferAsprintf(buf, "<initgroup>%s</initgroup>\n", def->os.initgroup); if (def->os.loader) - virDomainLoaderDefFormat(buf, def->os.loader); + virDomainLoaderDefFormat(buf, def->os.loader, xmlopt, flags); virBufferEscapeString(buf, "<kernel>%s</kernel>\n", def->os.kernel); virBufferEscapeString(buf, "<initrd>%s</initrd>\n", diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a4de46773c..f65558e82d 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2231,7 +2231,7 @@ struct _virDomainLoaderDef { virTristateBool readonly; virDomainLoader type; virTristateBool secure; - char *nvram; /* path to non-volatile RAM */ + virStorageSource *nvram; char *nvramTemplate; /* user override of path to master nvram */ }; diff --git a/src/qemu/qemu_cgroup.c b/src/qemu/qemu_cgroup.c index aa0c927578..d46c9ff36a 100644 --- a/src/qemu/qemu_cgroup.c +++ b/src/qemu/qemu_cgroup.c @@ -581,7 +581,8 @@ qemuSetupFirmwareCgroup(virDomainObj *vm) return -1; if (vm->def->os.loader->nvram && - qemuSetupImagePathCgroup(vm, vm->def->os.loader->nvram, false) < 0) + vm->def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE && + qemuSetupImagePathCgroup(vm, vm->def->os.loader->nvram->path, false) < 0) return -1; return 0; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index c836799888..649a6f5b55 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -9583,6 +9583,7 @@ qemuBuildDomainLoaderPflashCommandLine(virCommand *cmd, virQEMUCaps *qemuCaps) { g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; + g_autofree char *nvramPath = NULL; int unit = 0; if (loader->secure == VIR_TRISTATE_BOOL_YES) { @@ -9610,8 +9611,10 @@ qemuBuildDomainLoaderPflashCommandLine(virCommand *cmd, virCommandAddArgBuffer(cmd, &buf); if (loader->nvram) { + if (qemuGetDriveSourceString(loader->nvram, NULL, &nvramPath) < 0) + return; virBufferAddLit(&buf, "file="); - virQEMUBuildBufferEscapeComma(&buf, loader->nvram); + virQEMUBuildBufferEscapeComma(&buf, nvramPath); virBufferAsprintf(&buf, ",if=pflash,format=raw,unit=%d", unit); virCommandAddArg(cmd, "-drive"); diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 7180ae616b..303d9661a4 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -4452,8 +4452,13 @@ qemuDomainDefPostParse(virDomainDef *def, } if (virDomainDefHasOldStyleROUEFI(def) && - !def->os.loader->nvram) - qemuDomainNVRAMPathFormat(cfg, def, &def->os.loader->nvram); + (!def->os.loader->nvram || !def->os.loader->nvram->path)) { + + if (!def->os.loader->nvram) + def->os.loader->nvram = g_new0(virStorageSource, 1); + def->os.loader->nvram->type = VIR_STORAGE_TYPE_FILE; + qemuDomainNVRAMPathFormat(cfg, def, &def->os.loader->nvram->path); + } if (qemuDomainDefAddDefaultDevices(driver, def, qemuCaps) < 0) return -1; @@ -11133,15 +11138,23 @@ qemuDomainInitializePflashStorageSource(virDomainObj *vm) pflash0->nodeformat = g_strdup("libvirt-pflash0-format"); pflash0->nodestorage = g_strdup("libvirt-pflash0-storage"); - if (def->os.loader->nvram) { pflash1 = virStorageSourceNew(); - pflash1->type = VIR_STORAGE_TYPE_FILE; pflash1->format = VIR_STORAGE_FILE_RAW; - pflash1->path = g_strdup(def->os.loader->nvram); + pflash1->path = g_strdup(def->os.loader->nvram->path); pflash1->readonly = false; pflash1->nodeformat = g_strdup("libvirt-pflash1-format"); pflash1->nodestorage = g_strdup("libvirt-pflash1-storage"); + + if (def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE) { + pflash1->type = VIR_STORAGE_TYPE_FILE; + } else if (def->os.loader->nvram->type == VIR_STORAGE_TYPE_NETWORK) { + pflash1->protocol = def->os.loader->nvram->protocol; + pflash1->hosts = g_new0(virStorageNetHostDef, 1); + pflash1->nhosts = 1; + pflash1->hosts[0].name = def->os.loader->nvram->hosts[0].name; + pflash1->hosts[0].port = def->os.loader->nvram->hosts[0].port; + } } priv->pflash0 = g_steal_pointer(&pflash0); diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index b7e83c769a..4f4098975c 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6605,8 +6605,9 @@ qemuDomainUndefineFlags(virDomainPtr dom, } } - if (vm->def->os.loader && vm->def->os.loader->nvram) { - nvram_path = g_strdup(vm->def->os.loader->nvram); + if (vm->def->os.loader && vm->def->os.loader->nvram && + vm->def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE) { + nvram_path = g_strdup(vm->def->os.loader->nvram->path); } else if (vm->def->os.firmware == VIR_DOMAIN_OS_DEF_FIRMWARE_EFI) { qemuDomainNVRAMPathFormat(cfg, vm->def, &nvram_path); } diff --git a/src/qemu/qemu_firmware.c b/src/qemu/qemu_firmware.c index 51223faadf..fa99c7d217 100644 --- a/src/qemu/qemu_firmware.c +++ b/src/qemu/qemu_firmware.c @@ -1192,13 +1192,16 @@ qemuFirmwareEnableFeatures(virQEMUDriver *driver, VIR_FREE(def->os.loader->nvramTemplate); def->os.loader->nvramTemplate = g_strdup(flash->nvram_template.filename); - if (!def->os.loader->nvram) - qemuDomainNVRAMPathFormat(cfg, def, &def->os.loader->nvram); + if (!def->os.loader->nvram) { + def->os.loader->nvram = g_new0(virStorageSource, 1); + def->os.loader->nvram->type = VIR_STORAGE_TYPE_FILE; + qemuDomainNVRAMPathFormat(cfg, def, &def->os.loader->nvram->path); + } VIR_DEBUG("decided on firmware '%s' template '%s' NVRAM '%s'", def->os.loader->path, def->os.loader->nvramTemplate, - def->os.loader->nvram); + def->os.loader->nvram->path); break; case QEMU_FIRMWARE_DEVICE_KERNEL: @@ -1364,8 +1367,12 @@ qemuFirmwareFillDomain(virQEMUDriver *driver, * its path in domain XML) but no template for NVRAM was * specified and the varstore doesn't exist ... */ if (!virDomainDefHasOldStyleROUEFI(def) || - def->os.loader->nvramTemplate || - (!reset_nvram && virFileExists(def->os.loader->nvram))) + def->os.loader->nvramTemplate || + (def->os.loader->nvram && + !reset_nvram && ((def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE + && virFileExists(def->os.loader->nvram->path)) || + (def->os.loader->nvram->type == VIR_STORAGE_TYPE_NETWORK && + def->os.loader->nvram->path)))) return 0; /* ... then we want to consult JSON FW descriptors first, diff --git a/src/qemu/qemu_namespace.c b/src/qemu/qemu_namespace.c index 23681b14a4..18a24635ad 100644 --- a/src/qemu/qemu_namespace.c +++ b/src/qemu/qemu_namespace.c @@ -572,8 +572,9 @@ qemuDomainSetupLoader(virDomainObj *vm, case VIR_DOMAIN_LOADER_TYPE_PFLASH: *paths = g_slist_prepend(*paths, g_strdup(loader->path)); - if (loader->nvram) - *paths = g_slist_prepend(*paths, g_strdup(loader->nvram)); + if (loader->nvram && + loader->nvram->type == VIR_STORAGE_TYPE_FILE) + *paths = g_slist_prepend(*paths, g_strdup(loader->nvram->path)); break; case VIR_DOMAIN_LOADER_TYPE_NONE: diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 1ed60917ea..c8a25dbe91 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -4475,7 +4475,8 @@ qemuPrepareNVRAM(virQEMUDriver *driver, struct qemuPrepareNVRAMHelperData data; if (!loader || !loader->nvram || - (virFileExists(loader->nvram) && !reset_nvram)) + loader->nvram->type != VIR_STORAGE_TYPE_FILE || + (virFileExists(loader->nvram->path) && !reset_nvram)) return 0; master_nvram_path = loader->nvramTemplate; @@ -4507,7 +4508,7 @@ qemuPrepareNVRAM(virQEMUDriver *driver, data.srcFD = srcFD; data.srcPath = master_nvram_path; - if (virFileRewrite(loader->nvram, + if (virFileRewrite(loader->nvram->path, S_IRUSR | S_IWUSR, cfg->user, cfg->group, qemuPrepareNVRAMHelper, diff --git a/src/security/security_dac.c b/src/security/security_dac.c index e9e316551e..66c36c57a3 100644 --- a/src/security/security_dac.c +++ b/src/security/security_dac.c @@ -1971,7 +1971,8 @@ virSecurityDACRestoreAllLabel(virSecurityManager *mgr, } if (def->os.loader && def->os.loader->nvram && - virSecurityDACRestoreFileLabel(mgr, def->os.loader->nvram) < 0) + def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE && + virSecurityDACRestoreFileLabel(mgr, def->os.loader->nvram->path) < 0) rc = -1; if (def->os.kernel && @@ -2182,8 +2183,9 @@ virSecurityDACSetAllLabel(virSecurityManager *mgr, } if (def->os.loader && def->os.loader->nvram && + def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE && virSecurityDACSetOwnership(mgr, NULL, - def->os.loader->nvram, + def->os.loader->nvram->path, user, group, true) < 0) return -1; diff --git a/src/security/security_selinux.c b/src/security/security_selinux.c index 6f02baf2ce..1b6b67e8c7 100644 --- a/src/security/security_selinux.c +++ b/src/security/security_selinux.c @@ -2806,7 +2806,8 @@ virSecuritySELinuxRestoreAllLabel(virSecurityManager *mgr, } if (def->os.loader && def->os.loader->nvram && - virSecuritySELinuxRestoreFileLabel(mgr, def->os.loader->nvram, true) < 0) + def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE && + virSecuritySELinuxRestoreFileLabel(mgr, def->os.loader->nvram->path, true) < 0) rc = -1; if (def->os.kernel && @@ -3212,8 +3213,9 @@ virSecuritySELinuxSetAllLabel(virSecurityManager *mgr, /* This is different than kernel or initrd. The nvram store * is really a disk, qemu can read and write to it. */ if (def->os.loader && def->os.loader->nvram && + def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE && secdef && secdef->imagelabel && - virSecuritySELinuxSetFilecon(mgr, def->os.loader->nvram, + virSecuritySELinuxSetFilecon(mgr, def->os.loader->nvram->path, secdef->imagelabel, true) < 0) return -1; diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index 1f1cce8b3d..5803f9ff8d 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -1006,8 +1006,9 @@ get_files(vahControl * ctl) if (vah_add_file(&buf, ctl->def->os.loader->path, "rk") != 0) goto cleanup; - if (ctl->def->os.loader && ctl->def->os.loader->nvram) - if (vah_add_file(&buf, ctl->def->os.loader->nvram, "rwk") != 0) + if (ctl->def->os.loader && ctl->def->os.loader->nvram && + ctl->def->os.loader->nvram->type == VIR_STORAGE_TYPE_FILE) + if (vah_add_file(&buf, ctl->def->os.loader->nvram->path, "rwk") != 0) goto cleanup; for (i = 0; i < ctl->def->ngraphics; i++) { diff --git a/src/vbox/vbox_common.c b/src/vbox/vbox_common.c index acd18494d3..d6a482d28b 100644 --- a/src/vbox/vbox_common.c +++ b/src/vbox/vbox_common.c @@ -994,10 +994,10 @@ vboxSetBootDeviceOrder(virDomainDef *def, struct _vboxDriver *data, VIR_DEBUG("def->os.cmdline %s", def->os.cmdline); VIR_DEBUG("def->os.root %s", def->os.root); if (def->os.loader) { - VIR_DEBUG("def->os.loader->path %s", def->os.loader->path); - VIR_DEBUG("def->os.loader->readonly %d", def->os.loader->readonly); - VIR_DEBUG("def->os.loader->type %d", def->os.loader->type); - VIR_DEBUG("def->os.loader->nvram %s", def->os.loader->nvram); + VIR_DEBUG("def->os.loader->path %s", def->os.loader->path); + VIR_DEBUG("def->os.loader->readonly %d", def->os.loader->readonly); + VIR_DEBUG("def->os.loader->type %d", def->os.loader->type); + VIR_DEBUG("def->os.loader->nvram->path %s", def->os.loader->nvram->path); } VIR_DEBUG("def->os.bootloader %s", def->os.bootloader); VIR_DEBUG("def->os.bootloaderArgs %s", def->os.bootloaderArgs); diff --git a/tests/qemuxml2argvdata/bios-nvram-network.args b/tests/qemuxml2argvdata/bios-nvram-network.args new file mode 100644 index 0000000000..05075b45de --- /dev/null +++ b/tests/qemuxml2argvdata/bios-nvram-network.args @@ -0,0 +1,35 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/tmp/lib/domain--1-test-bios \ +USER=test \ +LOGNAME=test \ +XDG_DATA_HOME=/tmp/lib/domain--1-test-bios/.local/share \ +XDG_CACHE_HOME=/tmp/lib/domain--1-test-bios/.cache \ +XDG_CONFIG_HOME=/tmp/lib/domain--1-test-bios/.config \ +QEMU_AUDIO_DRV=none \ +/usr/bin/qemu-system-x86_64 \ +-name guest=test-bios,debug-threads=on \ +-S \ +-object secret,id=masterKey0,format=raw,file=/tmp/lib/domain--1-test-bios/master-key.aes \ +-machine pc,usb=off,dump-guest-core=off \ +-accel tcg \ +-drive file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on \ +-drive file=iscsi://example.org:6000/iqn.1992-01.com.example,if=pflash,format=raw,unit=1 \ +-m 1024 \ +-realtime mlock=off \ +-smp 1,sockets=1,cores=1,threads=1 \ +-uuid 362d1fc1-df7d-193e-5c18-49a71bd1da66 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-test-bios/monitor.sock,server=on,wait=off \ +-mon chardev=charmonitor,id=monitor,mode=control \ +-rtc base=utc \ +-no-shutdown \ +-boot menu=on,strict=on \ +-usb \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-ide0-0-0 \ +-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \ +-device usb-tablet,id=input0,bus=usb.0,port=1 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 \ +-msg timestamp=on diff --git a/tests/qemuxml2argvdata/bios-nvram-network.xml b/tests/qemuxml2argvdata/bios-nvram-network.xml new file mode 100644 index 0000000000..0e1e23fd31 --- /dev/null +++ b/tests/qemuxml2argvdata/bios-nvram-network.xml @@ -0,0 +1,40 @@ +<domain type='qemu'> + <name>test-bios</name> + <uuid>362d1fc1-df7d-193e-5c18-49a71bd1da66</uuid> + <memory unit='KiB'>1048576</memory> + <currentMemory unit='KiB'>1048576</currentMemory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='x86_64' machine='pc'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader> + <nvram> + <source protocol='iscsi' name='iqn.1992-01.com.example'> + <host name='example.org' port='6000'/> + </source> + </nvram> + <boot dev='hd'/> + <bootmenu enable='yes'/> + </os> + <features> + <acpi/> + </features> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>restart</on_crash> + <devices> + <emulator>/usr/bin/qemu-system-x86_64</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <controller type='usb' index='0'/> + <controller type='ide' index='0'/> + <controller type='pci' index='0' model='pci-root'/> + <input type='tablet' bus='usb'/> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index e7fecb24d3..7ec45ec74b 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1189,6 +1189,7 @@ mymain(void) QEMU_CAPS_DEVICE_ISA_SERIAL); DO_TEST_NOCAPS("bios-nvram"); DO_TEST_PARSE_ERROR_NOCAPS("bios-nvram-no-path"); + DO_TEST_NOCAPS("bios-nvram-network"); DO_TEST_CAPS_LATEST("bios-nvram-rw"); DO_TEST_CAPS_LATEST("bios-nvram-rw-implicit"); DO_TEST("bios-nvram-secure", -- 2.25.1

2 3

[PATCH] virConnectDomainEventRegisterAny: correct docs
by John Levon 28 Mar '22

28 Mar '22

The callback ID can be zero, not necessarily positive; correct the comment to reflect this. Signed-off-by: John Levon <levon(a)movementarian.org> --- src/libvirt-domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index a197618673..53efb60406 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -9486,7 +9486,7 @@ virDomainMigrateStartPostCopy(virDomainPtr domain, * The reference can be released once the object is no longer required * by calling virDomainFree(). * - * The return value from this method is a positive integer identifier + * The return value from this method is a non-negative integer identifier * for the callback. To unregister a callback, this callback ID should * be passed to the virConnectDomainEventDeregisterAny() method. * -- 2.32.0

3 2

[PATCH] virfile: Report error when changing pipe size fails
by Michal Privoznik 28 Mar '22

28 Mar '22

When changing the size of pipe that virFileWrapperFdNew() creates we start at 1MiB and if that fails because it's above the system wide limit we get EPERM and continue with half of the size. However, we might get another error in which case we should report proper system error and return failure from virFileWrapperFdNew(). Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/util/virfile.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index 12b359d550..130b0fbace 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -217,7 +217,7 @@ struct _virFileWrapperFd { * * OS note: only for linux, on other OS this is a no-op. */ -static void +static int virFileWrapperSetPipeSize(int fd) { int sz; @@ -230,21 +230,24 @@ virFileWrapperSetPipeSize(int fd) continue; /* retry with half the size */ } if (rv < 0) { - break; + virReportSystemError(errno, "%s", + _("unable to set pipe size")); + return -1; } VIR_DEBUG("fd %d pipe size adjusted to %d", fd, sz); - return; + return 0; } VIR_WARN("unable to set pipe size, data transfer might be slow: %s", g_strerror(errno)); + return 0; } # else /* !__linux__ */ -static void +static int virFileWrapperSetPipeSize(int fd G_GNUC_UNUSED) { - return; + return 0; } # endif /* !__linux__ */ @@ -323,6 +326,9 @@ virFileWrapperFdNew(int *fd, const char *name, unsigned int flags) if (virPipe(pipefd) < 0) goto error; + if (virFileWrapperSetPipeSize(pipefd[output]) < 0) + goto error; + if (!(iohelper_path = virFileFindResource("libvirt_iohelper", abs_top_builddir "/src", LIBEXECDIR))) @@ -330,8 +336,6 @@ virFileWrapperFdNew(int *fd, const char *name, unsigned int flags) ret->cmd = virCommandNewArgList(iohelper_path, name, NULL); - virFileWrapperSetPipeSize(pipefd[output]); - if (output) { virCommandSetInputFD(ret->cmd, pipefd[0]); virCommandSetOutputFD(ret->cmd, fd); -- 2.34.1

3 3

Request to move my GitLab custom executor repo under the libvirt umbrella
by Erik Skultety 28 Mar '22

28 Mar '22

So, we successfully enabled functional testing utilizing GitLab's custom executor feature. The code responsible for creating template images and provisioning and tearing down the VM workers currently lives under my GitLab account [1] and I think it would be better if we hosted it under the libvirt umbrella, especially since the project should serve as a uniform way of setting up custom runners for anyone in the libvirt community. If you think this is not a good idea, then please speak up and raise your concerns. Note there's still quite some work to be done with the project in general, but I'd be happy for any feedback in terms of what needs to be done prior moving the project to the libvirt namespace. Regards, Erik [1] https://gitlab.com/eskultety/libvirt-gitlab-executor

2 3

[libvirt RFCv4] virfile: set pipe size in virFileWrapperFdNew to improve throughput
by Claudio Fontana 28 Mar '22

28 Mar '22

currently the only user of virFileWrapperFdNew is the qemu driver; virsh save is very slow with a default pipe size. This change improves throughput by ~400% on fast nvme or ramdisk. Best value currently measured is 1MB, which happens to be also the kernel default for the pipe-max-size. Signed-off-by: Claudio Fontana <cfontana(a)suse.de> --- src/util/virfile.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) see v2 at https://listman.redhat.com/archives/libvir-list/2022-March/229423.html Changes v3 -> v4: * changed INFO and WARN messages to DEBUG (Daniel) Changes v2 -> v3: * removed reading of max-pipe-size from procfs, instead make multiple attempts on EPERM with smaller sizes. In the regular case, this should succeed on the first try. (Daniel) Changes v1 -> v2: * removed VIR_FILE_WRAPPER_BIG_PIPE, made the new pipe resizing unconditional (Michal) * moved code to separate functions (Michal) * removed ternary op, disliked in libvirt (Michal) * added #ifdef __linux__ (Ani Sinha) * try smallest value between currently best measured value (1MB) and the pipe-max-size setting. If pipe-max-size cannot be read, try kernel default max (1MB). (Daniel) diff --git a/src/util/virfile.c b/src/util/virfile.c index a04f888e06..87539be0b9 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -201,6 +201,50 @@ struct _virFileWrapperFd { }; #ifndef WIN32 + +#ifdef __linux__ + +/** + * virFileWrapperSetPipeSize: + * @fd: the fd of the pipe + * + * Set best pipe size on the passed file descriptor for bulk transfers of data. + * + * default pipe size (usually 64K) is generally not suited for large transfers + * to fast devices. A value of 1MB has been measured to improve virsh save + * by 400% in ideal conditions. We retry multiple times with smaller sizes + * on EPERM to account for possible small values of /proc/sys/fs/pipe-max-size. + * + * OS note: only for linux, on other OS this is a no-op. + */ +static void +virFileWrapperSetPipeSize(int fd) +{ + int sz; + + for (sz = 1024 * 1024; sz >= 64 * 1024; sz /= 2) { + int rv = fcntl(fd, F_SETPIPE_SZ, sz); + if (rv < 0 && errno == EPERM) { + VIR_DEBUG("EPERM trying to set fd %d pipe size to %d", fd, sz); + continue; /* retry with half the size */ + } + if (rv < 0) { + break; + } + VIR_DEBUG("fd %d pipe size adjusted to %d", fd, sz); + return; + } + virReportSystemError(errno, "%s", _("unable to set pipe size, data transfer might be slow")); +} + +#else /* !__linux__ */ +static void virFileWrapperSetPipeSize(int fd) +{ + return; +} +#endif /* !__linux__ */ + + /** * virFileWrapperFdNew: * @fd: pointer to fd to wrap @@ -282,6 +326,8 @@ virFileWrapperFdNew(int *fd, const char *name, unsigned int flags) ret->cmd = virCommandNewArgList(iohelper_path, name, NULL); + virFileWrapperSetPipeSize(pipefd[!output]); + if (output) { virCommandSetInputFD(ret->cmd, pipefd[0]); virCommandSetOutputFD(ret->cmd, fd); -- 2.35.1

3 4

[build-fix PATCH] util: virfile: Fix indentation of preprocessor directives
by Peter Krempa 28 Mar '22

28 Mar '22

stderr: cppi: /home/pipo/libvirt/src/util/virfile.c: line 205: not properly indented cppi: /home/pipo/libvirt/src/util/virfile.c: line 243: not properly indented cppi: /home/pipo/libvirt/src/util/virfile.c: line 249: not properly indented build-aux/syntax-check.mk: incorrect preprocessor indentation make: *** [/home/pipo/libvirt/build-aux/syntax-check.mk:565: sc_preprocessor_indentation] Error 1 Fixes: c61d1e9ba0a0bec18fdb0bdd485060dc27a4e5cc Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Pushed. src/util/virfile.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/util/virfile.c b/src/util/virfile.c index 6f2af35201..12b359d550 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -202,7 +202,7 @@ struct _virFileWrapperFd { #ifndef WIN32 -#ifdef __linux__ +# ifdef __linux__ /** * virFileWrapperSetPipeSize: @@ -240,13 +240,13 @@ virFileWrapperSetPipeSize(int fd) g_strerror(errno)); } -#else /* !__linux__ */ +# else /* !__linux__ */ static void virFileWrapperSetPipeSize(int fd G_GNUC_UNUSED) { return; } -#endif /* !__linux__ */ +# endif /* !__linux__ */ /** -- 2.35.1

1 0

[PATCH v2 0/6] qemu: Add qemu-namespace XML to override device properties (-set replacement)
by Peter Krempa 28 Mar '22

28 Mar '22

v2: - patch 1/7 from last time was pushed - XML changes: - top level element renamed to "qemu:override" to accomodate possible extension in the future - wrapped 'qemu:props' meant to override the frontend under a 'qemu:frontend' element - code changes: - renamed few bits (dropped 'Device') from code that can theoretically be reused - split out some functions for enhanced extensibility in the future Peter Krempa (6): docs: drvqemu: Document overriding of device properties qemu: domain: Add XML namespace code for overriding device config conf: Introduce VIR_DOMAIN_TAINT_CUSTOM_DEVICE and use it in qemu qemuBuildDeviceCommandlineFromJSON: Pass 'virDomainDef' into the function qemu: command: Override device definition according to the namespace config NEWS: Mention the qemu device property override feature NEWS.rst | 7 + docs/drvqemu.rst | 57 +++++ src/conf/domain_conf.c | 2 + src/conf/domain_conf.h | 1 + src/conf/schemas/domaincommon.rng | 42 ++++ src/qemu/qemu_command.c | 133 +++++++---- src/qemu/qemu_domain.c | 221 ++++++++++++++++++ src/qemu/qemu_domain.h | 33 +++ .../qemu-ns.x86_64-4.0.0.args | 2 +- .../qemu-ns.x86_64-latest.args | 2 +- tests/qemuxml2argvdata/qemu-ns.xml | 14 ++ .../qemu-ns.x86_64-latest.xml | 14 ++ 12 files changed, 478 insertions(+), 50 deletions(-) -- 2.35.1

2 8

[PATCH 0/2] Revert two patches I've pushed mistakenly
by Michal Privoznik 28 Mar '22

28 Mar '22

Firstly, QEMU part is not merged which alone should have prevented me from pushing these patches. Secondly, in the end QEMU is fixing the bug in a different manner that does not require any libvirt work. https://lists.gnu.org/archive/html/qemu-devel/2022-03/msg06077.html https://lists.gnu.org/archive/html/qemu-devel/2022-03/msg06188.html Michal Prívozník (2): Revert "domain: add tsc.on_reboot element" Revert "qemu: add support for tsc.on_reboot element" docs/formatdomain.rst | 4 -- src/conf/domain_conf.c | 17 ------- src/conf/domain_conf.h | 10 ----- src/conf/schemas/domaincommon.rng | 8 ---- src/qemu/qemu_capabilities.c | 2 - src/qemu/qemu_capabilities.h | 1 - src/qemu/qemu_command.c | 11 ----- src/qemu/qemu_validate.c | 45 ++++++++----------- .../caps_7.0.0.x86_64.replies | 4 -- .../caps_7.0.0.x86_64.xml | 1 - .../cpu-tsc-clear-on-reset.args | 32 ------------- .../cpu-tsc-clear-on-reset.x86_64-7.0.0.args | 34 -------------- .../cpu-tsc-clear-on-reset.xml | 35 --------------- tests/qemuxml2argvtest.c | 2 - 14 files changed, 19 insertions(+), 187 deletions(-) delete mode 100644 tests/qemuxml2argvdata/cpu-tsc-clear-on-reset.args delete mode 100644 tests/qemuxml2argvdata/cpu-tsc-clear-on-reset.x86_64-7.0.0.args delete mode 100644 tests/qemuxml2argvdata/cpu-tsc-clear-on-reset.xml -- 2.34.1

2 3

[PATCH v2 0/7] Do async IO when starting swtpm emulator
by Michal Privoznik 28 Mar '22

28 Mar '22

v2 of: https://listman.redhat.com/archives/libvir-list/2022-March/229433.html diff to v1: - More fixes - New test case, yay! Michal Prívozník (7): qemu_tpm: Do async IO when starting swtpm emulator vircommand: Document virCommandSetSendBuffer() behaviour wrt daemonize vircommand: Don't set nonblocking FDs in virCommandDoAsyncIO() commandtest: Use unsigned char in test27() virCommandSetSendBuffer: Take double pointer of @buffer commandtest: Test virCommandSetSendBuffer() with virCommandDoAsyncIO() commandtest: Use virTestCompareToFile() in checkoutput() src/qemu/qemu_tpm.c | 3 +- src/util/vircommand.c | 12 ++-- src/util/vircommand.h | 2 +- tests/commanddata/test29.log | 19 ++++++ tests/commandtest.c | 110 +++++++++++++++++++++++++++++------ 5 files changed, 121 insertions(+), 25 deletions(-) create mode 100644 tests/commanddata/test29.log -- 2.34.1

4 10

[PATCH 0/2] domain, qemu: add support for clearing TSC on reset
by Paolo Bonzini 28 Mar '22

28 Mar '22

Some versions of Windows hang on reboot if their TSC value is greater than 2^54. The workaround is to reset the TSC to a small value, and is implemented by QEMU and ESXi. This series adds a domain attribute for this, and implements it in QEMU. Paolo Paolo Bonzini (2): domain: add tsc.on_reboot element qemu: add support for tsc.on_reboot element docs/formatdomain.rst | 4 +++ src/conf/domain_conf.c | 22 ++++++++++++ src/conf/domain_conf.h | 10 ++++++ src/conf/schemas/domaincommon.rng | 9 +++++ src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 4 +++ src/qemu/qemu_validate.c | 36 +++++++++++++------ .../caps_7.0.0.x86_64.replies | 4 +++ .../caps_7.0.0.x86_64.xml | 1 + ...uency.args => cpu-tsc-clear-on-reset.args} | 2 +- ... cpu-tsc-clear-on-reset.x86_64-7.0.0.args} | 6 ++-- ...equency.xml => cpu-tsc-clear-on-reset.xml} | 2 +- tests/qemuxml2argvtest.c | 2 ++ 14 files changed, 89 insertions(+), 16 deletions(-) copy tests/qemuxml2argvdata/{cpu-tsc-frequency.args => cpu-tsc-clear-on-reset.args} (97%) copy tests/qemuxml2argvdata/{virtio-rng-builtin.x86_64-latest.args => cpu-tsc-clear-on-reset.x86_64-7.0.0.args} (77%) copy tests/qemuxml2argvdata/{cpu-tsc-frequency.xml => cpu-tsc-clear-on-reset.xml} (95%) -- 2.35.1

2 8

[libvirt PATCH] iohelper: some refactoring
by Claudio Fontana 26 Mar '22

26 Mar '22

while doing research with alternative implementations of runIO, it seemed necessary to do some refactoring, in order to separate parameter setting from the actual copy, so that alternative copy methods can be researched and hopefully eventually implemented. No functional changes are expected. Signed-off-by: Claudio Fontana <cfontana(a)suse.de> --- src/util/iohelper.c | 193 +++++++++++++++++++++++++++----------------- 1 file changed, 118 insertions(+), 75 deletions(-) diff --git a/src/util/iohelper.c b/src/util/iohelper.c index 2c91bf4f93..c04c97af27 100644 --- a/src/util/iohelper.c +++ b/src/util/iohelper.c @@ -45,21 +45,38 @@ # define O_DIRECT 0 #endif -static int -runIO(const char *path, int fd, int oflags) +struct runIOParams { + bool isBlockDev; + bool isDirect; + bool isWrite; + int fdin; + const char *fdinname; + int fdout; + const char *fdoutname; +}; + +/* runIOCopy: execute the IO copy based on the passed parameters */ +/** + * runIOCopy: + * @p: the IO parameters + * + * Execute the copy based on the passed parameters. + * + * Returns: size transfered, or < 0 on error. + * Errors: -1 = read/write error + * -2 = read error + * -3 = write error + * -4 = truncate error + */ + +static ssize_t +runIOCopy(const struct runIOParams p) { g_autofree void *base = NULL; /* Location to be freed */ char *buf = NULL; /* Aligned location within base */ size_t buflen = 1024*1024; intptr_t alignMask = 64*1024 - 1; - int ret = -1; - int fdin, fdout; - const char *fdinname, *fdoutname; - unsigned long long total = 0; - bool direct = O_DIRECT && ((oflags & O_DIRECT) != 0); - off_t end = 0; - struct stat sb; - bool isBlockDev = false; + ssize_t total = 0; #if WITH_POSIX_MEMALIGN if (posix_memalign(&base, alignMask + 1, buflen)) @@ -71,50 +88,6 @@ runIO(const char *path, int fd, int oflags) buf = (char *) (((intptr_t) base + alignMask) & ~alignMask); #endif - if (fstat(fd, &sb) < 0) { - virReportSystemError(errno, - _("Unable to access file descriptor %d path %s"), - fd, path); - goto cleanup; - } - isBlockDev = S_ISBLK(sb.st_mode); - - switch (oflags & O_ACCMODE) { - case O_RDONLY: - fdin = fd; - fdinname = path; - fdout = STDOUT_FILENO; - fdoutname = "stdout"; - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ - if (!isBlockDev && direct && ((end = lseek(fd, 0, SEEK_CUR)) != 0)) { - virReportSystemError(end < 0 ? errno : EINVAL, "%s", - _("O_DIRECT read needs entire seekable file")); - goto cleanup; - } - break; - case O_WRONLY: - fdin = STDIN_FILENO; - fdinname = "stdin"; - fdout = fd; - fdoutname = path; - /* To make the implementation simpler, we give up on any - * attempt to use O_DIRECT in a non-trivial manner. */ - if (!isBlockDev && direct && (end = lseek(fd, 0, SEEK_END)) != 0) { - virReportSystemError(end < 0 ? errno : EINVAL, "%s", - _("O_DIRECT write needs empty seekable file")); - goto cleanup; - } - break; - - case O_RDWR: - default: - virReportSystemError(EINVAL, - _("Unable to process file with flags %d"), - (oflags & O_ACCMODE)); - goto cleanup; - } - while (1) { ssize_t got; @@ -124,53 +97,123 @@ runIO(const char *path, int fd, int oflags) * writes will be aligned. * In other cases using saferead reduces number of syscalls. */ - if (fdin == fd && direct) { - if ((got = read(fdin, buf, buflen)) < 0 && + if (!p.isWrite && p.isDirect) { + if ((got = read(p.fdin, buf, buflen)) < 0 && errno == EINTR) continue; } else { - got = saferead(fdin, buf, buflen); - } - - if (got < 0) { - virReportSystemError(errno, _("Unable to read %s"), fdinname); - goto cleanup; + got = saferead(p.fdin, buf, buflen); } + if (got < 0) + return -2; if (got == 0) break; total += got; /* handle last write size align in direct case */ - if (got < buflen && direct && fdout == fd) { + if (got < buflen && p.isDirect && p.isWrite) { ssize_t aligned_got = (got + alignMask) & ~alignMask; memset(buf + got, 0, aligned_got - got); - if (safewrite(fdout, buf, aligned_got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), fdoutname); - goto cleanup; + if (safewrite(p.fdout, buf, aligned_got) < 0) { + return -3; } - - if (!isBlockDev && ftruncate(fd, total) < 0) { - virReportSystemError(errno, _("Unable to truncate %s"), fdoutname); - goto cleanup; + if (!p.isBlockDev && ftruncate(p.fdout, total) < 0) { + return -4; } - break; } + } + return total; +} - if (safewrite(fdout, buf, got) < 0) { - virReportSystemError(errno, _("Unable to write %s"), fdoutname); +static int +runIO(const char *path, int fd, int oflags) +{ + int ret = -1; + off_t total = 0; + struct stat sb; + struct runIOParams p; + + if (fstat(fd, &sb) < 0) { + virReportSystemError(errno, + _("Unable to access file descriptor %d path %s"), + fd, path); + goto cleanup; + } + p.isBlockDev = S_ISBLK(sb.st_mode); + p.isDirect = O_DIRECT && (oflags & O_DIRECT); + + switch (oflags & O_ACCMODE) { + case O_RDONLY: + p.isWrite = false; + p.fdin = fd; + p.fdinname = path; + p.fdout = STDOUT_FILENO; + p.fdoutname = "stdout"; + break; + case O_WRONLY: + p.isWrite = true; + p.fdin = STDIN_FILENO; + p.fdinname = "stdin"; + p.fdout = fd; + p.fdoutname = path; + break; + case O_RDWR: + default: + virReportSystemError(EINVAL, + _("Unable to process file with flags %d"), + (oflags & O_ACCMODE)); + goto cleanup; + } + /* To make the implementation simpler, we give up on any + * attempt to use O_DIRECT in a non-trivial manner. */ + if (!p.isBlockDev && p.isDirect) { + off_t end; + if (p.isWrite) { + if ((end = lseek(fd, 0, SEEK_END)) != 0) { + virReportSystemError(end < 0 ? errno : EINVAL, "%s", + _("O_DIRECT write needs empty seekable file")); + goto cleanup; + } + } else if ((end = lseek(fd, 0, SEEK_CUR)) != 0) { + virReportSystemError(end < 0 ? errno : EINVAL, "%s", + _("O_DIRECT read needs entire seekable file")); goto cleanup; } } + total = runIOCopy(p); + + if (total < 0) { + switch (total) { + case -1: + virReportSystemError(errno, _("Unable to move data from %s to %s"), + p.fdinname, p.fdoutname); + break; + case -2: + virReportSystemError(errno, _("Unable to read %s"), p.fdinname); + break; + case -3: + virReportSystemError(errno, _("Unable to write %s"), p.fdoutname); + break; + case -4: + virReportSystemError(errno, _("Unable to truncate %s"), p.fdoutname); + break; + default: + virReportError(VIR_ERR_INTERNAL_ERROR, _("Unknown runIOCopy error %ld"), total); + break; + } + goto cleanup; + } + /* Ensure all data is written */ - if (virFileDataSync(fdout) < 0) { + if (virFileDataSync(p.fdout) < 0) { if (errno != EINVAL && errno != EROFS) { /* fdatasync() may fail on some special FDs, e.g. pipes */ - virReportSystemError(errno, _("unable to fsync %s"), fdoutname); + virReportSystemError(errno, _("unable to fsync %s"), p.fdoutname); goto cleanup; } } -- 2.35.1

1 1

[PATCH v2 00/19] Refactoring conf to use XPath
by Kristina Hanicova 25 Mar '22

25 Mar '22

This is v2 from https://listman.redhat.com/archives/libvir-list/2021-April/msg00617.html Changes since v1: - rebase to the lastest git master This series reworks the outdated way of parsing XML to parsing by XPath, which is more obvious and saves a few lines of code. Kristina Hanicova (19): conf: Propagate xmlXPathContextPtr into virDomainHostdevSubsysUSBDefParseXML() Refactoring virDomainHostdevSubsysUSBDefParseXML() to use XPath conf: Propagate xmlXPathContextPtr into virDomainBlkioDeviceParseXML() Refactoring virDomainBlkioDeviceParseXML() to use XPath conf: Propagate xmlXPathContextPtr into virDomainHostdevSubsysPCIDefParseXML() Refactoring virDomainHostdevSubsysPCIDefParseXML() to use XPath conf: Propagate xmlXPathContextPtr into virDomainLeaseDefParseXML() Refactoring virDomainLeaseDefParseXML() to use XPath Refactoring virDomainFSDefParseXML() to use XPath Refactoring virDomainNetDefParseXML() to use XPath conf: Propagate xmlXPathContextPtr into virDomainChrDefParseTargetXML() Refactoring virDomainChrDefParseTargetXML() to use XPath Refactoring virDomainChrSourceDefParseXML() to use XPath Refactoring virDomainChrDefParseXML() to use XPath Refactoring virDomainSmartcardDefParseXML() to use XPath Refactoring virDomainGraphicsDefParseXMLSpice() to use XPath conf: Propagate xmlXPathContextPtr into virDomainVideoDriverDefParseXML() Refactoring virDomainVideoDriverDefParseXML() to use XPath Refactoring virDomainVideoDefParseXML() to use XPath src/conf/domain_conf.c | 1641 +++++++++++++++++----------------------- 1 file changed, 707 insertions(+), 934 deletions(-) -- 2.30.2

3 26

[PATCH v2] meson: do not look for libparted if not necessary
by Paolo Bonzini 25 Mar '22

25 Mar '22

libparted_dep is not used if -Dstorage_disk=disabled. Do not bother looking for it if the disk storage backend was not requested. Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meson.build b/meson.build index eb13c7efa4..27c21d1d67 100644 --- a/meson.build +++ b/meson.build @@ -1013,7 +1013,7 @@ else endif libparted_version = '1.8.0' -libparted_dep = dependency('libparted', version: '>=' + libparted_version, required: false) +libparted_dep = dependency('libparted', version: '>=' + libparted_version, required: get_option('storage_disk')) libpcap_version = '1.5.0' if not get_option('libpcap').disabled() -- 2.35.1

2 1

[PATCH] meson: do not look for librbd/librados if not requested
by Paolo Bonzini 25 Mar '22

25 Mar '22

rbd_dep is not used if -Dstorage_rbd=disabled. Do not bother looking for the libraries that compose it if the rbd storage backend was not requested. Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- meson.build | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meson.build b/meson.build index 2c1d3bc64d..eb13c7efa4 100644 --- a/meson.build +++ b/meson.build @@ -1127,8 +1127,8 @@ parallels_sdk_dep = dependency('parallels-sdk', version: '>=' + parallels_sdk_ve pciaccess_version = '0.10.0' pciaccess_dep = dependency('pciaccess', version: '>=' + pciaccess_version, required: get_option('pciaccess')) -rbd_dep = cc.find_library('rbd', required: false) -rados_dep = cc.find_library('rados', required: false) +rbd_dep = cc.find_library('rbd', required: get_option('storage_rbd')) +rados_dep = cc.find_library('rados', required: get_option('storage_rbd')) if rbd_dep.found() and not cc.has_function('rbd_get_features', dependencies: rbd_dep) rbd_dep = dependency('', required: false) endif -- 2.35.1

2 1

[PATCH] meson: do not look for libdevmapper/libparted if not requested
by Paolo Bonzini 25 Mar '22

25 Mar '22

devmapper_dep and libparted_dep are not used if -Dstorage_disk=disabled. Do not bother looking for these libraries if the disk storage backend was not requested. Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- meson.build | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meson.build b/meson.build index eb13c7efa4..9d1f90fc80 100644 --- a/meson.build +++ b/meson.build @@ -933,7 +933,7 @@ if curl_dep.found() endif devmapper_version = '1.0.0' -devmapper_dep = dependency('devmapper', version: '>=' + devmapper_version, required: false) +devmapper_dep = dependency('devmapper', version: '>=' + devmapper_version, required: get_option('storage_disk')) if devmapper_dep.found() conf.set('WITH_DEVMAPPER', 1) endif @@ -1013,7 +1013,7 @@ else endif libparted_version = '1.8.0' -libparted_dep = dependency('libparted', version: '>=' + libparted_version, required: false) +libparted_dep = dependency('libparted', version: '>=' + libparted_version, required: get_option('storage_disk')) libpcap_version = '1.5.0' if not get_option('libpcap').disabled() -- 2.35.1

2 1

[PATCH] tests: add dependencies to meson declaration
by Paolo Bonzini 25 Mar '22

25 Mar '22

Make sure that all tests are run after the helpers and mocks are (re)built. This enables for example using "meson test" as the command line passed to "git bisect run". Signed-off-by: Paolo Bonzini <pbonzini(a)redhat.com> --- tests/meson.build | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tests/meson.build b/tests/meson.build index 8f92f2033f..fd78d8a2fd 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -191,8 +191,9 @@ test_file_wrapper_lib = static_library( dependencies: [ tests_dep ], ) +tests_deps = [] foreach mock : mock_libs - shared_library( + tests_deps += shared_library( mock['name'], mock.get('sources', [ '@0@.c'.format(mock['name']) ]), override_options: [ @@ -215,7 +216,7 @@ endforeach # Must not link to any libvirt modules - libc only otherwise external # libraries might unexpectedly leak file descriptors into commandhelper # invalidating the test logic assumptions. -executable( +tests_deps += executable( 'commandhelper', [ 'commandhelper.c' ], dependencies: [ @@ -227,7 +228,7 @@ executable( ) # This is a fake SSH we use from virnetsockettest -executable( +tests_deps += executable( 'ssh', [ 'ssh.c' ], dependencies: [ @@ -592,7 +593,7 @@ foreach data : tests # default meson timeout timeout = 30 endif - test(data['name'], test_bin, env: tests_env, timeout: timeout) + test(data['name'], test_bin, env: tests_env, timeout: timeout, depends: tests_deps) endforeach -- 2.35.1

2 1

[libvirt RFCv3] virfile: set pipe size in virFileWrapperFdNew to improve throughput
by Claudio Fontana 25 Mar '22

25 Mar '22

currently the only user of virFileWrapperFdNew is the qemu driver; virsh save is very slow with a default pipe size. This change improves throughput by ~400% on fast nvme or ramdisk. Best value currently measured is 1MB, which happens to be also the kernel default for the pipe-max-size. Signed-off-by: Claudio Fontana <cfontana(a)suse.de> --- see v2 at https://listman.redhat.com/archives/libvir-list/2022-March/229423.html Changes v2 -> v3: * removed reading of max-pipe-size from procfs, instead make multiple attempts on EPERM with smaller sizes. In the regular case, this should succeed on the first try. (Daniel) Changes v1 -> v2: * removed VIR_FILE_WRAPPER_BIG_PIPE, made the new pipe resizing unconditional (Michal) * moved code to separate functions (Michal) * removed ternary op, disliked in libvirt (Michal) * added #ifdef __linux__ (Ani Sinha) * try smallest value between currently best measured value (1MB) and the pipe-max-size setting. If pipe-max-size cannot be read, try kernel default max (1MB). (Daniel) src/util/virfile.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/src/util/virfile.c b/src/util/virfile.c index a04f888e06..876b865974 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -201,6 +201,51 @@ struct _virFileWrapperFd { }; #ifndef WIN32 + +#ifdef __linux__ + +/** + * virFileWrapperSetPipeSize: + * @fd: the fd of the pipe + * + * Set best pipe size on the passed file descriptor for bulk transfers of data. + * + * default pipe size (usually 64K) is generally not suited for large transfers + * to fast devices. A value of 1MB has been measured to improve virsh save + * by 400% in ideal conditions. We retry multiple times with smaller sizes + * on EPERM to account for possible small values of /proc/sys/fs/pipe-max-size. + * + * Return value is 0 on success, -1 and errno set on error. + * OS note: only for linux, on other OS this is a no-op. + */ +static int +virFileWrapperSetPipeSize(int fd) +{ + int sz; + + for (sz = 1024 * 1024; sz >= 64 * 1024; sz /= 2) { + int rv = fcntl(fd, F_SETPIPE_SZ, sz); + if (rv < 0 && errno == EPERM) { + continue; /* retry with half the size */ + } + if (rv < 0) { + break; + } + VIR_INFO("fd %d pipe size adjusted to %d", fd, sz); + return 0; + } + VIR_WARN("failed to set pipe size to %d (errno=%d)", sz, errno); + return -1; +} + +#else /* !__linux__ */ +static int virFileWrapperSetPipeSize(int fd) +{ + return 0; +} +#endif /* !__linux__ */ + + /** * virFileWrapperFdNew: * @fd: pointer to fd to wrap @@ -282,6 +327,10 @@ virFileWrapperFdNew(int *fd, const char *name, unsigned int flags) ret->cmd = virCommandNewArgList(iohelper_path, name, NULL); + if (virFileWrapperSetPipeSize(pipefd[!output]) < 0) { + virReportError(VIR_ERR_SYSTEM_ERROR, "%s", _("unable to set pipe size, data transfer might be slow")); + } + if (output) { virCommandSetInputFD(ret->cmd, pipefd[0]); virCommandSetOutputFD(ret->cmd, fd); -- 2.35.1

2 5

[PATCH 0/5] Generalize job enums across all hypervisors
by Kristina Hanicova 25 Mar '22

25 Mar '22

2 7