August 2024 - Devel - libvirt List Archives

[PATCH RFC v4 01/17] schema: Add new domain elements to support multiple throttle groups
by danielwuwy＠163.com 28 Aug '24

28 Aug '24

From: Chun Feng Wu <danielwuwy(a)163.com> Introduce schema for defining '<throttlegroups>' element which configures throttling groups which can be configured for multiple disks. * Refactor "diskIoTune" to extract common schema "iotune" * Add new elements '<throttlegroups>' * <ThrottleGroups> contains <ThrottleGroup> defintion, which references "iotune" Signed-off-by: Chun Feng Wu <danielwuwy(a)163.com> --- docs/formatdomain.rst | 26 +++ src/conf/schemas/domaincommon.rng | 274 ++++++++++++++++-------------- 2 files changed, 174 insertions(+), 126 deletions(-) diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 47d3e2125e..0022904a50 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -1957,6 +1957,32 @@ advertisements to the guest OS. (NB: Only qemu driver support) the guest OS itself can choose to circumvent the unavailability of the sleep states (e.g. S4 by turning off completely). +Disk Throttle Group Management +------------------------------ + +:since:`Since 10.7.0` it is possible to create multiple named throttle groups +and then reference them within ``throttlefilters``(sub-element of ``disk`` element) +to form filter chain in QEMU for specific disk. The limits(throttlegroups) are +shared within domain, hence the same group can be referenced by different filters. + +:: + + <domain> + ... + <throttlegroups> + <throttlegroup> + <group_name>limit0</group_name> + <total_bytes_sec>10000000</total_bytes_sec> + <read_iops_sec>400000</read_iops_sec> + <write_iops_sec>100000</write_iops_sec> + </throttlegroup> + </throttlegroups> + ... + </domain> + +``throttlegroup`` + It has the same sub-elements as ``iotune`` (See `Hard drives, floppy disks, CDROMs`_), + The difference is that <group_name> is required. Hypervisor features ------------------- diff --git a/src/conf/schemas/domaincommon.rng b/src/conf/schemas/domaincommon.rng index efb5f00d77..27cd9d9c8c 100644 --- a/src/conf/schemas/domaincommon.rng +++ b/src/conf/schemas/domaincommon.rng @@ -51,6 +51,7 @@ </zeroOrMore> <ref name="os"/> <ref name="clock"/> + <ref name="throttlegroups"/> <ref name="resources"/> <ref name="features"/> <ref name="events"/> @@ -6759,6 +6760,23 @@ </interleave> </element> </define> + <define name="throttlegroup"> + <element name="throttlegroup"> + <ref name="iotune"/> + </element> + </define> +  + <define name="throttlegroups"> + <optional> + <element name="throttlegroups"> + <zeroOrMore> + <ref name="throttlegroup"/> + </zeroOrMore> + </element> + </optional> + </define> <!-- A set of optional features: PAE, APIC, ACPI, GIC, TCG, HyperV Enlightenment, KVM features, paravirtual spinlocks and HAP support @@ -7726,134 +7744,138 @@ </element> </define> + <define name="iotune"> + <interleave> + <choice> + <element name="total_bytes_sec"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec_max"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec_max"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <optional> + <element name="size_iops_sec"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="group_name"> + <text/> + </element> + </optional> + <choice> + <element name="total_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_bytes_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + <choice> + <element name="total_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + <group> + <interleave> + <optional> + <element name="read_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + <optional> + <element name="write_iops_sec_max_length"> + <data type="unsignedLong"/> + </element> + </optional> + </interleave> + </group> + </choice> + </interleave> + </define> + <define name="diskIoTune"> <element name="iotune"> - <interleave> - <choice> - <element name="total_bytes_sec"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_bytes_sec"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_bytes_sec"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - <choice> - <element name="total_iops_sec"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_iops_sec"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_iops_sec"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - <choice> - <element name="total_bytes_sec_max"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_bytes_sec_max"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_bytes_sec_max"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - <choice> - <element name="total_iops_sec_max"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_iops_sec_max"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_iops_sec_max"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - <optional> - <element name="size_iops_sec"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="group_name"> - <text/> - </element> - </optional> - <choice> - <element name="total_bytes_sec_max_length"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_bytes_sec_max_length"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_bytes_sec_max_length"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - <choice> - <element name="total_iops_sec_max_length"> - <data type="unsignedLong"/> - </element> - <group> - <interleave> - <optional> - <element name="read_iops_sec_max_length"> - <data type="unsignedLong"/> - </element> - </optional> - <optional> - <element name="write_iops_sec_max_length"> - <data type="unsignedLong"/> - </element> - </optional> - </interleave> - </group> - </choice> - </interleave> + <ref name="iotune"/> </element> </define> -- 2.43.0

1 10

[PATCH RFC v4 00/17] Support throttle block filters
by danielwuwy＠163.com 28 Aug '24

28 Aug '24

From: Chun Feng Wu <danielwuwy(a)163.com> Hi, I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives." The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: domthrottlegroupset Add or update a throttling group. domthrottlegroupdel Delete a throttling group. domthrottlegroupinfo Get a throttling group. domthrottlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest. Note, this implementation requires flag "QEMU_CAPS_OBJECT_JSON". From QMP perspective, the sample flow works this way: - Throttle group creation: virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit0","limits":{"iops-total":200,"iops-read":0,"iops-total-max":200,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit1","limits":{"iops-total":250,"iops-read":0,"iops-total-max":250,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit2","limits":{"iops-total":300,"iops-read":0,"iops-total-max":300,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit012","limits":{"iops-total":400,"iops-read":0,"iops-total-max":400,"iops-total-max-length":1}}}' - Chain up filters during attaching disk to apply two filters(limit0 and limit012): virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"file","filename":"/virt/disks/vm1_disk_1.qcow2","node-name":"test-3-storage","auto-read-only":true,"discard":"unmap"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"node-name":"test-4-format","read-only":false,"driver":"qcow2","file":"test-3-storage","backing":null}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"driver":"throttle","node-name":"libvirt-5-filter","throttle-group": "limit0","file":"test-4-format"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"throttle","node-name":"libvirt-6-filter","throttle-group":"limit012","file":"libvirt-5-filter"}}' virsh qemu-monitor-command 1 '{"execute": "device_add", "arguments": {"driver":"virtio-blk-pci","scsi":false,"bus":"pci.0","addr":"0x5","drive":"libvirt-6-filter","id":"virtio-disk1"}}' This patchset includes: - schema: Add new domain elements to support multiple throttle groups - schema: Add new domain elements to support multiple throttle filters - config: Introduce ThrottleGroup and corresponding XML parsing - config: Introduce ThrottleFilter and corresponding XML parsing - qemu: monitor: Add support for ThrottleGroup operations - tests: Test qemuMonitorJSONGetThrottleGroup and qemuMonitorJSONUpdateThrottleGroup - remote: New APIs for ThrottleGroup lifecycle management - qemu: Refactor qemuDomainSetBlockIoTune to extract common methods - qemu: Implement qemu driver for throttle API - qemu: helper: throttle filter nodename and preparation processing - qemu: block: Support block disk along with throttle filters - config: validate: Verify iotune, throttle group and filter - qemuxmlconftest: Add 'throttlefilter' tests - test_driver: Test throttle group lifecycle APIs - virsh: Refactor iotune options for re-use - virsh: Add support for throttle group operations - virsh: Add option "throttle-groups" to "attach_disk" v4 changes: - Format commit msgs - Update version to '10.7.0' - Update formatdomain.rst to remove nodename chain sample which exists in QEMU, and add description that order within <throttlefilters> doesn't mater - delete "virDomainThrottleGroupFind", which is duplicate to "virDomainThrottleGroupByName" - Remove "id" within _virDomainThrottleFilterDef, which is not required - Avoid export for methods not used anywhere else - Add manpage for new virsh cmds and options - Update error code&msg - Consolidate validation logic into one commit - Format throttle filter nodename into disk/privateData/nodenames/nodename(type='throttle-filter') and parse it reversely - Add tests for statusxml(disk/privateData/nodenames/nodename) test regarding throttle filter nodename parse and format - Add more complicated test cases and negative case for throttle filter xml - Add a 'throttle-' prefix so that we clearly separate the throttle group objects into their own QEMU namespace. - Update virDomainGetThrottleGroup and virDomainDelThrottleGroup to remove TYPED_PARAM_STRING check - Update remote_protocol.x "REMOTE_PROC_DOMAIN_DEL_THROTTLE_GROUP" to set "@acl: domain:write" rather than "read" - Split commit of "qemu: Implement qemu driver for throttle API" into two commits - Check QEMU_CAPS_OBJECT_JSON when vm is virDomainObjIsActive(vm), and update error msg - Address leak in qemuBuildThrottleFiltersAttachPrepareBlockdev - Prefix new cmd with "dom", e.g. domthrottlegroupset - Refactor completer and "domthrottlegrouplist" to share the same method "virshGetThrottleGroupNames" - Update qemuBlockThrottleFilterAttachPrepareBlockdev to remove "data->filterAttached = true;" - Support CopyOnRead by re-org chain: device -> throttle -> copyOnRead Layer-> image chain Any comments/suggestions will be appriciated! Chun Feng Wu (17): schema: Add new domain elements to support multiple throttle groups schema: Add new domain elements to support multiple throttle filters config: Introduce ThrottleGroup and corresponding XML parsing config: Introduce ThrottleFilter and corresponding XML parsing qemu: monitor: Add support for ThrottleGroup operations tests: Test qemuMonitorJSONGetThrottleGroup and qemuMonitorJSONUpdateThrottleGroup remote: New APIs for ThrottleGroup lifecycle management qemu: Refactor qemuDomainSetBlockIoTune to extract common methods qemu: Implement qemu driver for throttle API qemu: helper: throttle filter nodename and preparation processing qemu: block: Support block disk along with throttle filters config: validate: Verify iotune, throttle group and filter qemuxmlconftest: Add 'throttlefilter' tests test_driver: Test throttle group lifecycle APIs virsh: Refactor iotune options for re-use virsh: Add support for throttle group operations virsh: Add option "throttle-groups" to "attach_disk" docs/formatdomain.rst | 47 ++ docs/manpages/virsh.rst | 135 +++- include/libvirt/libvirt-domain.h | 21 + src/conf/domain_conf.c | 398 ++++++++++ src/conf/domain_conf.h | 45 ++ src/conf/domain_validate.c | 119 ++- src/conf/schemas/domaincommon.rng | 293 ++++---- src/conf/virconftypes.h | 4 + src/driver-hypervisor.h | 22 + src/libvirt-domain.c | 174 +++++ src/libvirt_private.syms | 8 + src/libvirt_public.syms | 7 + src/qemu/qemu_block.c | 136 ++++ src/qemu/qemu_block.h | 49 ++ src/qemu/qemu_command.c | 180 +++++ src/qemu/qemu_command.h | 6 + src/qemu/qemu_domain.c | 73 +- src/qemu/qemu_driver.c | 619 +++++++++++++--- src/qemu/qemu_hotplug.c | 33 + src/qemu/qemu_monitor.c | 34 + src/qemu/qemu_monitor.h | 14 + src/qemu/qemu_monitor_json.c | 134 ++++ src/qemu/qemu_monitor_json.h | 14 + src/remote/remote_daemon_dispatch.c | 44 ++ src/remote/remote_driver.c | 40 ++ src/remote/remote_protocol.x | 48 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 452 ++++++++---- tests/qemumonitorjsontest.c | 86 +++ .../throttlefilter-in.xml | 392 ++++++++++ .../throttlefilter-out.xml | 392 ++++++++++ tests/qemuxmlactivetest.c | 1 + .../throttlefilter-invalid.x86_64-latest.err | 1 + .../throttlefilter-invalid.xml | 89 +++ .../throttlefilter.x86_64-latest.args | 55 ++ .../throttlefilter.x86_64-latest.xml | 105 +++ tests/qemuxmlconfdata/throttlefilter.xml | 95 +++ tests/qemuxmlconftest.c | 2 + tools/virsh-completer-domain.c | 82 +++ tools/virsh-completer-domain.h | 16 + tools/virsh-domain.c | 680 ++++++++++++++---- 41 files changed, 4648 insertions(+), 525 deletions(-) create mode 100644 tests/qemustatusxml2xmldata/throttlefilter-in.xml create mode 100644 tests/qemustatusxml2xmldata/throttlefilter-out.xml create mode 100644 tests/qemuxmlconfdata/throttlefilter-invalid.x86_64-latest.err create mode 100644 tests/qemuxmlconfdata/throttlefilter-invalid.xml create mode 100644 tests/qemuxmlconfdata/throttlefilter.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/throttlefilter.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/throttlefilter.xml -- 2.43.0

1 0

[PATCH] virschematest: Replace g_lstat() with virFileIsLink()
by Michal Privoznik 27 Aug '24

27 Aug '24

Inside of virschematest.c there's testSchemaDir() which iterates over dentries in given directory but skips some files: those without ".xml" suffix, hidden files, symlinks, etc. Now, symlinks are detected as g_lstat() + S_ISLNK() combo which works, except it fails to compile on mingw where is no concept of symlinks. Replace the combo with a call to virFileIsLink() which at least allows us to compile cleanly on mingw. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- The reason this isn't detected by our CI is that we explicitly build libvirt with -Dtests=disabled which I missed when debugging latest build error on mingw. tests/virschematest.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/virschematest.c b/tests/virschematest.c index 20ac495f4a..83bc945711 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -125,8 +125,7 @@ testSchemaDir(const char *schema, continue; if (ent->d_name[0] == '.') continue; - if (g_lstat(ent->d_name, &sb) >= 0 && - S_ISLNK(sb.st_mode)) + if (virFileIsLink(ent->d_name)) continue; if (filter && !g_regex_match(filter, ent->d_name, 0, NULL)) -- 2.44.2

2 1

[RFC PATCH] conf: check for migration job during domain start
by Sergey Dyasli 27 Aug '24

27 Aug '24

It's possible to hit the following situation during qemu p2p live migration: 1. qemu has live migrated and exited (making virDomainObjIsActive() return false) 2. the live migration job is still in progress, waiting for a confirmation from the remote libvirt daemon. This may last for a while with a presence of networking issues (up to keepalive timeout). Any attempt to start the domain again would fail with "domain is already being started" message which is misleading in this situation as it doesn't reflect what's really happening. Add a check for the migration job and report a different error message if the migration job is still running. Signed-off-by: Sergey Dyasli <sergey.dyasli(a)nutanix.com> --- src/conf/virdomainobjlist.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/conf/virdomainobjlist.c b/src/conf/virdomainobjlist.c index bb5807d00b42..166bbc5cfd57 100644 --- a/src/conf/virdomainobjlist.c +++ b/src/conf/virdomainobjlist.c @@ -301,9 +301,15 @@ virDomainObjListAddLocked(virDomainObjList *doms, goto error; } if (!vm->persistent) { - virReportError(VIR_ERR_OPERATION_INVALID, - _("domain '%1$s' is already being started"), - vm->def->name); + if (vm->job->asyncJob == VIR_ASYNC_JOB_MIGRATION_OUT) { + virReportError(VIR_ERR_OPERATION_INVALID, + _("domain '%1$s' is being migrated out"), + vm->def->name); + } else { + virReportError(VIR_ERR_OPERATION_INVALID, + _("domain '%1$s' is already being started"), + vm->def->name); + } goto error; } } -- 2.22.3

2 1

[PATCH] hypervisor: interface: Stub out virDomainCreateInBridgePortWithHelper using 'socketpair' on win32
by Peter Krempa 27 Aug '24

27 Aug '24

Mingw build failed after commit af87ee7927d3245582d82d36da25b4dc3b34465e as 'socketpair()' is not available on that platform. Stub out the function to return failure. Fixes: af87ee7927d3245582d82d36da25b4dc3b34465e Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- Pipeline: https://gitlab.com/pipo.sk/libvirt/-/pipelines/1428482965 (Ignore the broken Cirrus jobs. Something broke and it can't find master branch when building from my user. I blame github :D) src/hypervisor/domain_interface.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/src/hypervisor/domain_interface.c b/src/hypervisor/domain_interface.c index fc82624bb9..5cdba279fa 100644 --- a/src/hypervisor/domain_interface.c +++ b/src/hypervisor/domain_interface.c @@ -532,6 +532,7 @@ virDomainClearNetBandwidth(virDomainDef *def) * * Returns 0 in case of success or -1 on failure */ +#ifndef WIN32 static int virDomainCreateInBridgePortWithHelper(const char *bridgeHelperName, const char *brname, @@ -583,9 +584,9 @@ virDomainCreateInBridgePortWithHelper(const char *bridgeHelperName, virCommandPassFD(cmd, pair[1], VIR_COMMAND_PASS_FD_CLOSE_PARENT); virCommandClearCaps(cmd); -#ifdef CAP_NET_ADMIN +# ifdef CAP_NET_ADMIN virCommandAllowCap(cmd, CAP_NET_ADMIN); -#endif +# endif if (virCommandRunAsync(cmd, NULL) < 0) { *tapfd = -1; goto cleanup; @@ -626,6 +627,20 @@ virDomainCreateInBridgePortWithHelper(const char *bridgeHelperName, return *tapfd < 0 ? -1 : 0; } +#else /* WIN32 */ + +static int +virDomainCreateInBridgePortWithHelper(const char *bridgeHelperName G_GNUC_UNUSED, + const char *brname G_GNUC_UNUSED, + char **ifname G_GNUC_UNUSED, + int *tapfd G_GNUC_UNUSED, + unsigned int unusedflags G_GNUC_UNUSED) +{ + virReportSystemError(ENOSYS, "%s", + _("bridge port creation is not supported on this platform")); + return -1; +} +#endif /* virDomainInterfaceBridgeConnect: * @def: the definition of the VM -- 2.46.0

2 1

Discouraging use of virInterface* APIs
by Jim Fehlig 26 Aug '24

26 Aug '24

Laine's attempt long ago [1] to deprecate/obsolete the virInterface* APIs did not receive a standing ovation. However he raised many good points which are still valid today. If anything, netcf, the libvirt netcf backend, and the whole interface driver have become more stale. Personally, I wish we could deprecate and eventually obsolete the APIs and nuke the entire driver, but understand that's not desirable/realistic upstream. Given the state of the thing, I'd like to hear opinions on adding some words to the interface API page and virtinterfaced man page that describe the state of the APIs/driver and discourages their use. The motivation for this request comes from my efforts to take advantage of libvirt's modularization and not provide virtinterfaced (libvirt-daemon-driver-interface package) for certain use-cases. Having upstream documentation warning of the limitations and inactive development of the APIs/driver would help in convincing unsuspected users to find a better solution. And I think the use of "unsuspected" is correct :-). Projects closer to libvirt, like virt-manager, ditched use of the APIs long ago. But projects with less cross-pollination may not be aware of the general uselessness. As an example, we recently discovered cockpit-machines uses virConnectListAllInterfaces and virInterfaceGetXMLDesc. It's not clear to me why cockpit gathers this info. AFAICT it's not used for anything. The code was added long ago [2] with no info on purpose or what it's supposed to achieve. We've opened a github issue [3] to discuss removing use of these APIs. Pointing cockpit-machines developers to upstream libvirt documentation that describes the state of virInterface* and sets realistic expectation might help in convincing them to follow virt-managers footsteps and drop use of the APIs. I can provide the words of discouragement in the form of a patch if folks are receptive to the idea :-). Regards, Jim [1] https://listman.redhat.com/archives/libvir-list/2020-December/212791.html [2] https://github.com/cockpit-project/cockpit/pull/12465 [3] https://github.com/cockpit-project/cockpit-machines/issues/1777

2 1

[PATCH] remoteDispatchConnectOpen: Fix check for 'BHYVE' connection type
by Peter Krempa 26 Aug '24

26 Aug '24

From: aokblast <aokblast(a)FreeBSD.org> 'bhyveConnectGetType' (which is called from 'virConnectGetType') returns 'BHYVE' as the type, but the code in 'remoteDispatchConnectOpen' responsible for selecting the sub-driver URIs in modular deployment checks for 'bhyve' and thus would not properly fill the URIs to the sub-daemons. Signed-off-by: aokblast <aokblast(a)FreeBSD.org> Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/remote/remote_daemon_dispatch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/remote/remote_daemon_dispatch.c b/src/remote/remote_daemon_dispatch.c index cfffb40095..e812f5c3e9 100644 --- a/src/remote/remote_daemon_dispatch.c +++ b/src/remote/remote_daemon_dispatch.c @@ -2108,7 +2108,7 @@ remoteDispatchConnectOpen(virNetServer *server G_GNUC_UNUSED, STREQ(type, "Xen") || STREQ(type, "LXC") || STREQ(type, "VBOX") || - STREQ(type, "bhyve") || + STREQ(type, "BHYVE") || STREQ(type, "vz") || STREQ(type, "Parallels") || STREQ(type, "CH")) { -- 2.46.0

2 1

[PATCH] cpu_map: Add missing feature "avx-vnni-int16"
by Tim Wiederhake 26 Aug '24

26 Aug '24

Introduced in qemu commit 138c3377a9b27accec516b2c0da90dedef98a780. Signed-off-by: Tim Wiederhake <twiederh(a)redhat.com> --- src/cpu_map/x86_features.xml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/cpu_map/x86_features.xml b/src/cpu_map/x86_features.xml index 19904f2e22..08bf014604 100644 --- a/src/cpu_map/x86_features.xml +++ b/src/cpu_map/x86_features.xml @@ -463,6 +463,9 @@ <feature name='amx-complex'> <cpuid eax_in='0x00000007' ecx_in='0x00000001' edx='0x00000100'/> </feature> + <feature name='avx-vnni-int16'> + <cpuid eax_in='0x00000007' ecx_in='0x00000001' edx='0x00000400'/> + </feature> <feature name='prefetchiti'> <cpuid eax_in='0x00000007' ecx_in='0x00000001' edx='0x00004000'/> </feature> -- 2.43.0

2 1

[PATCH 0/2] Add support for new hv enlightenments
by Tim Wiederhake 26 Aug '24

26 Aug '24

Both were added in qemu 7.10. Add support for libvirt. Note that the feature names use dashes in qemu but underscores in libvirt domain conf xml to match other hv features, e.g. "vendor_id". Tim Wiederhake (2): hyperv: Support hv-emsr-bitmap enlightenment hyperv: Support hv-xmm-input enlightenment docs/formatdomain.rst | 4 ++++ src/conf/domain_conf.c | 8 ++++++++ src/conf/domain_conf.h | 2 ++ src/conf/schemas/domaincommon.rng | 10 ++++++++++ src/cpu/cpu_x86.c | 6 ++++++ src/cpu/cpu_x86_data.h | 2 ++ src/qemu/qemu_command.c | 10 ++++++++++ src/qemu/qemu_process.c | 2 ++ tests/domaincapsdata/qemu_8.0.0-q35.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.0.0-tcg.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.0.0.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.1.0-q35.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.1.0-tcg.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.1.0.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.2.0-q35.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.2.0-tcg.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_8.2.0.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.0.0-q35.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.0.0-tcg.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.0.0.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.1.0-q35.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.1.0-tcg.x86_64.xml | 2 ++ tests/domaincapsdata/qemu_9.1.0.x86_64.xml | 2 ++ tests/qemucapabilitiesdata/caps_8.0.0_x86_64.xml | 2 ++ tests/qemucapabilitiesdata/caps_8.1.0_x86_64.xml | 2 ++ tests/qemucapabilitiesdata/caps_8.2.0_x86_64.xml | 2 ++ tests/qemucapabilitiesdata/caps_9.0.0_x86_64.xml | 2 ++ tests/qemucapabilitiesdata/caps_9.1.0_x86_64.xml | 2 ++ tests/qemuxmlconfdata/hyperv-off.x86_64-latest.xml | 2 ++ tests/qemuxmlconfdata/hyperv-off.xml | 2 ++ tests/qemuxmlconfdata/hyperv.x86_64-latest.args | 2 +- tests/qemuxmlconfdata/hyperv.x86_64-latest.xml | 2 ++ tests/qemuxmlconfdata/hyperv.xml | 2 ++ 33 files changed, 93 insertions(+), 1 deletion(-) -- 2.43.0

2 4

[PATCH RFC v3 00/16] Support throttle block filters
by wucf＠linux.ibm.com 23 Aug '24

23 Aug '24

From: Chun Feng Wu <wucf(a)linux.ibm.com> Hi, I am thinking to leverage "throttle block filter" in QEMU to support more flexible I/O limits(e.g. tiered I/O groups), one sample provided by QEMU doc is: https://github.com/qemu/qemu/blob/master/docs/throttle.txt "For example, let's say that we have three different drives and we want to set I/O limits for each one of them and an additional set of limits for the combined I/O of all three drives." The implementation idea is to - Define throttle groups(limit) in domain - Define throttle filter to reference throttle group within disk - Within domain disk, throttle filters references multiple throttle groups to form filter chain to apply multiple limits in QEMU like above sample - Add new virsh cmds for throttle group management: throttlegroupset Add or update a throttling group. throttlegroupdel Delete a throttling group. throttlegroupinfo Get a throttling group. throttlegrouplist list all domain throttlegroups - Update "attach-disk" to add one more option "--throttle-groups" to apply throttle filters e.g. "virsh attach-disk $VM_ID ${DISK_PATH}/vm1_disk_2.qcow2 vdd --driver qemu --subdriver qcow2 --targetbus virtio --throttle-groups limit2,limit012" - I chose above semantics as I felt they're appropriate, if there are better ones please kindly suggest. Note, this implementation requires flag "QEMU_CAPS_OBJECT_JSON". From QMP perspective, the sample flow works this way: - Throttle group creation: virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit0","limits":{"iops-total":200,"iops-read":0,"iops-total-max":200,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit1","limits":{"iops-total":250,"iops-read":0,"iops-total-max":250,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit2","limits":{"iops-total":300,"iops-read":0,"iops-total-max":300,"iops-total-max-length":1}}}' virsh qemu-monitor-command 1 '{"execute":"object-add", "arguments":{"qom-type":"throttle-group","id":"limit012","limits":{"iops-total":400,"iops-read":0,"iops-total-max":400,"iops-total-max-length":1}}}' - Chain up filters during attaching disk to apply two filters(limit0 and limit012): virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"file","filename":"/virt/disks/vm1_disk_1.qcow2","node-name":"test-3-storage","auto-read-only":true,"discard":"unmap"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"node-name":"test-4-format","read-only":false,"driver":"qcow2","file":"test-3-storage","backing":null}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments":{"driver":"throttle","node-name":"libvirt-5-filter","throttle-group": "limit0","file":"test-4-format"}}' virsh qemu-monitor-command 1 '{"execute":"blockdev-add", "arguments": {"driver":"throttle","node-name":"libvirt-6-filter","throttle-group":"limit012","file":"libvirt-5-filter"}}' virsh qemu-monitor-command 1 '{"execute": "device_add", "arguments": {"driver":"virtio-blk-pci","scsi":false,"bus":"pci.0","addr":"0x5","drive":"libvirt-6-filter","id":"virtio-disk1"}}' This patchset includes: - Throttle group XML schema definition in patch 1 - Throttle filter XML schema definition in patch 2 - Throttle group struct definition, parsing and formating in patch 3 - Throttle filter struct definition, parsing and formating in patch 4 - New QMP processing to update and get throttle group in patch 5&6 - New API definition and implementation in patch 7 - QEMU driver implementation in patch 8 - Hotplug processing for throttle filters in patch 9 - Extract common iotune validation in patch 10 - qemuProcessLaunch flow implemenation for throttle group in patch 11 - qemuProcessLaunch flow implemenation for throttle filter in patch 12 - Domain XML test for processing throttle groups and filters in patch 13 - Test new implemented driver in patch 14 - New virsh cmd implementation for group in patch 15 - Update Virsh cmd "attach_disk" to include throttle filters in patch 16 v3 changes: - re-org commits by splitting changes containing throttle group and filters - update commits msgs - move schema commits to be the first ones - refactor "diskIoTune" to extract common schema "iotune" - add new tests for throttle groups and filters in qemuxmlconftest - check flag "QEMU_CAPS_OBJECT_JSON" when preparing "-object"(qemu: command: Support throttle groups during qemuProcessLaunch ) or creating throttle group (qemu: Implement qemu driver for throttle API) - when creating throttle group through "object-add" (qemu: Implement qemu driver for throttle API), reuse "qemuMonitorAddObject" to check if "objectAddNoWrap"("props") is requried - remove "virObject parent;" in "_virDomainThrottleFilterDef" in domain_conf.h - remove "virDomainThrottleGroupIndexByName" in both domain_conf.h and domain_conf.c - remove "virDomainThrottleFilterDefNew" in domain_conf.c - update "virDomainThrottleFilterDefFree" to use "g_free" rather than "VIR_FREE" in domain_conf.c - update "virDomainDiskThrottleFilterDefParse" to remove "xmlXPathContextPtr ctx" parameter and check NULL against "filter->group_name" in domain_conf.c - use "virBufferEscapeString" instead of "virBufferAsprintf" in "virDomainDiskDefFormatThrottleFilterChain" - use "group->val > 0" instead of "if (group->val)" in FORMAT_THROTTLE_GROUP - remove NULL check for "group->group_name" since virBufferEscapeString checked NULL already in "virDomainThrottleGroupFormat" - I haven't added new conf module (src/conf/virdomainthrottle.c/h) because "virDomainThrottleGroupDef" is alias of "_virDomainBlockIoTuneInfo", try to avoid circular dependency - remove "NULLSTR" in qemuMonitorUpdateThrottleGroup and qemuMonitorGetThrottleGroup in qemu_monitor.c - refactor "qemuMonitorMakeThrottleGroupLimits" to use virJSONValueObjectAdd in qemu_monitor_json.c - use "g_strdup_printf" to avoid static buffers in "qemuMonitorJSONGetThrottleGroup" - remove virReportError after qemuMonitorJSONGetReply in "qemuMonitorJSONGetThrottleGroup" to avoid overriding error - remove "VIR_DOMAIN_THROTTLE_GROUP" in libvirt/include/libvirt/libvirt-domain.h - update "virDomainGetThrottleGroup" to not first query the number of parameters, - update "remote_domain_get_throttle_group_args" and "remote_domain_get_throttle_group_ret" to remove "nparams" in src/remote_protocol-structs, also updated "remote_domain_get_throttle_group_args" and "remote_domain_get_throttle_group_ret" in src/remote/remote_protocol.x - update parameter "virTypedParameterPtr params" to be "virTypedParameterPtr *params" in "virDrvDomainGetThrottleGroup" in driver-hypervisor.h - update "qemuDomainSetThrottleGroup" to not query number of parameters first - remove wrapper "qemuDomainThrottleGroupByName" and "qemuDomainSetThrottleGroupDefaults" - refactor "qemuDomainSetThrottleGroup" and "qemuDomainSetBlockIoTune" to use common logic - update "qemuDomainDelThrottleGroup" to use VIR_JOB_MODIFY by referencing "qemuDomainHotplugDelIOThread" - check if group is still being used by some filter(qemuDomainCheckThrottleGroupRef) during deletion - replace "ThrottleFilterChain" with "ThrottleFilters" - update "qemuDomainDiskGetTopNodename" to take top throttle node name if disk has throttles, and reuse "qemuDomainDiskGetBackendAlias" in "qemuBuildDiskDeviceProps" to get top node name as "drive" - after enabling throttlerfilter and if disk has throttlefilters, during blockcommit, the top node name is not "libvirt-x-format" anymore, instead, top node name referencies top filter like "libvirt-x-filter" - add check "cdrom device with throttle filters isn't supported" - delete "filternodenameindex" and reuse "nodenameindex" to generate index for throttle nodes - refactor detaching filters by adding "qemuBuildThrottleFiltersDetachPrepareBlockdev" to just build parameters for "blockdev-del" - refactor "testDomainSetBlockIoTune" and "testDomainSetThrottleGroup" to use common logic Any comments/suggestions will be appriciated! Chun Feng Wu (16): schema: Add new domain elements to support multiple throttle groups schema: Add new domain elements to support multiple throttle filters config: Introduce ThrottleGroup and corresponding XML parsing config: Introduce ThrottleFilter and corresponding XML parsing qemu: monitor: Add support for ThrottleGroup operations tests: Test qemuMonitorJSONGetThrottleGroup and qemuMonitorJSONUpdateThrottleGroup remote: New APIs for ThrottleGroup lifecycle management qemu: Implement qemu driver for throttle API qemu: hotplug: Support hot attach and detach block disk along with throttle filters config: validate: Refactor disk iotune validation for reuse qemu: command: Support throttle groups during qemuProcessLaunch qemu: command: Support throttle filters during qemuProcessLaunch qemuxmlconftest: Add 'throttlefilter' tests test_driver: Test throttle group lifecycle APIs virsh: Add support for throttle group operations virsh: Add option "throttle-groups" to "attach_disk" docs/formatdomain.rst | 48 ++ include/libvirt/libvirt-domain.h | 21 + src/conf/domain_conf.c | 376 +++++++++++ src/conf/domain_conf.h | 51 ++ src/conf/domain_validate.c | 118 +++- src/conf/schemas/domaincommon.rng | 293 +++++---- src/conf/virconftypes.h | 4 + src/driver-hypervisor.h | 22 + src/libvirt-domain.c | 196 ++++++ src/libvirt_private.syms | 9 + src/libvirt_public.syms | 7 + src/qemu/qemu_block.c | 131 ++++ src/qemu/qemu_block.h | 53 ++ src/qemu/qemu_command.c | 182 ++++++ src/qemu/qemu_command.h | 12 + src/qemu/qemu_domain.c | 39 +- src/qemu/qemu_domain.h | 8 + src/qemu/qemu_driver.c | 617 +++++++++++++++--- src/qemu/qemu_hotplug.c | 33 + src/qemu/qemu_monitor.c | 34 + src/qemu/qemu_monitor.h | 14 + src/qemu/qemu_monitor_json.c | 150 +++++ src/qemu/qemu_monitor_json.h | 14 + src/remote/remote_daemon_dispatch.c | 44 ++ src/remote/remote_driver.c | 40 ++ src/remote/remote_protocol.x | 48 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 452 +++++++++---- tests/qemumonitorjsontest.c | 86 +++ .../throttlefilter.x86_64-latest.args | 43 ++ .../throttlefilter.x86_64-latest.xml | 65 ++ tests/qemuxmlconfdata/throttlefilter.xml | 55 ++ tests/qemuxmlconftest.c | 1 + tools/virsh-completer-domain.c | 64 ++ tools/virsh-completer-domain.h | 5 + tools/virsh-domain.c | 453 ++++++++++++- 36 files changed, 3447 insertions(+), 369 deletions(-) create mode 100644 tests/qemuxmlconfdata/throttlefilter.x86_64-latest.args create mode 100644 tests/qemuxmlconfdata/throttlefilter.x86_64-latest.xml create mode 100644 tests/qemuxmlconfdata/throttlefilter.xml -- 2.34.1

3 53