March 2021 - Devel - libvirt List Archives

[PATCH v6 0/5] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate
by Hao Wang 05 Mar '21

05 Mar '21

V5 -> V6: split DomainGetDirtyRateInfo(domdirtyrate) API into two parts: 1. DomainStartDirtyRateCalc(domdirtyrate-calc) for starting dirty rate calculation; 2. qemuDomainGetStatsDirtyRate(domstats --dirtyrate) for querying dirty rate infomation. V4 -> V5: squash 1/7 and bits of 5/7 and 6/7 into 2/7 in v4 (to be 1/5 in v5) squash left of 5/7 into 4/7 in v4 (to be 3/5 in v5) add VIR_DOMAIN_DIRTYRATE_DEFAULT flag remove redundant error report rename virsh api to "domdirtyrate" use vshTablePtr for virsh api output add description in docs/manpages/virsh.rst other format optimize V3 -> V4: define flags to unsigned int fix some compile warnings V2 -> V3: reorganize patchset to fix compile warning V1 -> V2: replace QEMU_JOB_ASYNC with QEMU_JOB_QUERY Sometimes domain's memory dirty rate is expected by user in order to decide whether it's proper to be migrated out or not. We have already completed the QEMU part of the capability: https://patchew.org/QEMU/1600237327-33618-1-git-send-email-zhengchuan@huawe… And this serial of patches introduce the corresponding LIBVIRT part: 1. Calculating Introduce a new API DomainStartDirtyRateCalc and corresponding virsh api (domdirtyrate-calc) for starting dirty rate calculation by calling qmp 'calc-dirty-rate'. # virsh domdirtyrate-calc <domain> [--seconds <sec>] 2. Querying Introduce command 'virsh domstats --dirtyrate' for reporting memory dirty rate infomation by calling qmp 'query-dirty-rate'. The info is listed as: Domain: 'vm0' dirtyrate.calc_status=measured dirtyrate.calc_start_time=502814 dirtyrate.calc_period=1 dirtyrate.megabytes_per_second=2 Hao Wang (5): migration/dirtyrate: Introduce virDomainStartDirtyRateCalc API migration/dirtyrate: Implement qemuDomainStartDirtyRateCalc migration/dirtyrate: Introduce domdirtyrate-calc virsh api migration/dirtyrate: Implement qemuMonitorQueryDirtyRate migration/dirtyrate: Introduce command 'virsh domstats --dirtyrate' docs/manpages/virsh.rst | 33 ++++++++- include/libvirt/libvirt-domain.h | 5 ++ src/driver-hypervisor.h | 6 ++ src/libvirt-domain.c | 57 +++++++++++++++ src/libvirt_public.syms | 5 ++ src/qemu/qemu_driver.c | 115 +++++++++++++++++++++++++++++++ src/qemu/qemu_monitor.c | 24 +++++++ src/qemu/qemu_monitor.h | 18 +++++ src/qemu/qemu_monitor_json.c | 88 +++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 8 +++ src/remote/remote_driver.c | 1 + src/remote/remote_protocol.x | 14 +++- src/remote_protocol-structs | 6 ++ tools/virsh-domain-monitor.c | 7 ++ tools/virsh-domain.c | 63 +++++++++++++++++ 15 files changed, 447 insertions(+), 3 deletions(-) -- 2.23.0

2 10

[PATCH 0/4] Replace virFileMakePath*() with g_mkdir_with_parents()
by Michal Privoznik 04 Mar '21

04 Mar '21

2 7

[PATCH 00/33] Remove the now misleading virReportOOMError infra
by Peter Krempa 04 Mar '21

04 Mar '21

'virReportOOMError' is nowadays mostly useless since an OOM error will trigger yet another allocation failure in the logger when attempting to log the error. Additionally it was also used in few places which can fail with other failures than OOM. To prevent both errorneous usage types, let's remove it completely. In the rare case we'd want to allocate a massive buffer and want to fail gracefully we still can use virReportError with VIR_ERR_NO_MEMORY. The series starts by some cleanup, then removes error handling from places where it's now dead code as we'll abort before the reporting, then converts some real OOM scenarios to abort() directly and lastly fixes the error message for cases where other errors are possible and lastly removes virReportOOMError. Pipelines are stuck, so hopefully it will go through: https://gitlab.com/pipo.sk/libvirt/-/pipelines/261161989 Peter Krempa (33): Remove useless comments for VIR_FROM_THIS definition util: xml: Introduce autoptr cleanup support for 'xmlNode' virDomainDefSetMetadata: Avoid temporary variable for string copy virCommandAddEnv: Make stealing of argument more obvious virCommandAddArgBuffer: Simplify clearing of @buf virCPUx86DataParse: Don't check error from x86FeatureNames virhostcputest: linuxCPUStatsCompareFiles: Don't check return value of virBufferContentAndReset virBuildPath: Remove return value lxc_process: Remove OOM handling from logging setup virDomainDefSetMetadata: Rework memory handling util: vircommand: Remove OOM handling virCloseCallbacksGetForConn: Remove OOM handling virfirewall: Don't check OOM in ADD_ARG macro virfirewall: Remove OOM checks from virFirewallStartTransaction virfirewall: virFirewallAddRuleFullV: Remove OOM check from VIR_APPEND_ELEMENT virfirewall: Remove impossible OOM error reporting util: virnetlink: Add wrapper for 'nlmsg_alloc_simple' util: xml: Add virXMLBufferCreate wrapper util: xml: Add wrapper for 'xmlNewNode' Don't report OOM error on xmlCopyNode failure virXMLXPathContextNew: abort() on allocation failure virXMLParseHelper: abort() on allocation failure util: virprocess: abort() on CPU_ALLOC failure virURIFormat: abort() on failure util: iohelper: Don't handle OOM from posix_memalign hyperv: abort() failure of wsmc_fault_new() vbox: abort() on allocation failure in UTF8<->UTF16 conversion libxl: abort() on failure of libxl_cpu_bitmap_alloc() virVBoxSnapshotConfSaveVboxFile: abort() on failure to allocate xmlDoc and comment util: json: Report non-OOM error on yajl failure storage: Don't report OOM error on failure of glfs_new virVMXConvertToUTF8: Report non-OOM error on failure of xmlBufferCreateStatic util: virerror: Remove virReportOOMError build-aux/syntax-check.mk | 8 -- docs/internals/command.html.in | 8 +- src/conf/domain_conf.c | 62 ++++----- src/conf/network_conf.c | 5 +- src/cpu/cpu_x86.c | 6 +- src/hyperv/hyperv_wmi.c | 8 +- src/hypervisor/virclosecallbacks.c | 15 +-- src/libvirt_private.syms | 3 +- src/libxl/libxl_conf.c | 6 +- src/libxl/libxl_driver.c | 7 +- src/lxc/lxc_process.c | 9 +- src/storage/storage_backend_gluster.c | 3 +- .../storage_file_backend_gluster.c | 3 +- src/test/test_driver.c | 6 +- src/util/iohelper.c | 7 +- src/util/vircommand.c | 119 +++++------------- src/util/virerror.c | 22 ---- src/util/virerror.h | 8 -- src/util/virfcp.c | 3 +- src/util/virfile.c | 7 +- src/util/virfile.h | 2 +- src/util/virfirewall.c | 62 ++------- src/util/virhook.c | 14 +-- src/util/virjson.c | 6 +- src/util/virnetdev.c | 18 +-- src/util/virnetdevbridge.c | 10 +- src/util/virnetdevip.c | 15 +-- src/util/virnetdevvportprofile.c | 6 +- src/util/virnetlink.c | 38 +++--- src/util/virnetlink.h | 4 + src/util/virpci.c | 12 +- src/util/virprocess.c | 12 +- src/util/virscsi.c | 1 - src/util/virscsivhost.c | 1 - src/util/viruri.c | 8 +- src/util/virxml.c | 44 ++++--- src/util/virxml.h | 8 ++ src/vbox/vbox_common.c | 20 --- src/vbox/vbox_common.h | 15 ++- src/vbox/vbox_snapshot_conf.c | 46 ++----- src/vmx/vmx.c | 9 +- tests/virhostcputest.c | 5 +- tools/virsh-domain.c | 5 +- 43 files changed, 214 insertions(+), 462 deletions(-) -- 2.29.2

3 39

[PATCH] tests: Don't return -1 as program return value
by Peter Krempa 04 Mar '21

04 Mar '21

In cases we use -1 for failure internally we still must return EXIT_FAILURE. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- tests/qemucommandutiltest.c | 3 +-- tests/virnetdevbandwidthtest.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/qemucommandutiltest.c b/tests/qemucommandutiltest.c index 6291c3b787..03947d87bc 100644 --- a/tests/qemucommandutiltest.c +++ b/tests/qemucommandutiltest.c @@ -146,8 +146,7 @@ mymain(void) "server.2.type=tcp," "server.2.host=example.com"); - return ret; - + return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE; } VIR_TEST_MAIN(mymain) diff --git a/tests/virnetdevbandwidthtest.c b/tests/virnetdevbandwidthtest.c index 2e76af3d0c..6d0443876c 100644 --- a/tests/virnetdevbandwidthtest.c +++ b/tests/virnetdevbandwidthtest.c @@ -156,7 +156,7 @@ mymain(void) TC " filter add dev eth0 parent ffff: protocol all u32 match u32 0 0 " "police rate 5kbps burst 7kb mtu 64kb drop flowid :1\n")); - return ret; + return ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE; } VIR_TEST_MAIN_PRELOAD(mymain, VIR_TEST_MOCK("virnetdevbandwidth")) -- 2.29.2

2 1

[PATCH] conf: Rewrite network virtio driver formatting using virXMLFormatElement
by Peter Krempa 04 Mar '21

04 Mar '21

Simplify the logic picking which element form to format by using virBuffers for the partial properties and virXMLFormatElement for combining them. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/conf/domain_conf.c | 93 ++++++++++++++++-------------------------- 1 file changed, 35 insertions(+), 58 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 05b6cb3000..0220db794e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -25428,108 +25428,97 @@ virDomainActualNetDefFormat(virBufferPtr buf, static void -virDomainVirtioNetGuestOptsFormat(char **outstr, +virDomainVirtioNetGuestOptsFormat(virBufferPtr buf, virDomainNetDefPtr def) { - g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; if (def->driver.virtio.guest.csum) { - virBufferAsprintf(&buf, "csum='%s' ", + virBufferAsprintf(buf, " csum='%s'", virTristateSwitchTypeToString(def->driver.virtio.guest.csum)); } if (def->driver.virtio.guest.tso4) { - virBufferAsprintf(&buf, "tso4='%s' ", + virBufferAsprintf(buf, " tso4='%s'", virTristateSwitchTypeToString(def->driver.virtio.guest.tso4)); } if (def->driver.virtio.guest.tso6) { - virBufferAsprintf(&buf, "tso6='%s' ", + virBufferAsprintf(buf, " tso6='%s'", virTristateSwitchTypeToString(def->driver.virtio.guest.tso6)); } if (def->driver.virtio.guest.ecn) { - virBufferAsprintf(&buf, "ecn='%s' ", + virBufferAsprintf(buf, " ecn='%s'", virTristateSwitchTypeToString(def->driver.virtio.guest.ecn)); } if (def->driver.virtio.guest.ufo) { - virBufferAsprintf(&buf, "ufo='%s' ", + virBufferAsprintf(buf, " ufo='%s'", virTristateSwitchTypeToString(def->driver.virtio.guest.ufo)); } - virBufferTrim(&buf, " "); - - *outstr = virBufferContentAndReset(&buf); } static void -virDomainVirtioNetHostOptsFormat(char **outstr, +virDomainVirtioNetHostOptsFormat(virBufferPtr buf, virDomainNetDefPtr def) { - g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; if (def->driver.virtio.host.csum) { - virBufferAsprintf(&buf, "csum='%s' ", + virBufferAsprintf(buf, " csum='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.csum)); } if (def->driver.virtio.host.gso) { - virBufferAsprintf(&buf, "gso='%s' ", + virBufferAsprintf(buf, " gso='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.gso)); } if (def->driver.virtio.host.tso4) { - virBufferAsprintf(&buf, "tso4='%s' ", + virBufferAsprintf(buf, " tso4='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.tso4)); } if (def->driver.virtio.host.tso6) { - virBufferAsprintf(&buf, "tso6='%s' ", + virBufferAsprintf(buf, " tso6='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.tso6)); } if (def->driver.virtio.host.ecn) { - virBufferAsprintf(&buf, "ecn='%s' ", + virBufferAsprintf(buf, " ecn='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.ecn)); } if (def->driver.virtio.host.ufo) { - virBufferAsprintf(&buf, "ufo='%s' ", + virBufferAsprintf(buf, " ufo='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.ufo)); } if (def->driver.virtio.host.mrg_rxbuf) { - virBufferAsprintf(&buf, "mrg_rxbuf='%s' ", + virBufferAsprintf(buf, " mrg_rxbuf='%s'", virTristateSwitchTypeToString(def->driver.virtio.host.mrg_rxbuf)); } - virBufferTrim(&buf, " "); - - *outstr = virBufferContentAndReset(&buf); } static void -virDomainVirtioNetDriverFormat(char **outstr, +virDomainVirtioNetDriverFormat(virBufferPtr buf, virDomainNetDefPtr def) { - g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER; if (def->driver.virtio.name) { - virBufferAsprintf(&buf, " name='%s'", + virBufferAsprintf(buf, " name='%s'", virDomainNetBackendTypeToString(def->driver.virtio.name)); } if (def->driver.virtio.txmode) { - virBufferAsprintf(&buf, " txmode='%s'", + virBufferAsprintf(buf, " txmode='%s'", virDomainNetVirtioTxModeTypeToString(def->driver.virtio.txmode)); } if (def->driver.virtio.ioeventfd) { - virBufferAsprintf(&buf, " ioeventfd='%s'", + virBufferAsprintf(buf, " ioeventfd='%s'", virTristateSwitchTypeToString(def->driver.virtio.ioeventfd)); } if (def->driver.virtio.event_idx) { - virBufferAsprintf(&buf, " event_idx='%s'", + virBufferAsprintf(buf, " event_idx='%s'", virTristateSwitchTypeToString(def->driver.virtio.event_idx)); } if (def->driver.virtio.queues) - virBufferAsprintf(&buf, " queues='%u'", def->driver.virtio.queues); + virBufferAsprintf(buf, " queues='%u'", def->driver.virtio.queues); if (def->driver.virtio.rx_queue_size) - virBufferAsprintf(&buf, " rx_queue_size='%u'", + virBufferAsprintf(buf, " rx_queue_size='%u'", def->driver.virtio.rx_queue_size); if (def->driver.virtio.tx_queue_size) - virBufferAsprintf(&buf, " tx_queue_size='%u'", + virBufferAsprintf(buf, " tx_queue_size='%u'", def->driver.virtio.tx_queue_size); - virDomainVirtioOptionsFormat(&buf, def->virtio); - - *outstr = virBufferContentAndReset(&buf); + virDomainVirtioOptionsFormat(buf, def->virtio); } @@ -25834,30 +25823,18 @@ virDomainNetDefFormat(virBufferPtr buf, virBufferEscapeString(buf, "<model type='%s'/>\n", virDomainNetGetModelString(def)); if (virDomainNetIsVirtioModel(def)) { - g_autofree char *str = NULL; - g_autofree char *gueststr = NULL; - g_autofree char *hoststr = NULL; - - virDomainVirtioNetDriverFormat(&str, def); - virDomainVirtioNetGuestOptsFormat(&gueststr, def); - virDomainVirtioNetHostOptsFormat(&hoststr, def); - - if (!gueststr && !hoststr) { - if (str) - virBufferAsprintf(buf, "<driver%s/>\n", str); - } else { - if (str) - virBufferAsprintf(buf, "<driver%s>\n", str); - else - virBufferAddLit(buf, "<driver>\n"); - virBufferAdjustIndent(buf, 2); - if (hoststr) - virBufferAsprintf(buf, "<host %s/>\n", hoststr); - if (gueststr) - virBufferAsprintf(buf, "<guest %s/>\n", gueststr); - virBufferAdjustIndent(buf, -2); - virBufferAddLit(buf, "</driver>\n"); - } + g_auto(virBuffer) driverAttrBuf = VIR_BUFFER_INITIALIZER; + g_auto(virBuffer) driverChildBuf = VIR_BUFFER_INIT_CHILD(buf); + g_auto(virBuffer) hostAttrBuf = VIR_BUFFER_INITIALIZER; + g_auto(virBuffer) guestAttrBuf = VIR_BUFFER_INITIALIZER; + + virDomainVirtioNetDriverFormat(&driverAttrBuf, def); + virDomainVirtioNetGuestOptsFormat(&guestAttrBuf, def); + virDomainVirtioNetHostOptsFormat(&hostAttrBuf, def); + + virXMLFormatElement(&driverChildBuf, "host", &hostAttrBuf, NULL); + virXMLFormatElement(&driverChildBuf, "guest", &guestAttrBuf, NULL); + virXMLFormatElement(buf, "driver", &driverAttrBuf, &driverChildBuf); } } if (def->backend.tap || def->backend.vhost) { -- 2.29.2

2 1

[PATCH] virBufferAdd: Ensure that the buffer is initialized also when len == 0
by Peter Krempa 04 Mar '21

04 Mar '21

There's an optimization in virBufferAdd which returns early when the length of the added string is 0 (given that auto-indent is disabled). The optimization causes inconsistent behaviour between these two cases: virBufferAdd(buf, "", 0); // this doesn't initialize the buffer and virBufferAdd(buf, "", -1); //this initializes the buffer Since using an empty string is used to prime the buffer to an empty string it can be confusing. Remove the optimization. This fixes such a wrong initialization done in x86FeatureNames. Note that our code in many places expects that if no virBuffer APIs are used on a buffer object, then NULL should be retured, so we can't always prime the buffer to an empty string. Signed-off-by: Peter Krempa <pkrempa(a)redhat.com> --- src/util/virbuffer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/util/virbuffer.c b/src/util/virbuffer.c index ebb4108b5d..b9ddf1f5c9 100644 --- a/src/util/virbuffer.c +++ b/src/util/virbuffer.c @@ -156,7 +156,7 @@ virBufferApplyIndent(virBufferPtr buf) void virBufferAdd(virBufferPtr buf, const char *str, int len) { - if (!str || !buf || (len == 0 && buf->indent == 0)) + if (!str || !buf) return; virBufferInitialize(buf); -- 2.29.2

2 1

[PATCH 0/6] util: virerror: Avoid stack'd buffers for error message formatting
by Peter Krempa 04 Mar '21

04 Mar '21

We don't need OOM resiliency nowadays. Peter Krempa (6): util: virerror: Don't use stack'd buffers in error report helpers util: virerror: Avoid a copy of the error messages util: virprocess: Use local maximum error message size qemuProcessReportLogError: Remove unnecessary math for max error message qemuProcessReportLogError: Don't mark "%s: %s" as translatable util: virerror: Remove VIR_ERROR_MAX_LENGTH macro src/qemu/qemu_process.c | 12 +-- src/util/virerror.c | 188 ++++++++++++++++++++++------------------ src/util/virerror.h | 2 - src/util/virprocess.c | 10 ++- 4 files changed, 116 insertions(+), 96 deletions(-) -- 2.29.2

2 8

[PATCH v2 0/6] use g_autofree and g_autoptr
by Kristina Hanicova 04 Mar '21

04 Mar '21

This is v2 from https://listman.redhat.com/archives/libvir-list/2021-February/msg01117.html I split commit 3/4 into more commits and left out autofreeing outer array as suggested by Jan Tomko. Patches 2/6 and 6/6 (previously 2/4 and 4/4) have already been reviewed-by and I am sending them for completness. This uses g_autofree (if possible) in files: vsh-table, netdev_bandwidth_conf and g_autoptr instead of virNetDevBandwidthFree where possible. Kristina Hanicova (6): vsh-table: Use g_autofree where possible netdev_bandwidth_conf: Use g_autofree where possible netdev_bandwidth_conf: Refractor virNetDevBandwidthParse() Use g_steal_pointer where possible Use g_autoptr instead of virNetDevBandwidthFree where possible netdev_bandwidth_conf: Remove redundant variables/labels src/conf/netdev_bandwidth_conf.c | 67 ++++++++++++-------------------- src/network/bridge_driver.c | 6 +-- src/qemu/qemu_driver.c | 11 ++---- src/test/test_driver.c | 3 +- src/util/virnetdevbandwidth.h | 2 + tests/virnetdevbandwidthtest.c | 3 +- tools/vsh-table.c | 13 ++----- 7 files changed, 39 insertions(+), 66 deletions(-) -- 2.29.2

2 8

[libvirt PATCH] docs: Spell out our policy concerning minor releases
by Andrea Bolognani 04 Mar '21

04 Mar '21

We've already applied this policy on multiple occasions, but it's good to have it written down so that there can be no confusion. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- docs/platforms.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/platforms.rst b/docs/platforms.rst index b0ce0c3737..6d55728625 100644 --- a/docs/platforms.rst +++ b/docs/platforms.rst @@ -44,6 +44,7 @@ for the previous major version will be dropped 2 years after the new major version is released or when the vendor itself drops support, whichever comes first. In this context, third-party efforts to extend the lifetime of a distro are not considered, even when they are endorsed by the vendor (eg. Debian LTS). +Within each major release, only the most recent minor release is considered. For the purposes of identifying supported software versions available on Linux, the project will look at CentOS, Debian, Fedora, openSUSE, RHEL, SLES and -- 2.26.2

2 1

[libvirt PATCH 0/2] glib: Avoid yet another crash due to race in event loop code
by Martin Kletzander 04 Mar '21

04 Mar '21

Similarly to commit 0db4743645b7 we need to fix the other event loop as well. To be completely honest I do not completely understand all the details in the original workaround, so I just hope this is the best way to fix the issue. Hopefully-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1931331 Martin Kletzander (2): util: Move glib event loop workaround to glibcompat glib: Use safe glib event workaround in other event loops src/qemu/qemu_agent.c | 2 +- src/qemu/qemu_monitor.c | 2 +- src/util/glibcompat.c | 35 +++++++++++++++++++++++++++++++++++ src/util/glibcompat.h | 11 +++++++++++ src/util/vireventglib.c | 30 ------------------------------ src/util/vireventthread.c | 2 +- 6 files changed, 49 insertions(+), 33 deletions(-) -- 2.30.1

2 5