June 2018 - Devel - libvirt List Archives

[libvirt] [PATCH 0/2] Minor fixes for virTypedParams(De)Serialize
by Marc Hartmayer 07 Jul '18

07 Jul '18

Some minor fixes and two questions: * Is the first method, which is described in the documentation for virTypedParamsDeserialize, in sync with the actual code? ("Older APIs do not rely on deserializer allocating memory for @params, ...") * Do we also have to set *nparams = 0 in case of an error and the user has allocated the memory? (virTypedParamsDeserialize) Marc Hartmayer (2): virTypedParamsSerialize: minor fixes virTypedParamsDeserialize: minor fixes src/util/virtypedparam.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) -- 2.13.4

4 15

[libvirt] [RFCv2 PATCH 0/5] Introduce RDT memory bandwidth allocation support
by bing.niu＠intel.com 06 Jul '18

06 Jul '18

From: Bing Niu <bing.niu(a)intel.com> This series is to introduce RDT memory bandwidth allocation support by extending current virresctrl implementation. The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate control over memory bandwidth available per-core. This feature provides a method to control applications which may be over-utilizing bandwidth relative to their priority in environments such as the data-center. The details can be found in Intel's SDM 17.19.7. Kernel supports MBA through resctrl file system same as CAT. Each resctrl group have a MB parameter to control how much memory bandwidth it can utilize in unit of percentage. In this series, MBA is enabled by enhancing existing virresctrl implementation. The policy employed for MBA is similar with CAT: The sum of each MBA group's bandwidth dose not exceed 100%. The enhancement of virresctrl include two parts: Part 1: Add two new structures virResctrlInfoMB and virResctrlAllocMB for collecting host system MBA capability and domain memory bandwidth allocation. Those two structures are the extension of existing virResctrlInfo and virResctrlAlloc. With them, virresctrl framework can support MBA and CAT concurrently. Each virResctrlAlloc represent a resource allocation including CAT, or MBA, or CAT&MBA. The policy of MBA is that: total memory bandwidth of each resctrl group, created by virresctrl, does not exceed to 100%. Part 2: On XML part, add new elements to host capabilities query and domain allocation to support memory bandwidth allocation. ---------------------------------------------------------------------------------------------- For host capabilities XML, new XML format like below example, <host> ..... <memory> <node id='0' cpus='0-19'> <control granularity='10' min ='10' maxAllocs='8'/> </node> </memory> </host> granularity --- memory bandwidth granularity min --- minimum memory bandwidth allowed maxAllocs --- maximum concurrent memory bandwidth allocation allowed. ---------------------------------------------------------------------------------------------- For domain XML, new format as below example <domain type='kvm' id='2'> ...... <cputune> ...... <shares>1024</shares> <memorytune vcpus='0-1'> <node id='0' bandwidth='20'/> </memorytune> </cputune> ...... </domain> id --- node where memory bandwidth allocation will happen bandwidth --- bandwidth allocated in percentage ---------------------------------------------------------------------------------------------- Changelog: v1->v2: Jano's comment: 1. put renaming parts into separated patches. 2. set the initial return value as -1. 3. using full name in structure definition. 4. do not use VIR_CACHE_TYPE_LAST for memory bandwidth allocation formatting. Pavel's comment: 1. add host capabilities XML for memory bandwidth allocation. 2. do not mix use cachetune section in XML for memory bandwidth allocation in domain XML. define a dedicated one for memory bandwidth allocation. Bing Niu (5): util: Rename and packing parts of virresctrl util: Add memory bandwidth support to resctrl conf: rename cachetune to restune conf: Introduce cputune/memorytune to support memory bandwidth allocation conf: add memory bandwidth allocation capability of host src/conf/capabilities.c | 112 +++++++++++ src/conf/capabilities.h | 11 ++ src/conf/domain_conf.c | 304 ++++++++++++++++++++++++++--- src/conf/domain_conf.h | 10 +- src/libvirt_private.syms | 4 +- src/qemu/qemu_process.c | 18 +- src/util/virresctrl.c | 496 ++++++++++++++++++++++++++++++++++++++++++++--- src/util/virresctrl.h | 35 +++- 8 files changed, 918 insertions(+), 72 deletions(-) -- 2.7.4

3 16

[libvirt] [PATCH 0/9] Resolve libvirtd hang on termination with connected long running client
by John Ferlan 06 Jul '18

06 Jul '18

RFC: https://www.redhat.com/archives/libvir-list/2018-January/msg00318.html Adjustments since RFC... Patches 1&2: No change, were already R-B'd Patch 3: Removed code as noted in code review, update commit message Patch 4: From old series removed, see below for more details Patch 9: no change NB: Patches 5-8 and 10 from Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com> are removed as they seemed to not be necessary Replaced the former patch 4 with series of patches to (slowly) provide support to disable new connections, handle removing waiting jobs, causing the waiting workers to quit, and allow any running jobs to complete. As it turns out, waiting for running jobs to complete cannot be done from the virNetServerClose callbacks because that means the event loop processing done during virNetServerRun will not allow any currently long running worker job thread a means to complete. So when virNetDaemonQuit is called as a result of the libvirtd signal handlers for SIG{QUIT|INT|TERM}, instead of just causing virNetServerRun to quit immediately, alter to using a quitRequested flag and then use that quitRequested flag to check for long running worker threads before causing the event loop to quit resulting in libvirtd being able to run through the virNetDaemonClose processing. John Ferlan (9): libvirtd: Alter refcnt processing for domain server objects libvirtd: Alter refcnt processing for server program objects netserver: Remove ServiceToggle during ServerDispose util: Introduce virThreadPoolDrain rpc: Introduce virNetServerQuitRequested rpc: Introduce virNetServerWorkerCount rpc: Alter virNetDaemonQuit processing docs: Add news article for libvirtd issue APPLY ONLY FOR TESTING PURPOSES daemon/libvirtd.c | 43 +++++++++++++++++++++++--------- docs/news.xml | 12 +++++++++ src/libvirt_private.syms | 1 + src/libvirt_remote.syms | 2 ++ src/qemu/qemu_driver.c | 5 ++++ src/rpc/virnetdaemon.c | 45 +++++++++++++++++++++++++++++++++- src/rpc/virnetserver.c | 52 ++++++++++++++++++++++++++++++++++++--- src/rpc/virnetserver.h | 4 +++ src/util/virthreadpool.c | 64 ++++++++++++++++++++++++++++++++++++++++-------- src/util/virthreadpool.h | 2 ++ 10 files changed, 204 insertions(+), 26 deletions(-) -- 2.13.6

5 29

[libvirt] [PATCHv1 00/12] BaselineHypervisorCPU using QEMU QMP exchanges
by Chris Venteicher 06 Jul '18

06 Jul '18

Some architectures (S390) depend on QEMU to compute baseline CPU model and expand a models feature set. Interacting with QEMU requires starting the QEMU process and completing one or more query-cpu-model-baseline QMP exchanges with QEMU in addition to a query-cpu-model-expansion QMP exchange to expand all features in the model. See "s390x CPU models: exposing features" patch set on Qemu-devel for discussion of QEMU aspects. This is part of resolution of: https://bugzilla.redhat.com/show_bug.cgi?id=1511999 Signed-off-by: Chris Venteicher <cventeic(a)redhat.com> Chris Venteicher (12): qemu_monitor_json: qemuMonitorCPUModelInfo / JSON conversion qemu_monitor: Introduce qemuMonitorCPUModelInfoInit and qemuMonitorCPUModelInfoFreeContents qemu_monitor: Introduce qemuMonitorGetCPUModelBaseline (query-cpu-model-baseline) qemu_monitor: Indicate when CPUModelInfo props report migratablity qemu_monitor: CPUModelExpansion on CPUModel with both name and properties cpu_conf: Calculate virCPUDef List Length function qemu_capabilities: virCPUDef / qemuMonitorCPUModelInfo conversions qemu_capabilities: QMPCommandPtr without shadow qmperr qemu_capabilities: Persist QEMU instance over multiple QMP Cmds qemu_capabilities: Introduce virQEMUCapsNewQMPCommandConnection qemu_capabilities: Introduce virQEMUCapsQMPBaselineCPUModel (baseline using QEMU) qemu_driver: BaselineHypervisorCPU supports S390 using QEMU/QMP src/conf/cpu_conf.c | 15 ++ src/conf/cpu_conf.h | 3 + src/libvirt_private.syms | 1 + src/qemu/qemu_capabilities.c | 335 +++++++++++++++++++++++++++++------ src/qemu/qemu_capabilities.h | 33 ++++ src/qemu/qemu_driver.c | 66 ++++++- src/qemu/qemu_monitor.c | 90 +++++++++- src/qemu/qemu_monitor.h | 16 +- src/qemu/qemu_monitor_json.c | 226 +++++++++++++++++++---- src/qemu/qemu_monitor_json.h | 14 +- tests/cputest.c | 7 +- 11 files changed, 683 insertions(+), 123 deletions(-) -- 2.17.1

2 20

[libvirt] [PATCH v3] Loop through all resolved addresses in virNetSocketNewListenTCP
by Olaf Hering 05 Jul '18

05 Jul '18

Currently virNetSocketNewListenTCP bails out early under the following conditions: - the hostname resolves to at least one IPv4 and at least one IPv6 address - the local interfaces have that one IPv4 address assigned, but not any of the IPv6 addresses - the local interfaces have just IPv6 link-local addresses In this case the resolver returns not only the IPv4 addresses but also IPv6. Binding the IPv6 address will obviously fail. But this terminates the entire loop, even if binding to IPv4 succeeded. To fix this error, just keep going and loop through all returned addresses. In case none of the attempts to bind to some address succeeded, try to report some appropriate error. Signed-off-by: Olaf Hering <olaf(a)aepfle.de> --- v3: more whitespace fixes, as suggested by Daniel P. Berrang�� v2: whitespace fixes, as suggested by John Ferlan src/rpc/virnetsocket.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index 7087abec9c..3a055acd39 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -382,11 +382,8 @@ int virNetSocketNewListenTCP(const char *nodename, #endif if (bind(fd, runp->ai_addr, runp->ai_addrlen) < 0) { - if (errno != EADDRINUSE) { - virReportSystemError(errno, "%s", _("Unable to bind to port")); - goto error; - } - addrInUse = true; + if (errno == EADDRINUSE) + addrInUse = true; VIR_FORCE_CLOSE(fd); runp = runp->ai_next; continue; @@ -409,14 +406,14 @@ int virNetSocketNewListenTCP(const char *nodename, fd = -1; } - if (nsocks == 0 && familyNotSupported) { - virReportSystemError(EAFNOSUPPORT, "%s", _("Unable to bind to port")); - goto error; - } - - if (nsocks == 0 && - addrInUse) { - virReportSystemError(EADDRINUSE, "%s", _("Unable to bind to port")); + if (nsocks == 0) { + if (familyNotSupported) + errno = EAFNOSUPPORT; + else if (addrInUse) + errno = EADDRINUSE; + else + errno = EDESTADDRREQ; + virReportSystemError(errno, "%s", _("Unable to bind to port")); goto error; }

3 7

[libvirt] [PATCH 0/2] domain_addr: move qemu function and rename
by Anya Harter 03 Jul '18

03 Jul '18

qemuDomainCCWAddrSetCreateFromDomain moved from qemu_domain_addr to domain_addr and rename to virDomainCCWAddrSetCreateFromDomain because it is not a qemu-specific function Anya Harter (2): qemu: move qemuDomainCCWAddrSetCreateFromDomain domain_addr: refactor qemuDomainCCWAddrSetCreateFromDomain src/conf/domain_addr.c | 24 ++++++++++++++++++++++++ src/conf/domain_addr.h | 4 ++++ src/qemu/qemu_domain_address.c | 26 ++------------------------ src/qemu/qemu_domain_address.h | 4 ---- src/qemu/qemu_hotplug.c | 4 ++-- 5 files changed, 32 insertions(+), 30 deletions(-) -- 2.17.1

2 4

[libvirt] [PATCH 0/5] viriscsi: Couple of fixes
by Michal Privoznik 03 Jul '18

03 Jul '18

I've noticed that I'm unable to start my iscsi pool if I put my own IQN into the pool XML. This is very surprising because iscsiadm would read it from initiatorname.conf file, right? Well, no. If IQN is provided in pool XML the code calls slightly different iscsiadm commands (e.g. to set up iSCSI interface) and we don't pass the interface everywhere we should. Michal Privoznik (5): virStorageBackendIQNFound: Fix ret value assignment virStorageBackendIQNFound: Rename out label virStorageBackendIQNFound: Rework iscsiadm output parsing virISCSIScanTargets: Honour iSCSI interface virISCSIScanTargets: Allow making targets persistent src/storage/storage_backend_iscsi.c | 5 +- src/util/viriscsi.c | 197 +++++++++++++++++++++++------------- src/util/viriscsi.h | 2 + tests/viriscsitest.c | 3 +- 4 files changed, 137 insertions(+), 70 deletions(-) -- 2.16.4

4 21

[libvirt] [PATCH] qemuDomainSaveMemory: Don't enforce dynamicOwnership
by Michal Privoznik 03 Jul '18

03 Jul '18

https://bugzilla.redhat.com/show_bug.cgi?id=1589115 When doing a memory snapshot qemuOpenFile() is used. This means that the file where memory is saved is firstly attempted to be created under root:root (because that's what libvirtd is running under) and if this fails the second attempt is done under domain's uid:gid. This does not make much sense - qemu is given opened FD so it does not need to access the file. Moreover, if dynamicOwnership is set in qemu.conf and the file lives on a squashed NFS this is deadly combination and very likely to fail. The fix consists of using: qemuOpenFileAs(fallback_uid = cfg->user, fallback_gid = cfg->group, dynamicOwnership = false) In other words, dynamicOwnership is turned off for memory snapshot (chown() will still be attempted if the file does not live on NFS) and instead of using domain DAC label, configured user:group is set as fallback. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- src/qemu/qemu_driver.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 129bacdd34..6af7e6e171 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -3223,6 +3223,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, unsigned int flags, qemuDomainAsyncJob asyncJob) { + virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver); bool bypassSecurityDriver = false; bool needUnlink = false; int ret = -1; @@ -3241,9 +3242,10 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, goto cleanup; } } - fd = qemuOpenFile(driver, vm, path, - O_WRONLY | O_TRUNC | O_CREAT | directFlag, - &needUnlink, &bypassSecurityDriver); + + fd = qemuOpenFileAs(cfg->user, cfg->group, false, path, + O_WRONLY | O_TRUNC | O_CREAT | directFlag, + &needUnlink, &bypassSecurityDriver); if (fd < 0) goto cleanup; @@ -3283,6 +3285,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver, cleanup: VIR_FORCE_CLOSE(fd); virFileWrapperFdFree(wrapperFd); + virObjectUnref(cfg); if (ret < 0 && needUnlink) unlink(path); -- 2.16.4

3 2

[libvirt] [PATCH 0/2] domain_addr: delete unused functions
by Anya Harter 02 Jul '18

02 Jul '18

each commit references the commit where the last use was removed Anya Harter (2): domain_addr: delete virDomainCCWAddressReleaseAddr domain_addr: delete virDomainVirtioSerialAddrRelease src/conf/domain_addr.c | 68 ---------------------------------------- src/conf/domain_addr.h | 8 ----- src/libvirt_private.syms | 2 -- 3 files changed, 78 deletions(-) -- 2.17.1

2 3

[libvirt] [PATCH] qemu: don't use chardev FD passing with standalone args
by Daniel P. Berrangé 02 Jul '18

02 Jul '18

When using domxml-to-native, we must generate CLI args that can be used in a standalone scenario. This means no FD passing can be used. To achieve this we must clear the QEMU_CAPS_CHARDEV_FD_PASS capability bit. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- src/qemu/qemu_process.c | 5 +++++ src/qemu/qemu_process.h | 2 ++ 2 files changed, 7 insertions(+) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index ac32dafcbe..40d35cbe6b 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -5124,6 +5124,9 @@ qemuProcessInit(virQEMUDriverPtr driver, vm->def->os.machine))) goto cleanup; + if (flags & VIR_QEMU_PROCESS_START_STANDALONE) + virQEMUCapsClear(priv->qemuCaps, QEMU_CAPS_CHARDEV_FD_PASS); + if (qemuDomainUpdateCPU(vm, updatedCPU, &origCPU) < 0) goto cleanup; @@ -6632,6 +6635,8 @@ qemuProcessCreatePretendCmd(virQEMUDriverPtr driver, flags |= VIR_QEMU_PROCESS_START_PRETEND; flags |= VIR_QEMU_PROCESS_START_NEW; + if (standalone) + flags |= VIR_QEMU_PROCESS_START_STANDALONE; if (qemuProcessInit(driver, vm, NULL, QEMU_ASYNC_JOB_NONE, !!migrateURI, flags) < 0) diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h index 531c2a0cc7..07ce3a9915 100644 --- a/src/qemu/qemu_process.h +++ b/src/qemu/qemu_process.h @@ -81,6 +81,8 @@ typedef enum { VIR_QEMU_PROCESS_START_PRETEND = 1 << 3, VIR_QEMU_PROCESS_START_NEW = 1 << 4, /* internal, new VM is starting */ VIR_QEMU_PROCESS_START_GEN_VMID = 1 << 5, /* Generate a new VMID */ + VIR_QEMU_PROCESS_START_STANDALONE = 1 << 6, /* Require CLI args to be usable standalone, + ie no FD passing and the like */ } qemuProcessStartFlags; int qemuProcessStart(virConnectPtr conn, -- 2.17.1

3 3