[libvirt] [RFCv2 PATCH 0/5] Introduce RDT memory bandwidth allocation support
by bing.niu＠intel.com
From: Bing Niu <bing.niu(a)intel.com>
This series is to introduce RDT memory bandwidth allocation support by extending
current virresctrl implementation.
The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate
control over memory bandwidth available per-core. This feature provides a method to
control applications which may be over-utilizing bandwidth relative to their priority
in environments such as the data-center. The details can be found in Intel's SDM 17.19.7.
Kernel supports MBA through resctrl file system same as CAT. Each resctrl group have a
MB parameter to control how much memory bandwidth it can utilize in unit of percentage.
In this series, MBA is enabled by enhancing existing virresctrl implementation. The
policy employed for MBA is similar with CAT: The sum of each MBA group's bandwidth
dose not exceed 100%.
The enhancement of virresctrl include two parts:
Part 1: Add two new structures virResctrlInfoMB and virResctrlAllocMB for collecting
host system MBA capability and domain memory bandwidth allocation. Those two
structures are the extension of existing virResctrlInfo and virResctrlAlloc. With
them, virresctrl framework can support MBA and CAT concurrently. Each virResctrlAlloc
represent a resource allocation including CAT, or MBA, or CAT&MBA. The policy of MBA is
that: total memory bandwidth of each resctrl group, created by virresctrl, does not
exceed to 100%.
Part 2: On XML part, add new elements to host capabilities query and domain allocation to support
memory bandwidth allocation.
----------------------------------------------------------------------------------------------
For host capabilities XML, new XML format like below example,
<host>
.....
<memory>
<node id='0' cpus='0-19'>
<control granularity='10' min ='10' maxAllocs='8'/>
</node>
</memory>
</host>
granularity --- memory bandwidth granularity
min --- minimum memory bandwidth allowed
maxAllocs --- maximum concurrent memory bandwidth allocation allowed.
----------------------------------------------------------------------------------------------
For domain XML, new format as below example
<domain type='kvm' id='2'>
......
<cputune>
......
<shares>1024</shares>
<memorytune vcpus='0-1'>
<node id='0' bandwidth='20'/>
</memorytune>
</cputune>
......
</domain>
id --- node where memory bandwidth allocation will happen
bandwidth --- bandwidth allocated in percentage
----------------------------------------------------------------------------------------------
Changelog:
v1->v2:
Jano's comment: 1. put renaming parts into separated patches.
2. set the initial return value as -1.
3. using full name in structure definition.
4. do not use VIR_CACHE_TYPE_LAST for memory bandwidth allocation formatting.
Pavel's comment: 1. add host capabilities XML for memory bandwidth allocation.
2. do not mix use cachetune section in XML for memory bandwidth allocation in
domain XML. define a dedicated one for memory bandwidth allocation.
Bing Niu (5):
util: Rename and packing parts of virresctrl
util: Add memory bandwidth support to resctrl
conf: rename cachetune to restune
conf: Introduce cputune/memorytune to support memory bandwidth
allocation
conf: add memory bandwidth allocation capability of host
src/conf/capabilities.c | 112 +++++++++++
src/conf/capabilities.h | 11 ++
src/conf/domain_conf.c | 304 ++++++++++++++++++++++++++---
src/conf/domain_conf.h | 10 +-
src/libvirt_private.syms | 4 +-
src/qemu/qemu_process.c | 18 +-
src/util/virresctrl.c | 496 ++++++++++++++++++++++++++++++++++++++++++++---
src/util/virresctrl.h | 35 +++-
8 files changed, 918 insertions(+), 72 deletions(-)
--
2.7.4
5 years, 10 months
[libvirt] [PATCH 0/9] Resolve libvirtd hang on termination with connected long running client
by John Ferlan
RFC:
https://www.redhat.com/archives/libvir-list/2018-January/msg00318.html
Adjustments since RFC...
Patches 1&2: No change, were already R-B'd
Patch 3: Removed code as noted in code review, update commit message
Patch 4: From old series removed, see below for more details
Patch 9: no change
NB: Patches 5-8 and 10 from Nikolay Shirokovskiy <nshirokovskiy(a)virtuozzo.com>
are removed as they seemed to not be necessary
Replaced the former patch 4 with series of patches to (slowly) provide
support to disable new connections, handle removing waiting jobs, causing
the waiting workers to quit, and allow any running jobs to complete.
As it turns out, waiting for running jobs to complete cannot be done
from the virNetServerClose callbacks because that means the event loop
processing done during virNetServerRun will not allow any currently
long running worker job thread a means to complete.
So when virNetDaemonQuit is called as a result of the libvirtd signal
handlers for SIG{QUIT|INT|TERM}, instead of just causing virNetServerRun
to quit immediately, alter to using a quitRequested flag and then use
that quitRequested flag to check for long running worker threads before
causing the event loop to quit resulting in libvirtd being able to run
through the virNetDaemonClose processing.
John Ferlan (9):
libvirtd: Alter refcnt processing for domain server objects
libvirtd: Alter refcnt processing for server program objects
netserver: Remove ServiceToggle during ServerDispose
util: Introduce virThreadPoolDrain
rpc: Introduce virNetServerQuitRequested
rpc: Introduce virNetServerWorkerCount
rpc: Alter virNetDaemonQuit processing
docs: Add news article for libvirtd issue
APPLY ONLY FOR TESTING PURPOSES
daemon/libvirtd.c | 43 +++++++++++++++++++++++---------
docs/news.xml | 12 +++++++++
src/libvirt_private.syms | 1 +
src/libvirt_remote.syms | 2 ++
src/qemu/qemu_driver.c | 5 ++++
src/rpc/virnetdaemon.c | 45 +++++++++++++++++++++++++++++++++-
src/rpc/virnetserver.c | 52 ++++++++++++++++++++++++++++++++++++---
src/rpc/virnetserver.h | 4 +++
src/util/virthreadpool.c | 64 ++++++++++++++++++++++++++++++++++++++++--------
src/util/virthreadpool.h | 2 ++
10 files changed, 204 insertions(+), 26 deletions(-)
--
2.13.6
5 years, 10 months
[libvirt] [PATCHv1 00/12] BaselineHypervisorCPU using QEMU QMP exchanges
by Chris Venteicher
Some architectures (S390) depend on QEMU to compute baseline CPU model and
expand a models feature set.
Interacting with QEMU requires starting the QEMU process and completing one or
more query-cpu-model-baseline QMP exchanges with QEMU in addition to a
query-cpu-model-expansion QMP exchange to expand all features in the model.
See "s390x CPU models: exposing features" patch set on Qemu-devel for discussion
of QEMU aspects.
This is part of resolution of: https://bugzilla.redhat.com/show_bug.cgi?id=1511999
Signed-off-by: Chris Venteicher <cventeic(a)redhat.com>
Chris Venteicher (12):
qemu_monitor_json: qemuMonitorCPUModelInfo / JSON conversion
qemu_monitor: Introduce qemuMonitorCPUModelInfoInit and
qemuMonitorCPUModelInfoFreeContents
qemu_monitor: Introduce qemuMonitorGetCPUModelBaseline
(query-cpu-model-baseline)
qemu_monitor: Indicate when CPUModelInfo props report migratablity
qemu_monitor: CPUModelExpansion on CPUModel with both name and
properties
cpu_conf: Calculate virCPUDef List Length function
qemu_capabilities: virCPUDef / qemuMonitorCPUModelInfo conversions
qemu_capabilities: QMPCommandPtr without shadow qmperr
qemu_capabilities: Persist QEMU instance over multiple QMP Cmds
qemu_capabilities: Introduce virQEMUCapsNewQMPCommandConnection
qemu_capabilities: Introduce virQEMUCapsQMPBaselineCPUModel (baseline
using QEMU)
qemu_driver: BaselineHypervisorCPU supports S390 using QEMU/QMP
src/conf/cpu_conf.c | 15 ++
src/conf/cpu_conf.h | 3 +
src/libvirt_private.syms | 1 +
src/qemu/qemu_capabilities.c | 335 +++++++++++++++++++++++++++++------
src/qemu/qemu_capabilities.h | 33 ++++
src/qemu/qemu_driver.c | 66 ++++++-
src/qemu/qemu_monitor.c | 90 +++++++++-
src/qemu/qemu_monitor.h | 16 +-
src/qemu/qemu_monitor_json.c | 226 +++++++++++++++++++----
src/qemu/qemu_monitor_json.h | 14 +-
tests/cputest.c | 7 +-
11 files changed, 683 insertions(+), 123 deletions(-)
--
2.17.1
5 years, 10 months
[libvirt] [PATCH v3] Loop through all resolved addresses in virNetSocketNewListenTCP
by Olaf Hering
Currently virNetSocketNewListenTCP bails out early under the following
conditions:
- the hostname resolves to at least one IPv4 and at least one IPv6
address
- the local interfaces have that one IPv4 address assigned, but not any
of the IPv6 addresses
- the local interfaces have just IPv6 link-local addresses
In this case the resolver returns not only the IPv4 addresses but also
IPv6. Binding the IPv6 address will obviously fail. But this terminates
the entire loop, even if binding to IPv4 succeeded.
To fix this error, just keep going and loop through all returned
addresses. In case none of the attempts to bind to some address
succeeded, try to report some appropriate error.
Signed-off-by: Olaf Hering <olaf(a)aepfle.de>
---
v3:
more whitespace fixes, as suggested by Daniel P. Berrang��
v2:
whitespace fixes, as suggested by John Ferlan
src/rpc/virnetsocket.c | 23 ++++++++++-------------
1 file changed, 10 insertions(+), 13 deletions(-)
diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c
index 7087abec9c..3a055acd39 100644
--- a/src/rpc/virnetsocket.c
+++ b/src/rpc/virnetsocket.c
@@ -382,11 +382,8 @@ int virNetSocketNewListenTCP(const char *nodename,
#endif
if (bind(fd, runp->ai_addr, runp->ai_addrlen) < 0) {
- if (errno != EADDRINUSE) {
- virReportSystemError(errno, "%s", _("Unable to bind to port"));
- goto error;
- }
- addrInUse = true;
+ if (errno == EADDRINUSE)
+ addrInUse = true;
VIR_FORCE_CLOSE(fd);
runp = runp->ai_next;
continue;
@@ -409,14 +406,14 @@ int virNetSocketNewListenTCP(const char *nodename,
fd = -1;
}
- if (nsocks == 0 && familyNotSupported) {
- virReportSystemError(EAFNOSUPPORT, "%s", _("Unable to bind to port"));
- goto error;
- }
-
- if (nsocks == 0 &&
- addrInUse) {
- virReportSystemError(EADDRINUSE, "%s", _("Unable to bind to port"));
+ if (nsocks == 0) {
+ if (familyNotSupported)
+ errno = EAFNOSUPPORT;
+ else if (addrInUse)
+ errno = EADDRINUSE;
+ else
+ errno = EDESTADDRREQ;
+ virReportSystemError(errno, "%s", _("Unable to bind to port"));
goto error;
}
5 years, 10 months
[libvirt] [PATCH 0/2] domain_addr: move qemu function and rename
by Anya Harter
qemuDomainCCWAddrSetCreateFromDomain moved from qemu_domain_addr to
domain_addr and rename to virDomainCCWAddrSetCreateFromDomain because
it is not a qemu-specific function
Anya Harter (2):
qemu: move qemuDomainCCWAddrSetCreateFromDomain
domain_addr: refactor qemuDomainCCWAddrSetCreateFromDomain
src/conf/domain_addr.c | 24 ++++++++++++++++++++++++
src/conf/domain_addr.h | 4 ++++
src/qemu/qemu_domain_address.c | 26 ++------------------------
src/qemu/qemu_domain_address.h | 4 ----
src/qemu/qemu_hotplug.c | 4 ++--
5 files changed, 32 insertions(+), 30 deletions(-)
--
2.17.1
5 years, 10 months
[libvirt] [PATCH 0/5] viriscsi: Couple of fixes
by Michal Privoznik
I've noticed that I'm unable to start my iscsi pool if I put my
own IQN into the pool XML. This is very surprising because
iscsiadm would read it from initiatorname.conf file, right? Well,
no. If IQN is provided in pool XML the code calls slightly
different iscsiadm commands (e.g. to set up iSCSI interface) and
we don't pass the interface everywhere we should.
Michal Privoznik (5):
virStorageBackendIQNFound: Fix ret value assignment
virStorageBackendIQNFound: Rename out label
virStorageBackendIQNFound: Rework iscsiadm output parsing
virISCSIScanTargets: Honour iSCSI interface
virISCSIScanTargets: Allow making targets persistent
src/storage/storage_backend_iscsi.c | 5 +-
src/util/viriscsi.c | 197 +++++++++++++++++++++++-------------
src/util/viriscsi.h | 2 +
tests/viriscsitest.c | 3 +-
4 files changed, 137 insertions(+), 70 deletions(-)
--
2.16.4
5 years, 10 months
[libvirt] [PATCH] qemuDomainSaveMemory: Don't enforce dynamicOwnership
by Michal Privoznik
https://bugzilla.redhat.com/show_bug.cgi?id=1589115
When doing a memory snapshot qemuOpenFile() is used. This means
that the file where memory is saved is firstly attempted to be
created under root:root (because that's what libvirtd is running
under) and if this fails the second attempt is done under
domain's uid:gid. This does not make much sense - qemu is given
opened FD so it does not need to access the file. Moreover, if
dynamicOwnership is set in qemu.conf and the file lives on a
squashed NFS this is deadly combination and very likely to fail.
The fix consists of using:
qemuOpenFileAs(fallback_uid = cfg->user,
fallback_gid = cfg->group,
dynamicOwnership = false)
In other words, dynamicOwnership is turned off for memory
snapshot (chown() will still be attempted if the file does not
live on NFS) and instead of using domain DAC label, configured
user:group is set as fallback.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
src/qemu/qemu_driver.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 129bacdd34..6af7e6e171 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -3223,6 +3223,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver,
unsigned int flags,
qemuDomainAsyncJob asyncJob)
{
+ virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver);
bool bypassSecurityDriver = false;
bool needUnlink = false;
int ret = -1;
@@ -3241,9 +3242,10 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver,
goto cleanup;
}
}
- fd = qemuOpenFile(driver, vm, path,
- O_WRONLY | O_TRUNC | O_CREAT | directFlag,
- &needUnlink, &bypassSecurityDriver);
+
+ fd = qemuOpenFileAs(cfg->user, cfg->group, false, path,
+ O_WRONLY | O_TRUNC | O_CREAT | directFlag,
+ &needUnlink, &bypassSecurityDriver);
if (fd < 0)
goto cleanup;
@@ -3283,6 +3285,7 @@ qemuDomainSaveMemory(virQEMUDriverPtr driver,
cleanup:
VIR_FORCE_CLOSE(fd);
virFileWrapperFdFree(wrapperFd);
+ virObjectUnref(cfg);
if (ret < 0 && needUnlink)
unlink(path);
--
2.16.4
5 years, 10 months
[libvirt] [PATCH 0/2] domain_addr: delete unused functions
by Anya Harter
each commit references the commit where the last use was removed
Anya Harter (2):
domain_addr: delete virDomainCCWAddressReleaseAddr
domain_addr: delete virDomainVirtioSerialAddrRelease
src/conf/domain_addr.c | 68 ----------------------------------------
src/conf/domain_addr.h | 8 -----
src/libvirt_private.syms | 2 --
3 files changed, 78 deletions(-)
--
2.17.1
5 years, 10 months
[libvirt] [PATCH] qemu: don't use chardev FD passing with standalone args
by Daniel P. Berrangé
When using domxml-to-native, we must generate CLI args that can be used
in a standalone scenario. This means no FD passing can be used. To
achieve this we must clear the QEMU_CAPS_CHARDEV_FD_PASS capability bit.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
src/qemu/qemu_process.c | 5 +++++
src/qemu/qemu_process.h | 2 ++
2 files changed, 7 insertions(+)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index ac32dafcbe..40d35cbe6b 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -5124,6 +5124,9 @@ qemuProcessInit(virQEMUDriverPtr driver,
vm->def->os.machine)))
goto cleanup;
+ if (flags & VIR_QEMU_PROCESS_START_STANDALONE)
+ virQEMUCapsClear(priv->qemuCaps, QEMU_CAPS_CHARDEV_FD_PASS);
+
if (qemuDomainUpdateCPU(vm, updatedCPU, &origCPU) < 0)
goto cleanup;
@@ -6632,6 +6635,8 @@ qemuProcessCreatePretendCmd(virQEMUDriverPtr driver,
flags |= VIR_QEMU_PROCESS_START_PRETEND;
flags |= VIR_QEMU_PROCESS_START_NEW;
+ if (standalone)
+ flags |= VIR_QEMU_PROCESS_START_STANDALONE;
if (qemuProcessInit(driver, vm, NULL, QEMU_ASYNC_JOB_NONE,
!!migrateURI, flags) < 0)
diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h
index 531c2a0cc7..07ce3a9915 100644
--- a/src/qemu/qemu_process.h
+++ b/src/qemu/qemu_process.h
@@ -81,6 +81,8 @@ typedef enum {
VIR_QEMU_PROCESS_START_PRETEND = 1 << 3,
VIR_QEMU_PROCESS_START_NEW = 1 << 4, /* internal, new VM is starting */
VIR_QEMU_PROCESS_START_GEN_VMID = 1 << 5, /* Generate a new VMID */
+ VIR_QEMU_PROCESS_START_STANDALONE = 1 << 6, /* Require CLI args to be usable standalone,
+ ie no FD passing and the like */
} qemuProcessStartFlags;
int qemuProcessStart(virConnectPtr conn,
--
2.17.1
5 years, 10 months
[libvirt] [PATCH v2 0/2] {lxc, util}: Fix issues with NULL argument 'type' for mount().
by Julio Faracco
This serie fixes the argument 'type' of syscall mount(). Valgrind is
throwing an issue when that parameter 'type' of the syscall is NULL.
The best option to fix this issue is replace NULL to "none"
filesystem. For more details about the Valgrind bug, see the commit
message at 794b576c.
Julio Faracco (2):
lxc: moving 'type' argument to avoid issues with mount() syscall.
util: moving 'type' argument to avoid issues with mount() syscall.
src/lxc/lxc_container.c | 26 +++++++++++++-------------
src/util/vircgroup.c | 2 +-
2 files changed, 14 insertions(+), 14 deletions(-)
--
2.17.1
5 years, 10 months
