[libvirt] [PATCH] disable vCPU pinning with TCG mode
by Daniel P. Berrange
Although QMP returns info about vCPU threads in TCG mode, the
data it returns is mostly lies. Only the first vCPU has a valid
thread_id returned. The thread_id given for the other vCPUs is
in fact the main emulator thread. All vCPUs actually run under
the same thread in TCG mode.
Our vCPU pinning code is not at all able to cope with this
so if you try to set CPU affinity per-vCPU you end up with
wierd errors
error: Failed to start domain instance-00000007
error: cannot set CPU affinity on process 24365: Invalid argument
Since few people will care about the performance of TCG with
strict CPU pinning, lets just disable that for now, so we get
a clear error message
error: Failed to start domain instance-00000007
error: Requested operation is not valid: cpu affinity is not supported
---
src/qemu/qemu_process.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index b067f18..e2ccc4e 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -2231,6 +2231,40 @@ qemuProcessDetectVcpuPIDs(virQEMUDriverPtr driver,
int ncpupids;
qemuDomainObjPrivatePtr priv = vm->privateData;
+ /*
+ * Current QEMU *can* report info about host threads mapped
+ * to vCPUs, but it is not in a manner we can correctly
+ * deal with. The TCG CPU emulation does have a separate vCPU
+ * thread, but it runs every vCPU in that same thread. So it
+ * is impossible to setup different affinity per thread.
+ *
+ * What's more the 'query-cpus' command returns bizarre
+ * data for the threads. It gives the TCG thread for the
+ * vCPU 0, but for vCPUs 1-> N, it actually replies with
+ * the main process thread ID.
+ *
+ * The result is that when we try to set affinity for
+ * vCPU 1, it will actually change the affinity of the
+ * emulator thread :-( When you try to set affinity for
+ * vCPUs 2, 3.... it will fail if the affinity was
+ * different from vCPU 1.
+ *
+ * We *could* allow vcpu pinning with TCG, if we made the
+ * restriction that all vCPUs had the same mask. This would
+ * at least let us separate emulator from vCPUs threads, as
+ * we do for KVM. It would need some changes to our cgroups
+ * CPU layout though, and error reporting for the config
+ * restrictions.
+ *
+ * Just disable CPU pinning with TCG until someone wants
+ * to try to do this hard work.
+ */
+ if (vm->def->virtType == VIR_DOMAIN_VIRT_QEMU) {
+ priv->nvcpupids = 0;
+ priv->vcpupids = NULL;
+ return 0;
+ }
+
if (qemuDomainObjEnterMonitorAsync(driver, vm, asyncJob) < 0)
return -1;
/* failure to get the VCPU<-> PID mapping or to execute the query
--
2.1.0
10 years, 4 months
[libvirt] [PATCH] Don't setup fake CPU pids for old QEMU
by Daniel P. Berrange
The code assumes that def->vcpus == nvcpupids, so when we setup
fake CPU pids for old QEMU with nvcpupids == 1, we cause the
later code to read off the end of the array. This has fun results
like sche_setaffinity(0, ...) which changes libvirtd's own CPU
affinity, or even better sched_setaffinity($RANDOM, ...) which
changes the affinity of a random OS process.
---
src/qemu/qemu_process.c | 9 ++++-----
src/util/virprocess.c | 1 +
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index d683918..b067f18 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -2240,10 +2240,8 @@ qemuProcessDetectVcpuPIDs(virQEMUDriverPtr driver,
qemuDomainObjExitMonitor(driver, vm);
virResetLastError();
- priv->nvcpupids = 1;
- if (VIR_ALLOC_N(priv->vcpupids, priv->nvcpupids) < 0)
- return -1;
- priv->vcpupids[0] = vm->pid;
+ priv->nvcpupids = 0;
+ priv->vcpupids = NULL;
return 0;
}
qemuDomainObjExitMonitor(driver, vm);
@@ -2462,7 +2460,8 @@ qemuProcessSetVcpuAffinities(virDomainObjPtr vm)
virDomainVcpuPinDefPtr pininfo;
int n;
int ret = -1;
-
+ VIR_DEBUG("Setting affinity on CPUs nvcpupin=%zu nvcpus=%d nvcpupids=%d",
+ def->cputune.nvcpupin, def->vcpus, priv->nvcpupids);
if (!def->cputune.nvcpupin)
return 0;
diff --git a/src/util/virprocess.c b/src/util/virprocess.c
index 0c8a32f..d0a1500 100644
--- a/src/util/virprocess.c
+++ b/src/util/virprocess.c
@@ -399,6 +399,7 @@ int virProcessSetAffinity(pid_t pid, virBitmapPtr map)
{
size_t i;
bool set = false;
+ VIR_DEBUG("Set process affinity on %lld\n", (long long)pid);
# ifdef CPU_ALLOC
/* New method dynamically allocates cpu mask, allowing unlimted cpus */
int numcpus = 1024;
--
2.1.0
10 years, 4 months
[libvirt] [PATCH] qemu: Create memory-backend-{ram, file} iff needed
by Michal Privoznik
Libvirt BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1175397
QEMU BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1170093
In qemu there are two interesting arguments:
1) -numa to create a guest NUMA node
2) -object memory-backend-{ram,file} to tell qemu which memory
region on which host's NUMA node it should allocate the guest
memory from.
Combining these two together we can instruct qemu to create a
guest NUMA node that is tied to a host NUMA node. And it works
just fine. However, depending on machine type used, there might
be some issued during migration when OVMF is enabled (see QEMU
BZ). While this truly is a QEMU bug, we can help avoiding it. The
problem lies within the memory backend objects somewhere. Having
said that, fix on our side consists on putting those objects on
the command line if and only if needed. For instance, while
previously we would construct this (in all ways correct) command
line:
-object memory-backend-ram,size=256M,id=ram-node0 \
-numa node,nodeid=0,cpus=0,memdev=ram-node0
now we create just:
-numa node,nodeid=0,cpus=0,mem=256
because the backend object is obviously not tied to any specific
host NUMA node.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
src/qemu/qemu_command.c | 16 +++++++++-------
.../qemuxml2argvdata/qemuxml2argv-hugepages-pages3.args | 3 +--
.../qemuxml2argv-numatune-memnode-no-memory.args | 3 +--
3 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index 11e7cff..d5679de 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -6660,6 +6660,7 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg,
}
for (i = 0; i < def->cpu->ncells; i++) {
+ virDomainHugePagePtr hugepage = NULL;
unsigned long long cellmem = VIR_DIV_UP(def->cpu->cells[i].mem, 1024);
def->cpu->cells[i].mem = cellmem * 1024;
virMemAccess memAccess = def->cpu->cells[i].memAccess;
@@ -6681,7 +6682,6 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg,
if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_RAM) ||
virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_FILE)) {
virDomainNumatuneMemMode mode;
- virDomainHugePagePtr hugepage = NULL;
const char *policy = NULL;
mode = virDomainNumatuneGetMode(def->numatune, i);
@@ -6798,8 +6798,12 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg,
virBufferAsprintf(&buf, ",policy=%s", policy);
}
- virCommandAddArg(cmd, "-object");
- virCommandAddArgBuffer(cmd, &buf);
+ if (hugepage || nodemask) {
+ virCommandAddArg(cmd, "-object");
+ virCommandAddArgBuffer(cmd, &buf);
+ } else {
+ virBufferFreeAndReset(&buf);
+ }
} else {
if (memAccess) {
virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
@@ -6819,12 +6823,10 @@ qemuBuildNumaArgStr(virQEMUDriverConfigPtr cfg,
virBufferAdd(&buf, tmpmask, -1);
}
- if (virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_RAM) ||
- virQEMUCapsGet(qemuCaps, QEMU_CAPS_OBJECT_MEMORY_FILE)) {
+ if (hugepage || nodemask)
virBufferAsprintf(&buf, ",memdev=ram-node%zu", i);
- } else {
+ else
virBufferAsprintf(&buf, ",mem=%llu", cellmem);
- }
virCommandAddArgBuffer(cmd, &buf);
}
diff --git a/tests/qemuxml2argvdata/qemuxml2argv-hugepages-pages3.args b/tests/qemuxml2argvdata/qemuxml2argv-hugepages-pages3.args
index 27b3f8e..f81947e 100644
--- a/tests/qemuxml2argvdata/qemuxml2argv-hugepages-pages3.args
+++ b/tests/qemuxml2argvdata/qemuxml2argv-hugepages-pages3.args
@@ -1,7 +1,6 @@
LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \
/usr/bin/qemu -S -M pc -m 1024 -smp 2 \
--object memory-backend-ram,size=256M,id=ram-node0 \
--numa node,nodeid=0,cpus=0,memdev=ram-node0 \
+-numa node,nodeid=0,cpus=0,mem=256 \
-object memory-backend-file,prealloc=yes,\
mem-path=/dev/hugepages1G/libvirt/qemu,size=768M,id=ram-node1 \
-numa node,nodeid=1,cpus=1,memdev=ram-node1 \
diff --git a/tests/qemuxml2argvdata/qemuxml2argv-numatune-memnode-no-memory.args b/tests/qemuxml2argvdata/qemuxml2argv-numatune-memnode-no-memory.args
index b0e274c..2addf97 100644
--- a/tests/qemuxml2argvdata/qemuxml2argv-numatune-memnode-no-memory.args
+++ b/tests/qemuxml2argvdata/qemuxml2argv-numatune-memnode-no-memory.args
@@ -2,7 +2,6 @@ LC_ALL=C PATH=/bin HOME=/home/test USER=test LOGNAME=test QEMU_AUDIO_DRV=none \
/usr/bin/kvm -S -M pc -m 64 -smp 2 \
-object memory-backend-ram,size=32M,id=ram-node0,host-nodes=3,policy=preferred \
-numa node,nodeid=0,cpus=0,memdev=ram-node0 \
--object memory-backend-ram,size=32M,id=ram-node1 \
--numa node,nodeid=1,cpus=1,memdev=ram-node1 \
+-numa node,nodeid=1,cpus=1,mem=32 \
-nographic -monitor unix:/tmp/test-monitor,server,nowait \
-no-acpi -boot c -usb -net none -serial none -parallel none
--
2.0.4
10 years, 4 months
[libvirt] [libvirt-test-API][PATCH V4 0/4] Add test case for virconnect V4
by Jincheng Miao
V3->V4:
Remove getSysinfo() check for lxc connection.
Fix minor problems.
V2->V3:
Refactor connection_nodeinfo.
Change the way of getting version number.
V1->V2:
Seperate check functions in each test case.
Improve log message.
V1:
Add test case for virconnect
Add test case for nodeinfo of virconnect
Add connection_version test case
Add conf file of virconnect test
jmiao (4):
Add test case for virConnect
Add connection_nodeinfo test case
Add connection_version test case
Add test_connection.conf
cases/test_connection.conf | 31 +++++++
repos/virconn/__init__.py | 0
repos/virconn/connection_attributes.py | 92 +++++++++++++++++++++
repos/virconn/connection_nodeinfo.py | 146 +++++++++++++++++++++++++++++++++
repos/virconn/connection_version.py | 119 +++++++++++++++++++++++++++
5 files changed, 388 insertions(+)
create mode 100644 cases/test_connection.conf
create mode 100644 repos/virconn/__init__.py
create mode 100644 repos/virconn/connection_attributes.py
create mode 100644 repos/virconn/connection_nodeinfo.py
create mode 100644 repos/virconn/connection_version.py
--
1.8.3.1
10 years, 4 months
[libvirt] [PATCH v2] Buffer size too small when reading sysinfo
by Boris Fiuczynski
On a system with 160 CPUs the /proc/cpuinfo size grows beyond
the currently set limit of 10KB causing an internal error.
This patch increases the buffer size to 1MB.
Signed-off-by: Boris Fiuczynski <fiuczy(a)linux.vnet.ibm.com>
---
src/util/virsysinfo.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/util/virsysinfo.c b/src/util/virsysinfo.c
index 1bb6392..789c3bc 100644
--- a/src/util/virsysinfo.c
+++ b/src/util/virsysinfo.c
@@ -50,7 +50,7 @@ static const char *sysinfoCpuinfo = "/proc/cpuinfo";
#define SYSINFO_SMBIOS_DECODER sysinfoDmidecode
#define SYSINFO sysinfoSysinfo
#define CPUINFO sysinfoCpuinfo
-#define CPUINFO_FILE_LEN (10*1024) /* 10KB limit for /proc/cpuinfo file */
+#define CPUINFO_FILE_LEN (1024*1024) /* 1MB limit for /proc/cpuinfo file */
/* only to be used test programs, therefore not in sysinfo.h */
extern void virSysinfoSetup(const char *dmidecode, const char *sysinfo,
--
1.9.3
10 years, 4 months
[libvirt] [PATCH] qemu: fix memory leak in blockinfo
by Eric Blake
Coverity flagged commit 0282ca45 as introducing a memory leak;
in all my refactoring to make capacity probing conditional on
whether the image is non-raw, I missed deleting the unconditional
probe.
* src/qemu/qemu_driver.c (qemuStorageLimitsRefresh): Drop
redundant assignment.
Signed-off-by: Eric Blake <eblake(a)redhat.com>
---
Pushing as a trivial fix.
src/qemu/qemu_driver.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 89578e1..ebbb656 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -11116,9 +11116,6 @@ qemuStorageLimitsRefresh(virQEMUDriverPtr driver,
buf, len)) < 0)
goto cleanup;
}
- if (!(meta = virStorageFileGetMetadataFromBuf(src->path, buf, len,
- format, NULL)))
- goto cleanup;
if (format == VIR_STORAGE_FILE_RAW)
src->capacity = src->physical;
else if ((meta = virStorageFileGetMetadataFromBuf(src->path, buf,
--
1.9.3
10 years, 4 months
[libvirt] [PATCH v2 00/12] more progress towards backing chain allocation stats
by Eric Blake
v1 was here:
https://www.redhat.com/archives/libvir-list/2014-December/msg00370.html
ACKed patches from that series have been pushed. In this series,
3, 4, and 6 are new, and others try to address some of the feedback
and deal with rebased design decisions that resulted. I still don't
have things reporting quite as nicely as I would like (it turns out
that we HAVE to stat() a file for an online domain to learn its
allocation, and that for block devices, we HAVE to open()/lseek() to
learn its physical size; meanwhile, I still want to fix
virDomainGetBlockInfo to avoid read()ing a file while qemu is
active by reusing the code that getStats uses). But I'm posting
another round now, to hopefully get early patches ACKed and into
the tree, and to demonstrate that I now have recursive stat
collection for an active domain relying solely on qemu rather than
read()ing the backing files directly.
Eric Blake (12):
qemu: refactor blockinfo job handling
qemu: let blockinfo reuse virStorageSource
getstats: prepare monitor collection for recursion
getstats: perform recursion in monitor collection
getstats: rearrange blockinfo gathering
qemu: fix bugs in blockstats
qemu: refactor blockinfo data gathering
getstats: report block sizes for offline domains
getstats: prepare for dynamic block.count stat
getstats: add new flag for block backing chain
getstats: split block stats reporting for easier recursion
getstats: start crawling backing chain for qemu
include/libvirt/libvirt-domain.h | 24 +-
src/libvirt-domain.c | 7 +-
src/qemu/qemu_domain.c | 19 ++
src/qemu/qemu_domain.h | 1 +
src/qemu/qemu_driver.c | 477 ++++++++++++++++++++++++---------------
src/qemu/qemu_migration.c | 3 +-
src/qemu/qemu_monitor.c | 24 +-
src/qemu/qemu_monitor.h | 6 +-
src/qemu/qemu_monitor_json.c | 291 ++++++++++++++----------
src/qemu/qemu_monitor_json.h | 6 +-
src/util/virstoragefile.c | 3 +-
src/util/virstoragefile.h | 3 +-
tools/virsh-domain-monitor.c | 7 +
tools/virsh.pod | 8 +-
14 files changed, 558 insertions(+), 321 deletions(-)
--
1.9.3
10 years, 4 months
[libvirt] [PATCH v2] qemu: bulk stats: add pcpu placement information
by Francesco Romani
This patch adds the information about the physical cpu
placement of virtual cpus for bulk stats.
This is the only difference in output with the
virDomainGetVcpus() API.
Management software, like oVirt, needs this information
to properly manage NUMA configurations.
Signed-off-by: Francesco Romani <fromani(a)redhat.com>
---
src/libvirt-domain.c | 2 ++
src/qemu/qemu_driver.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c
index cb76d8c..e84f6a8 100644
--- a/src/libvirt-domain.c
+++ b/src/libvirt-domain.c
@@ -10888,6 +10888,8 @@ virConnectGetDomainCapabilities(virConnectPtr conn,
* from virVcpuState enum.
* "vcpu.<num>.time" - virtual cpu time spent by virtual CPU <num>
* as unsigned long long.
+ * "vcpu.<num>.physical" - real CPU number on which virtual CPU <num> is
+ * running, as int. -1 if offline.
*
* VIR_DOMAIN_STATS_INTERFACE: Return network interface statistics.
* The typed parameter keys are in this format:
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 830fca7..b62cabf 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -18348,6 +18348,15 @@ qemuDomainGetStatsVcpu(virQEMUDriverPtr driver ATTRIBUTE_UNUSED,
param_name,
cpuinfo[i].cpuTime) < 0)
goto cleanup;
+
+ snprintf(param_name, VIR_TYPED_PARAM_FIELD_LENGTH,
+ "vcpu.%zu.physical", i);
+ if (virTypedParamsAddInt(&record->params,
+ &record->nparams,
+ maxparams,
+ param_name,
+ cpuinfo[i].cpu) < 0)
+ goto cleanup;
}
ret = 0;
--
1.9.3
10 years, 4 months
[libvirt] [PATCH 00/11] Fix vm usage after ExitMonitor
by Ján Tomko
Check if the domain died while we were in the monitor
in most of the places (I didn't look at the Snapshot and Blockjob
APIs yet).
In some places, we've been accessing the data after it could've been
freed by qemuProcessStop (if the monitor API itself returned success,
but the domain was cleaned up because somehow qemuProcessStop
got the vm lock before the ExitMonitor call).
A more likely crash happens if qemu crashes in one of the APIs
with multiple Monitor calls, like the DetachDevice APIs
and AttachCharDevice.
Also, some occurences of writing the status XML for a dead domain
are fixed.
https://bugzilla.redhat.com/show_bug.cgi?id=1161024
Ján Tomko (11):
Introduce qemuDomainObjExitMonitorAlive
Mark the domain as active in qemuhotplugtest
Fix vm usage after ExitMonitor in UpdateDeviceList
Fix vm usage after ExitMonitor on device removal
Fix error message on redirdev caps detection
Fix vm usage after ExitMonitor on AttachDevice
Fix vm usage after ExitMonitor in DetachDevice
Fix monitor usage in qemuDomainHotplugVcpus
Fix vm usage after ExitMonitor in qemu driver
Fix vm usage after ExitMonitor in qemu migration
Fix vm usage after ExitMonitor in qemu process
src/qemu/qemu_command.c | 6 +-
src/qemu/qemu_domain.c | 20 +++++-
src/qemu/qemu_domain.h | 3 +
src/qemu/qemu_driver.c | 49 ++++++++++----
src/qemu/qemu_hotplug.c | 169 +++++++++++++++++++++++++++++++---------------
src/qemu/qemu_migration.c | 41 +++++++----
src/qemu/qemu_process.c | 65 ++++++++++++------
tests/qemuhotplugtest.c | 4 ++
8 files changed, 251 insertions(+), 106 deletions(-)
--
2.0.4
10 years, 4 months