December 2016 - Devel - libvirt List Archives

[libvirt] [PATCH 0/6] Qemu: s390: Cpu Model Support
by Jason J. Herne 07 Dec '16

07 Dec '16

This patch set enables cpu model support for s390. The user can now set exact cpu models, query supported models via virsh domcapabilities, and use host-model and host-passthrough modes. The end result is that migration is safer because Qemu will perform runnability checking on the destination host and quit with an error if the guest's cpu model is not supported. Big Thanks for Jiri and Eduardo for being patient and answering our questions while we figured out what we were doing! Collin L. Walling (5): s390: Stop reporting "host" for host model qemu: qmp query-cpu-model-expansion command qemu-caps: Get host model directly from Qemu when available qemu: migration: warn if migrating with host-passthrough qemu: command: Support new cpu feature argument syntax Jason J. Herne (1): s390: Cpu driver support for update and compare po/POTFILES.in | 1 + src/cpu/cpu_s390.c | 61 ++++++++++++++++++++---- src/qemu/qemu_capabilities.c | 109 +++++++++++++++++++++++++++++++++++++++++-- src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 10 +++- src/qemu/qemu_migration.c | 4 ++ src/qemu/qemu_monitor.c | 60 ++++++++++++++++++++++++ src/qemu/qemu_monitor.h | 22 +++++++++ src/qemu/qemu_monitor_json.c | 94 +++++++++++++++++++++++++++++++++++++ src/qemu/qemu_monitor_json.h | 6 +++ 10 files changed, 353 insertions(+), 15 deletions(-) -- 1.9.1

3 16

[libvirt] [PATCH v3 0/3] gluster: cache glfs connection object per volume
by Prasanna Kumar Kalever 07 Dec '16

07 Dec '16

v3: Address comments by Daniel and Peter on v2 * Split the patch to 3 parts Patch 1: change the virStorageNetHostDef type Patch 2: optimize calls to virStorageFileInit and friends Patch 3: add the caching for glfs * Thanks to Daniel, this version make all the methods as thread safe * Thanks to Peter for pointing use of virObjectLockable, this had simplified rest of the parts a lot. v2: Address review comments from Peter on v1 * Rebased on latest master * Changes to commit msg * Introduce storage API's for Register and Unregister of volume * During qemu process Start/Stop and snapshot create * Check Transport and Port type * Atomic element add/del to list and ref counting Pending: Treating IP and FQDN belong to same host v1: Initial patch Prasanna Kumar Kalever (3): util: change the virStorageNetHostDef type storage: optimize calls to virStorageFileInit and friends gluster: cache glfs connection object per volume src/conf/domain_conf.c | 46 +++--- src/qemu/qemu_command.c | 65 ++++---- src/qemu/qemu_domain.c | 2 +- src/qemu/qemu_domain.h | 5 + src/qemu/qemu_driver.c | 40 +++-- src/qemu/qemu_parse_command.c | 44 +++--- src/qemu/qemu_process.c | 45 ++++++ src/qemu/qemu_process.h | 4 + src/storage/storage_backend_fs.c | 2 + src/storage/storage_backend_gluster.c | 270 ++++++++++++++++++++++++++++++---- src/storage/storage_driver.c | 23 +-- src/util/virstoragefile.c | 85 ++++++----- src/util/virstoragefile.h | 20 ++- tests/virstoragetest.c | 2 +- 14 files changed, 477 insertions(+), 176 deletions(-) -- 2.7.4

3 10

[libvirt] Libvirt domain event usage and consistency
by Roman Mohr 07 Dec '16

07 Dec '16

Hi, I recently started to use the libvirt domain events. With them I increase the responsiveness of my VM state wachers. In general it works pretty well. I just listen to the events and do a periodic resync to cope with missed events. While watching the events I ran into a few interesting situations I wanted to share. The points 1-3 describe some minor issues or irregularities. Point 4 is about the fact that domain and state updates are not versioned which makes it very hard to stay in sync with libvirt when using events. My libvirt version is 1.2.18.4. 1) Event order seems to be weird on startup: When listening for VM lifecycle events I get this order: {"event_type": "Started", "timestamp": "2016-11-25T11:59:53.209326Z", "reason": "Booted", "domain_name": "generic", "domain_id": "8ff7047b-fb46-44ff-a4c6-7c20c73ab86e"} {"event_type": "Defined", "timestamp": "2016-11-25T11:59:53.435530Z", "reason": "Added", "domain_name": "generic", "domain_id": "8ff7047b-fb46-44ff-a4c6-7c20c73ab86e"} It is strange that a VM already boots before it is defined. Is this the intended order? 2) Defining a VM with VIR_DOMAIN_START_PAUSED gives me this event order {"event_type": "Defined", "timestamp": "2016-11-25T12:02:44.037817Z", "reason": "Added", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} {"event_type": "Resumed", "timestamp": "2016-11-25T12:02:44.813104Z", "reason": "Unpaused", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} {"event_type": "Started", "timestamp": "2016-11-25T12:02:44.813733Z", "reason": "Booted", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} This boot-order makes it hard to track active domains by listening to life-cycle events. One could theoretically still always fetch the VM state in the event callback and check the state, but if the state is not immediately transferred with the event itself, it can already be outdated, so this might be racy (intransparent for the libvirt bindings user), and as described in (3) currently not even possible. In general the real existing events seem to differ quite significantly from the described life-cycle in [1]. 3) "Defined" event is triggered before the domain is completely defined {"event_type": "Defined", "timestamp": "2016-11-25T12:02:44.037817Z", "reason": "Added", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} {"event_type": "Resumed", "timestamp": "2016-11-25T12:02:44.813104Z", "reason": "Unpaused", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} {"event_type": "Started", "timestamp": "2016-11-25T12:02:44.813733Z", "reason": "Booted", "domain_name": "core_node", "domain_id": "b9906489-6d5b-40f8-a742-ca71b2b84277"} When I try to process the first event and do a xmldump I get: Event: [Code-42] [Domain-10] Domain not found: no domain with matching uuid 'b9906489-6d5b-40f8-a742-ca71b2b84277' (core_node) So it seems like I get the event before the domain is completely ready. 4) There libvirt domain description is not versioned I would expect that every time I update a domainxml (update from third party entity), or an event is generated (update from libvirt), that the resource version of a Domain is increased and that I get this resource version when I do a xmldump or when I get an event. Without this there is afaik no way to stay in sync with libvirt, even if you do regular polling of all domains. The main issue here is that I can never know if events in the queue arrived before my latest domain resync or after it. Also not that this is not about delivery guarantees of events. It is just about having a consistent view of a VM and the individual event. If I have resource versions, I can decide if an event is still interesting for me or not, which is exactly what I need to solve the syncing problem above. When I do a complete relisting of all domains to syn, I know which version I got and I can then see on every event if it is newer or older. If along side with the event, the domain xml, the VM state, and the resource version would be sent to a client, it would be even better. Then, whenever there is a new event for a VM in the queue, I can be sure that this domainxml I see is the one which triggered the event. This xml is then a complete representation for this revision number. Would be nice to hear your thoughts to these points. Best Regards, Roman [1] https://wiki.libvirt.org/page/VM_lifecycle#States_that_a_guest_domain_can_b…

3 10

[libvirt] [PATCH v3] cgroup: Use system reported "unlimited" value for comparison
by Viktor Mihajlovski 07 Dec '16

07 Dec '16

With kernel 3.18 (since commit 3e32cb2e0a12b6915056ff04601cf1bb9b44f967) the "unlimited" value for cgroup memory limits has changed once again as its byte value is now computed from a page counter. The new "unlimited" value reported by the cgroup fs is therefore 2**51-1 pages which is (VIR_DOMAIN_MEMORY_PARAM_UNLIMITED - 3072). This results e.g. in virsh memtune displaying 9007199254740988 instead of unlimited for the limits. This patch uses the value of memory.limit_in_bytes from the cgroup memory root which is the system's "real" unlimited value for comparison. See also libvirt commit 231656bbeb9e4d3bedc44362784c35eee21cf0f4 for the history for kernel 3.12 and before. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- v2: - removed RFC verbiage from commit message - per Martin's review comment, cache the cgroup memory.limit_in_bytes - used the cgroup detection logic proposed by Martin, much nicer now indeed - other than initially planned, fall back to VIR_DOMAIN_MEMORY_PARAM_UNLIMITED in case of cgroup read failure, since the usual paranoia fits in nicely with the "already initialized" check. With the current code structure this will never be called when the memory controller is not configured or mounted anyway... - while at it, replaced the goto cleanups with direct returns as there's really no cleanup to be done in the GetMemoryxxxLimit functions v3: - use thread safe one time initialization for the "unlimited" value, to account for 64-bit architectures which do not guarantee atomic memory updates of long long values (which reportedly is the case for ARM). src/util/vircgroup.c | 73 +++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 55 insertions(+), 18 deletions(-) diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index f151193..b6affe3 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -60,6 +60,7 @@ #include "virsystemd.h" #include "virtypedparam.h" #include "virhostcpu.h" +#include "virthread.h" VIR_LOG_INIT("util.cgroup"); @@ -2452,6 +2453,51 @@ virCgroupGetBlkioDeviceWeight(virCgroupPtr group, } +/* + * Retrieve the "memory.limit_in_bytes" value from the memory controller + * root dir. This value cannot be modified by userspace and therefore + * is the maximum limit value supported by cgroups on the local system. + * Returns this value scaled to KB or falls back to the original + * VIR_DOMAIN_MEMORY_PARAM_UNLIMITED. Either way, remember the return + * value to avoid unnecessary cgroup filesystem access. + */ +static unsigned long long int virCgroupMemoryUnlimitedKB; +static virOnceControl virCgroupMemoryOnce = VIR_ONCE_CONTROL_INITIALIZER; + +static void +virCgroupMemoryOnceInit(void) +{ + virCgroupPtr group; + unsigned long long int mem_unlimited = 0ULL; + + if (virCgroupNew(-1, "/", NULL, -1, &group) < 0) + goto cleanup; + + if (!virCgroupHasController(group, VIR_CGROUP_CONTROLLER_MEMORY)) + goto cleanup; + + ignore_value(virCgroupGetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + "memory.limit_in_bytes", + &mem_unlimited)); + cleanup: + virCgroupFree(&group); + virCgroupMemoryUnlimitedKB = mem_unlimited >> 10; +} + +static unsigned long long int +virCgroupGetMemoryUnlimitedKB(void) +{ + if (virOnce(&virCgroupMemoryOnce, virCgroupMemoryOnceInit) < 0) + VIR_DEBUG("Init failed, will fall back to defaults."); + + if (virCgroupMemoryUnlimitedKB) + return virCgroupMemoryUnlimitedKB; + else + return VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; +} + + /** * virCgroupSetMemory: * @@ -2534,20 +2580,17 @@ int virCgroupGetMemoryHardLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; - int ret = -1; if (virCgroupGetValueU64(group, VIR_CGROUP_CONTROLLER_MEMORY, "memory.limit_in_bytes", &limit_in_bytes) < 0) - goto cleanup; + return -1; *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (*kb >= virCgroupGetMemoryUnlimitedKB()) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; - ret = 0; - cleanup: - return ret; + return 0; } @@ -2596,20 +2639,17 @@ int virCgroupGetMemorySoftLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; - int ret = -1; if (virCgroupGetValueU64(group, VIR_CGROUP_CONTROLLER_MEMORY, "memory.soft_limit_in_bytes", &limit_in_bytes) < 0) - goto cleanup; + return -1; *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (*kb >= virCgroupGetMemoryUnlimitedKB()) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; - ret = 0; - cleanup: - return ret; + return 0; } @@ -2658,20 +2698,17 @@ int virCgroupGetMemSwapHardLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; - int ret = -1; if (virCgroupGetValueU64(group, VIR_CGROUP_CONTROLLER_MEMORY, "memory.memsw.limit_in_bytes", &limit_in_bytes) < 0) - goto cleanup; + return -1; *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (*kb >= virCgroupGetMemoryUnlimitedKB()) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; - ret = 0; - cleanup: - return ret; + return 0; } -- 1.9.1

2 2

[libvirt] [libvirt-perl PATCH] Add blkdeviotune group_name parameter
by John Ferlan 06 Dec '16

06 Dec '16

Add VIR_DOMAIN_{BLOCK_IOTUNE|TUNABLE_BLKDEV}_GROUP_NAME parameters and the description. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- Changes | 2 +- Virt.xs | 2 ++ lib/Sys/Virt/Domain.pm | 5 +++++ 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/Changes b/Changes index 721f036..c9a36dd 100644 --- a/Changes +++ b/Changes @@ -2,7 +2,7 @@ Revision history for perl module Sys::Virt 3.0.0 2017-00-00 - - XXX + - Add group_name for block iotune 2.5.0 2016-12-05 diff --git a/Virt.xs b/Virt.xs index cbf2e7b..ac14feb 100644 --- a/Virt.xs +++ b/Virt.xs @@ -8275,6 +8275,7 @@ BOOT: REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_READ_IOPS_SEC_MAX, BLOCK_IOTUNE_READ_IOPS_SEC_MAX); REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX, BLOCK_IOTUNE_WRITE_IOPS_SEC_MAX); REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_SIZE_IOPS_SEC, BLOCK_IOTUNE_SIZE_IOPS_SEC); + REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_GROUP_NAME, BLOCK_IOTUNE_GROUP_NAME); REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH, BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH); REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH, BLOCK_IOTUNE_READ_BYTES_SEC_MAX_LENGTH); REGISTER_CONSTANT_STR(VIR_DOMAIN_BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH, BLOCK_IOTUNE_WRITE_BYTES_SEC_MAX_LENGTH); @@ -8491,6 +8492,7 @@ BOOT: REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_BYTES_SEC_MAX, TUNABLE_BLKDEV_TOTAL_BYTES_SEC_MAX); REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_TOTAL_IOPS_SEC_MAX, TUNABLE_BLKDEV_TOTAL_IOPS_SEC_MAX); REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_SIZE_IOPS_SEC, TUNABLE_BLKDEV_SIZE_IOPS_SEC); + REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_GROUP_NAME, TUNABLE_BLKDEV_GROUP_NAME); REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_READ_BYTES_SEC_MAX_LENGTH, TUNABLE_BLKDEV_READ_BYTES_SEC_MAX_LENGTH); REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_READ_IOPS_SEC_MAX_LENGTH, TUNABLE_BLKDEV_READ_IOPS_SEC_MAX_LENGTH); REGISTER_CONSTANT_STR(VIR_DOMAIN_TUNABLE_BLKDEV_WRITE_BYTES_SEC_MAX_LENGTH, TUNABLE_BLKDEV_WRITE_BYTES_SEC_MAX_LENGTH); diff --git a/lib/Sys/Virt/Domain.pm b/lib/Sys/Virt/Domain.pm index ba58f26..e11440d 100644 --- a/lib/Sys/Virt/Domain.pm +++ b/lib/Sys/Virt/Domain.pm @@ -2566,6 +2566,11 @@ The maximum I/O operations written per second. The maximum I/O operations per second +=item Sys::Virt::Domain::BLOCK_IOTUNE_GROUP_NAME + +A string representing a group name to allow sharing of I/O +throttling quota between multiple drives + =item Sys::Virt::Domain::BLOCK_IOTUNE_TOTAL_BYTES_SEC_MAX_LENGTH The duration in seconds allowed for maximum total bytes processed per second. -- 2.7.4

2 1

[libvirt] [PATCH] qemu: capabilities: Add gluster.debug_level detection for 2.8.0+
by Peter Krempa 06 Dec '16

06 Dec '16

Qemu 2.8.0+ changes arguments structure for blockdev-add in the effort to make it finally stable. Since libvirt recently added the detection of gluster debug support relying on the old syntax we need to add the new as well. --- src/qemu/qemu_capabilities.c | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + 2 files changed, 2 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index df417f2..214f848 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -1725,6 +1725,7 @@ static struct virQEMUCapsStringFlags virQEMUCapsObjectPropsUSBNECXHCI[] = { /* see documentation for virQEMUCapsQMPSchemaGetByPath for the query format */ static struct virQEMUCapsStringFlags virQEMUCapsQMPSchemaQueries[] = { { "blockdev-add/arg-type/options/+gluster/debug-level", QEMU_CAPS_GLUSTER_DEBUG_LEVEL}, + { "blockdev-add/arg-type/+gluster/debug-level", QEMU_CAPS_GLUSTER_DEBUG_LEVEL}, }; struct virQEMUCapsObjectTypeProps { diff --git a/tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml b/tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml index f5e467c..60c53bc 100644 --- a/tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml +++ b/tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml @@ -195,6 +195,7 @@ <flag name='ivshmem-plain'/> <flag name='ivshmem-doorbell'/> <flag name='query-qmp-schema'/> + <flag name='gluster.debug_level'/> <flag name='vhost-scsi'/> <flag name='drive-iotune-group'/> <version>2007091</version> -- 2.10.2

2 3

[libvirt] [PATCH] perf: add one more perf event support
by Nitesh Konkar 06 Dec '16

06 Dec '16

With current perf framework, this patch adds support and documentation for branch instructions perf event. Signed-off-by: Nitesh Konkar <nitkon12(a)linux.vnet.ibm.com> --- docs/formatdomain.html.in | 6 ++++ docs/schemas/domaincommon.rng | 1 + include/libvirt/libvirt-domain.h | 10 +++++++ src/libvirt-domain.c | 38 +++++++++++++------------ src/qemu/qemu_driver.c | 1 + src/util/virperf.c | 6 +++- src/util/virperf.h | 9 +++--- tests/genericxml2xmlindata/generic-perf.xml | 1 + tools/virsh.pod | 43 +++++++++++++++-------------- 9 files changed, 72 insertions(+), 43 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 6bd02cc..259b2c6 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -1927,6 +1927,7 @@ <event name='instructions' enabled='yes'/> <event name='cache_references' enabled='no'/> <event name='cache_misses' enabled='no'/> + <event name='branch_instructions' enabled='no'/> </perf> ... </pre> @@ -1972,6 +1973,11 @@ <td>the count of cache misses by applications running on the platform</td> <td><code>perf.cache_misses</code></td> </tr> + <tr> + <td><code>hardware_instructions</code></td> + <td>the count of hardware instructions by applications running on the platform</td> + <td><code>perf.hardware_instructions</code></td> + </tr> </table> <h3><a name="elementsDevices">Devices</a></h3> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index bb903ef..5fdc036 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -427,6 +427,7 @@ <value>instructions</value> <value>cache_references</value> <value>cache_misses</value> + <value>branch_instructions</value> </choice> </attribute> <attribute name="enabled"> diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h index a8435ab..aad2541 100644 --- a/include/libvirt/libvirt-domain.h +++ b/include/libvirt/libvirt-domain.h @@ -2125,6 +2125,16 @@ void virDomainStatsRecordListFree(virDomainStatsRecordPtr *stats); */ # define VIR_PERF_PARAM_CPU_CYCLES "cpu_cycles" +/** + * VIR_PERF_PARAM_BRANCH_INSTRUCTIONS: + * + * Macro for typed parameter name that represents branch instructions + * perf event which can be used to measure the count of branch instructions + * by applications running on the platform. It corresponds to the + * "perf.branch_instructions" field in the *Stats APIs. + */ +# define VIR_PERF_PARAM_BRANCH_INSTRUCTIONS "branch_instructions" + int virDomainGetPerfEvents(virDomainPtr dom, virTypedParameterPtr *params, int *nparams, diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c index ce199f0..c12c87f 100644 --- a/src/libvirt-domain.c +++ b/src/libvirt-domain.c @@ -11195,24 +11195,26 @@ virConnectGetDomainCapabilities(virConnectPtr conn, * * VIR_DOMAIN_STATS_PERF: Return perf event statistics. * The typed parameter keys are in this format: - * "perf.cmt" - the usage of l3 cache (bytes) by applications running on the - * platform as unsigned long long. It is produced by cmt perf - * event. - * "perf.mbmt" - the total system bandwidth (bytes/s) from one level of cache - * to another as unsigned long long. It is produced by mbmt perf - * event. - * "perf.mbml" - the amount of data (bytes/s) sent through the memory controller - * on the socket as unsigned long long. It is produced by mbml - * perf event. - * "perf.cache_misses" - the count of cache misses as unsigned long long. - * It is produced by cache_misses perf event. - * "perf.cache_references" - the count of cache hits as unsigned long long. - * It is produced by cache_references perf event. - * "perf.instructions" - The count of instructions as unsigned long long. - * It is produced by instructions perf event. - * "perf.cpu_cycles" - The count of cpu cycles (total/elapsed) as an - * unsigned long long. It is produced by cpu_cycles - * perf event. + * "perf.cmt" - the usage of l3 cache (bytes) by applications running on the + * platform as unsigned long long. It is produced by cmt perf + * event. + * "perf.mbmt" - the total system bandwidth (bytes/s) from one level of cache + * to another as unsigned long long. It is produced by mbmt perf + * event. + * "perf.mbml" - the amount of data (bytes/s) sent through the memory controller + * on the socket as unsigned long long. It is produced by mbml + * perf event. + * "perf.cache_misses" - the count of cache misses as unsigned long long. + * It is produced by cache_misses perf event. + * "perf.cache_references" - the count of cache hits as unsigned long long. + * It is produced by cache_references perf event. + * "perf.instructions" - The count of instructions as unsigned long long. + * It is produced by instructions perf event. + * "perf.cpu_cycles" - The count of cpu cycles (total/elapsed) as an + * unsigned long long. It is produced by cpu_cycles + * perf event. + * "perf.branch_instructions" - The count of branch instructions as unsigned long long. + * It is produced by branch_instructions perf event. * * Note that entire stats groups or individual stat fields may be missing from * the output in case they are not supported by the given hypervisor, are not diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 3517aa2..c7fad30 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -9852,6 +9852,7 @@ qemuDomainSetPerfEvents(virDomainPtr dom, VIR_PERF_PARAM_INSTRUCTIONS, VIR_TYPED_PARAM_BOOLEAN, VIR_PERF_PARAM_CACHE_REFERENCES, VIR_TYPED_PARAM_BOOLEAN, VIR_PERF_PARAM_CACHE_MISSES, VIR_TYPED_PARAM_BOOLEAN, + VIR_PERF_PARAM_BRANCH_INSTRUCTIONS, VIR_TYPED_PARAM_BOOLEAN, NULL) < 0) return -1; diff --git a/src/util/virperf.c b/src/util/virperf.c index 5d57962..635faf1 100644 --- a/src/util/virperf.c +++ b/src/util/virperf.c @@ -40,7 +40,8 @@ VIR_LOG_INIT("util.perf"); VIR_ENUM_IMPL(virPerfEvent, VIR_PERF_EVENT_LAST, "cmt", "mbmt", "mbml", "cpu_cycles", "instructions", - "cache_references", "cache_misses"); + "cache_references", "cache_misses", + "branch_instructions"); struct virPerfEvent { int type; @@ -85,6 +86,9 @@ static struct virPerfEventAttr attrs[] = { {.type = VIR_PERF_EVENT_CACHE_MISSES, .attrType = PERF_TYPE_HARDWARE, .attrConfig = PERF_COUNT_HW_CACHE_MISSES}, + {.type = VIR_PERF_EVENT_BRANCH_INSTRUCTIONS, + .attrType = PERF_TYPE_HARDWARE, + .attrConfig = PERF_COUNT_HW_BRANCH_INSTRUCTIONS}, }; typedef struct virPerfEventAttr *virPerfEventAttrPtr; diff --git a/src/util/virperf.h b/src/util/virperf.h index 3fca2d0..e43f332 100644 --- a/src/util/virperf.h +++ b/src/util/virperf.h @@ -32,10 +32,11 @@ typedef enum { VIR_PERF_EVENT_MBMT, /* Memory Bandwidth Monitoring Total */ VIR_PERF_EVENT_MBML, /* Memory Bandwidth Monitor Limit for controller */ - VIR_PERF_EVENT_CPU_CYCLES, /* Count of CPU Cycles (total/elapsed) */ - VIR_PERF_EVENT_INSTRUCTIONS, /* Count of instructions for application */ - VIR_PERF_EVENT_CACHE_REFERENCES, /* Cache hits by applications */ - VIR_PERF_EVENT_CACHE_MISSES, /* Cache misses by applications */ + VIR_PERF_EVENT_CPU_CYCLES, /* Count of CPU Cycles (total/elapsed) */ + VIR_PERF_EVENT_INSTRUCTIONS, /* Count of instructions for application */ + VIR_PERF_EVENT_CACHE_REFERENCES, /* Cache hits by applications */ + VIR_PERF_EVENT_CACHE_MISSES, /* Cache misses by applications */ + VIR_PERF_EVENT_BRANCH_INSTRUCTIONS, /* Count of branch instructions by applications*/ VIR_PERF_EVENT_LAST } virPerfEventType; diff --git a/tests/genericxml2xmlindata/generic-perf.xml b/tests/genericxml2xmlindata/generic-perf.xml index a914133..92e5847 100644 --- a/tests/genericxml2xmlindata/generic-perf.xml +++ b/tests/genericxml2xmlindata/generic-perf.xml @@ -20,6 +20,7 @@ <event name='instructions' enabled='yes'/> <event name='cache_references' enabled='no'/> <event name='cache_misses' enabled='no'/> + <event name='branch_instructions' enabled='yes'/> </perf> <devices> </devices> diff --git a/tools/virsh.pod b/tools/virsh.pod index 247d235..aa5b756 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -945,7 +945,8 @@ I<--perf> returns the statistics of all enabled perf events: "perf.cpu_cycles" - the count of cpu cycles (total/elapsed), "perf.instructions" - the count of instructions, "perf.cache_references" - the count of cache hits, -"perf.cache_misses" - the count of caches misses +"perf.cache_misses" - the count of caches misses, +"perf.branch_instructions" - the count of branch instructions See the B<perf> command for more details about each event. @@ -2270,25 +2271,27 @@ performance event. B<eventSpec> is a string list of one or more events separated by commas. Valid event names are as follows: B<Valid perf event names> - cmt - A PQos (Platform Qos) feature to monitor the - usage of cache by applications running on the - platform. - mbmt - Provides a way to monitor the total system - memory bandwidth between one level of cache - and another. - mbml - Provides a way to limit the amount of data - (bytes/s) send through the memory controller - on the socket. - cache_misses - Provides the count of cache misses by - applications running on the platform. - cache_references - Provides the count of cache hits by - applications running on th e platform. - instructions - Provides the count of instructions executed - by applications running on the platform. - cpu_cycles - Provides the count of cpu cycles - (total/elapsed). May be used with - instructions in order to get a cycles - per instruction. + cmt - A PQos (Platform Qos) feature to monitor the + usage of cache by applications running on the + platform. + mbmt - Provides a way to monitor the total system + memory bandwidth between one level of cache + and another. + mbml - Provides a way to limit the amount of data + (bytes/s) send through the memory controller + on the socket. + cache_misses - Provides the count of cache misses by + applications running on the platform. + cache_references - Provides the count of cache hits by + applications running on th e platform. + instructions - Provides the count of instructions executed + by applications running on the platform. + cpu_cycles - Provides the count of cpu cycles + (total/elapsed). May be used with + instructions in order to get a cycles + per instruction. + branch_instructions - Provides the count of branch instructions + executed by applications running on the platform. B<Note>: The statistics can be retrieved using the B<domstats> command using the I<--perf> flag. -- 1.9.3

3 2

[libvirt] [PATCH v5 0/2] List only online cpus for vcpupin/emulatorpin when vcpu placement static
by Nitesh Konkar 06 Dec '16

06 Dec '16

Currently when the vcpu placement is static and cpuset is not specified, CPU Affinity shows 0.. CPUMAX. This patchset will result in display of only online CPU's under CPU Affinity on linux. Fixes the following Bug: virsh dumpxml Fedora <domain type='kvm' id='4'> <name>Fedora</name> <uuid>aecf3e5e-6f9a-42a3-9d6a-223a75569a66</uuid> <maxMemory slots='32' unit='KiB'>3145728</maxMemory> <memory unit='KiB'>524288</memory> <currentMemory unit='KiB'>524288</currentMemory> <vcpu placement='static' current='8'>160</vcpu> <resource> <partition>/machine</partition> </resource> ..................... ....................... ......................... <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+0</label> <imagelabel>+0:+0</imagelabel> </seclabel> </domain> lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-2,4-7 Off-line CPU(s) list: 3 Thread(s) per core: 1 Core(s) per socket: 7 Socket(s): 1 .......... .......... NUMA node0 CPU(s): 0-2,4-7 NUMA node1 CPU(s): cat /sys/devices/system/cpu/online 0-2,4-7 Before Patch virsh vcpupin Fedora VCPU: CPU Affinity ---------------------------------- 0: 0-7 1: 0-7 ... ... 158: 0-7 159: 0-7 virsh emulatorpin Fedora emulator: CPU Affinity ---------------------------------- *: 0-7 After Patch virsh vcpupin Fedora VCPU: CPU Affinity ---------------------------------- 0: 0-2,4-7 1: 0-2,4-7 ... ... 158: 0-2,4-7 159: 0-2,4-7 virsh emulatorpin Fedora emulator: CPU Affinity ---------------------------------- *: 0-2,4-7 Nitesh Konkar (2): conf: List only online cpus for virsh vcpupin conf: List only online cpus for virsh emulatorpin src/conf/domain_conf.c | 6 ++++++ src/qemu/qemu_driver.c | 5 +++++ 2 files changed, 11 insertions(+) -- 2.1.0

3 7

[libvirt] [RFC PATCH] cgroup: Use system reported "unlimited" value for comparison
by Viktor Mihajlovski 06 Dec '16

06 Dec '16

With kernel 3.18 (since commit 3e32cb2e0a12b6915056ff04601cf1bb9b44f967) the "unlimited" value for cgroup memory limits has changed once again as its byte value is now computed from a page counter. The new "unlimited" value reported by the cgroup fs is therefore 2**51-1 pages which is (VIR_DOMAIN_MEMORY_PARAM_UNLIMITED - 3072). This results e.g. in virsh memtune displaying 9007199254740988 instead of unlimited for the limits. This patch uses the value of memory.limit_in_bytes from the cgroup memory root which is the system's "real" unlimited value for comparison. See also libvirt commit 231656bbeb9e4d3bedc44362784c35eee21cf0f4 for the history for kernel 3.12 and before. I've tested this on F24 with the following configurations: - no memory cgroup controller mounted - memory cgroup controller mounted but not configured for libvirt - memory cgroup controller mounted and configured The first two fail as expected (and as before), the third case works as expected. Testing on other kernel versions highly welcome! Not perfect yet in that we still provide a fallback to the old value. We might consider failing right away if we can't get the system value. I'd be inclined to do that, since we're probably facing principal cgroup issues in this case. Further, it's not the most efficient implementation. Obviously, the unlimited value can be read once and cached. However, I'd like to see the question above resolved first. Signed-off-by: Viktor Mihajlovski <mihajlov(a)linux.vnet.ibm.com> --- src/util/vircgroup.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/src/util/vircgroup.c b/src/util/vircgroup.c index f151193..969dca5 100644 --- a/src/util/vircgroup.c +++ b/src/util/vircgroup.c @@ -2452,6 +2452,40 @@ virCgroupGetBlkioDeviceWeight(virCgroupPtr group, } +/* + * Retrieve the "memory.limit_in_bytes" value from the memory controller + * root dir. This value cannot be modified by userspace and therefore + * is the maximum limit value supported by cgroups on the local system. + */ +static int +virCgroupGetMemoryUnlimited(unsigned long long int * mem_unlimited) +{ + int ret = -1; + virCgroupPtr group; + + if (VIR_ALLOC(group)) + goto cleanup; + + if (virCgroupDetectMounts(group)) + goto cleanup; + + if (!group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].mountPoint) + goto cleanup; + + if (VIR_STRDUP(group->controllers[VIR_CGROUP_CONTROLLER_MEMORY].placement, + "/.") < 0) + goto cleanup; + + ret = virCgroupGetValueU64(group, + VIR_CGROUP_CONTROLLER_MEMORY, + "memory.limit_in_bytes", + mem_unlimited); + cleanup: + virCgroupFree(&group); + return ret; +} + + /** * virCgroupSetMemory: * @@ -2534,6 +2568,7 @@ int virCgroupGetMemoryHardLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; + long long unsigned int unlimited_in_bytes; int ret = -1; if (virCgroupGetValueU64(group, @@ -2541,9 +2576,13 @@ virCgroupGetMemoryHardLimit(virCgroupPtr group, unsigned long long *kb) "memory.limit_in_bytes", &limit_in_bytes) < 0) goto cleanup; - *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (virCgroupGetMemoryUnlimited(&unlimited_in_bytes) < 0) + unlimited_in_bytes = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED << 10; + + if (limit_in_bytes == unlimited_in_bytes) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; + else + *kb = limit_in_bytes >> 10; ret = 0; cleanup: @@ -2596,6 +2635,7 @@ int virCgroupGetMemorySoftLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; + long long unsigned int unlimited_in_bytes; int ret = -1; if (virCgroupGetValueU64(group, @@ -2603,9 +2643,13 @@ virCgroupGetMemorySoftLimit(virCgroupPtr group, unsigned long long *kb) "memory.soft_limit_in_bytes", &limit_in_bytes) < 0) goto cleanup; - *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (virCgroupGetMemoryUnlimited(&unlimited_in_bytes) < 0) + unlimited_in_bytes = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED << 10; + + if (limit_in_bytes == unlimited_in_bytes) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; + else + *kb = limit_in_bytes >> 10; ret = 0; cleanup: @@ -2658,6 +2702,7 @@ int virCgroupGetMemSwapHardLimit(virCgroupPtr group, unsigned long long *kb) { long long unsigned int limit_in_bytes; + long long unsigned int unlimited_in_bytes; int ret = -1; if (virCgroupGetValueU64(group, @@ -2665,9 +2710,13 @@ virCgroupGetMemSwapHardLimit(virCgroupPtr group, unsigned long long *kb) "memory.memsw.limit_in_bytes", &limit_in_bytes) < 0) goto cleanup; - *kb = limit_in_bytes >> 10; - if (*kb > VIR_DOMAIN_MEMORY_PARAM_UNLIMITED) + if (virCgroupGetMemoryUnlimited(&unlimited_in_bytes) < 0) + unlimited_in_bytes = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED << 10; + + if (limit_in_bytes == unlimited_in_bytes) *kb = VIR_DOMAIN_MEMORY_PARAM_UNLIMITED; + else + *kb = limit_in_bytes >> 10; ret = 0; cleanup: -- 1.9.1

2 3

[libvirt] [PATCH python] libvirt-override: fix setBlockIoTune failure passed
by xieyingtai＠huawei.com 06 Dec '16

06 Dec '16

From: Yingtai Xie <xieyingtai(a)huawei.com> virDomainGetBlockIoTune() will be called before setBlockIoTune, as a result, it leads to a failure if set flags with VIR_DOMAIN_AFFECT_LIVE | VIR_DOMAIN_AFFECT_CONFIG which is an reasonable option. Signed-off-by: Yingtai Xie <xieyingtai(a)huawei.com> --- libvirt-override.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/libvirt-override.c b/libvirt-override.c index 2de95ce..caa52fb 100644 --- a/libvirt-override.c +++ b/libvirt-override.c @@ -4805,6 +4805,7 @@ libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, Py_ssize_t size = 0; const char *disk; unsigned int flags; + unsigned int flags1 = 0; virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OzOI:virDomainSetBlockIoTune", @@ -4822,7 +4823,7 @@ libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, } LIBVIRT_BEGIN_ALLOW_THREADS; - i_retval = virDomainGetBlockIoTune(domain, disk, NULL, &nparams, flags); + i_retval = virDomainGetBlockIoTune(domain, disk, NULL, &nparams, flags1); LIBVIRT_END_ALLOW_THREADS; if (i_retval < 0) @@ -4838,7 +4839,7 @@ libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, return PyErr_NoMemory(); LIBVIRT_BEGIN_ALLOW_THREADS; - i_retval = virDomainGetBlockIoTune(domain, disk, params, &nparams, flags); + i_retval = virDomainGetBlockIoTune(domain, disk, params, &nparams, flags1); LIBVIRT_END_ALLOW_THREADS; if (i_retval < 0) { -- 1.8.3.1

3 2