- Devel - libvirt List Archives

[libvirt] Automatically affinitize hostdev interrupts to vm vCpus (qemu/kvm)
by Mooney, Sean K 28 Aug '14

28 Aug '14

Hi I would like to ask for comments and propose a possible new libvirt feature. Problem statement: At present when you boot a vm via libvirt it is possible to pin vm vCPUs to host CPUs to improve performance in the guest under certain conditions. If hostdev interrupts are not pined to a specific core/cpuset suboptimal processing of irqs may occur reducing the performance of both guest and host. I would like to propose extending libvirt to automatically pin interrupts for hostdev devices if they are present in the guest. By affinitizing interrupts, cache line sharing between the specified interrupt and guest can be achieved. If CPU affinity for the guest, as set by the cpuset parameter, is intelligently chosen to place the guest on the same numa node as the hostdev, cross socket traffic can be mitigated. As a result, latency which would be introduce if the interrupt processing was scheduled to a non-local numa node CPU can be reduced via interrupt pinning. Proposed change: * util/virpci and util/virhostdev will be extended to retrieve IRQ and msi_interupt information from sysfs. * util/virinterupt will be created * util/virinterupt will implement managing interrupt affinity via /proc/irq/x/smp_affinity * qemuProcessInitCpuAffinity will be extended to conditionally affinitize hostdev interrupts to vm vCpus when hostdevs are present in the vm definition. Alternative implementation: In addition to the above changes the hostdev element could be extended to include a cpuset attribute: * if the cpuset is auto: the interrupts would be pinned to the same cpuset as the vms vCPU * if a cpuset is specified: the interrupts would be affinitized as per the set cpuset * if the cpuset is native: the interrupts will not be pinned and the current behaviour will not be changed. * If the cpuset attribute is not present either the behaviour of auto or native could be used as a default. o Using auto as the default would allow transparent use of the feature. o Using native as the default would allow no changes to any existing deployments unless the feature is requested. Any feedback is welcomed. Regards Sean. -------------------------------------------------------------- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

2 2

[libvirt] Proposition for the implementation of new features for Hyper-V driver
by Adrien Kantcheff 28 Aug '14

28 Aug '14

Dear libvirt developers, During my final student training in the French company Bull, with a previous student (Simon RASTELLO), we developed new features for Hyper-V driver. For a project called OpenCloudware, our work was to bring new functionalities using libvirt API in order to do basic actions on virtual machines hosted on Hyper-V. You may be interested in pushing our developments in the official release. The libvirt driver already provides functions to enumerate and get WMI classes by sending WQL requests to WMI provider and also to call WMI class methods. For that last kind of communication, a method requires input parameters. There are two kinds of argument: basic (integer, string...) and complex (objects, end point references, embedded instances). Actually, just the first argument passing mode is available with libvirt driver. But the second one is very useful because there is many WMI methods with complex types parameters, and for the moment it constraints developers to call WMI methods only with basic types parameters. So in order to expand WMI methods calls, we have implemented the second argument passing mode. Thanks to this new argument passing mode, we are available to set new functionalities. Our contributions include: * hyperv_driver.c o hypervDomainDefineXML o hypervDomainCreateXML o hypervDomainUndefine o hypervDomainUndefineFlags o hypervDomainShutdown o hypervDomainShutdownFlags o hypervDomainGetVcpus o hypervDomainGetVcpusFlags o hypervConnectGetMaxVcpus o hypervDomainGetMaxVcpus o hypervDomainSetVcpus o hypervDomainSetVcpusFlags o hypervDomainSetMemory o hypervDomainSetMemoryFlags o hypervDomainSetMaxMemory o hypervNodeGetFreeMemory o hypervDomainAttachDevice o hypervDomainAttachDeviceFlags o hypervDomainGetSchedulerParameters o hypervDomainGetSchedulerParametersFlags o hypervDomainGetSchedulerType o hypervConnectGetCapabilities o hypervConnectGetVersion o hypervDomainSetAutostart o hypervDomainGetAutostart * hyperv_network.c o hypervConnectNumOfNetworks o hypervConnectListNetworks o hypervConnectNumOfDefinedNetworks o hypervConnectListDefinedNetworks o hypervNetworkLookupByName o hypervNetworkGetXMLDesc * hyperv_storage_driver.c o hypervConnectNumOfStoragePools (stub) o hypervConnectListStoragePools (stub) o hypervConnectNumOfDefinedStoragePools (stub) o hypervStoragePoolLookupByName (stub) o hypervStorageVolLookupByPath (stub) * hyperv_private.h o Add mutex to protect against concurrent calls o Add virDomainXMLOptionPtr to parse Domain XML * hyperv_wmi.h o Structures for complex arguments: objects, EPR (end point references) and embedded instances * hyperv_wmi.c o Methods to invoke WMI methods with complex arguments * hyperv_wmi_generator.input o CIM_DataFile o Win32_ComputerSystemProduct o Msvm_VirtualSystemManagementService o Msvm_VirtualSystemGlobalSettingData o Msvm_ResourceAllocationSettingData o Msvm_AllocationCapabilities o Msvm_VirtualSwitch o Msvm_SwitchPort o Msvm_SyntheticEthernetPortSettingData o Msvm_VirtualSwitchManagementService o Win32_OperatingSystem o Win32_PerfFormattedData_HvStats_HyperVHypervisorVirtualProcessor o Win32_PerfRawData_HvStats_HyperVHypervisorVirtualProcessor * hyperv_wmi_generator.py o Add CIM_DataFile class header to be generated o Add tab classes and types to be generated o Add a function to print header types * openwsman.h o Add ws_xml_create_doc signature o Add xml_parser_get_root signature Attach files contain sources and an pdf of our contributions. I'm ending my training this week but I'm available to answer any questions you may have. You can use my personal email: adrien.kantcheff(a)gmail.com<mailto:adrien.kantcheff@gmail.com> I also put in copy of this email my tutors Yves VINTER and Christian BOURGEOIS. Feel free to contact them as well. Best regards, Adrien KANTCHEFF Bull

2 1

[libvirt] run qemu-agent-command via binding (ruby/python/php)
by Vasiliy Tolstov 28 Aug '14

28 Aug '14

Hi! Does it possible(featured, planned) to run some qemu-agent-command via libvirt binding (i'm interesting on ruby and php).? I'm understand that i can connect via socket and run it, but it very usable to get this ability inside binding. -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru jabber: vase(a)selfip.ru

3 4

[libvirt] [PATCHv4] qemu: Implement bulk stats API and one of the stats groups to return
by Peter Krempa 28 Aug '14

28 Aug '14

Implement the API function for virDomainListGetStats and virConnectGetAllDomainStats in a modular way and implement the VIR_DOMAIN_STATS_STATE group of statistics. Although it may look like the function looks universal I'd rather not expose it to other drivers as the coming stats groups are likely to do qemu specific stuff to obtain the stats. --- Notes: Version 4: - fixed handling and error checking of @stats - domain filtering flags are now rejected when passing in a domain list src/qemu/qemu_driver.c | 198 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 198 insertions(+) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 73959da..45a080b 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -17190,6 +17190,203 @@ qemuConnectGetDomainCapabilities(virConnectPtr conn, } +static int +qemuDomainGetStatsState(virDomainObjPtr dom, + virDomainStatsRecordPtr record, + int *maxparams, + unsigned int privflags ATTRIBUTE_UNUSED) +{ + if (virTypedParamsAddInt(&record->params, + &record->nparams, + maxparams, + "state.state", + dom->state.state) < 0) + return -1; + + if (virTypedParamsAddInt(&record->params, + &record->nparams, + maxparams, + "state.reason", + dom->state.reason) < 0) + return -1; + + return 0; +} + + +typedef int +(*qemuDomainGetStatsFunc)(virDomainObjPtr dom, + virDomainStatsRecordPtr record, + int *maxparams, + unsigned int flags); + +struct qemuDomainGetStatsWorker { + qemuDomainGetStatsFunc func; + unsigned int stats; +}; + +static struct qemuDomainGetStatsWorker qemuDomainGetStatsWorkers[] = { + { qemuDomainGetStatsState, VIR_DOMAIN_STATS_STATE}, + { NULL, 0 } +}; + + +static int +qemuDomainGetStatsCheckSupport(unsigned int *stats, + bool enforce) +{ + unsigned int supportedstats = 0; + size_t i; + + for (i = 0; qemuDomainGetStatsWorkers[i].func; i++) + supportedstats |= qemuDomainGetStatsWorkers[i].stats; + + if (*stats == 0) { + *stats = supportedstats; + return 0; + } + + if (enforce && + *stats & ~supportedstats) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, + _("Stats types bits 0x%x are not supported by this daemon"), + *stats & ~supportedstats); + return -1; + } + + *stats &= supportedstats; + return 0; +} + + +static int +qemuDomainGetStats(virConnectPtr conn, + virDomainObjPtr dom, + unsigned int stats, + virDomainStatsRecordPtr *record, + unsigned int flags) +{ + int maxparams = 0; + virDomainStatsRecordPtr tmp; + size_t i; + int ret = -1; + + if (VIR_ALLOC(tmp) < 0) + goto cleanup; + + for (i = 0; qemuDomainGetStatsWorkers[i].func; i++) { + if (stats & qemuDomainGetStatsWorkers[i].stats) { + if (qemuDomainGetStatsWorkers[i].func(dom, tmp, &maxparams, + flags) < 0) + goto cleanup; + } + } + + if (!(tmp->dom = virGetDomain(conn, dom->def->name, dom->def->uuid))) + goto cleanup; + + *record = tmp; + tmp = NULL; + ret = 0; + + cleanup: + if (tmp) { + virTypedParamsFree(tmp->params, tmp->nparams); + VIR_FREE(tmp); + } + + return ret; +} + + + +static int +qemuConnectGetAllDomainStats(virConnectPtr conn, + virDomainPtr *doms, + unsigned int ndoms, + unsigned int stats, + virDomainStatsRecordPtr **retStats, + unsigned int flags) +{ + virQEMUDriverPtr driver = conn->privateData; + virDomainPtr *domlist = NULL; + virDomainObjPtr dom = NULL; + virDomainStatsRecordPtr *tmpstats = NULL; + bool enforce = !!(flags & VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS); + int ntempdoms; + int nstats = 0; + size_t i; + int ret = -1; + + if (ndoms) + virCheckFlags(VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS, -1); + else + virCheckFlags(VIR_CONNECT_LIST_DOMAINS_FILTERS_ACTIVE | + VIR_CONNECT_LIST_DOMAINS_FILTERS_PERSISTENT | + VIR_CONNECT_LIST_DOMAINS_FILTERS_STATE | + VIR_CONNECT_GET_ALL_DOMAINS_STATS_ENFORCE_STATS, -1); + + if (virConnectGetAllDomainStatsEnsureACL(conn) < 0) + return -1; + + if (qemuDomainGetStatsCheckSupport(&stats, enforce) < 0) + return -1; + + if (!ndoms) { + unsigned int lflags = flags & (VIR_CONNECT_LIST_DOMAINS_FILTERS_ACTIVE | + VIR_CONNECT_LIST_DOMAINS_FILTERS_PERSISTENT | + VIR_CONNECT_LIST_DOMAINS_FILTERS_STATE); + + if ((ntempdoms = virDomainObjListExport(driver->domains, + conn, + &domlist, + virConnectGetAllDomainStatsCheckACL, + lflags)) < 0) + goto cleanup; + + ndoms = ntempdoms; + doms = domlist; + } + + if (VIR_ALLOC_N(tmpstats, ndoms + 1) < 0) + goto cleanup; + + for (i = 0; i < ndoms; i++) { + virDomainStatsRecordPtr tmp = NULL; + + if (!(dom = qemuDomObjFromDomain(doms[i]))) + continue; + + if (!domlist && + !virConnectGetAllDomainStatsCheckACL(conn, dom->def)) + continue; + + if (qemuDomainGetStats(conn, dom, stats, &tmp, flags) < 0) + goto cleanup; + + if (tmp) + tmpstats[nstats++] = tmp; + + virObjectUnlock(dom); + dom = NULL; + } + + *retStats = tmpstats; + tmpstats = NULL; + + ret = nstats; + + cleanup: + if (dom) + virObjectUnlock(dom); + + virDomainStatsRecordListFree(tmpstats); + virDomainListFree(domlist); + + return ret; +} + + static virDriver qemuDriver = { .no = VIR_DRV_QEMU, .name = QEMU_DRIVER_NAME, @@ -17387,6 +17584,7 @@ static virDriver qemuDriver = { .domainSetTime = qemuDomainSetTime, /* 1.2.5 */ .nodeGetFreePages = qemuNodeGetFreePages, /* 1.2.6 */ .connectGetDomainCapabilities = qemuConnectGetDomainCapabilities, /* 1.2.7 */ + .connectGetAllDomainStats = qemuConnectGetAllDomainStats, /* 1.2.8 */ }; -- 2.0.2

2 2

[libvirt] [PATCHv3 0/5] Implement bulk stats API
by Peter Krempa 28 Aug '14

28 Aug '14

New iteration of the series with a few improvements Peter Krempa (5): conf: Add helper to free domain list lib: Add few flags for the bulk stats APIs remote: Implement bulk domain stats APIs in the remote driver qemu: Implement bulk stats API and one of the stats groups to return virsh: Implement command to excercise the bulk stats APIs daemon/remote.c | 86 +++++++++++++++++++ include/libvirt/libvirt.h.in | 15 ++++ src/conf/domain_conf.c | 31 +++++-- src/conf/domain_conf.h | 2 + src/libvirt.c | 29 ++++++- src/libvirt_private.syms | 1 + src/qemu/qemu_driver.c | 175 +++++++++++++++++++++++++++++++++++++++ src/remote/remote_driver.c | 84 +++++++++++++++++++ src/remote/remote_protocol.x | 25 +++++- src/remote_protocol-structs | 22 +++++ tools/virsh-domain-monitor.c | 191 +++++++++++++++++++++++++++++++++++++++++++ tools/virsh.pod | 34 ++++++++ 12 files changed, 685 insertions(+), 10 deletions(-) -- 2.0.2

2 13

[libvirt] [PATCH 0/3] Coverity patches to resolve RESOURCE_LEAK
by Wang Rui 28 Aug '14

28 Aug '14

I did coverity scan for libvirt-1.2.8 as John Ferlan did. He has sent many patches about RESOURCE_LEAK. I picked the other errors left to fix. There are also many errors to analyze and fix in the future. Wang Rui (3): util: Resolve Coverity RESOURCE_LEAK tests: Resolve Coverity RESOURCE_LEAK qemu_capabilities: Resolve Coverity RESOURCE_LEAK src/qemu/qemu_capabilities.c | 4 +++- src/util/virpci.c | 1 + tests/shunloadtest.c | 1 + 3 files changed, 5 insertions(+), 1 deletion(-) -- 1.7.12.4

2 4

[libvirt] [PATCH 0/4] Introduce support for virtio-blk-pci iothreads
by John Ferlan 28 Aug '14

28 Aug '14

Introduce iothreads support to libvirt. These will be used to facilitate adding an iothread attribute to a support disk which will enable having a dedicated event loop thread for the disk. IOThreads are a QEMU feature recently added (2.1) as a replacement for the virtio-blk data plane functionality that's been in tech preview since 1.4. Followup patches will add API's in order to list the IOThreads and eventually be able to assign IOThreads to specific CPU's if so desired. This set of patches should cover at least the bare minimum in order to allow modifying domain XML in order to use the feature. John Ferlan (4): domain_conf: Introduce iothreads XML qemu: Add support for iothreads domain_conf: Add support for iothreads in disk definition qemu: Allow use of iothreads for disk definitions docs/formatdomain.html.in | 34 ++++++++++++++ docs/schemas/domaincommon.rng | 25 +++++++++++ src/conf/domain_conf.c | 52 +++++++++++++++++++++- src/conf/domain_conf.h | 4 ++ src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 48 ++++++++++++++++++++ src/qemu/qemu_command.h | 5 +++ src/qemu/qemu_hotplug.c | 11 +++++ .../qemuxml2argv-iothreads-disk.args | 17 +++++++ .../qemuxml2argv-iothreads-disk.xml | 40 +++++++++++++++++ tests/qemuxml2argvdata/qemuxml2argv-iothreads.args | 8 ++++ tests/qemuxml2argvdata/qemuxml2argv-iothreads.xml | 29 ++++++++++++ tests/qemuxml2argvtest.c | 4 ++ tests/qemuxml2xmltest.c | 2 + 15 files changed, 281 insertions(+), 1 deletion(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-iothreads-disk.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-iothreads-disk.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-iothreads.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-iothreads.xml -- 1.9.3

5 22

[libvirt] [PATCHv2] virsh: fix keepalive error msg, man page update
by Erik Skultety 28 Aug '14

28 Aug '14

resolves https://bugzilla.redhat.com/show_bug.cgi?id=1132305 --- tools/virsh.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/tools/virsh.c b/tools/virsh.c index 30a84c1..713c9a5 100644 --- a/tools/virsh.c +++ b/tools/virsh.c @@ -3470,18 +3470,32 @@ vshParseArgv(vshControl *ctl, int argc, char **argv) exit(EXIT_SUCCESS); break; case 'k': - if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0 || - keepalive < 0) { - vshError(ctl, _("option %s requires a positive numeric argument"), + if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0) { + vshError(ctl, + _("Invalid value for option %s"), + longindex == -1 ? "-k" : "--keepalive-interval"); + exit(EXIT_FAILURE); + } + + if (keepalive < 0) { + vshError(ctl, + _("option %s requires a positive integer argument"), longindex == -1 ? "-k" : "--keepalive-interval"); exit(EXIT_FAILURE); } ctl->keepalive_interval = keepalive; break; case 'K': - if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0 || - keepalive < 0) { - vshError(ctl, _("option %s requires a positive numeric argument"), + if (virStrToLong_i(optarg, NULL, 0, &keepalive) < 0) { + vshError(ctl, + _("Invalid value for option %s"), + longindex == -1 ? "-K" : "--keepalive-count"); + exit(EXIT_FAILURE); + } + + if (keepalive < 0) { + vshError(ctl, + _("option %s requires a positive integer argument"), longindex == -1 ? "-K" : "--keepalive-count"); exit(EXIT_FAILURE); } -- 1.9.3

2 2

[libvirt] [PATCH] qemu: call endjob in RevertToSnapshot
by Jincheng Miao 28 Aug '14

28 Aug '14

In qemuDomainRevertToSnapshot(), it will check snap->def->state. But when the state is PMSUSPENDED/NOSTATE/BLOCKED, it forgets to call qemuDomainObjEndJob. Signed-off-by: Jincheng Miao <jmiao(a)redhat.com> --- src/qemu/qemu_driver.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index f5b17ba..2db507f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -14294,7 +14294,7 @@ static int qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s", _("qemu doesn't support reversion of snapshot taken in " "PMSUSPENDED state")); - goto cleanup; + goto endjob; case VIR_DOMAIN_NOSTATE: case VIR_DOMAIN_BLOCKED: @@ -14303,7 +14303,7 @@ static int qemuDomainRevertToSnapshot(virDomainSnapshotPtr snapshot, _("Invalid target domain state '%s'. Refusing " "snapshot reversion"), virDomainStateTypeToString(snap->def->state)); - goto cleanup; + goto endjob; } ret = 0; -- 1.7.1

2 5

[libvirt] [PATCH 00/10] Resolve some Coverity found issues
by John Ferlan 27 Aug '14

27 Aug '14

After installing a new version of Coverity - my daily checker has found many more (>140) "new" issues. Figured I'd start tackling the list bit by bit. I didn't dig into when any of these were introduced - with so many on the list it'd be overwhelming :-) John Ferlan (10): parallels: Resolve Coverity USE_AFTER_FREE xen_common: Resolve Coverity USE_AFTER_FREE xen_xm: Resolve Coverity USE_AFTER_FREE storage_driver: Resolve Coverity REVERSE_INULL domain_conf: Resolve Coverity REVERSE_INULL qemu_command: Resolve Coverity REVERSE_INULL domain_conf: Resolve Coverity REVERSE_INULL qemu_driver: Resolve Coverity CONSTANT_EXPRESSION_RESULT libxl_migration: Resolve Coverity NULL_RETURNS qemu_capabilities: Resolve Coverity NULL_RETURNS src/conf/domain_conf.c | 18 +++++++++--------- src/libxl/libxl_migration.c | 1 - src/parallels/parallels_network.c | 4 +--- src/qemu/qemu_capabilities.c | 5 ++++- src/qemu/qemu_command.c | 2 +- src/qemu/qemu_driver.c | 4 ++-- src/storage/storage_driver.c | 6 ++++-- src/xenconfig/xen_common.c | 5 ++--- src/xenconfig/xen_xm.c | 1 + 9 files changed, 24 insertions(+), 22 deletions(-) -- 1.9.3

5 19