April 2017 - Devel - libvirt List Archives

[libvirt] [PATCH v3 00/16] qemu: migration: show disks stats for nbd migration
by Nikolay Shirokovskiy 20 Jul '17

20 Jul '17

diff from v2: ============ 1. Fix style issues. 2. Rework patch for fetching job info (save logic to use temporary variable when drop vm lock) 3. Update disk stats when block jobs are canceled. 4. Adress a few more corner cases. This patch series add disks stats to domain job info(stats) as well as to migration completed event in case nbd scheme is used. There is little nuisance with qcow2 disks (which is the main scenario I guess) tied to the way qemu reports stats for this type of disks. For example if we have 64G disk filled only to 1G then stats start from 63G and will grow up to 64G on completion. The same way disk stats will be reported by this patch. I guess the better way to express the situation is to say we have 64G 'total', and have 'processed' field grow from 0G to 1G, like in case of memory stats. [1] is the example of completed memory stats of empty guest domain, which show difference between processed and total. There can be a workaround like getting initial blockjob offset position as a zero but is is rather ugly and racy and like uses undocumented behaviour. [1] memory migration stats example Memory processed: 3.307 MiB Memory remaining: 0.000 B Memory total: 1.032 GiB The above is applied to qemu 2.6 at least. Patches that were explicitly ACKed in previous review (up to style issues) marked with A. Nikolay Shirokovskiy (16): qemu: drop code for VIR_DOMAIN_JOB_BOUNDED and timeRemaining A qemu: introduce qemu domain job status A qemu: introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY A qemu: drop QEMU_MIGRATION_COMPLETED_UPDATE_STATS A qemu: drop excessive zero-out in qemuMigrationFetchJobStatus qemu: refactor fetching migration stats A qemu: simplify getting completed job stats qemu: fail querying destination migration statistics always qemu: start all async job with job status active qemu: introduce migrating job status qemu: always get job condition on getting job stats qemu: migrate: show disks stats on job info requests qemu: support getting disks stats during stopping block jobs qemu: migation: resolve race on getting job info and stopping block jobs qemu: migrate: copy disks stats to completed job qemu: migration: don't expose incomplete job as complete src/qemu/qemu_blockjob.c | 1 + src/qemu/qemu_domain.c | 38 +++++-- src/qemu/qemu_domain.h | 16 ++- src/qemu/qemu_driver.c | 95 ++++++++-------- src/qemu/qemu_migration.c | 236 ++++++++++++++++++++++++++------------- src/qemu/qemu_migration.h | 15 ++- src/qemu/qemu_migration_cookie.c | 7 +- src/qemu/qemu_monitor.c | 5 +- src/qemu/qemu_monitor.h | 4 +- src/qemu/qemu_monitor_json.c | 4 +- src/qemu/qemu_process.c | 10 +- tests/qemumonitorjsontest.c | 1 + 12 files changed, 273 insertions(+), 159 deletions(-) -- 1.8.3.1

2 19

[libvirt] [PATCH] qemu: Use iohelper during restore
by Shivaprasad G Bhat 07 Jul '17

07 Jul '17

Commit afe6e58 & c4caab53 made necessary changes to use io-helpers during save and restore. The commit c4caab53 missed to remove the redundant check in qemuDomainSaveImageOpen() because of which virFileWrapperFdNew() is not called if bypass_cache is false. Signed-off-by: Shivaprasad G Bhat <sbhat(a)linux.vnet.ibm.com> --- src/qemu/qemu_driver.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 516a851..ac89372 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -6150,9 +6150,11 @@ qemuDomainSaveImageOpen(virQEMUDriverPtr driver, virDomainDefPtr def = NULL; int oflags = open_write ? O_RDWR : O_RDONLY; virCapsPtr caps = NULL; + unsigned int wrapperFlags = VIR_FILE_WRAPPER_NON_BLOCKING; if (bypass_cache) { int directFlag = virFileDirectFdFlag(); + wrapperFlags |= VIR_FILE_WRAPPER_BYPASS_CACHE; if (directFlag < 0) { virReportError(VIR_ERR_OPERATION_FAILED, "%s", _("bypass cache unsupported by this system")); @@ -6166,9 +6168,8 @@ qemuDomainSaveImageOpen(virQEMUDriverPtr driver, if ((fd = qemuOpenFile(driver, NULL, path, oflags, NULL, NULL)) < 0) goto error; - if (bypass_cache && - !(*wrapperFd = virFileWrapperFdNew(&fd, path, - VIR_FILE_WRAPPER_BYPASS_CACHE))) + if (wrapperFd && + !(*wrapperFd = virFileWrapperFdNew(&fd, path, wrapperFlags))) goto error; if (saferead(fd, &header, sizeof(header)) != sizeof(header)) {

3 4

[libvirt] [PATCH 0/8] Multiple cleanups within interfaceobj and interface driver
by John Ferlan 26 May '17

26 May '17

Sensing a theme lately yet? More adjustments in preparation for having virobject code handle the bulk of the object management code. Far less in this series than the other two... The Clone code was a bit unique and required more of a rewrite than a refigure since now we have two allocated lists rather than two static lists. John Ferlan (8): interface: Consistently use 'obj' for a virInterfaceObjPtr interface: Remove some unnecessary goto's for Interface tests interface: Use virInterfaceDefPtr rather than deref from virInterfaceObjPtr interface: Make _virInterfaceObj struct private interface: Make _virInterfaceObjList struct private interface: Rename some virInterfaceObj* API's interface: Clean up virInterfaceObjListFindByMACString interface: Introduce virInterfaceObjNew src/conf/virinterfaceobj.c | 232 +++++++++++++++++++++++++++++---------------- src/conf/virinterfaceobj.h | 65 ++++++------- src/libvirt_private.syms | 15 +-- src/test/test_driver.c | 143 ++++++++++++++-------------- 4 files changed, 262 insertions(+), 193 deletions(-) -- 2.9.3

2 22

[libvirt] [PATCH] conf: Resolve corner case on fc_host deletion
by John Ferlan 24 May '17

24 May '17

https://bugzilla.redhat.com/show_bug.cgi?id=1420740 Testing found an inventive way to cause an error at shutdown by providing the parent name for the fc host creation using the "same name" as the HBA. Since the code thus assumed the parent host name provided was the parent HBA and just extracted out the host number and sent that along to the vport_destroy this avoided checks made for equality. So just add the equality check to that path to resolve. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- src/conf/node_device_conf.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/src/conf/node_device_conf.c b/src/conf/node_device_conf.c index 85cfd83..3f995da 100644 --- a/src/conf/node_device_conf.c +++ b/src/conf/node_device_conf.c @@ -2127,17 +2127,25 @@ virNodeDeviceDeleteVport(virConnectPtr conn, goto cleanup; } + if (virAsprintf(&scsi_host_name, "scsi_%s", name) < 0) + goto cleanup; + /* If at startup time we provided a parent, then use that to * get the parent_host value; otherwise, we have to determine * the parent scsi_host which we did not save at startup time */ if (fchost->parent) { + /* Someone provided a parent string at startup time that + * was the same as the scsi_host - meaning we have a pool + * backed to an HBA, so there won't be a vHBA to delete */ + if (STREQ(scsi_host_name, fchost->parent)) { + ret = 0; + goto cleanup; + } + if (virSCSIHostGetNumber(fchost->parent, &parent_host) < 0) goto cleanup; } else { - if (virAsprintf(&scsi_host_name, "scsi_%s", name) < 0) - goto cleanup; - if (!(vhba_parent = virNodeDeviceGetParentName(conn, scsi_host_name))) goto cleanup; -- 2.9.3

2 3

[libvirt] [PATCH 0/1] Enable Travis build status icon
by Claudio André 17 May '17

17 May '17

Follow up of the recent 'work in progress' announced in "Enable CI build testing with Travis" mailing message. Claudio André (1): Enable Travis CI build status icon README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 README.md -- 2.11.0

3 6

[libvirt] [PATCH v3 0/5] Hyper-V method invocation
by Sri Ramanujam 17 May '17

17 May '17

Changes from v2: * Correctly manage and free invocation parameters * Fixed a couple of other memory leaks found while fixing the above issue * Minor code changes from review I also forgot to mention previously that we have test servers available for reviewers as we introduce new functionality to the driver. Let me know if you want access to them :) Sri Ramanujam (5): hyperv: Functions to work with invocation parameters. hyperv: Generate object property type information. hyperv: add hypervInvokeMethod hyperv: support virDomainSendKey hyperv: Add support for virDomainSetMemory src/hyperv/hyperv_driver.c | 181 +++++++ src/hyperv/hyperv_wmi.c | 891 ++++++++++++++++++++++++++++++++++ src/hyperv/hyperv_wmi.h | 93 +++- src/hyperv/hyperv_wmi_classes.h | 19 + src/hyperv/hyperv_wmi_generator.input | 116 +++++ src/hyperv/hyperv_wmi_generator.py | 15 +- src/hyperv/openwsman.h | 4 + 7 files changed, 1317 insertions(+), 2 deletions(-) -- 2.9.3

3 12

[libvirt] [RFC] qemu: monitor: do not report error on shutdown
by Christian Ehrhardt 16 May '17

16 May '17

If a shutdown is expected because it was triggered via libvirt we can also expect the monitor to close. In those cases do not report an internal error like: "internal error: End of file from qemu monitor" Signed-off-by: Christian Ehrhardt <christian.ehrhardt(a)canonical.com> --- src/qemu/qemu_monitor.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c index d71f84c..352cc8a 100644 --- a/src/qemu/qemu_monitor.c +++ b/src/qemu/qemu_monitor.c @@ -63,6 +63,7 @@ struct _qemuMonitor { int fd; int watch; int hasSendFD; + int willhangup; virDomainObjPtr vm; @@ -691,8 +692,10 @@ qemuMonitorIO(int watch, int fd, int events, void *opaque) if (events & VIR_EVENT_HANDLE_HANGUP) { hangup = true; if (!error) { - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("End of file from qemu monitor")); + if (!mon->willhangup) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("End of file from qemu monitor")); + } eof = true; events &= ~VIR_EVENT_HANDLE_HANGUP; } @@ -731,7 +734,7 @@ qemuMonitorIO(int watch, int fd, int events, void *opaque) if (mon->lastError.code != VIR_ERR_OK) { /* Already have an error, so clear any new error */ virResetLastError(); - } else { + } else if (!mon->willhangup) { virErrorPtr err = virGetLastError(); if (!err) virReportError(VIR_ERR_INTERNAL_ERROR, "%s", @@ -1282,6 +1285,7 @@ qemuMonitorEmitShutdown(qemuMonitorPtr mon) { int ret = -1; VIR_DEBUG("mon=%p", mon); + mon->willhangup = 1; QEMU_MONITOR_CALLBACK(mon, ret, domainShutdown, mon->vm); return ret; -- 2.7.4

2 2

[libvirt] [PATCH v2 00/38] Implement sparse streams for libvirt
by Michal Privoznik 15 May '17

15 May '17

v2 of: https://www.redhat.com/archives/libvir-list/2017-April/msg00671.html All the patches can be found on my github: https://github.com/zippy2/libvirt/tree/sparse_iohelper5 diff to v1: - Do not drop the pipe in 06/38. Use it for signaling to the event loop that there's a message in the queue waiting to be processed. - Dropped the error report in virNetClientStreamRecvPacket in 31/38. It was useless. Michal Privoznik (38): fdstreamtest: Rename tempdir fdstreamtest: Print more info on read failure fdstream: s/struct virFDStreamData */virFDStreamDataPtr/ virFDStreamData: Turn into virObjectLockable virfdstream: Drop iohelper in favour of a thread virfdstream: Use messages instead of pipe iohelper: Remove unused mode util: Introduce virFileInData Introduce virStreamRecvFlags Implement virStreamRecvFlags to some drivers Introduce virStreamSkip Introduce virStreamHoleSize Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag Introduce virStreamSparseRecvAll Introduce virStreamSparseSendAll Introduce virStreamInData virNetClientStreamNew: Track origin stream Track if stream is skippable RPC: Introduce virNetStreamSkip Introduce VIR_NET_STREAM_SKIP message type Teach wireshark plugin about VIR_NET_STREAM_SKIP daemon: Introduce virNetServerProgramSendStreamSkip virnetclientstream: Introduce virNetClientStreamSendSkip daemon: Implement VIR_NET_STREAM_SKIP handling virnetclientstream: Introduce virNetClientStreamHandleSkip remote_driver: Implement virStreamSkip virNetClientStreamRecvPacket: Introduce @flags argument Introduce virNetClientStreamHoleSize remote: Implement virStreamHoleSize virNetClientStream: Wire up VIR_NET_STREAM_SKIP remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE daemonStreamHandleRead: Wire up seekable stream daemon: Don't call virStreamInData so often daemon: Don't call virStreamInData so often gendispatch: Introduce @sparseflag for our calls Introduce virStorageVol{Download,Upload}Flags virsh: Implement sparse stream to vol-download virsh: Implement sparse stream to vol-upload daemon/remote.c | 2 +- daemon/stream.c | 147 +++++- daemon/stream.h | 3 +- include/libvirt/libvirt-storage.h | 9 + include/libvirt/libvirt-stream.h | 102 +++- src/driver-stream.h | 23 + src/esx/esx_stream.c | 16 +- src/libvirt-storage.c | 4 +- src/libvirt-stream.c | 456 +++++++++++++++++ src/libvirt_internal.h | 3 + src/libvirt_private.syms | 2 + src/libvirt_public.syms | 9 + src/libvirt_remote.syms | 3 + src/remote/remote_driver.c | 89 +++- src/remote/remote_protocol.x | 2 + src/rpc/gendispatch.pl | 21 +- src/rpc/virnetclient.c | 1 + src/rpc/virnetclientstream.c | 201 +++++++- src/rpc/virnetclientstream.h | 17 +- src/rpc/virnetprotocol.x | 16 +- src/rpc/virnetserverprogram.c | 33 ++ src/rpc/virnetserverprogram.h | 7 + src/storage/storage_driver.c | 4 +- src/storage/storage_util.c | 10 +- src/util/iohelper.c | 72 +-- src/util/virfdstream.c | 926 ++++++++++++++++++++++++++++------- src/util/virfdstream.h | 2 +- src/util/virfile.c | 81 +++ src/util/virfile.h | 3 + src/virnetprotocol-structs | 4 + tests/fdstreamtest.c | 12 +- tests/virfiletest.c | 203 ++++++++ tools/virsh-util.c | 65 +++ tools/virsh-util.h | 29 ++ tools/virsh-volume.c | 50 +- tools/virsh.pod | 6 +- tools/wireshark/src/packet-libvirt.c | 48 ++ tools/wireshark/src/packet-libvirt.h | 2 + 38 files changed, 2369 insertions(+), 314 deletions(-) -- 2.10.2

3 93

[libvirt] [PATCH] RFE: virsh: add domxml-to-native <fmt> [--domain DOMAIN] option
by Dan 15 May '17

15 May '17

Bug 835476 RFE: virsh: add domxml-to-native --domain option (for existing VM) [1] virsh DOMAIN COMMAND domxml-to-native did not support domain (id|uuid|name) as input for generating hypervisor agent native command, instead only supported XML input from STDIN. Here in this patch, it supports the following syntax: domxml-to-native <format> { [--domain DOMAIN] | [XML] }, i.e., it supports either designating domain (domain id, uuid, or name), or path to XML domain configuration file; NOTE that it deprecated existing STDIN input passing XML functionality. It was tested on the test mock driver and QEMU. [1]. https://bugzilla.redhat.com/show_bug.cgi?id=835476 Dan --- tools/virsh-domain.c | 68 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 55 insertions(+), 13 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index db8accfe4..9ac855b19 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -60,6 +60,7 @@ #include "virsh-nodedev.h" #include "viruri.h" + /* Gnulib doesn't guarantee SA_SIGINFO support. */ #ifndef SA_SIGINFO # define SA_SIGINFO 0 @@ -9811,9 +9812,13 @@ static const vshCmdOptDef opts_domxmltonative[] = { .flags = VSH_OFLAG_REQ, .help = N_("target config data type format") }, + {.name = "domain", + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ_OPT, + .help = N_("domain name, id or uuid") + }, {.name = "xml", .type = VSH_OT_DATA, - .flags = VSH_OFLAG_REQ, .help = N_("xml data file to export from") }, {.name = NULL} @@ -9822,31 +9827,68 @@ static const vshCmdOptDef opts_domxmltonative[] = { static bool cmdDomXMLToNative(vshControl *ctl, const vshCmd *cmd) { - bool ret = true; const char *format = NULL; const char *xmlFile = NULL; - char *configData; - char *xmlData; + const char *domain = NULL; + char *configData = NULL; + char *xmlData = NULL; unsigned int flags = 0; + unsigned int domflags = 0; virshControlPtr priv = ctl->privData; + virDomainPtr dom = NULL; - if (vshCommandOptStringReq(ctl, cmd, "format", &format) < 0 || - vshCommandOptStringReq(ctl, cmd, "xml", &xmlFile) < 0) + if (vshCommandOptStringReq(ctl, cmd, "format", &format) < 0) return false; + + if (vshCommandOptStringReq(ctl, cmd, "domain", &domain) < 0) + return false; - if (virFileReadAll(xmlFile, VSH_MAX_XML_FILE, &xmlData) < 0) - return false; + if (vshCommandOptStringReq(ctl, cmd, "xml", &xmlFile) < 0) + return false; + + VSH_EXCLUSIVE_OPTIONS_VAR(domain, xmlFile); + + if (domain) { + domflags = VIRSH_BYID | VIRSH_BYUUID | VIRSH_BYNAME; + dom = virshLookupDomainBy(ctl, domain, domflags); + } + + if (!dom && !xmlFile) { + vshError(ctl, _("need either xml file input or domain")); + return false; + } + + if(dom){ + xmlData = virDomainGetXMLDesc(dom, flags); + if (xmlData == NULL) { + virshDomainFree(dom); + return false; + } + } + + if (xmlFile) { + if (virFileReadAll(xmlFile, VSH_MAX_XML_FILE, &xmlData) < 0) { + vshReportError(ctl); + VIR_FREE(xmlData); + return false; + } + } configData = virConnectDomainXMLToNative(priv->conn, format, xmlData, flags); + if (configData != NULL) { - vshPrint(ctl, "%s", configData); - VIR_FREE(configData); + vshPrint(ctl, "%s", configData); + VIR_FREE(xmlData); + VIR_FREE(configData); + virDomainFree(dom); + return true; } else { - ret = false; + VIR_FREE(xmlData); + virDomainFree(dom); + return false; } - VIR_FREE(xmlData); - return ret; + return false; } /* -- 2.12.2

2 2

[libvirt] error in docs https://libvirt.org/html/libvirt-libvirt-domain.html
by Vasiliy Tolstov 12 May '17

12 May '17

virDomainMigrateFlags have errors VIR_MIGRATE_LIVE have invalid comment and as i see all comments are shifted down by one point. -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

3 4