November 2017 - Devel - libvirt List Archives

[libvirt] [PATCH] remote: Fix libvirtd service memory leak when libvirt client was terminated unexpectly
by xinhua.Cao 06 Nov '17

06 Nov '17

base on commit 2033e8cc119454bc4273e8a41e66c899c60ba58b and fe8f1c8b8650c8ab5e7d27338f4538582651bd14, we solve libvirt coredump problem, but it introduce a memory leak sense. the sense follow 1. one client register a domain event such as reboot event 2. and client was terminated unexpectly, then this client will not free at libvirtd service program. remoteDispatchConnectDomainEventCallbackRegisterAny reference the client, but when client was terminated before it call deRegisterAny, the reference of client will not reduced to zero. so the memory leak take place. this patch will deRegister all event when client was close. --- daemon/remote.c | 47 +++++++++++++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index 3f7d2d3..2b5a18b 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -1686,25 +1686,16 @@ void remoteRelayConnectionClosedEvent(virConnectPtr conn ATTRIBUTE_UNUSED, int r VIR_WARN("unexpected %s event deregister failure", name); \ } \ VIR_FREE(eventCallbacks); \ + neventCallbacks = 0; \ } while (0); -/* - * You must hold lock for at least the client - * We don't free stuff here, merely disconnect the client's - * network socket & resources. - * We keep the libvirt connection open until any async - * jobs have finished, then clean it up elsewhere - */ -void remoteClientFreeFunc(void *data) +static void +remoteFreePrivCallbacks(void *data) { struct daemonClientPrivate *priv = data; /* Deregister event delivery callback */ - if (priv->conn) { - virIdentityPtr sysident = virIdentityGetSystem(); - - virIdentitySetCurrent(sysident); - + if (priv && priv->conn) { DEREG_CB(priv->conn, priv->domainEventCallbacks, priv->ndomainEventCallbacks, virConnectDomainEventDeregisterAny, "domain"); @@ -1723,6 +1714,26 @@ void remoteClientFreeFunc(void *data) DEREG_CB(priv->conn, priv->qemuEventCallbacks, priv->nqemuEventCallbacks, virConnectDomainQemuMonitorEventDeregister, "qemu monitor"); + } +} +#undef DEREG_CB + +/* + * You must hold lock for at least the client + * We don't free stuff here, merely disconnect the client's + * network socket & resources. + * We keep the libvirt connection open until any async + * jobs have finished, then clean it up elsewhere + */ +void remoteClientFreeFunc(void *data) +{ + struct daemonClientPrivate *priv = data; + + if (priv) { + virIdentityPtr sysident = virIdentityGetSystem(); + + virIdentitySetCurrent(sysident); + remoteFreePrivCallbacks(priv); if (priv->closeRegistered) { if (virConnectUnregisterCloseCallback(priv->conn, @@ -1734,18 +1745,18 @@ void remoteClientFreeFunc(void *data) virIdentitySetCurrent(NULL); virObjectUnref(sysident); + VIR_FREE(priv); } - - VIR_FREE(priv); } -#undef DEREG_CB - static void remoteClientCloseFunc(virNetServerClientPtr client) { struct daemonClientPrivate *priv = virNetServerClientGetPrivateData(client); - daemonRemoveAllClientStreams(priv->streams); + if (priv) { + daemonRemoveAllClientStreams(priv->streams); + remoteFreePrivCallbacks(priv); + } } -- 2.8.3

3 6

[libvirt] [PATCH 0/2] Removing backend support when net interface is user/direct/hostdev.
by Julio Faracco 05 Nov '17

05 Nov '17

The <backend> xml tag is not supported for some interface types and the virsh command 'attach-device' permits to add <backend> settings to all of them. These commits avoid <backend> for user, direct and hostdev interface types. Julio Faracco (2): tests: removing backend xml tag inside some test cases. conf: network user/direct/hostdev do not support backend tag. src/conf/domain_conf.c | 10 ++++++++++ tests/qemuxml2argvdata/qemuxml2argv-tap-vhost-incorrect.xml | 2 -- tests/qemuxml2argvdata/qemuxml2argv-tap-vhost.xml | 1 - tests/qemuxml2argvtest.c | 6 +++--- .../qemuxml2xmloutdata/qemuxml2xmlout-tap-vhost-incorrect.xml | 2 -- tests/qemuxml2xmloutdata/qemuxml2xmlout-tap-vhost.xml | 1 - 6 files changed, 13 insertions(+), 9 deletions(-) -- 2.7.4

2 4

[libvirt] [PATCH go-xml] Improve DomainChardevSource.
by Brandon Bergren 04 Nov '17

04 Nov '17

* Add missing attributes to DomainChardevSource. * Change Path to be omitempty because several chardev types do not use it. * Document which chardev types use which attributes (for convenience) I was mainly interested in making it possible to define nmdm devices from go code, but I figured I would add the other missing attributes while I was here. --- domain.go | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/domain.go b/domain.go index f4ef35c..debc5b2 100644 --- a/domain.go +++ b/domain.go @@ -256,9 +256,15 @@ type DomainInterface struct { } type DomainChardevSource struct { - Mode string `xml:"mode,attr,omitempty"` - Path string `xml:"path,attr"` - Append string `xml:"append,attr,omitempty"` + Mode string `xml:"mode,attr,omitempty"` // tcp,udp,unix + Path string `xml:"path,attr,omitempty"` // file,pty,dev,pipe + Channnel string `xml:"channel,attr,omitempty"` // spiceport + Append string `xml:"append,attr,omitempty"` // file + Host string `xml:"host,attr,omitempty"` // tcp,udp + Service string `xml:"service,attr,omitempty"` // tcp,udp + TLS string `xml:"tls,attr,omitempty"` // tcp(QEMU) + Master string `xml:"master,attr,omitempty"` // nmdm + Slave string `xml:"slave,attr,omitempty"` // nmdm } type DomainChardevTarget struct { -- 2.14.1

1 1

Re: [libvirt] How to best handle the reoccurring of rom changes breaking cross version migrations?
by Christian Ehrhardt 03 Nov '17

03 Nov '17

Ping - since there wasn't any reply so far - any best practices one could share? Let me add a TL;DR: - bump of ipxe rom versions change the size of virtio-net-pci.rom - that breaks on migration "Length mismatch" I'd guess the size of that rom has to be fixed up on the fly, but if that is really ok and how/where is the question. Also to +1 on bad things for today - I made this a cross post to libvirt in case there is one that has done that in the past. On Mon, Aug 28, 2017 at 4:36 PM, Christian Ehrhardt < christian.ehrhardt(a)canonical.com> wrote: > Hi, > migration issues due to rom changes seem to occur over and over in past > years [1], [2],[3],[4],[5]. > From the past I know several workarounds (like just truncating to the > bigger size) but all have various deficiencies. > > But OTOH rom's will always change due to fixes in them. And recently I > found one such change [6] that affects the next Ubuntu release and wonder > what the ?right?, well lets say best way to fix it would be. > Current issue: > Length mismatch: 0000:00:03.0/virtio-net-pci.rom: 0x40000 in != 0x80000: > Invalid argument > Due to efi-virtio.rom growing over 256k due to an update to a newer ipxe > from upstream. > > I beg your pardon (for not being educated enough on this yet), but I want > to avoid to go a route that is fixing it in a sub-optimal way and ask for > some guidance. There might be better ways in the modern qemu of today than > there were in the past. > TL;DR I look for the best way (if any) to declare in the new qemu: "I know > that the old size was smaller, let me fix that up on migration". > > I'll try on my own as I expect the machine type structures/mechanisms (and > we have Ubuntu specific types that could encapsulate the rom status from > the ipxe package to be coupled with the type) have all that is needed. > Yet me almost randomly trying around there surely isn't the best way to go > - so if there is some existing case or a short hint at what in there might > be the best way to fixup a changed rom size on a migration I'd be very > happy to hear about that. > > [1]: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1536331 > [2]: https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01881.html > [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1293566 > [4]: https://bugzilla.redhat.com/show_bug.cgi?id=1090093 > [5]: https://forge.univention.org/bugzilla/show_bug.cgi?id=38877 > [6]: https://bugs.launchpad.net/ubuntu/+source/ipxe/+bug/1713490 > > P.S: As everybody else I don't mind so much on reverse migration to older > releases > > -- > Christian Ehrhardt > Software Engineer, Ubuntu Server > Canonical Ltd > -- Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd

3 3

[libvirt] [REPOST PATCH v6 0/8] Use secret objects to pass iSCSI passwords
by John Ferlan 03 Nov '17

03 Nov '17

Reposting after 3.9.0 release and refresh to current HEAD. Had to move the news.xml into the 3.10.0 section as well and address a couple of merge conflicts. Original: https://www.redhat.com/archives/libvir-list/2017-October/msg01012.html Copy of Original cover letter: v5: https://www.redhat.com/archives/libvir-list/2017-October/msg00228.html FWIW: AFAICT this series does not need/require changes that Peter has posted for continuing blocked-add saga related to user-specified backing chains: https://www.redhat.com/archives/libvir-list/2017-October/msg00956.html Changes since v5: * Some patches pushed as part of Peter Krempa's work and the two patches to perform the parsing of auth and encryption data in virStorageSource were pushed separately * Removed patches dealing with qemuDomainStorageSourceCopy and virDomainDiskStorageSourceNew Patches 9->16 reworked Patch1: (previous patch 10) - Rework logic to remove the need to pass around the @xmlopt for virStorageSourcePtr allocation and instead VIR_ALLOC(iscsisrc->src) directly as other consumers do. Patch2: (previous patch 11) - Use the qemuDomainStorageSourcePrivatePtr and friend macro QEMU_DOMAIN_STORAGE_SOURCE_PRIVATE to manage the iscsisrc->src data making sure to ensure that srcPriv (e.g. privateData) exists. Patch3: (previous patch 12) - Only minor merge related changes. Patch4: (previous patch 13) - Merge related plus additional check to ensure srcPriv exists before dereference secinfo Patch5: (previous patch 14) - No change Patch 6: (previous patch 15) - Merge related changes, plus checks for srcPriv before deref secinfo NB: Testing note - I did ensure at this point if the password secret capability check fails that the code will still do the right thing. Patch7: (previous patch 9) - No change... Presented since it wasn't ACK'd before Patch8: (previous patch16) - No change John Ferlan (8): conf,qemu: Replace iscsisrc fields with virStorageSourcePtr qemu: Use private storage source for iscsi instead of private hostdev qemu: Remove private hostdev qemu: Refactor qemuBuildSCSIiSCSIHostdevDrvStr slightly qemu: Get capabilities to use iscsi password-secret argument qemu: Use secret objects to pass iSCSI passwords docs: Add news article regarding auth/encryption placement docs: Add news article to describe iSCSI usage of secret object docs/news.xml | 23 ++++++ src/conf/domain_conf.c | 58 ++++++++-------- src/conf/domain_conf.h | 9 +-- src/lxc/lxc_native.c | 2 +- src/qemu/qemu_block.c | 65 ++++++++++++++++- src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 75 +++++++++++++++----- src/qemu/qemu_command.h | 3 +- src/qemu/qemu_domain.c | 81 +++++++--------------- src/qemu/qemu_domain.h | 14 ---- src/qemu/qemu_hotplug.c | 52 +++++++++++++- src/qemu/qemu_parse_command.c | 4 +- src/vbox/vbox_common.c | 2 +- src/xenconfig/xen_common.c | 2 +- src/xenconfig/xen_sxpr.c | 2 +- src/xenconfig/xen_xl.c | 2 +- tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64le.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + ...xml2argv-disk-drive-network-iscsi-auth-AES.args | 41 +++++++++++ ...uxml2argv-disk-drive-network-iscsi-auth-AES.xml | 43 ++++++++++++ ...ml2argv-hostdev-scsi-virtio-iscsi-auth-AES.args | 45 ++++++++++++ ...xml2argv-hostdev-scsi-virtio-iscsi-auth-AES.xml | 48 +++++++++++++ tests/qemuxml2argvtest.c | 10 +++ tests/virhostdevtest.c | 2 +- 28 files changed, 451 insertions(+), 140 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-iscsi-auth-AES.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-iscsi-auth-AES.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-scsi-virtio-iscsi-auth-AES.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-hostdev-scsi-virtio-iscsi-auth-AES.xml -- 2.13.6

1 8

[libvirt] [PATCH] cpu_map: Add cqm alternative name for cmt
by Jiri Denemark 03 Nov '17

03 Nov '17

Linux kernel shows our "cmt" feature as "cqm". Let's mention the name in the cpu_map.xml to make it easier to find. Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/cpu/cpu_map.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/cpu/cpu_map.xml b/src/cpu/cpu_map.xml index 96f4ce60d..e5da7a887 100644 --- a/src/cpu/cpu_map.xml +++ b/src/cpu/cpu_map.xml @@ -224,7 +224,7 @@ <feature name='rtm'> <cpuid eax_in='0x07' ecx_in='0x00' ebx='0x00000800'/> </feature> - <feature name='cmt'> + <feature name='cmt'>  <cpuid eax_in='0x07' ecx_in='0x00' ebx='0x00001000'/> </feature> <feature name='mpx'> -- 2.14.3

3 2

[libvirt] [PATCH 0/9] qemu: command: -drive formatting cleanups (blockdev-add saga)
by Peter Krempa 03 Nov '17

03 Nov '17

Kill some ugly code. Peter Krempa (9): util: Fix condition check in virDiskNameToIndex qemu: command: Remove dead code when formatting -drive qemu: command: Move disk index validation closer to usage qemu: command: Directly report bus type in qemuBuildDriveStrValidate qemu: command: Refactor qemuBuildDriveStrValidate to make qemuCaps optional qemu: command: Merge checks from qemuBuildDriveStrValidate to qemuCheckDiskConfig qemu: command: Refactor blkiotune checks to tolerate NULL qemuCaps qemu: command: Move blkiotune checks to qemuCheckDiskConfig qemu: command: Move disk serial validation to checker function src/qemu/qemu_command.c | 485 +++++++++++++++++++++++------------------------- src/qemu/qemu_command.h | 3 +- src/qemu/qemu_driver.c | 2 +- src/util/virutil.c | 2 +- 4 files changed, 236 insertions(+), 256 deletions(-) -- 2.14.3

2 13

[libvirt] [PATCH] conf: Don't inline virDomainNetTypeSharesHostView
by Jiri Denemark 03 Nov '17

03 Nov '17

When coverage build is enabled, gcc complains about it: In file included from qemu/qemu_agent.h:29:0, from qemu/qemu_driver.c:47: qemu/qemu_driver.c: In function 'qemuDomainSetInterfaceParameters': ./conf/domain_conf.h:3397:1: error: inlining failed in call to 'virDomainNetTypeSharesHostView': call is unlikely and code size would grow [-Werror=inline] virDomainNetTypeSharesHostView(const virDomainNetDef *net) ^ Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Notes: Shouldn't we just kill all (or most of) our worthless usage of the inline keyword? src/conf/domain_conf.c | 36 ++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 37 +++---------------------------------- src/libvirt_private.syms | 1 + 3 files changed, 40 insertions(+), 34 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 77c20c697..394afb0d8 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -28082,3 +28082,39 @@ virDomainGenerateMachineName(const char *drivername, virBufferCheckError(&buf); return virBufferContentAndReset(&buf); } + + +/** + * virDomainNetTypeSharesHostView: + * @net: interface + * + * Some types of interfaces "share" the host view. For instance, + * for macvtap interface, every domain RX is the host RX too. And + * every domain TX is host TX too. IOW, for some types of + * interfaces guest and host are on the same side of RX/TX + * barrier. This is important so that we set up QoS correctly and + * report proper stats. + */ +bool +virDomainNetTypeSharesHostView(const virDomainNetDef *net) +{ + virDomainNetType actualType = virDomainNetGetActualType(net); + switch (actualType) { + case VIR_DOMAIN_NET_TYPE_DIRECT: + case VIR_DOMAIN_NET_TYPE_ETHERNET: + return true; + case VIR_DOMAIN_NET_TYPE_USER: + case VIR_DOMAIN_NET_TYPE_VHOSTUSER: + case VIR_DOMAIN_NET_TYPE_SERVER: + case VIR_DOMAIN_NET_TYPE_CLIENT: + case VIR_DOMAIN_NET_TYPE_MCAST: + case VIR_DOMAIN_NET_TYPE_NETWORK: + case VIR_DOMAIN_NET_TYPE_BRIDGE: + case VIR_DOMAIN_NET_TYPE_INTERNAL: + case VIR_DOMAIN_NET_TYPE_HOSTDEV: + case VIR_DOMAIN_NET_TYPE_UDP: + case VIR_DOMAIN_NET_TYPE_LAST: + break; + } + return false; +} diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 38de70b15..171f34078 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -3382,40 +3382,9 @@ virDomainGenerateMachineName(const char *drivername, int id, const char *name, bool privileged); -/** - * virDomainNetTypeSharesHostView: - * @net: interface - * - * Some types of interfaces "share" the host view. For instance, - * for macvtap interface, every domain RX is the host RX too. And - * every domain TX is host TX too. IOW, for some types of - * interfaces guest and host are on the same side of RX/TX - * barrier. This is important so that we set up QoS correctly and - * report proper stats. - */ -static inline bool -virDomainNetTypeSharesHostView(const virDomainNetDef *net) -{ - virDomainNetType actualType = virDomainNetGetActualType(net); - switch (actualType) { - case VIR_DOMAIN_NET_TYPE_DIRECT: - case VIR_DOMAIN_NET_TYPE_ETHERNET: - return true; - case VIR_DOMAIN_NET_TYPE_USER: - case VIR_DOMAIN_NET_TYPE_VHOSTUSER: - case VIR_DOMAIN_NET_TYPE_SERVER: - case VIR_DOMAIN_NET_TYPE_CLIENT: - case VIR_DOMAIN_NET_TYPE_MCAST: - case VIR_DOMAIN_NET_TYPE_NETWORK: - case VIR_DOMAIN_NET_TYPE_BRIDGE: - case VIR_DOMAIN_NET_TYPE_INTERNAL: - case VIR_DOMAIN_NET_TYPE_HOSTDEV: - case VIR_DOMAIN_NET_TYPE_UDP: - case VIR_DOMAIN_NET_TYPE_LAST: - break; - } - return false; -} + +bool +virDomainNetTypeSharesHostView(const virDomainNetDef *net); bool virDomainDefLifecycleActionAllowed(virDomainLifecycle type, diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 448d962b2..811d9053e 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -444,6 +444,7 @@ virDomainNetInsert; virDomainNetRemove; virDomainNetRemoveHostdev; virDomainNetTypeFromString; +virDomainNetTypeSharesHostView; virDomainNetTypeToString; virDomainNostateReasonTypeFromString; virDomainNostateReasonTypeToString; -- 2.14.3

2 1

[libvirt] [PATCH 00/12] qemu: block: Prepare for user-specified backing chains (blockdev-add saga)
by Peter Krempa 03 Nov '17

03 Nov '17

When users will specify backing chain, we need to take into account what was passed in them and/or inherit the data from the parents as we are copying the data (labels, etc...) from the parent disk source. Peter Krempa (12): storage: Extract common code to retrieve driver backend for support check storage: Add feature check for storage file backend supporting access check storage: Extract error reporting for broken chains security: selinux: Pass parent storage source into image labeling helper security: dac: Take parent security label into account security: selinux: Take parent security label into account qemu: domain: Simplify using DAC permissions of top of backing chain qemu: domain: Extract setup for disk source secrets qemu: domain: Destroy secrets for complete backing chain qemu: domain: Remove pointless alias check qemu: domain: Prepare TLS data for the whole backing chain qemu: domain: skip chain detection to end of backing chain src/qemu/qemu_domain.c | 177 ++++++++++++++++++++++++++-------------- src/qemu/qemu_domain.h | 6 +- src/qemu/qemu_driver.c | 6 +- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_process.c | 2 +- src/security/security_dac.c | 38 +++++++-- src/security/security_selinux.c | 26 +++--- src/storage/storage_source.c | 110 ++++++++++++++++--------- src/storage/storage_source.h | 5 ++ 9 files changed, 246 insertions(+), 126 deletions(-) -- 2.14.1

2 26

[libvirt] storage lock issues
by Cedric Bosdonnat 02 Nov '17

02 Nov '17

Hi all, I'm investigating a crash in the storage driver, and it seems the solution is not as obvious as I wanted it. Basically what happens is that libvirtd crashes due to the pool refresh thread clearing the list of pool volumes while someone is adding a new volume. Here is the valgrind log I have for that: http://paste.opensuse.org/58733866 I tried adding a call to virStoragePoolObjLock() in the virStorageVolPoolRefreshThread(), since the storageVolCreateXML() appears to have it, but I'm getting a dead lock. Here are the waiting threads I got from gdb: http://paste.opensuse.org/45961070 Any idea what would be a proper fix for that? My vision of the storage drivers locks is too limited it seems ;) -- Cedric

2 5