March 2016 - Devel - libvirt List Archives

[libvirt] [PATCH 0/3] Add a domain masterKey secret for qemu
by John Ferlan 29 Mar '16

29 Mar '16

Differences to RFC: http://www.redhat.com/archives/libvir-list/2016-March/msg00966.html (many, but highlights are): * Patch2 is Patch1 and I figured out the 'qom-list-types' magic in order to fill in the .replies file * Use of qemuDomainObjPrivate rather than passing 'masterKey' * The masterKey is not stored in XML output nor is it stored as a base64 value * The masterKey is created during qemuProcessPrepareHost right after the libDir is created. It is only created if the capability exists. * The masterKey is removed during qemuProcessStop. There is no call to qemuDomainMasterKeyRemove from qemuDomainObjPrivateFree since the assumption is the *Stop code will handle things. * Created a qemuDomainMasterKeyReadFile to handle reading the master.key file during qemuProcessReconnect rather than the prior code which would have taken the masterKey from the XML file in base64 format. * The MasterKeyCreate and MasterKeyRemove incorporate the file manipulation as well. The *Create code will save the base64 encoded secret. The callers need only call those API's. * The masterKey is generated using gnutls_rnd if that's available (following examples w/ HAVE_GNUTLS_CRYPTO_H); otherwise, the fallback position is the virRandomBits. * Moved alias code to qemu_alias.c instead of qemu_domain.c. Notes on comments where I didn't change things: * Since qemuBuildCommandLine doesn't get the qemuDomainObjPrivate, I kept the qemuBuildHasMasterKey checking the capability bit only. Other *CommandLine callers would use the same decision point. If the key was available, then "assume" it's already present on the command line since there's no failure path from building the master key command line. * I kept the write "0" to the key file just before deletion. I wasn't sure with file caches being as they can be if there could ever be that 100% guarantee that what was unlink()'d really "went away" completely. Something I noted and wasn't "clear" how to resolve, so I followed existing practices of not checking if something exists: * The recent changes to have a qemuProcessCreatePretendCmd which will allow libDir and channelTargetDir creation using "/tmp/", but not actually creating the directory inhibit this code from being able to check if the path to the masterKey exists before sending it to qemu. Now if the 'flags' was passed to qemuBuildCommandLine and the VIR_QEMU_PROCESS_START_PRETEND could be checked while building the MasterKey command line, then I could have the check in. Otherwise, I just have to assume everything's worked "right" up to this point. A secondary note on that... Why "/tmp/" - shouldn't have been "tmp/"? John Ferlan (3): qemu: Add capability bit for qemu secret object qemu: Create domain master key qemu: Introduce qemuBuildMasterKeyCommandLine src/qemu/qemu_alias.c | 17 ++ src/qemu/qemu_alias.h | 3 + src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 68 ++++++ src/qemu/qemu_domain.c | 269 +++++++++++++++++++++ src/qemu/qemu_domain.h | 13 + src/qemu/qemu_process.c | 11 + tests/qemucapabilitiesdata/caps_2.6.0-1.caps | 1 + tests/qemucapabilitiesdata/caps_2.6.0-1.replies | 3 + .../qemuxml2argvdata/qemuxml2argv-master-key.args | 23 ++ tests/qemuxml2argvdata/qemuxml2argv-master-key.xml | 30 +++ tests/qemuxml2argvtest.c | 2 + 13 files changed, 443 insertions(+) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-master-key.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-master-key.xml -- 2.5.0

3 10

[libvirt] [PATCH v4 0/8] Add perf and Intel CMT feature support
by Qiaowei Ren 29 Mar '16

29 Mar '16

The series mainly adds Intel CMT feature support into libvirt. CMT is new introduced PQos (Platform Qos) feature to monitor the usage of cache by applications running on the platform. Currently CMT patches has been merged into Linux kernel mainline. The CMT implementation in Linux kernel is based on perf mechanism and there is no support for perf in libvirt, and so this series firstly add perf support into libvirt, including two public API and a set of util interfaces. And based on these APIs and interfaces, thie series implements CMT perf event support. Current state: * 1/8 - perf: add new public APIs for perf event - ACKed by Daniel * 2/8 - perf: implement the remote protocol for perf event - ACKed by Daniel * 3/8 - perf: implement a set of util functions for perf event - ACKed by Daniel * 4/8 - qemu_driver: add support to perf event - restart perf in qemuProcessReconnect and qemuProcessAttach in patch 6/8 * 5/8 - perf: add new xml element - ACKed by Daniel * 6/8 - perf: reenable perf events when libvirtd restart - add qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach() * 7/8 - virsh: implement new command to support perf - ACKed by Daniel * 8/8 - virsh: extend domstats command - ACKed by Daniel TODO: 1. add support for new APIs into libvirt-python library. Changes since v1: * change perf APIs implementation to match new style of the API. * add new xml element * reenable all perf events previously enabled when libvirtd daemon restart. * redesign perf util functions. Changes since v2: * add an example XML file to the test suite. * add virPerfReadEvent(). * change 'perf' xml element to new style. * change 'perf' command to new stype. Changes since v3: * move qemuDomainPerfRestart() into qemuProcessReconnect() and qemuProcessAttach(). Qiaowei Ren (8): perf: add new public APIs for perf event perf: implement the remote protocol for perf event perf: implement a set of util functions for perf event qemu_driver: add support to perf event perf: add new xml element perf: reenable perf events when libvirtd restart virsh: implement new command to support perf virsh: extend domstats command daemon/remote.c | 47 ++++ docs/schemas/domaincommon.rng | 27 +++ include/libvirt/libvirt-domain.h | 19 ++ include/libvirt/virterror.h | 2 + src/Makefile.am | 1 + src/conf/domain_conf.c | 111 ++++++++++ src/conf/domain_conf.h | 10 + src/driver-hypervisor.h | 12 + src/libvirt-domain.c | 93 ++++++++ src/libvirt_private.syms | 12 + src/libvirt_public.syms | 2 + src/qemu/qemu_domain.h | 3 + src/qemu/qemu_driver.c | 159 +++++++++++++ src/qemu/qemu_process.c | 44 ++++ src/remote/remote_driver.c | 39 ++++ src/remote/remote_protocol.x | 30 ++- src/remote_protocol-structs | 18 ++ src/util/virerror.c | 2 + src/util/virperf.c | 307 ++++++++++++++++++++++++++ src/util/virperf.h | 63 ++++++ tests/domainschemadata/domain-perf-simple.xml | 20 ++ tools/virsh-domain-monitor.c | 7 + tools/virsh-domain.c | 128 +++++++++++ tools/virsh.pod | 27 ++- 24 files changed, 1180 insertions(+), 3 deletions(-) create mode 100644 src/util/virperf.c create mode 100644 src/util/virperf.h create mode 100644 tests/domainschemadata/domain-perf-simple.xml -- 1.9.1

3 14

[libvirt] [PATCH] qemu: Fix /proc/**/stat parsing
by Jiri Denemark 29 Mar '16

29 Mar '16

Since commit v1.3.2-119-g1e34a8f which enabled debug-threads in QEMU qemuGetProcessInfo would fail to parse stats for any thread with a space in its name. https://bugzilla.redhat.com/show_bug.cgi?id=1316803 Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_driver.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 7e01459..dec99ca 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -1460,7 +1460,7 @@ qemuGetProcessInfo(unsigned long long *cpuTime, int *lastCpu, long *vm_rss, if (!pidinfo || fscanf(pidinfo, /* pid -> stime */ - "%*d %*s %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %llu %llu" + "%*d (%*[^)]) %*c %*d %*d %*d %*d %*d %*u %*u %*u %*u %*u %llu %llu" /* cutime -> endcode */ "%*d %*d %*d %*d %*d %*d %*u %*u %ld %*u %*u %*u" /* startstack -> processor */ -- 2.7.4

2 1

[libvirt] [PATCH 0/6] virsh: Add support for byte granularity and scaled integers for virsh block APIs
by Peter Krempa 29 Mar '16

29 Mar '16

Quite some time ago we've added support for byte granularity for block job bandwidth. Make it work in virsh and add support for scaled integers too. Peter Krempa (6): vsh: Tweak error message for scaled integers vsh: Refactor vshCommandOptScaledInt virsh: blockjob: Support --bytes and scaled integers as bandwidth virsh: blockcommit: Support --bytes and scaled integers virsh: blockcopy: Support --bytes and scaled integers virsh: blockpull: Support --bytes and scaled integers tests/virsh-optparse | 6 ++--- tools/virsh-domain.c | 73 ++++++++++++++++++++++++++++++++++++---------------- tools/virsh.pod | 37 ++++++++++++++------------ tools/vsh.c | 66 +++++++++++++++++++++++++++++++++++++++++------ tools/vsh.h | 4 +++ 5 files changed, 137 insertions(+), 49 deletions(-) -- 2.7.3

2 10

[libvirt] Plan for next release
by Daniel Veillard 29 Mar '16

29 Mar '16

I'm a bit behind, I forgot ! I think we should freeze tomorrow (Wed 30), then make an rc2 on Friday and shoot for a release next Monday (Apr 4), works for everybody ? Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

2 1

[libvirt] [PATCH] tools: virt-host-validate: fix KVM check on s390
by Bjoern Walk 29 Mar '16

29 Mar '16

Since kernel version 4.5, s390 has the 'sie' flag to declare its hardware virtualization support. Let's make virt-host-validate aware so this check is passed correctly. Signed-off-by: Bjoern Walk <bwalk(a)linux.vnet.ibm.com> Reviewed-by: Boris Fiuczynski <fiuczy(a)linux.vnet.ibm.com> --- tools/virt-host-validate-qemu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/virt-host-validate-qemu.c b/tools/virt-host-validate-qemu.c index a9f6c1e..01fb24b 100644 --- a/tools/virt-host-validate-qemu.c +++ b/tools/virt-host-validate-qemu.c @@ -31,7 +31,8 @@ int virHostValidateQEMU(void) virHostMsgCheck("QEMU", "%s", ("for hardware virtualization")); if (virHostValidateHasCPUFlag("svm") || - virHostValidateHasCPUFlag("vmx")) { + virHostValidateHasCPUFlag("vmx") || + virHostValidateHasCPUFlag("sie")) { virHostMsgPass(); if (virHostValidateDeviceExists("QEMU", "/dev/kvm", VIR_HOST_VALIDATE_FAIL, -- 2.6.5

4 3

[libvirt] [PATCH 0/2] bhyve: tiny cleanup and fix
by Maxim Nestratov 29 Mar '16

29 Mar '16

Maxim Nestratov (2): bhyve: cleanup unnecessary variables bhyve: fix invalid hostsysinfo freeing src/bhyve/bhyve_driver.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) -- 2.4.3

2 5

[libvirt] [PATCH] qemu: tolerate panic device on S390
by Boris Fiuczynski 29 Mar '16

29 Mar '16

If a panic device is being defined without a model in a domain the default value is always overwritten with model ISA. An ISA bus does not exist on S390 and therefore specifying a panic device results in an unsupported configuration. Since the s390 architecture inherently provides a crash detection capability the panic device should be tolerated in the domain xml. This patch achieves the toleration by setting the model to the default value in case the architecture is S390. Signed-off-by: Boris Fiuczynski <fiuczy(a)linux.vnet.ibm.com> Reviewed-by: Cornelia Huck <cornelia.huck(a)de.ibm.com> --- src/qemu/qemu_domain.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index 9f9fae3..96561c0 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -1605,6 +1605,9 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, if (ARCH_IS_PPC64(def->os.arch) && STRPREFIX(def->os.machine, "pseries")) dev->data.panic->model = VIR_DOMAIN_PANIC_MODEL_PSERIES; + else if (ARCH_IS_S390(def->os.arch)) + /* since S390 does not support ISA use default for toleration */ + dev->data.panic->model = VIR_DOMAIN_PANIC_MODEL_DEFAULT; else dev->data.panic->model = VIR_DOMAIN_PANIC_MODEL_ISA; } -- 2.5.0

2 1

[libvirt] [PATCH] SCSI storage: Initialise the pool size parameters to zero while refreshing
by Nitesh Konkar 29 Mar '16

29 Mar '16

If the pool creation thread happens to detect the luns in the scsi target,the size parameters will be populated as part of the refresh called from storagePoolCreate(). The commit <4a85bf3e2f> added additional refresh for SCSI which should also have these values initialised to zero, otherwise the values would appear exactly double. A separate refresh would correct the values, but without an explicit pool-refresh, the allocation/capacity would appear exactly double after pool-create. --- src/storage/storage_backend_scsi.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/storage/storage_backend_scsi.c b/src/storage/storage_backend_scsi.c index 670cc4d..7c7d7d0 100644 --- a/src/storage/storage_backend_scsi.c +++ b/src/storage/storage_backend_scsi.c @@ -614,6 +614,9 @@ virStoragePoolFCRefreshThread(void *opaque) */ VIR_DEBUG("Attempt FC Refresh for pool='%s' name='%s' tries='%d'", pool->def->name, fchost_name, tries); + + pool->def->allocation = pool->def->capacity = pool->def->available = 0; + if (virStoragePoolObjIsActive(pool) && virGetSCSIHostNumber(fchost_name, &host) == 0 && virStorageBackendSCSITriggerRescan(host) == 0) { -- 1.8.3.1

2 1

[libvirt] [PATCH] docs: Document NSS module
by Michal Privoznik 29 Mar '16

29 Mar '16

While we have a wiki page describing the feature [1] since the feature is distributed in our .tar.gz we ought to document it. So I went ahead, copied the wiki page and reformatted so it fits our docs coding style. 1: http://wiki.libvirt.org/page/NSS_module Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- docs/nss.html.in | 141 +++++++++++++++++++++++++++++++++++++++++++++++++++ docs/sitemap.html.in | 4 ++ 2 files changed, 145 insertions(+) create mode 100644 docs/nss.html.in diff --git a/docs/nss.html.in b/docs/nss.html.in new file mode 100644 index 0000000..84ea8df --- /dev/null +++ b/docs/nss.html.in @@ -0,0 +1,141 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml"> + <body> + <h1>Libvirt NSS module</h1> + + <ul id="toc"></ul> + + <p> + When it comes to managing guests and executing commands inside them, logging + into guest operating system and doing the job is convenient. Users are used + to ssh in this case. Ideally: + </p> + + <code>ssh user@virtualMachine</code> + + <p> + would be nice. But depending on virtual network configuration it might not + be always possible. For instance, when using libvirt NATed network it's + dnsmasq (spawned by libvirt) who assigns IP addresses to domains. But by + default, the dnsmasq process is then not consulted when it comes to host + name translation. Users work around this problem by configuring their + libvirt network to assign static IP addresses and maintaining + <code>/etc/hosts</code> file in sync. But this puts needless burden onto + users. This is where NSS module comes handy. + </p> + + <h2><a name="Installation">Installation</a></h2> + + <p> + Installing the module is really easy: + </p> + +<pre> +# yum install libvirt-nss +</pre> + + <h2><a name="Configuration">Configuration</a></h2> + + <p> + Enabling the module is really easy. Just add <b>libvirt</b> into + <code>/etc/nsswitch.conf</code> file. For instance: + </p> + +<pre> +$ cat /etc/nsswitch.conf +# /etc/nsswitch.conf: +passwd: compat +shadow: compat +group: compat +hosts: files libvirt dns +# ... +</pre> + + <p> + So, in this specific case, whenever ssh program is looking up the host user + is trying to connect to, <b>files</b> module is consulted first (which + boils down to looking up the host name in <code>/etc/hosts</code> file), if + not found <b>libvirt</b> module is consulted then. The DNS is the last + effort then, if none of the previous modules matched the host in question. + Therefore users should consider the order in which they want the modules to + lookup given host name. + </p> + + <h2><a name="Internals">How does it work?</a></h2> + + <p> + Whenever an Unix process wants to do a host name translation + <a href="http://linux.die.net/man/3/gethostbyname"><code>gethostbyname()</code></a> + or some variant of it is called. This is a glibc function that takes a + string containing the host name, crunch it and produces a list of IP + addresses assigned to that host. Now, glibc developers made a really good + decision when implementing the internals of the function when they decided + to make the function pluggable. Since there can be several sources for the + records (e.g. <code>/etc/hosts</code> file, DNS, LDAP, etc.) it would not + make much sense to create one big implementation containing all possible + cases. What they have done instead is this pluggable mechanism. Small + plugins implementing nothing but specific technology for lookup process are + provided and the function then calls those plugins. There is just one + configuration file that instructs the lookup function in which order should + the plugins be called and which plugins should be loaded. For more info + reading <a href="https://en.wikipedia.org/wiki/Name_Service_Switch">wiki + page</a> is recommended. + </p> + + <p> + And this is point where libvirt comes in. Libvirt provides plugin for the + NSS ecosystem. For some time now libvirt keeps a list of assigned IP + addresses for libvirt networks. The NSS plugin does no more than search the + list trying to find matching record for given host name. When found, + matching IP address is returned to the caller. If not found, translation + process continues with the next plugin configured. At this point it is + important to stress the order in which plugins are called. Users should be + aware that a hostname might match in multiple plugins and right after first + match, translation process is terminated and no other plugin is consulted. + Therefore, if there are two different records for the same host name users + should carefully chose the lookup order. + </p> + + <h2><a name="Limitations">Limitations</a></h2> + + <ol> + <li>The libvirt NSS module matches only hostnames provided by guest. If + the libvirt name and one advertised by guest differs, the latter is + matched.</li> + <li>The module works only in that cases where IP addresses are assigned by + dnsmasq spawned by libvirt. Libvirt NATed networks are typical + example.</li> + </ol> + + <p> + These limitation are result of libvirt's internal implementation. While + libvirt can report IP addresses regardless of their origin, a public API + must be used to obtain those. However, for the API a connection object is + required. Doing that for every name translation request would be too + costly. Fortunately, libvirt spawns dnsmasq for NATed networks. Not only + that, it provides small executable that on each IP address space change + updates an internal list of addresses thus keeping it in sync. The NSS + module then merely consults the list trying to find the match. Users can + view the list themselves: + </p> + +<pre> +virsh net-dhcp-leases $network +</pre> + + <p> + where <code>$network</code> iterates through all running networks. So the module + does merely the same as + </p> + +<pre> +virsh domifaddr --source lease $domain +</pre> + + <p> + If there's no record for either of the aforementioned commands, it's very + likely that NSS module won't find anything and vice versa. + </p> + </body> +</html> diff --git a/docs/sitemap.html.in b/docs/sitemap.html.in index 7e3746e..22e410c 100644 --- a/docs/sitemap.html.in +++ b/docs/sitemap.html.in @@ -120,6 +120,10 @@ <a href="hooks.html">Hooks</a> <span>Hooks for system specific management</span> </li> + <li> + <a href="nss.html">NSS module</a> + <span>Enable domain host name translation to IP addresses</span> + </li> </ul> </li> <li> -- 2.7.3

2 1