October 2018 - Devel - libvirt List Archives

[libvirt] [libvirt PATCH] news: Update for 4.8.0 release
by Fabiano Fidêncio 01 Oct '18

01 Oct '18

Signed-off-by: Fabiano Fidêncio <fidencio(a)redhat.com> --- I went through the commits since last release and added a few notes which seem to be relevant for the news. In case you notice there's something missing, please, either send me the text to add it (and I'll re-sping this patch) and add the text before merging this one. --- docs/news.xml | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 3ed6ff8aeb..29b12cb04d 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -44,6 +44,50 @@ and virDomainPMWakeup APIs. </description> </change> + <change> + utils: Introduce monitor capability interface + </change> + <description> + The resource monitor has been introduced and it creates the interface + for getting the host capability of the resource monitor from the system + resource control. + The resource monitor takes the role of RDT monitoring groups and could + be used to monitor the resource consumption information. + </description> + <change> + conf: Introduce RDT monitor host capability + </change> + <description> + Introduce cache monitor (CMT) and memory bandwidth monitor (MBM). + For the former (CMT), the host capability is shown like: + <host> + ... + <cache> + <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0-5'> + <control granularity='768' min='1536' unit='KiB' type='both' maxAllocs='4'/> + </bank> + <monitor level='3' 'reuseThreshold'='270336' maxMonitors='176'> + <feature name='llc_occupancy'/> + </monitor> + </cache> + ... + </host> + + For the latter (MBM), the capability is shown like this: + <host> + ... + <memory_bandwidth> + <node id='1' cpus='6-11'> + <control granularity='10' min ='10' maxAllocs='4'/> + </node> + <monitor maxMonitors='176'> + <feature name='mbm_total_bytes'/> + <feature name='mbm_local_bytes'/> + </monitor> + </memory_bandwidth> + ... + </host> + </description> </section> <section title="Removed features"> <change> @@ -55,6 +99,15 @@ Drop support for these older versions and require Xen >= 4.6. </description> </change> + <change> + <summary> + nwfilter: Disallow binding creation in session mode + </summary> + <description> + Ensure that a filter binding creation is not attempted in session + mode and generate the proper error message. + </description> + </change> </section> <section title="Improvements"> <change> @@ -68,8 +121,43 @@ Guest Agent. </description> </change> + <change> + storage: Allow to use any format as input volume for encryption + </change> + <description> + Since v4.5.0 libvirt has support to use a 'raw' input volume for + encryption. From now on, let's not limit this to 'raw' only. + </description> + <change> + libxl: Add support to set shadow memory for any guest type, not only + HVM + </change> + <description> + PVH guests now can take advantage of using shadow memory. + </description> </section> <section title="Bug fixes"> + <change> + <summary> + virsh: Require explicit --domain for domxml-to-native + </summary> + <description> + The domxml-to-native virsh command accepts either --xml or --domain + option followed by a file or domain name respectively, The --domain + option is documented as required, which means an argument with no + option is treated as --xml. Commit v4.3.0-127-gd86531daf2 broke this + by making --domain optional and thus an argument with no option was + treated as --domain. + </description> + </change> + <change> + lxc_monitor: Avoid AB / BA lock race + </change> + <description> + A dealock situation could occur when autostarting a LXC domain 'guest' + due to two threads attempting to take opposing locks while holding + opposing locks (AB BA problem). + </description> </section> </release> <release version="v4.7.0" date="2018-09-03"> -- 2.19.0

1 1

[libvirt] [PATCH 0/7] Various Coverity based concerns
by John Ferlan 01 Oct '18

01 Oct '18

I'm sure it'll be felt one or two could just be false positives, but I have 35-40 of true false positives and it seems at least these go above just noise on the channel. Perhaps the most difficult one to immediately see was the libxl refcnt patch. That involves a little bit of theory and has been in my noise pile for a while until I noted that the @args is being added in a loop to a callback function that just Unref's it when done. So if there was more than 1 IP Address, then all sorts of fun things could happen. Without any change, the Alloc is matched by the Unref, but with the change we add a Ref to match each Unref in the I/O loop and we just ensure the Unref is done for the path that put @args into the I/O callback. I also think the nwfilter patch was "interesting" insomuch as it has my "favorite" 'if (int-value) {' condition. IOW, if not zero, then do something. What became "interesting" is that the virNWFilterIPAddrMapDelIPAddr could return -1 if the virHashLookup on @req->binding->portdevname returned NULL, so when "shrinking" the code to only call the instantiation for/when there was an IP Address found resolves a couple of issues in the code. John Ferlan (7): lxc: Only check @nparams in lxcDomainBlockStatsFlags libxl: Fix possible object refcnt issue tests: Inline a sysconf call for linuxCPUStatsToBuf util: Data overrun may lead to divide by zero tests: Alter logic in testCompareXMLToDomConfig tests: Use STRNEQ_NULLABLE nwfilter: Alter virNWFilterSnoopReqLeaseDel logic src/libxl/libxl_migration.c | 4 ++-- src/lxc/lxc_driver.c | 2 +- src/nwfilter/nwfilter_dhcpsnoop.c | 9 ++++----- src/util/virutil.c | 11 +++++------ tests/commandtest.c | 4 ++-- tests/libxlxml2domconfigtest.c | 11 +++++------ tests/virhostcputest.c | 12 ++++++++++-- 7 files changed, 29 insertions(+), 24 deletions(-) -- 2.17.1

2 8

[libvirt] [PATCH RFC] lxc: Include support to lxc version 3.0 and higher.
by Julio Faracco 01 Oct '18

01 Oct '18

This patch introduce the new settings for LXC 3.0 and higher. The older versions keep the compatibility to deprecated settings for LXC, but after release 3.0, the compatibility was removed. This commit adds the support to the refactored settings. Signed-off-by: Julio Faracco <jcfaracco(a)gmail.com> --- src/lxc/lxc_native.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/lxc/lxc_native.c b/src/lxc/lxc_native.c index cbdec723dd..c0f70ebb7d 100644 --- a/src/lxc/lxc_native.c +++ b/src/lxc/lxc_native.c @@ -200,7 +200,9 @@ lxcSetRootfs(virDomainDefPtr def, int type = VIR_DOMAIN_FS_TYPE_MOUNT; VIR_AUTOFREE(char *) value = NULL; - if (virConfGetValueString(properties, "lxc.rootfs", &value) <= 0) + if (virConfGetValueString(properties, "lxc.rootfs", &value) <= 0 && + /* Legacy config keys were removed after release 3.0. */ + virConfGetValueString(properties, "lxc.rootfs.path", &value) <= 0) return -1; if (STRPREFIX(value, "/dev/")) @@ -1041,7 +1043,9 @@ lxcParseConfigString(const char *config, if (VIR_STRDUP(vmdef->os.init, "/sbin/init") < 0) goto error; - if (virConfGetValueString(properties, "lxc.utsname", &value) <= 0 || + if ((virConfGetValueString(properties, "lxc.utsname", &value) <= 0 && + /* Legacy config keys were removed after release 3.0. */ + virConfGetValueString(properties, "lxc.uts.name", &value) <= 0) || VIR_STRDUP(vmdef->name, value) < 0) goto error; if (!vmdef->name && (VIR_STRDUP(vmdef->name, "unnamed") < 0)) @@ -1051,7 +1055,9 @@ lxcParseConfigString(const char *config, goto error; /* Look for fstab: we shouldn't have it */ - if (virConfGetValue(properties, "lxc.mount")) { + if (virConfGetValue(properties, "lxc.mount") || + /* Legacy config keys were removed after release 3.0. */ + virConfGetValue(properties, "lxc.mount.fstab")) { virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", _("lxc.mount found, use lxc.mount.entry lines instead")); goto error; -- 2.17.1

2 3

[libvirt] [PATCHv4 00/18] Introduce x86 Cache Monitoring Technology (CMT)
by Wang Huaqiang 01 Oct '18

01 Oct '18

This series of patches and the series already been merged introduce the x86 Cache Monitoring Technology (CMT) to libvirt by interacting with kernel resource control (resctrl) interface. CMT is one of the Intel(R) x86 CPU feature which belongs to the Resource Director Technology (RDT). CMT reports the occupancy of the last level cache, which is shared by all CPU cores. In the v1 series, an original and complete feature for CMT was introduced The v2 and v3 patches address the feature for the host capability of CMT. v4 is addressing the feature for monitoring VM vcpu thread set cache occupancy and reporting it through a virsh command. We have serval discussion about the enabling of CMT, please refer to following links for the RFCs. RFCv3 https://www.redhat.com/archives/libvir-list/2018-August/msg01213.html RFCv2 https://www.redhat.com/archives/libvir-list/2018-July/msg00409.html https://www.redhat.com/archives/libvir-list/2018-July/msg01241.html RFCv1 https://www.redhat.com/archives/libvir-list/2018-June/msg00674.html And the merged commits are list as below, for host capability of CMT. 6af8417415508c31f8ce71234b573b4999f35980 8f6887998bf63594ae26e3db18d4d5896c5f2cb4 58fcee6f3a2b7e89c21c1fb4ec21429c31a0c5b8 12093f1feaf8f5023dcd9d65dff111022842183d a5d293c18831dcf69ec6195798387fbb70c9f461 1. About reason why CMT is necessary in libvirt? The perf events of 'CMT, MBML, MBMT' have been phased out since Linux kernel commit c39a0e2c8850f08249383f2425dbd8dbe4baad69, in libvirt the perf based cmt,mbm will not work with the latest linux kernel. These patches add CMT feature to libvirt through kernel resctrlfs interface. 2 Create cache monitoring group (cache monitor). The main interface for creating monitoring group is through XML file. The proposed configuration is like: <cputune> <cachetune vcpus='1'> <cache id='0' level='3' type='code' size='7680' unit='KiB'/> <cache id='1' level='3' type='data' size='3840' unit='KiB'/> + <monitor level='3' vcpus='1'/> </cachetune> <cachetune vcpus='4-7'> + <monitor level='3' vcpus='4-6'/> </cachetune> </cputune> In above XML, created 2 cache resctrl allocation groups and 2 resctrl monitoring groups. The changes of cache monitor will be effective in next booting of VM. 2 Show CMT result through command 'domstats' Adding the interface in qemu to report this information for resource monitor group through command 'virsh domstats --cpu-total'. Below is a typical output: # virsh domstats 1 --cpu-total Domain: 'ubuntu16.04-base' ... cpu.cache.monitor.count=2 cpu.cache.0.name=vcpus_1 cpu.cache.0.vcpus=1 cpu.cache.0.bank.count=2 cpu.cache.0.bank.0.id=0 cpu.cache.0.bank.0.bytes=4505600 cpu.cache.0.bank.1.id=1 cpu.cache.0.bank.1.bytes=5586944 cpu.cache.1.name=vcpus_4-6 cpu.cache.1.vcpus=4,5,6 cpu.cache.1.bank.count=2 cpu.cache.1.bank.0.id=0 cpu.cache.1.bank.0.bytes=17571840 cpu.cache.1.bank.1.id=1 cpu.cache.1.bank.1.bytes=29106176 Changes in v4: v4 is addressing the feature for monitoring VM vcpu thread set cache occupancy and reporting it through a virsh command. - Introduced resctrl default allocation - Introduced resctrl monitor and default monitor Changes in v3: - Addressed John Ferlan's review. - Typo fixed. - Removed VIR_ENUM_DECL(virMonitor); Changes in v2: - Introduced MBM capability. - Capability layout changed * Moved <monitor> from cahe <bank> to <cache> * Renamed <Threshold> to <reuseThreshold> - Document for 'reuseThreshold' changed. - Introduced API virResctrlInfoGetMonitorPrefix - Added more tests, covering standalone CMT, fake new feature. - Creating CMT resource control group will be subsequent job. Wang Huaqiang (18): docs: Refactor schemas to support default allocation util: Introduce resctrl monitor for CMT util: Refactor code for adding PID to the resource group util: Add interface for adding PID to monitor util: Refactor code for determining allocation path util: Add monitor interface to determine path util: Refactor code for creating resctrl group util: Add interface for creating monitor group util: Add more interfaces for resctrl monitor util: Introduce default monitor conf: Refactor code for matching existing resctrls conf: Refactor virDomainResctrlAppend conf: Add resctrl monitor configuration Util: Add function for checking if monitor is running qemu: enable resctrl monitor in qemu conf: Add a 'id' to virDomainResctrlDef qemu: refactor qemuDomainGetStatsCpu qemu: Report cache occupancy (CMT) with domstats docs/schemas/domaincommon.rng | 14 +- src/conf/domain_conf.c | 327 ++++++++++-- src/conf/domain_conf.h | 12 + src/libvirt_private.syms | 12 + src/qemu/qemu_driver.c | 272 +++++++++- src/qemu/qemu_process.c | 41 +- src/util/virresctrl.c | 555 +++++++++++++++++++-- src/util/virresctrl.h | 49 ++ tests/genericxml2xmlindata/cachetune-cdp.xml | 3 + .../cachetune-colliding-monitor.xml | 30 ++ tests/genericxml2xmlindata/cachetune-small.xml | 7 + tests/genericxml2xmltest.c | 2 + 12 files changed, 1224 insertions(+), 100 deletions(-) create mode 100644 tests/genericxml2xmlindata/cachetune-colliding-monitor.xml -- 2.7.4

1 18