November 2017 - Devel - libvirt List Archives

[libvirt] [PATCH 00/12] qemu: block: Prepare for user-specified backing chains (blockdev-add saga)
by Peter Krempa 03 Nov '17

03 Nov '17

When users will specify backing chain, we need to take into account what was passed in them and/or inherit the data from the parents as we are copying the data (labels, etc...) from the parent disk source. Peter Krempa (12): storage: Extract common code to retrieve driver backend for support check storage: Add feature check for storage file backend supporting access check storage: Extract error reporting for broken chains security: selinux: Pass parent storage source into image labeling helper security: dac: Take parent security label into account security: selinux: Take parent security label into account qemu: domain: Simplify using DAC permissions of top of backing chain qemu: domain: Extract setup for disk source secrets qemu: domain: Destroy secrets for complete backing chain qemu: domain: Remove pointless alias check qemu: domain: Prepare TLS data for the whole backing chain qemu: domain: skip chain detection to end of backing chain src/qemu/qemu_domain.c | 177 ++++++++++++++++++++++++++-------------- src/qemu/qemu_domain.h | 6 +- src/qemu/qemu_driver.c | 6 +- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_process.c | 2 +- src/security/security_dac.c | 38 +++++++-- src/security/security_selinux.c | 26 +++--- src/storage/storage_source.c | 110 ++++++++++++++++--------- src/storage/storage_source.h | 5 ++ 9 files changed, 246 insertions(+), 126 deletions(-) -- 2.14.1

2 26

[libvirt] storage lock issues
by Cedric Bosdonnat 02 Nov '17

02 Nov '17

Hi all, I'm investigating a crash in the storage driver, and it seems the solution is not as obvious as I wanted it. Basically what happens is that libvirtd crashes due to the pool refresh thread clearing the list of pool volumes while someone is adding a new volume. Here is the valgrind log I have for that: http://paste.opensuse.org/58733866 I tried adding a call to virStoragePoolObjLock() in the virStorageVolPoolRefreshThread(), since the storageVolCreateXML() appears to have it, but I'm getting a dead lock. Here are the waiting threads I got from gdb: http://paste.opensuse.org/45961070 Any idea what would be a proper fix for that? My vision of the storage drivers locks is too limited it seems ;) -- Cedric

2 5

[libvirt] [PATCH 00/12] qemu: Add support for more migration parameters
by Jiri Denemark 02 Nov '17

02 Nov '17

QEMU is transforming existing special migration parameters (those which need dedicated QMP commands to be set or queried) into proper parameters handled by query-migrate-parameters and migrate-set-parameters. Even though we may still want to use the existing commands adding support for tha transformed parameters will help us clean all of them before migration and reset them to their original values after a failed migration. Thus we wouldn't need to add more ad-hoc code which resets some of them and ignores some others. Jiri Denemark (12): qemu: Generalize PARSE macro in qemuMonitorJSONGetMigrationParams qemu: Use macro for parsing string migration parameters qemu: Use macro for parsing ull migration parameters qemu: Generalize APPEND macro in qemuMonitorJSONSetMigrationParams qemu: Use macro for setting string migration parameters qemu: Drop giant if statement from qemuMonitorSetMigrationParams qemumonitorjsontest: Rename 1st CHECK macro in migration params test qemumonitorjsontest: Rename 2nd CHECK macro in migration params test qemu: Add support for setting downtime-limit migration parameter qemu: Rename TLS related migration parameters qemu: Add support for max-bandwidth migration parameter qemu: Add support for block-incremental migration parameter src/qemu/qemu_migration.c | 23 +++++----- src/qemu/qemu_monitor.c | 20 +++------ src/qemu/qemu_monitor.h | 10 ++++- src/qemu/qemu_monitor_json.c | 104 +++++++++++++++++++++++++++---------------- tests/qemumonitorjsontest.c | 46 ++++++++++++------- 5 files changed, 123 insertions(+), 80 deletions(-) -- 2.14.3

2 14

[libvirt] [PATCH] Post-release version bump to 3.10.0
by Jiri Denemark 02 Nov '17

02 Nov '17

Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- Pushed. configure.ac | 2 +- docs/news.xml | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index ebe3318e0..b2d991c3b 100644 --- a/configure.ac +++ b/configure.ac @@ -16,7 +16,7 @@ dnl You should have received a copy of the GNU Lesser General Public dnl License along with this library. If not, see dnl <http://www.gnu.org/licenses/>. -AC_INIT([libvirt], [3.9.0], [libvir-list(a)redhat.com] [], [https://libvirt.org] +AC_INIT([libvirt], [3.10.0], [libvir-list(a)redhat.com] [], [https://libvirt.org] AC_CONFIG_SRCDIR([src/libvirt.c]) AC_CONFIG_AUX_DIR([build-aux]) AC_CONFIG_HEADERS([config.h]) diff --git a/docs/news.xml b/docs/news.xml index c24cc3f99..ef855d895 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -33,6 +33,14 @@ --> <libvirt> + <release version="v3.10.0" date="unreleased"> + <section title="New features"> + </section> + <section title="Improvements"> + </section> + <section title="Bug fixes"> + </section> + </release> <release version="v3.9.0" date="2017-11-02"> <section title="New features"> <change> -- 2.14.3

1 0

[libvirt] Release of libvirt-3.9.0
by Daniel Veillard 02 Nov '17

02 Nov '17

It's out ! I tagged it on git and pushed signed tarball and rpms to the usual place: ftp://libvirt.org/libvirt/ I also made a 3.9.0 release of libvirt-python but it's virtually equivalent to 3.8.0 as no commit were made last month in that module. People are strongly encouraged to upgrade, as 3.9.0 includes a security fix, there is also a reasonable amount of user visible new features, improvement and bug fixes as usual: Security: - qemu: Ensure TLS clients always verify the server certificate While it's reasonable to turn off client certificate validation, as setting it up can be non-trivial, clients should always verify the server certificate to avoid MITM attacks. However, libvirt was using the same knob to control both checks, leading to CVE-2017-1000256 / LSN-2017-0002. New features: - Add capability to allow hot (un)plug of a domain watchdog device - Allow users to set device aliases Users can set aliases to domain devices and thus identify them easily. - qemu: Support multiqueue for virtio-blk Multiqueue support for virtio-blk has been available in QEMU ever since 2.7.0, and now libvirt guests can enable it. - Add virDomainSetLifecycleAction API Provided a new API to allow dynamic guest lifecycle control for guest reactions to poweroff, restart, or crash type events related to the domain XML on_poweroff, on_reboot, and on_crash elements. The virsh set-lifecycle-action command was created to control the actions. - qemu: Allow cold(un)plugging and hot(un)plugging input devices - net: Implement QoS for vhostuser Improvements: - Allow a logical volume to be create using LUKS A logical volume may be created using an encryption element using "luks" format. This does require a previously created secret to store the passphrase used to encrypt the volume Adding the volume to a domain can then either provide the secret or allow the consumer in the guest to provide the passphrase in order to decrypt the volume. - net: Ignore auto-generated MAC address when detaching an interface If the MAC address has not been specified by the user, libvirt will try and fill in the gaps by generating one; however, for some error paths that led to some confusing error messages, so when an auto-generated MAC address is specified the error message will not include the auto-generated MAC. - net: Enable MAC address lookup for virDomainInterfaceStats - apparmor: Several improvements Changes include permitting access to data about USB devices and dnsmasq instances, allowing spaces in guest names and many more. - cpu: Use CPU information obtained from QEMU when possible Recent QEMU versions can expose information about which CPU models are available and usable on the host; libvirt will now make use of such information whenever possible. - hyperv: Various improvements The error reported when clients can't connect to Hyper-V has been made more descriptive, and memory limits for guests are now mapped to more appropriate libvirt equivalents. - qemu: Report QEMU error on failed migration Instead of reporting a generic error, ask QEMU for a more detailed and thus hopefully more helpful one. - vbox: Implement autoport for RDP libvirt will now obtain the (dynamically allocated) RDP port number from VirtualBox itself, avoiding conflicts between multiple guests wanting to use RDP at the same time. - qemu: Allow rotation of small logs On a host where numerous unique instances are executed per day, it's quite possible that, even though each of the single log files are fairly small, collectively the quantity and volume may add tens of thousands of log files to the /var/log/libvirt/qemu/ directory. Removing the constraints that log have to be bigger than 100 KiB before they can be rotated solves the issue. Bug fixes: - Fix swapped interface statistics and QoS Due to internal implementation, reported statistics for some types of interfaces were swapped (RX appeared in TX and vice versa). Similarly, QoS was set in reversed way. - Properly resize local LUKS encrypted volume Resizing of a local LUKS encrypted volume will now use qemu-img to resize the volume. This will require configuring a secret for the LUKS encrypted volume. - qemu: Reserve PCI addresses for implicit i440fx devices Failing to do so causes the addresses to be considered usable by libvirt, which means they could be assigned to more than one device resulting in the guest failing to start. - spec: Restart libvirtd only at the end of the upgrade process Use %posttrans to make sure libvirtd is not restarted before all other components, such as the library itself and storage / hypervisor drivers, have already been upgraded. Thanks everybody for your help with this release, be it with patches, bug reports, ideas, reviews, docs, etc... Enjoy ! Daniel -- Daniel Veillard | Red Hat Developers Tools http://developer.redhat.com/ veillard(a)redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/

1 0

Re: [libvirt] networking problem
by msliu＠reynoldstocks.com 02 Nov '17

02 Nov '17

Hi Daniel, # ls /usr/share/libvirt/ cpu_map.xml libvirtLogo.png schemas No "network" folder found # locate default.xml /etc/libvirt/storage/default.xml /etc/libvirt/storage/autostart/default.xml /home/satimis/.config/libvirt/storage/default.xml /home/satimis/.config/libvirt/storage/autostart/default.xml /usr/share/gutenprint/5.2/xml/escp2/inputslots/default.xml Besides # ls /etc/libvirt/qemu/networks/ autostart NAT.xml # ls /etc/libvirt/qemu/networks/autostart/ NAT.xml There are 2 NAT.xml file of same content # cat /etc/libvirt/qemu/networks/NAT.xml  <network> <name>NAT</name> <uuid>2b550fe1-cff0-476e-aabe-f8a239231315</uuid> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:a6:37:e4'/> <domain name='NAT'/> <ip address='192.168.100.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.100.128' end='192.168.100.254'/> </dhcp> </ip> </network> # cat /etc/libvirt/qemu/networks/autostart/NAT.xml  <network> <name>NAT</name> <uuid>2b550fe1-cff0-476e-aabe-f8a239231315</uuid> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:a6:37:e4'/> <domain name='NAT'/> <ip address='192.168.100.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.100.128' end='192.168.100.254'/> </dhcp> </ip> </network> Regards SL -------- Original Message -------- Subject: Re: [libvirt] networking problem From: "Daniel P. Berrange" <berrange(a)redhat.com> Date: Thu, November 02, 2017 9:31 am To: Stefan Hajnoczi <stefanha(a)gmail.com> Cc: msliu(a)reynoldstocks.com, libvir-list(a)redhat.com, kvm(a)vger.kernel.org On Thu, Nov 02, 2017 at 09:24:22AM +0000, Stefan Hajnoczi wrote: > On Sun, Oct 29, 2017 at 04:07:09AM -0700, msliu(a)reynoldstocks.com wrote: > > I have performed following steps: > > > > $ virsh net-destroy default > > $ virsh net-undefine default > > > > Now I couldn't start guest with following warning popup: > > Error starting domain: Network not found: no network with matching name > > 'default' > > > > Please advise how to re-create 'default' instead of reinstalling all > > guests. > > This is a libvirt question. I have CCed the libvirt mailing list so you > can continue discussion there. > > On Red Hat-based distros try reinstalling the > libvirt-daemon-config-network package. On Debian-based distros try > reinstalling the libvirt-daemon-system package. This should restore the > /etc/libvirt/qemu/networks/default.xml file that you are missing. No need to re-install those RPMs - Libvirt leaves a copy of the stock XML file at /usr/share/libvirt/networks/default.xml. So just run virsh define /usr/share/libvirt/networks/default.xml virsh start default Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

1 0

[libvirt] [jenkins-ci PATCH 0/2] fix libosinfo rpm build
by Pavel Hrdina 02 Nov '17

02 Nov '17

Pavel Hrdina (2): jobs: rename check_env into job_env jobs: use job_env in all job templates jobs/autotools.yaml | 5 ++++- jobs/defaults.yaml | 2 +- jobs/perl-makemaker.yaml | 3 +++ projects/libosinfo.yaml | 2 +- projects/libvirt.yaml | 2 +- 5 files changed, 10 insertions(+), 4 deletions(-) -- 2.13.6

3 10

[libvirt] [PATCH v5 0/4] numa: describe sibling nodes distances
by Wim Ten Have 02 Nov '17

02 Nov '17

From: Wim ten Have <wim.ten.have(a)oracle.com> This patch extends guest domain administration adding support to advertise node sibling distances when configuring NUMA guests also referred to as vNUMA (Virtual NUMA). NUMA (Non-Uniform Memory Access), a method of configuring a cluster of nodes within a single multiprocessing system such that it shares processor local memory amongst others improving performance and the ability of the system to be expanded. A NUMA system could be illustrated as shown below. Within this 4-NODE system, every socket is equipped with its own distinct memory and some with I/O. Access to memory or I/O on remote nodes is only possible through the "Interconnect". This results in different performance for local and remote resources. In contrast to NUMA we recognize the flat SMP system where no concept of local or remote resource exists. The disadvantage of high socket count SMP systems is that the shared bus can easily become a performance bottleneck under high activity. +-------------+-------+ +-------+-------------+ |NODE0| | | | | |NODE3| | | CPU00 | CPU03 | | CPU12 | CPU15 | | | | | | | | | | | Mem +--- Socket0 ---<-------->--- Socket3 ---+ Mem | | | | | | | | | +-----+ CPU01 | CPU02 | | CPU13 | CPU14 | | | I/O | | | | | | | +-----+-------^-------+ +-------^-------+-----+ | | | Interconnect | | | +-------------v-------+ +-------v-------------+ |NODE1| | | | | |NODE2| | | CPU04 | CPU07 | | CPU08 | CPU11 | | | | | | | | | | | Mem +--- Socket1 ---<-------->--- Socket2 ---+ Mem | | | | | | | | | +-----+ CPU05 | CPU06 | | CPU09 | CPU10 | | | I/O | | | | | | | +-----+-------+-------+ +-------+-------+-----+ NUMA adds an intermediate level of memory shared amongst a few cores per socket as illustrated above, so that data accesses do not have to travel over a single bus. Unfortunately the way NUMA does this adds its own limitations. This, as visualized in the illustration above, happens when data is stored in memory associated with Socket2 and is accessed by a CPU (core) in Socket0. The processors use the "Interconnect" path to access resource on other nodes. These "Interconnect" hops add data access delays. It is therefore in our interest to describe the relative distances between nodes. The relative distances between nodes are described in the system's SLIT (System Locality Distance Information Table) which is part of the ACPI (Advanced Configuration and Power Interface) specification. On Linux systems the SLIT detail can be listed with help of the 'numactl -H' command. The above guest would show the following output. [root@f25 ~]# numactl -H available: 4 nodes (0-3) node 0 cpus: 0 1 2 3 node 0 size: 2007 MB node 0 free: 1931 MB node 1 cpus: 4 5 6 7 node 1 size: 1951 MB node 1 free: 1902 MB node 2 cpus: 8 9 10 11 node 2 size: 1998 MB node 2 free: 1910 MB node 3 cpus: 12 13 14 15 node 3 size: 2015 MB node 3 free: 1907 MB node distances: node 0 1 2 3 0: 10 21 31 21 1: 21 10 21 31 2: 31 21 10 21 3: 21 31 21 10 These patches extend core libvirt's XML description of NUMA cells to include NUMA distance information and propagate it to Xen guests via libxl. Recently qemu landed support for constructing the SLIT since commit 0f203430dd ("numa: Allow setting NUMA distance for different NUMA nodes"). The core libvirt extensions in this patch set could be used to propagate NUMA distances to qemu quests in the future. Wim ten Have (4): numa: describe siblings distances within cells xenconfig: add domxml conversions for xen-xl libxl: vnuma support xlconfigtest: add tests for numa cell sibling distances docs/formatdomain.html.in | 63 +++- docs/schemas/basictypes.rng | 7 + docs/schemas/cputypes.rng | 18 ++ src/conf/numa_conf.c | 359 ++++++++++++++++++++- src/conf/numa_conf.h | 20 ++ src/libvirt_private.syms | 5 + src/libxl/libxl_conf.c | 121 +++++++ src/libxl/libxl_driver.c | 3 +- src/xenconfig/xen_xl.c | 333 +++++++++++++++++++ .../test-fullvirt-vnuma-autocomplete.cfg | 26 ++ .../test-fullvirt-vnuma-autocomplete.xml | 85 +++++ .../test-fullvirt-vnuma-nodistances.cfg | 26 ++ .../test-fullvirt-vnuma-nodistances.xml | 53 +++ .../test-fullvirt-vnuma-partialdist.cfg | 26 ++ .../test-fullvirt-vnuma-partialdist.xml | 60 ++++ tests/xlconfigdata/test-fullvirt-vnuma.cfg | 26 ++ tests/xlconfigdata/test-fullvirt-vnuma.xml | 81 +++++ tests/xlconfigtest.c | 6 + 18 files changed, 1313 insertions(+), 5 deletions(-) create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-autocomplete.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-autocomplete.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-nodistances.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-partialdist.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma-partialdist.xml create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.cfg create mode 100644 tests/xlconfigdata/test-fullvirt-vnuma.xml -- 2.13.6

4 13

Re: [libvirt] networking problem
by msliu＠reynoldstocks.com 02 Nov '17

02 Nov '17

1 0

[libvirt] [PATCH] news: Update for 3.9.0 release
by Andrea Bolognani 02 Nov '17

02 Nov '17

Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- As usual, a lot of changes have made it to master without the release notes being updated at the same time. I'll push this tomorrow in the morning, so that it can make it into the release, unless it gets (N)ACKed earlier. docs/news.xml | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 6a864916d..3cf761d89 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -49,6 +49,36 @@ easily. </description> </change> + <change> + <summary> + qemu: Support multiqueue for virtio-blk + </summary> + <description> + Multiqueue support for <code>virtio-blk</code> has been available + in QEMU ever since 2.7.0, and now libvirt guests can enable it. + </description> + </change> + <change> + <summary> + Add virDomainSetLifecycleAction() API + </summary> + <description> + This new API, also exposed through the + <code>set-lifecycle-action</code> <code>virsh</code> command, allows + the user to dynamically control how the guest will react to being + powered off, being restarted or crashing. + </description> + </change> + <change> + <summary> + qemu: Allow cold(un)plugging and hot(un)plugging input devices + </summary> + </change> + <change> + <summary> + net: Implement QoS for vhostuser + </summary> + </change> </section> <section title="Improvements"> <change> @@ -64,6 +94,82 @@ in order to decrypt the volume. </description> </change> + <change> + <summary> + net: Ignore auto-generated MAC address when detaching an interface + </summary> + <description> + If the MAC address has not been specified by the user, libvirt will + try and fill in the gaps by generating one; however, that leads to + very confusing error messages, so it's not desiderable. + </description> + </change> + <change> + <summary> + net: Enable MAC address lookup for virDomainInterfaceStats() + </summary> + </change> + <change> + <summary> + apparmor: Several improvements + </summary> + <description> + Changes include permitting access to data about USB devices and + <code>dnsmasq</code> instances, allowing spaces in guest names and + many more. + </description> + </change> + <change> + <summary> + cpu: Use CPU information obtained from QEMU when possible + </summary> + <description> + Recent QEMU versions can expose information about which CPU models + are available and usable on the host; libvirt will now make use of + such information whenever possible. + </description> + </change> + <change> + <summary> + hyperv: Various improvements + </summary> + <description> + The error reported when clients can't connect to Hyper-V has been + made more descriptive, and memory limits for guests are now mapped + to more appropriate libvirt equivalents. + </description> + </change> + <change> + <summary> + qemu: Report QEMU error on failed migration + </summary> + <description> + Instead of reporting a generic error, ask QEMU for a more detailed + and thus hopefully more helpful one. + </description> + </change> + <change> + <summary> + vbox: Implement autoport for RDP + </summary> + <description> + libvirt will now obtain the (dynamically allocated) RDP port number + from VirtualBox itself, avoiding conflicts between multiple guests + wanting to use RDP at the same time. + </description> + </change> + <change> + <summary> + qemu: Allow rotation of small logs + </summary> + <description> + On a host where tons of unique instances are executed per day, it's + quite possible that while each of the single log files are quite + small, they will build up to quite a bit of clutter. Removing the + constraints that log have to be bigger than 100 KiB before they can + be rotated solves the issue. + </description> + </change> </section> <section title="Bug fixes"> <change> @@ -87,6 +193,38 @@ the LUKS encrypted volume. </description> </change> + <change> + <summary> + qemu: Reserve PCI addresses for implicit i440fx devices + </summary> + <description> + Failing to do so causes the addresses to be considered usable by + libvirt, which means they could be assigned to more than one device + resulting in the guest failing to start. + </description> + </change> + <change> + <summary> + qemu: Ensure TLS clients always verify the server certificate + </summary> + <description> + While it's reasonable to turn off client certificate validation, + as setting it up can be non-trivial, clients should always verify + the server certificate to avoid MITM attacks. libvirt was, however, + using the same knob to control both checks, leading to + CVE-2017-1000256 / LSN-2017-0002. + </description> + </change> + <change> + <summary> + spec: Restart libvirtd only at the end of the upgrade process + </summary> + <description> + Use <code>%posttrans</code> to make sure <code>libvirtd</code> + is not restarted before all other components, such as the library + itself and storage / hypervisor drivers, have already been upgraded. + </description> + </change> </section> </release> <release version="v3.8.0" date="2017-10-04"> -- 2.13.6

2 3