January 2019 - Devel - libvirt List Archives

[libvirt] [PATCH] virPortAllocatorSetUsed: ignore port 0
by Ján Tomko 24 Jan '19

24 Jan '19

Similar to what commit 86dba8f3 did for virPortAllocatorRelease, ignore port 0 in virPortAllocatorSetUsed. For all the reasonable use cases the callers already check that the port is non-zero, however if the port from the XML overflows unsigned short and turns into 0, it can be set as used by virPortAllocatorSetUsed but not released by virPortAllocatorRelease. Also skip port '0' in virPortAllocatorSetUsed to make this behavior symmetric. The serenity was disturbed by commit 5dbda5e9 which started using virPortAllocatorRelease instead of virPortAllocatorSetUsed (false). https://bugzilla.redhat.com/show_bug.cgi?id=1591645 Signed-off-by: Ján Tomko <jtomko(a)redhat.com> --- src/util/virportallocator.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/util/virportallocator.c b/src/util/virportallocator.c index db95a601c7..d48963c1ff 100644 --- a/src/util/virportallocator.c +++ b/src/util/virportallocator.c @@ -294,6 +294,9 @@ virPortAllocatorSetUsed(unsigned short port) if (!pa) return -1; + if (!port) + return 0; + virObjectLock(pa); if (virBitmapIsBitSet(pa->bitmap, port) || -- 2.16.4

2 1

[libvirt] [PATCH] docs: Add more iscsi-direct references to storage pages
by John Ferlan 24 Jan '19

24 Jan '19

Found that it was missing in formatstorage and had a few typos in the storage driver page. Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- noted while working on other stuff... Also although <initiator> is supported by the "iscsi" pool as well, it's not something ever documented since commit 6aabcb5bd (0.7.5). I could add a note that it's optional to use with the "iscsi" pool if desired. docs/formatstorage.html.in | 20 ++++++++++++++++---- docs/storage.html.in | 8 ++++---- 2 files changed, 20 insertions(+), 8 deletions(-) diff --git a/docs/formatstorage.html.in b/docs/formatstorage.html.in index b6bf3edbd2..979e170c08 100644 --- a/docs/formatstorage.html.in +++ b/docs/formatstorage.html.in @@ -18,7 +18,9 @@ The top level tag for a storage pool document is 'pool'. It has a single attribute <code>type</code>, which is one of <code>dir</code>, <code>fs</code>, <code>netfs</code>, <code>disk</code>, - <code>iscsi</code>, <code>logical</code>, <code>scsi</code> + <code>iscsi</code>, + <code>iscsi-direct</code>(<span class="since">since 4.7.0</span>), + <code>logical</code>, <code>scsi</code> (all <span class="since">since 0.4.1</span>), <code>mpath</code> (<span class="since">since 0.7.1</span>), <code>rbd</code> (<span class="since">since 0.9.13</span>), <code>sheepdog</code> @@ -125,11 +127,12 @@ <dt><code>device</code></dt> <dd>Provides the source for pools backed by physical devices (pool types <code>fs</code>, <code>logical</code>, <code>disk</code>, - <code>iscsi</code>, <code>zfs</code>, <code>vstorage</code>). + <code>iscsi</code>, <code>iscsi-direct</code>, <code>zfs</code>, + <code>vstorage</code>). May be repeated multiple times depending on backend driver. Contains a required attribute <code>path</code> which is either the fully qualified path to the block device node or for <code>iscsi</code> - the iSCSI Qualified Name (IQN). + or <code>iscsi-direct</code> the iSCSI Qualified Name (IQN). <span class="since">Since 0.4.1</span> <p>An optional attribute <code>part_separator</code> for each <code>path</code> may be supplied. Valid values for the attribute @@ -334,6 +337,7 @@ <dt><code>host</code></dt> <dd>Provides the source for pools backed by storage from a remote server (pool types <code>netfs</code>, <code>iscsi</code>, + <code>iscsi-direct</code>, <code>rbd</code>, <code>sheepdog</code>, <code>gluster</code>). Will be used in combination with a <code>directory</code> or <code>device</code> element. Contains an attribute <code>name</code> @@ -348,11 +352,19 @@ server. See the <a href="storage.html">storage driver page</a> for any restrictions for specific storage backends. <span class="since">Since 0.4.1</span></dd> + <dt><code>initiator</code></dt> + <dd>Required by the <code>iscsi-direct</code> pool in order to provide + the iSCSI Qualified Name (IQN) to communicate with the pool's + <code>device</code> target IQN. There is one sub-element + <code>iqn</code> with the <code>name</code> attribute to describe + the IQN for the initiator. + <span class="since">Since 4.7.0</span></dd> <dt><code>auth</code></dt> <dd>If present, the <code>auth</code> element provides the authentication credentials needed to access the source by the setting of the <code>type</code> attribute (pool - types <code>iscsi</code>, <code>rbd</code>). The <code>type</code> + types <code>iscsi</code>, <code>iscsi-direct</code>, <code>rbd</code>). + The <code>type</code> must be either "chap" or "ceph". Use "ceph" for Ceph RBD (Rados Block Device) network sources and use "iscsi" for CHAP (Challenge-Handshake Authentication Protocol) iSCSI diff --git a/docs/storage.html.in b/docs/storage.html.in index 9adcc2a87f..c8617aace5 100644 --- a/docs/storage.html.in +++ b/docs/storage.html.in @@ -437,9 +437,9 @@ <h2><a id="StorageBackendISCSIDirect">iSCSI direct pool</a></h2> <p> - This is a variant of the iSCSI pool. Instead of unsing iscsiadm, it uses + This is a variant of the iSCSI pool. Instead of using iscsiadm, it uses libiscsi. - It require a host, a path which is the target iqn and an initiator iqn. + It requires a host, a path which is the target IQN, and an initiator IQN. </p> <h3>Example pool input</h3> @@ -457,12 +457,12 @@ <h3>Valid pool format types</h3> <p> - The iSCSI volume pool does not use the pool format type element. + The iSCSI direct volume pool does not use the pool format type element. </p> <h3>Valid volume format types</h3> <p> - The iSCSI volume pool does not use the volume format type element. + The iSCSI direct volume pool does not use the volume format type element. </p> <h2><a id="StorageBackendSCSI">SCSI pool</a></h2> -- 2.20.1

2 1

[libvirt] [PATCH v3 00/11] Allow adding mountOpts to the storage pool mount command
by John Ferlan 24 Jan '19

24 Jan '19

v2: https://www.redhat.com/archives/libvir-list/2019-January/msg00164.html Changes since v2: * Split out the noexec, nodev, nosuid, and (new) ro into the first 3 patches as "known" or "supported" mount options that can be added. I avoided just adding them noexec, nodev, and nosuid by default. * Add patches 4-6 to essentially support the nfsvers option * The rest of the patches are reworked to add support for namespace at the pool level (as opposed to the v2 at the source level). John Ferlan (11): conf: Add some defined NFS Storage Pool mount options storage: Add mountOpts to the storage pool mount command line virsh: Add source-mount-opts for pool commands conf: Add optional NFS Source Pool <protocol ver='n'/> option storage: Add the nfsvers to the command line virsh: Add source-protocol-ver for pool commands conf: Introduce virStoragePoolXMLNamespace nfs: Add infrastructure to manage XML namespace options docs,tests: Add schema, description, and tests for NFS namespace storage: Add NFS storage pool namespace options to command line rbd: Utilize storage pool namespace to manage config options docs/formatstorage.html.in | 137 +++++++++++++++ docs/schemas/storagepool.rng | 70 ++++++++ src/conf/storage_conf.c | 120 ++++++++++++- src/conf/storage_conf.h | 40 +++++ src/libvirt_private.syms | 1 + src/storage/storage_backend_fs.c | 128 ++++++++++++++ src/storage/storage_backend_rbd.c | 166 +++++++++++++++++- src/storage/storage_util.c | 47 +++++ src/storage/storage_util.h | 14 ++ tests/Makefile.am | 4 +- .../pool-netfs-mountopts.argv | 1 + .../pool-netfs-ns-mountopts.argv | 2 + .../pool-netfs-protocol-ver.argv | 1 + tests/storagepoolxml2argvtest.c | 6 + .../pool-netfs-mountopts.xml | 24 +++ .../pool-netfs-ns-mountopts.xml | 29 +++ .../pool-netfs-protocol-ver.xml | 21 +++ .../pool-rbd-ns-configopts.xml | 17 ++ .../pool-netfs-mountopts.xml | 24 +++ .../pool-netfs-ns-mountopts.xml | 29 +++ .../pool-netfs-protocol-ver.xml | 21 +++ .../pool-rbd-ns-configopts.xml | 20 +++ tests/storagepoolxml2xmltest.c | 9 + tools/virsh-pool.c | 44 ++++- tools/virsh.pod | 10 ++ 25 files changed, 977 insertions(+), 8 deletions(-) create mode 100644 tests/storagepoolxml2argvdata/pool-netfs-mountopts.argv create mode 100644 tests/storagepoolxml2argvdata/pool-netfs-ns-mountopts.argv create mode 100644 tests/storagepoolxml2argvdata/pool-netfs-protocol-ver.argv create mode 100644 tests/storagepoolxml2xmlin/pool-netfs-mountopts.xml create mode 100644 tests/storagepoolxml2xmlin/pool-netfs-ns-mountopts.xml create mode 100644 tests/storagepoolxml2xmlin/pool-netfs-protocol-ver.xml create mode 100644 tests/storagepoolxml2xmlin/pool-rbd-ns-configopts.xml create mode 100644 tests/storagepoolxml2xmlout/pool-netfs-mountopts.xml create mode 100644 tests/storagepoolxml2xmlout/pool-netfs-ns-mountopts.xml create mode 100644 tests/storagepoolxml2xmlout/pool-netfs-protocol-ver.xml create mode 100644 tests/storagepoolxml2xmlout/pool-rbd-ns-configopts.xml -- 2.20.1

3 15

[libvirt] [PATCH 00/11] qemu: format CCID controllers after USB hubs
by Ján Tomko 24 Jan '19

24 Jan '19

https://bugzilla.redhat.com/show_bug.cgi?id=1599971 Ján Tomko (11): qemu: move out legacy USB controller formatting qemu: exit early if USB_CONTROLLER_MODEL_NONE is present Add qemuBuildDomainForbidLegacyUSBController qemu: separate counting of legacy USB controllers qemu: separate counting of USB controllers qemuBuildControllerDevStr: remove nusbcontroller argument qemu: Introduce qemuBuildControllersByTypeCommandLine qemu: format CCID controllers after USB hubs qemuBuildControllersByTypeCommandLine: free devstr in the cleanup section rename qemuBuildControllerDevCommandLine qemuBuildControllersCommandLine: use i instead of j as the counter src/qemu/qemu_command.c | 222 +++++++++++------- src/qemu/qemu_command.h | 3 +- src/qemu/qemu_hotplug.c | 2 +- tests/qemuxml2argvdata/chardev-reconnect.args | 2 +- tests/qemuxml2argvdata/controller-order.args | 2 +- tests/qemuxml2argvdata/name-escape.args | 2 +- .../smartcard-controller.args | 2 +- .../smartcard-host-certificates-database.args | 2 +- .../smartcard-host-certificates.args | 2 +- tests/qemuxml2argvdata/smartcard-host.args | 2 +- .../smartcard-passthrough-spicevmc.args | 2 +- .../smartcard-passthrough-tcp.args | 2 +- tests/qemuxml2argvdata/user-aliases.args | 2 +- 13 files changed, 150 insertions(+), 97 deletions(-) -- 2.20.1

2 13

[libvirt] [python PATCH 0/3] travis: misc fixes to build
by Daniel P. Berrangé 24 Jan '19

24 Jan '19

Fix the travis build which has been broken for a while Daniel P. Berrangé (3): travis: remove python 2.6 and 3.6 from build matrix travis: switch from trusty to xenial dist travis: add testing of 5.0.0 release .travis.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.20.1

2 4

[libvirt] [PATCH v2 00/17] implement cgroups v2 devices support
by Pavel Hrdina 24 Jan '19

24 Jan '19

In cgroups v2 there is no devices controller, BPF should be used instead. Changes in v2: - fixed build on bsd and older kernels without cgroup BPF - cgroup bpf devices code moved to separate file Documentation for eBPF: <http://man7.org/linux/man-pages/man2/bpf.2.html> <https://www.kernel.org/doc/Documentation/networking/filter.txt> <https://docs.cilium.io/en/v1.3/bpf/> Pavel Hrdina (17): util: introduce virbpf helpers vircgroup: introduce virCgroupV2DevicesAvailable vircgroup: introduce virCgroupV2DevicesAttachProg vircgroup: introduce virCgroupV2DevicesDetectProg vircgroup: introduce virCgroupV2DevicesCreateProg vircgroup: introduce virCgroupV2DevicesPrepareProg vircgroup: introduce virCgroupV2DevicesRemoveProg vircgroup: introduce virCgroupV2DeviceGetPerms vircgroup: introduce virCgroupV2DevicesGetKey vircgroup: introduce virCgroupV2AllowDevice vircgroup: introduce virCgroupV2DenyDevice vircgroup: introduce virCgroupV2AllowAllDevices vircgroup: introduce virCgroupV2DenyAllDevices vircgroup: workaround devices in hybrid mode vircgroupv2: detech BPF program before removing cgroup vircgroupv2: use dummy process to workaround kernel bug with systemd vircgroupmock: mock virCgroupV2DevicesAvailable configure.ac | 6 + include/libvirt/virterror.h | 1 + src/Makefile.am | 2 + src/libvirt_private.syms | 27 ++ src/lxc/lxc_cgroup.c | 1 + src/qemu/qemu_cgroup.c | 6 +- src/util/Makefile.inc.am | 4 + src/util/virbpf.c | 437 +++++++++++++++++++++ src/util/virbpf.h | 271 +++++++++++++ src/util/vircgroup.c | 19 +- src/util/vircgroup.h | 1 + src/util/vircgroupbackend.h | 3 +- src/util/vircgrouppriv.h | 12 + src/util/vircgroupv1.c | 9 +- src/util/vircgroupv2.c | 119 +++++- src/util/vircgroupv2devices.c | 625 ++++++++++++++++++++++++++++++ src/util/vircgroupv2devices.h | 57 +++ src/util/virerror.c | 1 + src/util/virsystemd.c | 2 +- src/util/virsystemd.h | 2 + tests/vircgroupdata/hybrid.parsed | 2 +- tests/vircgroupmock.c | 7 + tests/vircgrouptest.c | 4 +- 23 files changed, 1608 insertions(+), 10 deletions(-) create mode 100644 src/util/virbpf.c create mode 100644 src/util/virbpf.h create mode 100644 src/util/vircgroupv2devices.c create mode 100644 src/util/vircgroupv2devices.h -- 2.20.1

3 25

[libvirt] [PATCH 0/3] Improve block job handling docs
by Peter Krempa 24 Jan '19

24 Jan '19

Modernize documentation for our APIs and clean up some old cruft. Some of those may be slightly controversial. Peter Krempa (3): qemu: Don't reject making domain persistent if block copy is running lib: Clarify that any block job may block VM save or device detach lib: domain: Emphasise that users should wait for block job READY state via events src/libvirt-domain.c | 33 +++++++++++++-------------------- src/qemu/qemu_driver.c | 6 ------ 2 files changed, 13 insertions(+), 26 deletions(-) -- 2.20.1

2 5

[libvirt] [PATCH v2 0/4] Restructure firewall rules for virtual networks into private chains
by Daniel P. Berrangé 24 Jan '19

24 Jan '19

v1: https://www.redhat.com/archives/libvir-list/2018-November/msg00018.html The virtual networks in NAT mode are supposed to only allow outbound network access for guests. Unfortunately due to ordering of the firewall rules libvirt creates, when you have multiple virtual networks, guests on the more recently created virtual networks can connect to guests on old virtual networks. This was reported way back in 2008 but we always thought the fix would be very complicated to deal with, so we've been putting it off forever. In parallel with this there's also been a long standing desire since 2009 to move our firewall rules out of the builtin chains, to libvirt private chains. This is to make it easier for admins to use hook scripts to setup rules in the builtin chains that take priority over rules libvirt creates. In implementing the changes to use private chains, I suddenly realized that fixing the network to network traffic blocking problem was trivial if I grouped the forwarding rules into three distinct sets. So this series finally fixes an annoying 10 year old bug, and implements a 9 year old RFE. It may take us a while, but we'll get to your bugs eventually ;-) Changed in v2: - Detect whether chains already exist before creating them - Only try to delete legacy rules from builtin chain during startup Daniel P. Berrangé (4): network: add platform driver callbacks around firewall reload util: pass layer into firewall query callback util: create private chains for virtual network firewall rules util: move virtual network firwall rules into private chains src/libvirt_private.syms | 2 + src/network/bridge_driver.c | 13 +- src/network/bridge_driver_linux.c | 32 +++ src/network/bridge_driver_nop.c | 11 + src/network/bridge_driver_platform.h | 3 + src/nwfilter/nwfilter_ebiptables_driver.c | 17 +- src/util/virfirewall.c | 2 +- src/util/virfirewall.h | 1 + src/util/viriptables.c | 254 +++++++++++++++--- src/util/viriptables.h | 4 + .../nat-default-linux.args | 32 +-- .../nat-ipv6-linux.args | 48 ++-- .../nat-many-ips-linux.args | 60 ++--- .../nat-no-dhcp-linux.args | 46 ++-- .../nat-tftp-linux.args | 34 +-- .../route-default-linux.args | 22 +- tests/virfirewalltest.c | 3 +- 17 files changed, 405 insertions(+), 179 deletions(-) -- 2.19.2

3 9

[libvirt] [PATCH 0/3] Some random fixes
by Michal Privoznik 24 Jan '19

24 Jan '19

*** BLURB HERE *** Michal Prívozník (3): virPCIGetNetName: Initialize @netname to NULL virpci: Fix memleak in virPCIDeviceIterDevices domain_conf: Free egl render node in virDomainGraphicsDefFree src/conf/domain_conf.c | 3 +++ src/util/virnetdev.c | 2 -- src/util/virpci.c | 4 ++++ 3 files changed, 7 insertions(+), 2 deletions(-) -- 2.19.2

2 8

[libvirt] [PATCH] qemu-nbd: Deprecate qemu-nbd --partition
by Eric Blake 24 Jan '19

24 Jan '19

The existing qemu-nbd --partition code claims to handle logical partitions up to 8, since its introduction in 2008 (commit 7a5ca86). However, the implementation is bogus (actual MBR logical partitions form a sort of linked list, with one partition per extended table entry, rather than four logical partitions in a single extended table), making the code unlikely to work for anything beyond -P5 on actual guest images. What's more, the code does not support GPT partitions, which are becoming more popular, and maintaining device subsetting in both NBD and the raw device is unnecessary maintenance burden. And nbdkit has just added code to properly handle an arbitrary number of MBR partitions, along with its existing code for handling GPT partitions. Note that obtaining the offsets of a partition can be learned by using 'qemu-nbd -c /dev/nbd0 file.qcow2 && sfdisk --dump /dev/nbd0', but by the time you've done that, you might as well just mount /dev/nbd0p1 that the kernel creates for you. Start the clock on the deprecation cycle, with an example of how to write device subsetting without using -P. Signed-off-by: Eric Blake <eblake(a)redhat.com> --- qemu-deprecated.texi | 27 +++++++++++++++++++++++++++ qemu-nbd.texi | 6 ++++-- qemu-nbd.c | 2 ++ 3 files changed, 33 insertions(+), 2 deletions(-) diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi index 219206a836f..12f8b30943f 100644 --- a/qemu-deprecated.texi +++ b/qemu-deprecated.texi @@ -175,3 +175,30 @@ The above, converted to the current supported format: @subsubsection "irq": "" (since 3.0.0) The ``irq'' property is obsoleted. + +@section Related binaries + +@subsection qemu-nbd --partition (since 4.0.0) + +The ``qemu-nbd --partition $digit'' code (also spelled @option{-P}) +can only handle MBR partitions, and does not correctly handle logical +partitions beyond partition 5. If you know the relative position of +the partition (perhaps by using @code{sfdisk} or similar, either in +the guest or when mapping the entire device through /dev/nbd0 in the +host), you can achieve the effect of exporting just that subset of the +disk by use of the @option{--image-opts} option with a raw blockdev +using the @code{offset} and @code{size} parameters layered on top of +any other existing blockdev. + +For example, if partition 1 is 100MiB starting at 1MiB, the old command + +@example{qemu-nbd -P 1 -f qcow2 file.qcow2} + +can be rewritten as: + +@example{qemu-nbd --image-opts driver=raw,offset=1M,size=100M,file.driver=qcow2,file.backing.driver=file,file.backing.filename=file.qcow2} + +Alternatively, the @code{nbdkit} project provides a more powerful +partition filter on top of its nbd plugin, which can be used to select +an arbitrary MBR or GPT partition on top of any other full-image NBD +export. diff --git a/qemu-nbd.texi b/qemu-nbd.texi index 386bece4680..d0c51828149 100644 --- a/qemu-nbd.texi +++ b/qemu-nbd.texi @@ -56,8 +56,10 @@ auto-detecting. @item -r, --read-only Export the disk as read-only. @item -P, --partition=@var{num} -Only expose MBR partition @var{num}. Understands physical partitions -1-4 and logical partitions 5-8. +Deprecated: Only expose MBR partition @var{num}. Understands physical +partitions 1-4 and logical partition 5. New code should instead use +@option{--image-opts} with the raw driver wrapping a subset of the +original image. @item -B, --bitmap=@var{name} If @var{filename} has a qcow2 persistent bitmap @var{name}, expose that bitmap via the ``qemu:dirty-bitmap:@var{name}'' context diff --git a/qemu-nbd.c b/qemu-nbd.c index 1f7b2a03f5d..00c07fd27ea 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@ -787,6 +787,8 @@ int main(int argc, char **argv) flags &= ~BDRV_O_RDWR; break; case 'P': + warn_report("The '-P' option is deprecated; use --image-opts with " + "a raw device wrapper for subset exports instead"); if (qemu_strtoi(optarg, NULL, 0, &partition) < 0 || partition < 1 || partition > 8) { error_report("Invalid partition '%s'", optarg); -- 2.20.1

3 4