August 2017 - Devel - libvirt List Archives

[libvirt] [PATCH v5 0/9] Add support for Veritas HyperScale (VxHS) block device protocol
by Ashish Mittal 30 Aug '17

30 Aug '17

QEMU changes for VxHS (including TLS support) are already upstream. This series of patches adds support for VxHS block devices in libvirt. Patch 1 adds the base functionality for supporting VxHS protocol. Patches 2 and 3 add test cases for the base functionality. Patch 4 adds two new configuration options in qemu.conf to enable TLS for VxHS devices. Patch 5 implements the main TLS functionality. Patches 6 through 9 add test cases for the TLS functionality. This series has the following changes - (1) Rebased to latest master. (2) Most of the review comments for patch 1 have been incorporated. (3) Patches have been broken into smaller chunks TODO: Changes in response to review comments on the TLS functionality are still pending and will be addressed next. Ashish Mittal (9): Add support for Veritas HyperScale (VxHS) block device protocol Add a test case to verify generation of qemu command line args for a VxHS disk Add a test case to verify parsing of VxHS backing storage. conf: Introduce TLS options for VxHS block device clients Add TLS support for Veritas HyperScale (VxHS) block device protocol Add a test case to verify TLS arguments are added for VxHS disk Add a test case to verify TLS arguments are parsed correctly for a VxHS disk Add a test case to verify setting vxhs_tls=0 disables TLS for VxHS disks Add a test case to verify different TLS combinations for a VxHS disk docs/formatdomain.html.in | 31 ++++++++- docs/schemas/domaincommon.rng | 18 +++++ src/conf/domain_conf.c | 19 ++++++ src/libxl/libxl_conf.c | 1 + src/qemu/libvirtd_qemu.aug | 4 ++ src/qemu/qemu.conf | 23 +++++++ src/qemu/qemu_block.c | 78 ++++++++++++++++++++++ src/qemu/qemu_command.c | 76 +++++++++++++++++++++ src/qemu/qemu_conf.c | 7 ++ src/qemu/qemu_conf.h | 3 + src/qemu/qemu_driver.c | 3 + src/qemu/qemu_parse_command.c | 15 +++++ src/qemu/test_libvirtd_qemu.aug.in | 2 + src/util/virstoragefile.c | 53 ++++++++++++++- src/util/virstoragefile.h | 10 +++ src/xenconfig/xen_xl.c | 1 + ...ml2argv-disk-drive-network-tlsx509-err-vxhs.xml | 34 ++++++++++ ...-disk-drive-network-tlsx509-multidisk-vxhs.args | 43 ++++++++++++ ...v-disk-drive-network-tlsx509-multidisk-vxhs.xml | 56 ++++++++++++++++ ...muxml2argv-disk-drive-network-tlsx509-vxhs.args | 30 +++++++++ ...emuxml2argv-disk-drive-network-tlsx509-vxhs.xml | 34 ++++++++++ .../qemuxml2argv-disk-drive-network-vxhs.args | 27 ++++++++ .../qemuxml2argv-disk-drive-network-vxhs.xml | 34 ++++++++++ tests/qemuxml2argvtest.c | 10 +++ tests/virstoragetest.c | 21 ++++++ 25 files changed, 629 insertions(+), 4 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-err-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-multidisk-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-multidisk-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-tlsx509-vxhs.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-drive-network-vxhs.xml -- 2.5.5

3 21

Re: [libvirt] [RFC v2] [OVS/NOVA] Vhost-user backends cross-version migration support
by Eduardo Habkost 30 Aug '17

30 Aug '17

I'm CCing libvir-list and qemu-devel because I would like to get feedback from libvirt and QEMU developers too. On Tue, Aug 08, 2017 at 10:49:21PM +0300, Michael S. Tsirkin wrote: > On Tue, Jul 18, 2017 at 03:42:08PM +0200, Maxime Coquelin wrote: > > This is an revival from a thread I initiated earlier this year [0], that > > I had to postpone due to other priorities. > > > > First, I'd like to thanks reviewers of my first proposal, this new > > version tries to address the comments made: > > 1.This is Nova's role and not Libvirt's to query hosts supported > > compatibility modes and to select one, since Nova adds the vhost-user > > ports and has visibility on other hosts. Hence I remove libvirt ML and > > add Openstack one in the recipient list. > > 2. By default, the compatibility version selected is the most recent > > one, except if the admin selects on older compat version. > > > > The goal of this thread is to draft a solution based on the outcomes > > of discussions with contributors of the different parties (DPDK/OVS > > /Nova/...). > > > > I'm really interested on feedback from OVS & Nova contributors, > > as my experience with these projects is rather limited. > > > > Problem statement: > > ================== > > > > When migrating a VM from one host to another, the interfaces exposed by > > QEMU must stay unchanged in order to guarantee a successful migration. > > In the case of vhost-user interface, parameters like supported Virtio > > feature set, max number of queues, max vring sizes,... must remain > > compatible. Indeed, the frontend not being re-initialized, no > > re-negotiation happens at migration time. > > > > For example, we have a VM that runs on host A, which has its vhost-user > > backend advertising VIRTIO_F_RING_INDIRECT_DESC feature. Since the Guest > > also support this feature, it is successfully negotiated, and guest > > transmit packets using indirect descriptor tables, that the backend > > knows to handle. > > > > At some point, the VM is being migrated to host B, which runs an older > > version of the backend not supporting this VIRTIO_F_RING_INDIRECT_DESC > > feature. The migration would break, because the Guest still have the > > VIRTIO_F_RING_INDIRECT_DESC bit sets, and the virtqueue contains some > > decriptors pointing to indirect tables, that backend B doesn't know to > > handle. > > This is just an example about Virtio features compatibility, but other > > backend implementation details could cause other failures. (e.g. > > configurable queues sizes) > > > > What we need is to be able to query the destination host's backend to > > ensure migration is possible before it is initiated. > > This remided me strongly of the issues around the virtual CPU modeling > in KVM, see > https://wiki.qemu.org/index.php/Features/CPUModels#Querying_host_capabiliti… > > QEMU recently gained query-cpu-model-expansion to allow capability queries. > > Cc Eduardo accordingly. Eduardo, could you please take a look - > how is the problem solved on the KVM/VCPU side? Do the above > problem and solution for vhost look similar? (Sorry for taking so long to reply) CPU configuration in QEMU has the additional problem of features depending on host hardware and kernel capabilities (not just QEMU software capabilities). Do you have vhost-user features that depend on the host kernel or hardware too, or all of them just depend on the vhost-user backend software? If it depends only on software, a solution similar to how machine-types work in QEMU sound enough. If features depend on host kernel or host hardware too, it is a bit more complex: it means you need an interface to find out if each configurable feature/version is really available on the host. (In the case of CPU models, we started with an interface that reported which CPU models were runnable on the host. But as libvirt allows enabling/disabling individual CPU features, the interface had to be extended to report which CPU features were available/unavailable on the host.) * * * Now, there's one thing that seems very different here: the guest-visible interface is not defined only by QEMU, but also by the vhost-user backend. Is that correct? This means QEMU won't fully control the resulting guest ABI anymore. I would really prefer if we could keep libvirt+QEMU in control of the guest ABI as usual, making QEMU configure all the guest-visible vhost-user features. But I understand this would require additional interfaces between QEMU and libvirt, and extending the libvirt APIs. So, if QEMU is really not going to control the resulting guest ABI completely, can we at least provide a mechanism which QEMU can use to ask vhost-user for guest ABI details on migration, and block migration if vhost-user was misconfigured on the destination host when migrating? > > > The below proposal has been drafted based on how Qemu manages machine types: > > > > Proposal > > ======== > > > > The idea is to have a table of supported version strings in OVS, > > associated to key/value pairs. Nova or any other management tool could > > query OVS for the list of supported versions strings for each hosts. > > By default, the latest compatibility version will be selected, but the > > admin can select manually an older compatibility mode in order to ensure > > successful migration to an older destination host. > > > > Then, Nova would add OVS's vhost-user port with adding the selected > > version (compatibility mode) as an extra parameter. > > > > Before starting the VM migration, Nova will ensure both source and > > destination hosts' vhost-user interfaces run in the same compatibility > > modes, and will prevent it if this is not the case. > > > > For example host A runs OVS-2.7, and host B OVS-2.6. > > Host A's OVS-2.7 has an OVS-2.6 compatibility mode (e.g. with indirect > > descriptors disabled), which should be selected at vhost-user port add > > time to ensure migration will succeed to host B. > > > > Advantage of doing so is that Nova does not need any update if new keys > > are introduced (i.e. it does not need to know how the new keys have to > > be handled), all these checks remain in OVS's vhost-user implementation. > > > > Ideally, we would support per vhost-user interface compatibility mode, > > which may have an impact also on DPDK API, as the Virtio feature update > > API is global, and not per port. > > > > - Implementation: > > ----------------- > > > > Goal here is just to illustrate this proposal, I'm sure you will have > > good suggestion to improve it. > > In OVS vhost-user library, we would introduce a new structure, for > > example (neither compiled nor tested): > > > > struct vhostuser_compat { > > char *version; > > uint64_t virtio_features; > > uint32_t max_rx_queue_sz; > > uint32_t max_nr_queues; > > }; > > > > *version* field is the compatibility version string. It could be > > something like: "upstream.ovs-dpdk.v2.6". In case for example Fedora > > adds some more patches to its package that would break migration to > > upstream version, it could have a dedicated compatibility string: > > "fc26.ovs-dpdk.v2.6". In case OVS-v2.7 does not break compatibility with > > previous OVS-v2.6 version, then no need to create a new entry, just keep > > v2.6 one. > > > > *virtio_features* field is the Virtio features set for a given > > compatibility version. When an OVS tag is to be created, it would be > > associated to a DPDK version. The Virtio features for these version > > would be stored in this field. It would allow to upgrade the DPDK > > package for example from v16.07 to v16.11 without breaking migration. > > In case the distribution wants to benefit from latests Virtio > > features, it would have to create a new entry to ensure migration > > won't be broken. > > > > *max_rx_queue_sz* > > *max_nr_queues* fields are just here for example, don't think this is > > needed today. I just want to illustrate that we have to anticipate > > other parameters than the Virtio feature set, even if not necessary > > at the moment. > > > > We create a table with different compatibility versions in OVS > > vhost-user lib: > > > > static struct vhostuser_compat vu_compat[] = { > > { > > .version = "upstream.ovs-dpdk.v2.7", > > .virtio_features = 0x12045694, > > .max_rx_queue_sz = 512, > > }, > > { > > .version = "upstream.ovs-dpdk.v2.6", > > .virtio_features = 0x10045694, > > .max_rx_queue_sz = 1024, > > }, > > }; > > > > At some time during installation, or system init, the table would be > > parsed, and compatibility version strings would be stored into the OVS > > database, or a new tool would be created to list these strings, or a > > config file packaged with OVS stores the list of compatibiliy versions. > > > > Before launching the VM, Nova will query the version strings for the > > host so that the admin can select an older compatibility mode. If none > > selected by the admin, then the most recent one will be used by default, > > and passed to the OVS's add-port command as parameter. Note that if no > > compatibility mode is passed to the add-port command, the most recent > > one is selected by OVS as default. > > > > When the vhost-user connection is initiated, OVS would know in which > > compatibility mode to init the interface, for example by restricting the > > support Virtio features of the interface. > > > > Cheers, > > Maxime > > > > [0]: > > https://mail.openvswitch.org/pipermail/ovs-dev/2017-February/328257.html > > <b2a5501c-7df7-ad2a-002f-d731c445a502 at redhat.com> -- Eduardo

2 4

[libvirt] [PATCH v2 0/5] qemu: Default to video type=virtio for machvirt
by Cole Robinson 30 Aug '17

30 Aug '17

This series aim is to change the qemu machvirt video type default to virtio, but rather than continue to hack things into place in domain_conf.c, this rearranges things to allow drivers to set a video type default. Patches 1 is a small prep change in the qemu code Patches 2-4 are the plumbing to allow drivers to set their own default Patch 5 is the actual default change https://bugzilla.redhat.com/show_bug.cgi?id=1404112 v2: Rebase Push a few prep patches Drop the (virDomainVideoType) annotations Add patch #1 Cole Robinson (5): qemu: whitelist valid video types conf: domain: add VIDEO_TYPE_DEFAULT conf: domain: move video type validation to DeviceDefValidate qemu: Set default video type in qemu PostParse qemu: Default to video type=virtio for machvirt src/conf/domain_conf.c | 32 ++++++++------- src/conf/domain_conf.h | 1 + src/qemu/qemu_command.c | 3 ++ src/qemu/qemu_domain.c | 22 +++++++---- src/qemu/qemu_domain_address.c | 1 + tests/domaincapsschemadata/full.xml | 1 + .../qemuxml2argv-aarch64-video-default.args | 26 ++++++++++++ .../qemuxml2argv-aarch64-video-default.xml | 17 ++++++++ tests/qemuxml2argvtest.c | 6 +++ .../qemuxml2xmlout-aarch64-video-default.xml | 46 ++++++++++++++++++++++ tests/qemuxml2xmltest.c | 10 +++++ 11 files changed, 142 insertions(+), 23 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-aarch64-video-default.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-aarch64-video-default.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-aarch64-video-default.xml -- 2.13.5

2 15

[libvirt] [PATCH v2] qemu: set bind mode for chardev while parsing XML
by Pavel Hrdina 30 Aug '17

30 Aug '17

Currently while parsing domain XML we clear the UNIX path if it matches one of the auto-generated paths by libvirt. After that when the guest is started new path is generated but the mode is also changed to "bind". In the real-world use-case the mode should not change, it only happens if a user provides a mode='connect' and path that matches one of the auto-generated path or not provides a path at all. Before *reconnect* feature was introduced there was no issue, but with the new feature we need to make sure that it's used only with "connect" mode, therefore we need to move the mode change into parsing in order to have a proper error reported by validation code. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- src/qemu/qemu_domain.c | 13 +++++++++--- ...muxml2argv-chardev-reconnect-generated-path.xml | 23 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 3 +++ 3 files changed, 36 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bf57f94a50..cbee151f5d 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3766,8 +3766,17 @@ qemuDomainDeviceDefPostParse(virDomainDeviceDefPtr dev, /* clear auto generated unix socket path for inactive definitions */ if ((parseFlags & VIR_DOMAIN_DEF_PARSE_INACTIVE) && dev->type == VIR_DOMAIN_DEVICE_CHR) { - if (qemuDomainChrDefDropDefaultPath(dev->data.chr, driver) < 0) + virDomainChrDefPtr chr = dev->data.chr; + if (qemuDomainChrDefDropDefaultPath(chr, driver) < 0) goto cleanup; + + /* For UNIX chardev if no path is provided we generate one. + * This also implies that the mode is 'bind'. */ + if (chr->source && + chr->source->type == VIR_DOMAIN_CHR_TYPE_UNIX && + !chr->source->data.nix.path) { + chr->source->data.nix.listen = true; + } } if (dev->type == VIR_DOMAIN_DEVICE_VIDEO) { @@ -7441,8 +7450,6 @@ qemuDomainPrepareChannel(virDomainChrDefPtr channel, return -1; } - channel->source->data.nix.listen = true; - return 0; } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml b/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml new file mode 100644 index 0000000000..3cb67ecdcc --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml @@ -0,0 +1,23 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <devices> + <emulator>/usr/bin/qemu-system-i686</emulator> + <controller type='virtio-serial' index='1'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> + </controller> + <channel type='unix'> + <source mode='connect' path='/tmp/channel/domain-oldname/asdf'> + <reconnect enabled='yes' timeout='10'/> + </source> + <target type='virtio' name='asdf'/> + </channel> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0c0bd16f94..18f06e5aa5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1351,6 +1351,9 @@ mymain(void) DO_TEST_PARSE_ERROR("chardev-reconnect-invalid-timeout", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_CHARDEV_RECONNECT); + DO_TEST_PARSE_ERROR("chardev-reconnect-generated-path", + QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_CHARDEV_RECONNECT); DO_TEST("usb-controller", QEMU_CAPS_NODEFCONFIG); -- 2.13.5

2 1

[libvirt] [PATCH] qemu: set bind mode for chardev while parsing XML
by Pavel Hrdina 30 Aug '17

30 Aug '17

Currently while parsing domain XML we clear the UNIX path if it matches one of the auto-generated paths by libvirt. After that when the guest is started new path is generated but the mode is also changed to "bind". In the real-world use-case the mode should not change, it only happens if a user provides a mode='connect' and path that matches one of the auto-generated path or not provides a path at all. Before *reconnect* feature was introduced there was no issue, but with the new feature we need to make sure that it's used only with "connect" mode, therefore we need to move the mode change into parsing in order to have a proper error reported by validation code. Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com> --- src/conf/domain_conf.c | 8 ++++++++ src/qemu/qemu_domain.c | 6 +++--- ...muxml2argv-chardev-reconnect-generated-path.xml | 23 ++++++++++++++++++++++ tests/qemuxml2argvtest.c | 3 +++ 4 files changed, 37 insertions(+), 3 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index f7574d77b6..9fdd0f90fc 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -4421,6 +4421,14 @@ virDomainDeviceDefPostParseCommon(virDomainDeviceDefPtr dev, chr->target.port = maxport + 1; } + + /* For UNIX chardev if no path is provided we generate one. + * This also implies that the mode is 'bind'. */ + if (chr->source && + chr->source->type == VIR_DOMAIN_CHR_TYPE_UNIX && + !chr->source->data.nix.path) { + chr->source->data.nix.listen = true; + } } /* set default path for virtio-rng "random" backend to /dev/random */ diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bf57f94a50..ecad68d3bb 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -3489,8 +3489,10 @@ qemuDomainChrDefDropDefaultPath(virDomainChrDefPtr chr, regexp = virBufferContentAndReset(&buf); - if (virStringMatch(chr->source->data.nix.path, regexp)) + if (virStringMatch(chr->source->data.nix.path, regexp)) { VIR_FREE(chr->source->data.nix.path); + chr->source->data.nix.listen = true; + } ret = 0; cleanup: @@ -7441,8 +7443,6 @@ qemuDomainPrepareChannel(virDomainChrDefPtr channel, return -1; } - channel->source->data.nix.listen = true; - return 0; } diff --git a/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml b/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml new file mode 100644 index 0000000000..3cb67ecdcc --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-chardev-reconnect-generated-path.xml @@ -0,0 +1,23 @@ +<domain type='qemu'> + <name>QEMUGuest1</name> + <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid> + <memory unit='KiB'>219136</memory> + <vcpu placement='static'>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <devices> + <emulator>/usr/bin/qemu-system-i686</emulator> + <controller type='virtio-serial' index='1'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> + </controller> + <channel type='unix'> + <source mode='connect' path='/tmp/channel/domain-oldname/asdf'> + <reconnect enabled='yes' timeout='10'/> + </source> + <target type='virtio' name='asdf'/> + </channel> + <memballoon model='none'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 0c0bd16f94..18f06e5aa5 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1351,6 +1351,9 @@ mymain(void) DO_TEST_PARSE_ERROR("chardev-reconnect-invalid-timeout", QEMU_CAPS_NODEFCONFIG, QEMU_CAPS_CHARDEV_RECONNECT); + DO_TEST_PARSE_ERROR("chardev-reconnect-generated-path", + QEMU_CAPS_NODEFCONFIG, + QEMU_CAPS_CHARDEV_RECONNECT); DO_TEST("usb-controller", QEMU_CAPS_NODEFCONFIG); -- 2.13.5

2 1

[libvirt] [PATCH 0/5] chardev source reconnect cleanup and fixes
by Pavel Hrdina 30 Aug '17

30 Aug '17

Pavel Hrdina (5): tests: remove unused file conf: add reconnect to virDomainChrSourceDef(Copy|IsEqual) tests: don't use unix socket path that matches auto-generated path conf: make sure that chardev reconnect is formatted only for connect mode qemu: make sure that chardev reconnect is formatted only for connect mode src/conf/domain_conf.c | 24 ++++++++++++----- src/qemu/qemu_command.c | 8 +++--- .../qemuxml2argv-channel-reconnect.args | 31 ---------------------- .../qemuxml2argv-chardev-reconnect.args | 6 ++--- .../qemuxml2argv-chardev-reconnect.xml | 4 +-- 5 files changed, 26 insertions(+), 47 deletions(-) delete mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-reconnect.args -- 2.13.5

2 16

[libvirt] [PATCH v3 REBASE 00/16] qemu: migration: show disks stats for nbd migration
by Nikolay Shirokovskiy 30 Aug '17

30 Aug '17

diff from v2: ============ 1. Fix style issues. 2. Rework patch for fetching job info (save logic to use temporary variable when drop vm lock) 3. Update disk stats when block jobs are canceled. 4. Adress a few more corner cases. This patch series add disks stats to domain job info(stats) as well as to migration completed event in case nbd scheme is used. There is little nuisance with qcow2 disks (which is the main scenario I guess) tied to the way qemu reports stats for this type of disks. For example if we have 64G disk filled only to 1G then stats start from 63G and will grow up to 64G on completion. The same way disk stats will be reported by this patch. I guess the better way to express the situation is to say we have 64G 'total', and have 'processed' field grow from 0G to 1G, like in case of memory stats. [1] is the example of completed memory stats of empty guest domain, which show difference between processed and total. There can be a workaround like getting initial blockjob offset position as a zero but is is rather ugly and racy and like uses undocumented behaviour. Patches that were explicitly ACKed in previous review (up to style issues) marked with A. [1] memory migration stats example Memory processed: 3.307 MiB Memory remaining: 0.000 B Memory total: 1.032 GiB Nikolay Shirokovskiy (16): qemu: drop code for VIR_DOMAIN_JOB_BOUNDED and timeRemaining A qemu: introduce qemu domain job status A qemu: introduce QEMU_DOMAIN_JOB_STATUS_POSTCOPY A qemu: drop QEMU_MIGRATION_COMPLETED_UPDATE_STATS A qemu: drop excessive zero-out in qemuMigrationFetchJobStatus qemu: refactor fetching migration stats A qemu: simplify getting completed job stats qemu: fail querying destination migration statistics always qemu: start all async job with job status active qemu: introduce migrating job status qemu: always get job condition on getting job stats qemu: migrate: show disks stats on job info requests qemu: support getting disks stats during stopping block jobs qemu: migation: resolve race on getting job info and stopping block jobs qemu: migrate: copy disks stats to completed job qemu: migration: don't expose incomplete job as complete src/qemu/qemu_blockjob.c | 1 + src/qemu/qemu_domain.c | 38 +++++-- src/qemu/qemu_domain.h | 15 ++- src/qemu/qemu_driver.c | 94 ++++++++-------- src/qemu/qemu_migration.c | 236 ++++++++++++++++++++++++++------------- src/qemu/qemu_migration.h | 15 ++- src/qemu/qemu_migration_cookie.c | 7 +- src/qemu/qemu_monitor.c | 5 +- src/qemu/qemu_monitor.h | 4 +- src/qemu/qemu_monitor_json.c | 4 +- src/qemu/qemu_process.c | 11 +- tests/qemumonitorjsontest.c | 1 + 12 files changed, 272 insertions(+), 159 deletions(-) -- 1.8.3.1

2 34

[libvirt] [PATCH] Add qcow2 cache configuration
by Liu Qing 30 Aug '17

30 Aug '17

Random write IOPS will drop dramatically if qcow2 l2 cache could not cover the whole disk. This patch give libvirt user a chance to adjust the qcow2 cache configuration. Signed-off-by: Liu Qing <liuqing(a)huayun.com> --- src/conf/domain_conf.c | 30 ++++++++++++++++++++++++++++++ src/qemu/qemu_command.c | 6 ++++++ src/util/virstoragefile.c | 3 +++ src/util/virstoragefile.h | 4 ++++ 4 files changed, 43 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index d97aab4..06ca1de 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -8590,6 +8590,30 @@ virDomainDiskDefDriverParseXML(virDomainDiskDefPtr def, VIR_FREE(tmp); } + if ((tmp = virXMLPropString(cur, "l2-cache-size")) && + (virStrToLong_ui(tmp, NULL, 10, &def->src->l2_cache_size) < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid l2-cache-size attribute in disk driver element: %s"), + tmp); + goto cleanup; + } + + if ((tmp = virXMLPropString(cur, "refcount-cache-size")) && + (virStrToLong_ui(tmp, NULL, 10, &def->src->refcount_cache_size) < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid refcount-cache-size attribute in disk driver element: %s"), + tmp); + goto cleanup; + } + + if ((tmp = virXMLPropString(cur, "cache-clean-interval")) && + (virStrToLong_ui(tmp, NULL, 10, &def->src->cache_clean_interval) < 0)) { + virReportError(VIR_ERR_XML_ERROR, + _("Invalid cache-clean-interval attribute in disk driver element: %s"), + tmp); + goto cleanup; + } + if ((tmp = virXMLPropString(cur, "detect_zeroes")) && (def->detect_zeroes = virDomainDiskDetectZeroesTypeFromString(tmp)) <= 0) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, @@ -21887,6 +21911,12 @@ virDomainDiskDefFormat(virBufferPtr buf, virBufferAsprintf(&driverBuf, " iothread='%u'", def->iothread); if (def->detect_zeroes) virBufferAsprintf(&driverBuf, " detect_zeroes='%s'", detect_zeroes); + if (def->src->l2_cache_size > 0) + virBufferAsprintf(&driverBuf, " l2-cache-size='%u'", def->src->l2_cache_size); + if (def->src->refcount_cache_size > 0) + virBufferAsprintf(&driverBuf, " refcount-cache-size='%u'", def->src->refcount_cache_size); + if (def->src->cache_clean_interval > 0) + virBufferAsprintf(&driverBuf, " cache-clean-interval='%u'", def->src->cache_clean_interval); virDomainVirtioOptionsFormat(&driverBuf, def->virtio); diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index 9a27987..7996eed 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -1430,6 +1430,12 @@ qemuBuildDriveSourceStr(virDomainDiskDefPtr disk, qemuformat = "luks"; virBufferAsprintf(buf, "format=%s,", qemuformat); } + if (disk->src->format == VIR_STORAGE_FILE_QCOW2 && disk->src->l2_cache_size > 0) + virBufferAsprintf(buf, "l2-cache-size=%u,", disk->src->l2_cache_size); + if (disk->src->format == VIR_STORAGE_FILE_QCOW2 && disk->src->refcount_cache_size > 0) + virBufferAsprintf(buf, "refcount-cache-size=%u,", disk->src->refcount_cache_size); + if (disk->src->format == VIR_STORAGE_FILE_QCOW2 && disk->src->cache_clean_interval > 0) + virBufferAsprintf(buf, "cache-clean-interval=%u,", disk->src->cache_clean_interval); ret = 0; diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index fbc8245..c685331 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -2038,6 +2038,9 @@ virStorageSourceCopy(const virStorageSource *src, ret->physical = src->physical; ret->readonly = src->readonly; ret->shared = src->shared; + ret->l2_cache_size = src->l2_cache_size; + ret->refcount_cache_size = src->refcount_cache_size; + ret->cache_clean_interval = src->cache_clean_interval; /* storage driver metadata are not copied */ ret->drv = NULL; diff --git a/src/util/virstoragefile.h b/src/util/virstoragefile.h index 6c388b1..e7889d9 100644 --- a/src/util/virstoragefile.h +++ b/src/util/virstoragefile.h @@ -280,6 +280,10 @@ struct _virStorageSource { /* metadata that allows identifying given storage source */ char *nodeformat; /* name of the format handler object */ char *nodestorage; /* name of the storage object */ + + unsigned l2_cache_size; /* qcow2 l2 cache size */ + unsigned refcount_cache_size; /* qcow2 reference count table cache size */ + unsigned cache_clean_interval; /* clean unused cache entries interval */ }; -- 1.8.3.1

2 2

[libvirt] [PATCH for 3.7.0] docs: Document yet another limitation of tx_queue_size
by Michal Privoznik 30 Aug '17

30 Aug '17

https://bugzilla.redhat.com/show_bug.cgi?id=1484234 Turns out, only vhostuser type of interfaces are supported currently. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- I'd like to get this one in since the whole feature is being introduced in this release. docs/formatdomain.html.in | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 19e869f12..8ca7637a4 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -5213,7 +5213,10 @@ qemu-kvm -net nic,model=? /dev/null The default value is hypervisor dependent and may change across its releases. Moreover, some hypervisors may pose some restrictions on actual value. For instance, QEMU - v2.9 requires value to be a power of two from [256, 1024] range. + v2.9 requires value to be a power of two from [256, 1024] + range. In addition to that, this may work only for a subset of + interface types, e.g. aforementioned QEMU enables this option + only for <code>vhostuser</code> type. Since 3.7.0 (QEMU and KVM only) In general you should leave this option alone, unless you -- 2.13.5

2 4

[libvirt] [PATCH] rpc: avoid ssh interpreting malicious hostname as arguments
by Daniel P. Berrange 30 Aug '17

30 Aug '17

Inspired by the recent GIT / Mercurial security flaws (http://blog.recurity-labs.com/2017-08-10/scm-vulns) consider someone/something manages to feed libvirt a bogus URI such as: virsh -c qemu+ssh://-oProxyCommand=gnome-calculator/system In this case, the hosname "-oProxyCommand=gnome-calculator" will get interpreted as an argument to ssh, not a hostname. Fortunately, due to the set of args we have following the hostname, SSH will then interpret our bit of shell script that runs 'nc' on the remote host as a cipher name, which is clearly invalid. This makes ssh exit during argv parsing and so it never tries to run gnome-calculator. We are lucky this time, but lets be more paranoid, by using '--' to explicitly tell SSH when it has finished seeing command line options. This forces it to interpret "-oProxyCommand=gnome-calculator" as a hostname, and thus see a fail from hostname lookup. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/rpc/virnetsocket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index d228c8a8c..23089afef 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -868,7 +868,7 @@ int virNetSocketNewConnectSSH(const char *nodename, if (!netcat) netcat = "nc"; - virCommandAddArgList(cmd, nodename, "sh", "-c", NULL); + virCommandAddArgList(cmd, "--", nodename, "sh", "-c", NULL); virBufferEscapeShell(&buf, netcat); if (virBufferCheckError(&buf) < 0) { -- 2.13.5

3 2