- Devel - libvirt List Archives

[PATCH v2 0/7] qemu: block iotune group fixes/improvements
by Michal Privoznik 29 Jan '20

29 Jan '20

This series was originally posted by Nikolay here: https://www.redhat.com/archives/libvir-list/2020-January/msg00283.html I've delayed review too much. Sorry for that. To make it up to you, I've fixed some patches and resending the series. Basically, I want to know whether you agree with my adjustments. I will add Dan's Reviewed-by when merging. Nikolay Shirokovskiy (7): qemu: Move qemuDiskConfigBlkdeviotuneHas* to conf conf: expand iotune params if only group name is given qemu: propagate iotune settings to all disks in the group qemu: check iotune params same for all disk in group qemu: fix using defaults when setting persistent iotune params qemu: get defaults from iotune group we move disk into qemu: when leaving iotune group update xml properly src/conf/domain_conf.c | 108 +++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 22 ++++++++ src/libvirt_private.syms | 6 +++ src/qemu/qemu_command.c | 91 ++++++++++++++------------------- src/qemu/qemu_command.h | 3 ++ src/qemu/qemu_driver.c | 94 ++++++++++++++++++++++++++++++++-- src/qemu/qemu_hotplug.c | 2 +- tests/qemublocktest.c | 8 ++- 8 files changed, 275 insertions(+), 59 deletions(-) -- 2.24.1

2 9

[libvirt] [PATCH 0/7] qemu: block iotune group fixes/improvements
by Nikolay Shirokovskiy 29 Jan '20

29 Jan '20

Currently iotune group impl has several bugs/oddities this patchset aims to fix. Also patches (2) and (3) add/change functionality so that group iotune became easier/saner to use. TODO: add docs to API/virsh in respect to using iotune group Nikolay Shirokovskiy (7): conf: refactor virDomainBlockIoTuneInfoHas* conf: expand iotune params if only group name is given (2) qemu: check iotune params same for all disk in group qemu: fix using defaults when setting persistent iotune params qemu: propagate iotune settings to all disks in the group qemu: get defaults from iotune group we move disk into (3) qemu: when leaving iotune group update xml properly src/conf/domain_conf.c | 104 +++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 16 ++++++ src/libvirt_private.syms | 5 ++ src/qemu/qemu_command.c | 90 ++++++++++++++------------------- src/qemu/qemu_command.h | 3 ++ src/qemu/qemu_driver.c | 98 ++++++++++++++++++++++++++++++++++-- src/qemu/qemu_hotplug.c | 2 +- tests/qemublocktest.c | 2 +- 8 files changed, 262 insertions(+), 58 deletions(-) -- 2.23.0

3 17

[libvirt PATCH 00/10] Various cleanups (virtio-fs epopee)
by Ján Tomko 29 Jan '20

29 Jan '20

2 22

[PATCH v2 0/4] lxc: Add suport to network indexes for LXC 3.X
by Julio Faracco 28 Jan '20

28 Jan '20

This series implement support for network syntax settings for LXC 3.X. Now, indexes are part of the settings to define network interfaces. Each interface has its own index. The old style uses 'type' tag to differentiate each interface and with this, it could use one single structure. Now, network interfaces are a single array with interface pointers. Old: lxc.network.type = veth lxc.network.flags = up lxc.network.link = virbr0 New: lxc.net.0.type = veth lxc.net.0.flags = up lxc.net.0.link = virbr0 v1-v2: Moving sscanf to virStrToLong_ull according Daniel's suggestion. Julio Faracco (4): lxc: refactor lxcNetworkParseData pointers to use new structures lxc: add LXC version 3 network parser lxc: remove domain definition from lxc network struct tests: update LXC config dataset to support V3 indexes src/lxc/lxc_native.c | 189 ++++++++++++------ .../lxcconf2xml-ethernet-v3.config | 16 +- .../lxcconf2xml-fstab-v3.config | 10 +- .../lxcconf2xml-macvlannetwork-v3.config | 10 +- .../lxcconf2xml-simple-v3.config | 18 +- 5 files changed, 151 insertions(+), 92 deletions(-) -- 2.20.1

1 4

[PATCH 0/4] lxc: Add suport to network indexes for LXC 3.X
by Julio Faracco 28 Jan '20

28 Jan '20

This series implement support for network syntax settings for LXC 3.X. Now, indexes are part of the settings to define network interfaces. Each interface has its own index. The old style uses 'type' tag to differentiate each interface and with this, it could use one single structure. Now, network interfaces are a single array with interface pointers. Old: lxc.network.type = veth lxc.network.flags = up lxc.network.link = virbr0 New: lxc.net.0.type = veth lxc.net.0.flags = up lxc.net.0.link = virbr0 Julio Faracco (4): lxc: refactor lxcNetworkParseData pointers to use new structures. lxc: add LXC version 3 network parser. lxc: remove domain definition from lxc network struct tests: update LXC config dataset to support V3 indexes src/lxc/lxc_native.c | 183 +++++++++++------- .../lxcconf2xml-ethernet-v3.config | 16 +- .../lxcconf2xml-fstab-v3.config | 10 +- .../lxcconf2xml-macvlannetwork-v3.config | 10 +- .../lxcconf2xml-simple-v3.config | 18 +- 5 files changed, 145 insertions(+), 92 deletions(-) -- 2.20.1

2 9

[libvirt PATCH v2 0/6] <interface> <teaming> element (was: virtio failover / vfio auto-plug-on-migrate)
by Laine Stump 28 Jan '20

28 Jan '20

V1: https://www.redhat.com/archives/libvir-list/2020-January/msg00813.html This all used different names in V1 - in that incarnation the configuration was done using "failover" and "backupAlias" attributes added to the <driver> subelement of <interface>. But the resulting code was cumbersome and had little bits scattered all over the place due to needing it in both hostdev and interface parsing/formatting. In his review of V1, danpb suggesting just adding a new subelement for this configuration to free ourselves from the constraints of <driver> parsing/formatting. This ended up dramatically simplifying the code (hence the lack of V1's refactoring patches in V2, and a decrease in patch count from 12 to 6). During further discussion in email and on IRC, we decided that naming the element <failover> was too limiting, as it implied the behavior of what is, to libvirt, just two network devices that should be teamed/bonded together - it's completely up to the hypervisor and guest what is done with this information. In light of that, we decided to name the new subelement <teaming>, and to specify the two interfaces as "persistent" (an interface that will always remain plugged in) and "transient" (an interface that may be periodically unplugged (during migration, in the case of QEMU). So the virtio device will have <teaming type='persistent'/> and the hostdev device will have <teaming type='transient' persistent='ua-myvirtio'/> (note that the persistent interface must have <alias name='ua-myvirtio'/>) Given this config, libvirt will add "failover=on" to the device commandline arg for the virtio device, and "failover_pair_id=ua-myvirtio" to the arg for the hostdev device (and when a migration is requested, it will notice if there is a hostdev that has <teaming type='transient'/> set, and will allow the migration in this case, but still disallow migrations of domains with normal hostdevs). In response to these extra commandline options, QEMU will set some extra capabilities in the virtio device PCI capabilities data, and will also automatically unplug/re-plug the hostdev device before and after migration. In the guest, the virtio-net driver will notice the extra PCI capabilities and use this as a clue that it should search for another device with matching MAC address (NB: the guest driver requires the two devices to have matching MAC addresses) to join into a bond with the virtio-net device. This bond is hard-wired to always prefer the hostdev device whenever it is present, and use the virtio device as backup when the hostdev is unplugged. ---- As mentioned in a followup to the V1 cover letter, there is a regression in QEMU 4.2.0 that causes QEMU to segv when a hostdev is unplugged. That bug is fixed with this upstream QEMU patch: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=0446f8121723b134ca1d1ed0b73… Be sure to use a qemu build with this patch applied, or you may not even be able to start the guest! Also we've found that the DEVICE_DELETED event is never sent to libvirt when one of these hostdevs is manually unplugged, meaning that libvirt keeps the device marked as "in-use", and it therefore cannot be re-plugged to the guest until after a complete guest "power cycle". AFAIK there isn't yet a fix for that bug, so don't expect manual unplug of the device to work. Laine Stump (6): qemu: add capabilities flag for failover feature conf: parse/format <teaming> subelement of <interface> qemu: support interface <teaming> functionality qemu: allow migration with assigned PCI hostdev if <teaming> is set qemu: add wait-unplug to qemu migration status enum docs: document <interface> subelement <teaming> docs/formatdomain.html.in | 100 ++++++++++++++++++ docs/news.xml | 28 +++++ docs/schemas/domaincommon.rng | 19 ++++ src/conf/domain_conf.c | 45 ++++++++ src/conf/domain_conf.h | 14 +++ src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 9 ++ src/qemu/qemu_domain.c | 36 ++++++- src/qemu/qemu_migration.c | 53 +++++++++- src/qemu/qemu_monitor.c | 1 + src/qemu/qemu_monitor.h | 1 + src/qemu/qemu_monitor_json.c | 1 + .../caps_4.2.0.aarch64.xml | 1 + .../caps_4.2.0.x86_64.xml | 1 + .../net-virtio-teaming-network.xml | 37 +++++++ .../qemuxml2argvdata/net-virtio-teaming.args | 40 +++++++ tests/qemuxml2argvdata/net-virtio-teaming.xml | 50 +++++++++ tests/qemuxml2argvtest.c | 4 + .../net-virtio-teaming-network.xml | 51 +++++++++ .../qemuxml2xmloutdata/net-virtio-teaming.xml | 66 ++++++++++++ tests/qemuxml2xmltest.c | 6 ++ 22 files changed, 563 insertions(+), 7 deletions(-) create mode 100644 tests/qemuxml2argvdata/net-virtio-teaming-network.xml create mode 100644 tests/qemuxml2argvdata/net-virtio-teaming.args create mode 100644 tests/qemuxml2argvdata/net-virtio-teaming.xml create mode 100644 tests/qemuxml2xmloutdata/net-virtio-teaming-network.xml create mode 100644 tests/qemuxml2xmloutdata/net-virtio-teaming.xml -- 2.24.1

3 13

[PATCH v2 0/9] virsh: secret: Improve handling of secret value
by Peter Krempa 28 Jan '20

28 Jan '20

Improve handling of secrets for security. 'secret-passwd' command was dropped in favor of '--interactive' and few other changes requested at review. Peter Krempa (9): virsh: Work around virSecretFree quirks virsh: secret: Refactor cleanup in cmdSecretSetValue virsh: secret: Refactor cleanup in cmdSecretGetValue virsh: secret: Add --plain flag for secret-get-value virsh: secret: Add --file 'filename' support for secret-set-value virsh: secret: Print warning that passing secret on command-line is insecure virsh: secret: Add --plain switch for secret-set-value tools: virsh: Add --interactive flag for secret-set-value command docs: secret: Unify and sanitize examples on how to set secret value build-aux/syntax-check.mk | 2 +- docs/formatsecret.html.in | 88 +++++++++++++-------- docs/manpages/virsh.rst | 20 ++++- tools/virsh-completer-secret.c | 3 +- tools/virsh-secret.c | 140 ++++++++++++++++++++++++--------- tools/virsh-util.c | 11 +++ tools/virsh-util.h | 5 ++ 7 files changed, 195 insertions(+), 74 deletions(-) -- 2.24.1

3 21

[libvirt] [RFC PATCH 00/16] qemu: checkpoint: Add support for deleting checkpoints accross snapshots
by Peter Krempa 28 Jan '20

28 Jan '20

Posted as RFC because qemu probably doesn't reopen backing images for bitmap operations resulting in: $ virsh checkpoint-delete vm --checkpointname a error: Failed to delete checkpoint a error: internal error: unable to execute QEMU command 'transaction': Bitmap 'a' is readonly and cannot be modified Unfortunately this can't be done manually because 'blockdev-reopen' is still experimental. Peter Krempa (16): qemu: checkpoint: Store whether deleted checkpoint is current in a variable qemu: checkpoint: split out checkpoint deletion bitmaps qemu: checkpoint: rename disk->chkdisk in qemuCheckpointDiscardBitmaps qemu: checkpoint: rename disk->chkdisk in qemuCheckpointAddActions qemu: checkpoint: Use disk definition directly when creating checkpoint qemu: checkpoint: tolerate missing disks on checkpoint deletion qemu: domain: Remove unused qemuDomainDiskNodeFormatLookup qemu: checkpoint: Introduce helper to find checkpoint disk definition in parents qemu: checkpoint: Extract calculation of bitmap merging for checkpoint deletion tests: qemublock: Add test for checkpoint deletion bitmap merge tests: qemublock: Add few more test cases for checkpoint deletion tests: qemublock: Add synthetic snapshot+checkpoint test data qemu: checkpoint: Introduce support for deleting checkpoints accross snapshots tests: qemublock: Add checkpoint deletion test for deep backing chain tests: qemublock: Add checkpoint deletion tests for some special cases qemu: checkpoint: Track and relabel images for bitmap merging src/qemu/qemu_checkpoint.c | 329 +++++-- src/qemu/qemu_checkpoint.h | 9 + src/qemu/qemu_domain.c | 14 - src/qemu/qemu_domain.h | 3 - tests/qemublocktest.c | 102 +++ .../snapshots-synthetic-checkpoint.json | 827 ++++++++++++++++++ .../bitmap/snapshots-synthetic-checkpoint.out | 13 + .../checkpointdelete/basic-current-out.json | 29 + .../basic-intermediate1-out.json | 22 + .../basic-intermediate2-out.json | 22 + .../basic-intermediate3-out.json | 22 + .../checkpointdelete/basic-noparent-out.json | 9 + .../snapshots-current-out.json | 29 + .../snapshots-intermediate1-out.json | 24 + .../snapshots-intermediate2-out.json | 62 ++ .../snapshots-intermediate3-out.json | 61 ++ .../snapshots-noparent-out.json | 27 + ...hots-synthetic-checkpoint-current-out.json | 29 + ...ynthetic-checkpoint-intermediate1-out.json | 31 + ...ynthetic-checkpoint-intermediate2-out.json | 34 + ...ynthetic-checkpoint-intermediate3-out.json | 61 ++ ...ots-synthetic-checkpoint-noparent-out.json | 27 + 22 files changed, 1680 insertions(+), 106 deletions(-) create mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json create mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-current-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json -- 2.24.1

2 33

[libvirt PATCH 00/32] the gnulib saga: revenge of the gnus
by Daniel P. Berrangé 27 Jan '20

27 Jan '20

We currently have 25 gnulib modules in bootstrap.conf and after this series we're down to 7. Except this is a lie, because we've been failing to count the indirect dependencies, so after this series, libvirt is actually pulling in 41 modules. Still, that's better than the 115 modules we indirectly use before this series :-) I have actually eliminated the remaining modules too, but I'm having trouble with the event loop impl in the RPC client on Windows that I'm still debugging. I'll poist the remaining patches once I've figured out that problem. Daniel P. Berrangé (32): tests: stop setting $SHELL env variable util: add a virArchFromHost() impl for Windows util: add API for reading password from the console src: remove usage of strchrnul function build: generate configmake.h in root directory util: use getgrouplist() directly instead of mgetgroups tools: replace wcwidth() with g_unichar_* APIs src: remove unused sys/utsname.h includes util: explicitly include windows.h storage: remove use of stat-time.h headers src: implement APIs for passing FDs over UNIX sockets rpc: conditionalize signal handling src: only import sys/uio.h when journald is built src: replace mkdir() with g_mkdir() m4: disable polkit build on Windows util: conditionalize more of virCommand on WIN32 src: remove all traces of Cygwin support util: conditionalize virProcess APIs on Windows src: conditionalize use of net/if.h configure: add check for sys/ioctl.h src: conditionalize use of S_ISSOCK macro configure: request system specific extensions src: stop using O_DIRECTORY in resctrl src: ensure O_CLOEXEC is defined on Windows src: conditionalize use of F_DUPFD_CLOEXEC src: conditionalize use of O_DIRECT src: conditionalize use of O_BINARY src: conditionalize use of chown & stat constants src: convert all code to use virsocket.h tests: conditionalize use of SIGPIPE src: conditionalize EAI_ADDRFAMILY bootstrap: remove 18 more gnulib modules Makefile.am | 44 ++++++++ bootstrap.conf | 36 ------ build-aux/syntax-check.mk | 2 +- configure.ac | 35 +++--- m4/virt-compile-pie.m4 | 2 +- m4/virt-polkit.m4 | 4 + m4/virt-win-common.m4 | 8 +- m4/virt-win-cygwin.m4 | 32 ------ m4/virt-win-symbols.m4 | 4 +- m4/virt-win-windres.m4 | 4 +- m4/virt-xdr.m4 | 9 +- po/POTFILES.in | 1 - src/Makefile.am | 8 +- src/admin/Makefile.inc.am | 1 - src/conf/network_conf.c | 2 - src/esx/esx_util.c | 3 +- src/esx/esx_util.h | 1 - src/internal.h | 10 ++ src/libvirt-domain.c | 2 + src/libvirt.c | 9 +- src/libvirt_private.syms | 6 + src/libxl/libxl_conf.c | 2 - src/locking/Makefile.inc.am | 1 - src/logging/Makefile.inc.am | 1 - src/lxc/lxc_controller.c | 4 +- src/lxc/lxc_driver.c | 4 +- src/nwfilter/nwfilter_dhcpsnoop.c | 3 - src/nwfilter/nwfilter_learnipaddr.c | 4 +- src/openvz/openvz_conf.c | 4 +- src/qemu/qemu_agent.c | 4 +- src/qemu/qemu_conf.c | 3 +- src/qemu/qemu_driver.c | 3 +- src/qemu/qemu_interface.c | 4 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_monitor.c | 3 +- src/qemu/qemu_monitor_json.c | 4 +- src/remote/qemu_protocol.x | 1 - src/remote/remote_protocol.x | 2 +- src/rpc/Makefile.inc.am | 3 - src/rpc/genprotocol.pl | 2 +- src/rpc/virnetclient.c | 33 ++++-- src/rpc/virnetdaemon.c | 35 +++++- src/rpc/virnetdaemon.h | 4 + src/rpc/virnetprotocol.x | 2 +- src/rpc/virnetsocket.c | 42 ++++--- src/security/security_dac.c | 4 + src/security/security_manager.c | 2 + src/security/security_selinux.c | 16 +-- src/storage/storage_util.c | 24 +++- src/util/virarch.c | 52 ++++++++- src/util/virarptable.c | 1 - src/util/vircgroup.c | 10 +- src/util/vircommand.c | 165 +++++++++++++++++----------- src/util/virdnsmasq.c | 9 +- src/util/virfdstream.c | 13 +-- src/util/virfile.c | 58 ++++++++-- src/util/virhostcpu.c | 1 - src/util/virhostmem.c | 1 - src/util/virlog.c | 8 +- src/util/virnetdev.c | 1 - src/util/virnetdev.h | 4 +- src/util/virnetdevbridge.c | 8 +- src/util/virnetdevip.c | 4 +- src/util/virnetdevmacvlan.c | 3 +- src/util/virnetdevtap.c | 4 +- src/util/virnetdevvportprofile.c | 3 +- src/util/virnetlink.c | 3 +- src/util/virprocess.c | 103 +++++++++++++---- src/util/virresctrl.c | 2 +- src/util/virsocket.c | 142 +++++++++++++++++++++++- src/util/virsocket.h | 6 +- src/util/virsocketaddr.c | 2 - src/util/virsocketaddr.h | 20 +--- src/util/virsysinfo.c | 14 ++- src/util/virsystemd.c | 9 +- src/util/virutil.c | 76 +++++++++---- src/util/virutil.h | 2 + src/util/virxdrdefs.h | 12 +- src/vbox/vbox_MSCOMGlue.c | 6 +- tests/Makefile.am | 3 - tests/libxlmock.c | 2 +- tests/nsstest.c | 3 +- tests/testutils.c | 2 +- tests/virauthconfigtest.c | 2 + tests/virkeyfiletest.c | 2 + tests/virlockspacetest.c | 4 +- tests/virnetmessagetest.c | 2 + tests/virnetsockettest.c | 3 +- tests/virnettlscontexttest.c | 3 +- tests/virnettlshelpers.c | 1 - tests/virnettlssessiontest.c | 3 +- tests/virportallocatormock.c | 5 +- tests/virtestmock.c | 9 +- tests/virtimetest.c | 2 + tests/viruritest.c | 2 + tests/vshtabletest.c | 8 +- tools/nss/libvirt_nss.c | 7 +- tools/virsh-domain.c | 46 +++++++- tools/virt-login-shell.c | 6 +- tools/vsh-table.c | 2 +- tools/vsh.c | 12 +- 101 files changed, 859 insertions(+), 452 deletions(-) delete mode 100644 m4/virt-win-cygwin.m4 -- 2.24.1

2 49

[PATCH 0/3] Trivial fixes
by Andrea Bolognani 27 Jan '20

27 Jan '20

Pushed as - wait for it - trivial. Andrea Bolognani (3): docs: Fix link to virt-qemu-run(1) qemu_shim: Fix typos qemu_shim: Update temporary directory template docs/manpages/index.rst | 2 +- src/qemu/qemu_shim.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) -- 2.24.1

1 3