November 2018 - Devel - libvirt List Archives

[libvirt] [PATCH v2 0/8] Usb hub Cold(un)plug and hot(un)plug support
by Han Han 07 Nov '18

07 Nov '18

v1: https://www.redhat.com/archives/libvir-list/2018-October/msg00563.html Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1375423 Han Han (8): qemu: Allow coldplugging of hub device qemu: Allow coldunplugging of hub device qemu_alias: Refactor hub alias assignment for hotplug qemu_command: Make qemuBuildHubDevStr global private_syms: add virDomainHubDefFree to libvirt_private.syms qemu: implement usb hub device hotplug qemu: implement usb hub device hotunplug news: Cold(un)plug and hot(un)plug support for usb hub device docs/news.xml | 5 ++ src/conf/domain_conf.c | 30 +++++++++ src/conf/domain_conf.h | 3 + src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 22 +++++-- src/qemu/qemu_alias.h | 4 ++ src/qemu/qemu_command.c | 2 +- src/qemu/qemu_command.h | 4 ++ src/qemu/qemu_driver.c | 30 +++++++-- src/qemu/qemu_hotplug.c | 132 ++++++++++++++++++++++++++++++++++++++- src/qemu/qemu_hotplug.h | 12 ++++ 11 files changed, 236 insertions(+), 10 deletions(-) -- 2.19.1

2 21

[libvirt] [PATCH v3 00/20] Incremental Backup API additions
by Eric Blake 07 Nov '18

07 Nov '18

The following is the latest version of my API proposal for incremental backups, and matches the demo I will be presenting as part of my KVM Forum 2018 talk tomorrow afternoon: https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-e… The patches are also available via https://repo.or.cz/libvirt/ericb.git at the tag backup-v3. The qemu integration is still incomplete (I don't have push mode backups working, only pull mode; and there are still some things to fine-tune as I figure out how to better support external snapshots and block commit/stream), and I still haven't gotten any opinions on whether it is better to have: virDomainSnapshotCreateXML("<domainsnapshot>...") # existing virDomainBackupBegin("<domainbackup>...", "<domaincheckpoint>...") # this series virDomainSnapshotCheckpointCreateXML("<domainsnapshot>...", "<domaincheckpoint>...") # new or to make checkpoint creation part of the snapshot and backup XML, as in: virDomainSnapshotCreateXML("<domainsnapshot><domaincheckpoint>...</domainsnapshot>") # extension of existing virDomainBackupBegin("<domainbackup><domaincheckpoint>...</domainbackup>") # tweak to this series It's also obvious that we don't want to check in the qemu implementation until after Peter's blockdev work has landed AND qemu has gotten rid of the x- prefix in its QMP interactions, although the state of this series SHOULD be sufficient to prove that the proposed interface is at least flexible enough to adapt to whatever qemu tweaks land and whatever other hypervisors may support. Whether to accept the API now (to make backporting qemu support downstream across versions without rebasing to an even later date for the API) is a harder question; Jirka has already expressed distaste at the idea. There are probably still loads of things needing fixing in this version, I already know I did not address all of Peter's review comments on v2. My unit test coverage is not very complete on the XML parsing, and there are definitely some hacks in place to get things to the point of a working demo that need to be polished for robustness. Below is a diff since the v2 posting (patches 1-10 here, for later patches a comparison against the state of my tree at the time I posted v2): https://www.redhat.com/archives/libvir-list/2018-October/msg00700.html 001/20:[----] [--] 'snapshots: Avoid term 'checkpoint' for full system snapshot' 002/20:[----] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost' 003/20:[down] 'qemu: Allow optional export name during NBD export' 004/20:[----] [--] 'backup: Document nuances between different state capture APIs' 005/20:[----] [--] 'backup: Introduce virDomainCheckpointPtr' 006/20:[0012] [FC] 'backup: Document new XML for backups' 007/20:[----] [--] 'backup: Introduce virDomainCheckpoint APIs' 008/20:[0024] [FC] 'backup: Introduce virDomainBackup APIs' 009/20:[----] [--] 'backup: Add new domain:checkpoint access control' 010/20:[0001] [FC] 'backup: Implement backup APIs for remote driver' 011/20:[0007] [FC] 'wip: backup: Parse and output checkpoint XML' 012/20:[0199] [FC] 'wip: backup: Parse and output backup XML' 013/20:[----] [--] 'backup: Implement virsh support for checkpoints' 014/20:[----] [--] 'wip: backup: virsh support for backup' 015/20:[0223] [FC] 'backup: Add new qemu monitor interactions' 016/20:[0123] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs' 017/20:[down] 'wip: backup: Wire up qemu checkpoint commands over QMP' 018/20:[0072] [FC] 'wip: backup: qemu: Implement framework for backup job APIs' 019/20:[down] 'backup: Wire up qemu full pull backup commands over QMP' 020/20:[down] 'backup: implement qemu incremental pull backup' No direct dependency on this one, but included in my git tag: https://www.redhat.com/archives/libvir-list/2018-October/msg00899.html Eric Blake (20): snapshots: Avoid term 'checkpoint' for full system snapshot domain_conf: Expose virDomainStorageNetworkParseHost qemu: Allow optional export name during NBD export backup: Document nuances between different state capture APIs backup: Introduce virDomainCheckpointPtr backup: Document new XML for backups backup: Introduce virDomainCheckpoint APIs backup: Introduce virDomainBackup APIs backup: Add new domain:checkpoint access control backup: Implement backup APIs for remote driver wip: backup: Parse and output checkpoint XML wip: backup: Parse and output backup XML backup: Implement virsh support for checkpoints wip: backup: virsh support for backup backup: Add new qemu monitor interactions backup: qemu: Implement metadata tracking for checkpoint APIs wip: backup: Wire up qemu checkpoint commands over QMP wip: backup: qemu: Implement framework for backup job APIs backup: Wire up qemu full pull backup commands over QMP backup: implement qemu incremental pull backup include/libvirt/virterror.h | 6 +- src/util/virerror.c | 21 +- include/libvirt/libvirt-domain-checkpoint.h | 177 ++ include/libvirt/libvirt-domain-snapshot.h | 2 +- include/libvirt/libvirt-domain.h | 17 +- include/libvirt/libvirt.h | 3 +- src/access/viraccessperm.h | 8 +- src/conf/checkpoint_conf.h | 218 +++ src/conf/domain_conf.h | 16 +- src/datatypes.h | 31 +- src/driver-hypervisor.h | 74 +- src/qemu/qemu_conf.h | 2 + src/qemu/qemu_domain.h | 29 +- src/qemu/qemu_monitor.h | 17 + src/qemu/qemu_monitor_json.h | 19 + tools/virsh-checkpoint.h | 29 + tools/virsh-completer.h | 4 + tools/virsh-util.h | 3 + tools/virsh.h | 1 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 9 +- docs/domainstatecapture.html.in | 314 ++++ docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 291 ++++ docs/formatsnapshot.html.in | 33 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 185 ++ docs/schemas/domaincheckpoint.rng | 94 ++ examples/object-events/event-test.c | 3 + libvirt.spec.in | 3 + mingw-libvirt.spec.in | 6 + po/POTFILES | 3 + src/Makefile.am | 2 + src/access/viraccessperm.c | 5 +- src/conf/Makefile.inc.am | 2 + src/conf/checkpoint_conf.c | 1541 +++++++++++++++++ src/conf/domain_conf.c | 52 +- src/conf/snapshot_conf.c | 4 +- src/datatypes.c | 62 +- src/libvirt-domain-checkpoint.c | 925 ++++++++++ src/libvirt-domain-snapshot.c | 7 +- src/libvirt-domain.c | 8 +- src/libvirt_private.syms | 30 + src/libvirt_public.syms | 23 + src/qemu/qemu_conf.c | 5 + src/qemu/qemu_domain.c | 205 ++- src/qemu/qemu_driver.c | 1494 ++++++++++++++++- src/qemu/qemu_migration.c | 2 +- src/qemu/qemu_monitor.c | 73 +- src/qemu/qemu_monitor_json.c | 217 +++ src/qemu/qemu_process.c | 7 + src/remote/remote_daemon_dispatch.c | 15 + src/remote/remote_driver.c | 33 +- src/remote/remote_protocol.x | 239 ++- src/remote_protocol-structs | 129 ++ src/rpc/gendispatch.pl | 32 +- tests/Makefile.am | 15 +- tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/domaincheckpointxml2xmlin/empty.xml | 1 + tests/domaincheckpointxml2xmlin/sample.xml | 7 + tests/domaincheckpointxml2xmlout/empty.xml | 10 + tests/domaincheckpointxml2xmlout/sample.xml | 16 + tests/domaincheckpointxml2xmltest.c | 231 +++ tests/qemumonitorjsontest.c | 2 +- tests/virschematest.c | 4 + tools/Makefile.am | 3 +- tools/virsh-checkpoint.c | 1578 ++++++++++++++++++ tools/virsh-completer.c | 52 +- tools/virsh-domain.c | 6 +- tools/virsh-snapshot.c | 2 +- tools/virsh-util.c | 11 + tools/virsh.c | 2 + tools/virsh.pod | 14 +- 79 files changed, 8619 insertions(+), 118 deletions(-) create mode 100644 include/libvirt/libvirt-domain-checkpoint.h create mode 100644 src/conf/checkpoint_conf.h create mode 100644 tools/virsh-checkpoint.h create mode 100644 docs/domainstatecapture.html.in create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 src/conf/checkpoint_conf.c create mode 100644 src/libvirt-domain-checkpoint.c create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml create mode 100644 tests/domaincheckpointxml2xmlin/sample.xml create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml create mode 100644 tests/domaincheckpointxml2xmlout/sample.xml create mode 100644 tests/domaincheckpointxml2xmltest.c create mode 100644 tools/virsh-checkpoint.c -- 2.17.2

3 23

[libvirt] [PATCH] qemu: Fix debug message in qemuProcessHandleResume
by Jiri Denemark 07 Nov '18

07 Nov '18

The message talks about "resumed" state, which is a bit confusing. While we have a "resumed" event, the corresponding state is called "running". Signed-off-by: Jiri Denemark <jdenemar(a)redhat.com> --- src/qemu/qemu_process.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 59ca7cd333..4e93b2d741 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -721,7 +721,7 @@ qemuProcessHandleResume(qemuMonitorPtr mon ATTRIBUTE_UNUSED, if (virDomainObjGetState(vm, NULL) != VIR_DOMAIN_RUNNING) { eventDetail = qemuDomainRunningReasonToResumeEvent(reason); - VIR_DEBUG("Transitioned guest %s out of paused into resumed state, " + VIR_DEBUG("Transitioned guest %s out of paused into running state, " "reason '%s', event detail %d", vm->def->name, virDomainRunningReasonTypeToString(reason), eventDetail); -- 2.19.1

2 1

[libvirt] [PATCH v2 0/3] qemu: Restore lost shutdown reason
by John Ferlan 07 Nov '18

07 Nov '18

v1: https://www.redhat.com/archives/libvir-list/2018-October/msg00945.html Changes since v1: Patch 1 (NEW) - Set the priv->allowReboot prior to any error processing since we could/should be using that. Patch 2 (ADJUSTED LOGIC) - Rather than open code the "reason" to use the -no-shutdown flag, let's create a qemu_domain helper for that so that both the command line building and the Reconnection failure logic can use it. Patch 3 (NEW) - Let's narrow the window for using VIR_DOMAIN_SHUTOFF_CRASHED to just the period where we try to open a monitor channel. If we cannot do so, then "assume" the reason was crashed. There are a few open failure steps that may not exactly fit the model, but those are probably splitting hairs. John Ferlan (3): qemu: Move allow reboot check setting qemu: Restore lost shutdown reason qemu: Narrow the shutdown reconnection failure reason window src/qemu/qemu_command.c | 6 +----- src/qemu/qemu_domain.c | 17 +++++++++++++++++ src/qemu/qemu_domain.h | 3 +++ src/qemu/qemu_process.c | 27 ++++++++++++++++++--------- 4 files changed, 39 insertions(+), 14 deletions(-) -- 2.17.2

2 10

[libvirt] [jenkins-ci PATCH 0/4] Add Fedora 29, drop Fedora 27
by Andrea Bolognani 07 Nov '18

07 Nov '18

It's that time of the year again :) Andrea Bolognani (4): guests: Add Fedora 29 Build on Fedora 29 Stop building on Fedora 27 guests: Drop Fedora 27 guests/host_vars/libvirt-fedora-27/docker.yml | 2 - guests/host_vars/libvirt-fedora-29/docker.yml | 2 + .../install.yml | 2 +- .../main.yml | 2 +- guests/inventory | 2 +- guests/playbooks/build/jobs/defaults.yml | 4 +- .../playbooks/build/projects/libvirt-dbus.yml | 6 +- .../build/projects/libvirt-ocaml.yml | 2 +- .../build/projects/libvirt-sandbox.yml | 4 +- .../playbooks/build/projects/libvirt-tck.yml | 4 +- guests/playbooks/build/projects/libvirt.yml | 2 +- .../playbooks/build/projects/virt-manager.yml | 6 +- .../playbooks/build/projects/virt-viewer.yml | 1 + guests/vars/mappings.yml | 1 - guests/vars/vault.yml | 82 +++++++++---------- jobs/defaults.yaml | 4 +- projects/libvirt-dbus.yaml | 6 +- projects/libvirt-ocaml.yaml | 2 +- projects/libvirt-sandbox.yaml | 4 +- projects/libvirt-tck.yaml | 4 +- projects/libvirt.yaml | 2 +- projects/virt-manager.yaml | 6 +- projects/virt-viewer.yaml | 1 + 23 files changed, 76 insertions(+), 75 deletions(-) delete mode 100644 guests/host_vars/libvirt-fedora-27/docker.yml create mode 100644 guests/host_vars/libvirt-fedora-29/docker.yml rename guests/host_vars/{libvirt-fedora-27 => libvirt-fedora-29}/install.yml (66%) rename guests/host_vars/{libvirt-fedora-27 => libvirt-fedora-29}/main.yml (95%) -- 2.19.1

2 5

[libvirt] [PATCH] libxl: Properly dispose libxl_domain_config object
by Jim Fehlig 06 Nov '18

06 Nov '18

V2 of the libxl soft reset patch, which was pushed as commit da4b0fd9, dropped the hunk that disposed of the libxl_domain_config object. Add the missing hunk to properly dispose the object. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- Pushing as trivial... src/libxl/libxl_domain.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 6aca7eebf8..5fe3f44fbe 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -605,6 +605,7 @@ libxlDomainShutdownThread(void *opaque) virObjectEventStateQueue(driver->domainEventState, dom_event); libxl_event_free(cfg->ctx, ev); VIR_FREE(shutdown_info); + libxl_domain_config_dispose(&d_config); virObjectUnref(cfg); } -- 2.18.0

1 0

[libvirt] [PATCH v2] lxc: Include support to lxc version 3.0 or higher.
by Julio Faracco 06 Nov '18

06 Nov '18

This patch introduce the new settings for LXC 3.0 or higher. The older versions keep the compatibility to deprecated settings for LXC, but after release 3.0, the compatibility was removed. This commit adds the support to the refactored settings. Signed-off-by: Julio Faracco <jcfaracco(a)gmail.com> --- src/lxc/lxc_native.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/src/lxc/lxc_native.c b/src/lxc/lxc_native.c index cbdec723dd..8604cbaf46 100644 --- a/src/lxc/lxc_native.c +++ b/src/lxc/lxc_native.c @@ -200,7 +200,9 @@ lxcSetRootfs(virDomainDefPtr def, int type = VIR_DOMAIN_FS_TYPE_MOUNT; VIR_AUTOFREE(char *) value = NULL; - if (virConfGetValueString(properties, "lxc.rootfs", &value) <= 0) + if (virConfGetValueString(properties, "lxc.rootfs", &value) <= 0 && + /* Legacy config keys were removed after release 3.0. */ + virConfGetValueString(properties, "lxc.rootfs.path", &value) <= 0) return -1; if (STRPREFIX(value, "/dev/")) @@ -684,7 +686,9 @@ lxcCreateConsoles(virDomainDefPtr def, virConfPtr properties) virDomainChrDefPtr console; size_t i; - if (virConfGetValueString(properties, "lxc.tty", &value) <= 0) + if (virConfGetValueString(properties, "lxc.tty", &value) <= 0 && + /* Legacy config keys were removed after release 3.0. */ + virConfGetValueString(properties, "lxc.tty.max", &value) <= 0) return 0; if (virStrToLong_i(value, NULL, 10, &nbttys) < 0) { @@ -724,7 +728,7 @@ lxcIdmapWalkCallback(const char *name, virConfValuePtr value, void *data) char type; unsigned long start, target, count; - if (STRNEQ(name, "lxc.id_map") || !value->str) + if (STRNEQ(name, "lxc.id_map") || STRNEQ(name, "lxc.idmap") || !value->str) return 0; if (sscanf(value->str, "%c %lu %lu %lu", &type, @@ -1041,7 +1045,9 @@ lxcParseConfigString(const char *config, if (VIR_STRDUP(vmdef->os.init, "/sbin/init") < 0) goto error; - if (virConfGetValueString(properties, "lxc.utsname", &value) <= 0 || + if ((virConfGetValueString(properties, "lxc.utsname", &value) <= 0 && + /* Legacy config keys were removed after release 3.0. */ + virConfGetValueString(properties, "lxc.uts.name", &value) <= 0) || VIR_STRDUP(vmdef->name, value) < 0) goto error; if (!vmdef->name && (VIR_STRDUP(vmdef->name, "unnamed") < 0)) @@ -1051,7 +1057,9 @@ lxcParseConfigString(const char *config, goto error; /* Look for fstab: we shouldn't have it */ - if (virConfGetValue(properties, "lxc.mount")) { + if (virConfGetValue(properties, "lxc.mount") || + /* Legacy config keys were removed after release 3.0. */ + virConfGetValue(properties, "lxc.mount.fstab")) { virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", _("lxc.mount found, use lxc.mount.entry lines instead")); goto error; -- 2.17.1

3 2

[libvirt] [PATCH v4 0/2] Refactor virFileIsCDROM to check cdrom status
by Han Han 06 Nov '18

06 Nov '18

v3: https://www.redhat.com/archives/libvir-list/2018-July/msg00675.html v2: https://www.redhat.com/archives/libvir-list/2018-July/msg00655.html v1: https://www.redhat.com/archives/libvir-list/2018-July/msg00103.html Refactor virFileIsCDROM to virFileCheckCDROM and add new argument filled with cdrom status. Give more detailed when fail to read cdrom backend vol. Han Han (2): util: Refactor virFileIsCDROM to virFileCheckCDROM storage: Improve error handling on cdrom backend src/libvirt_private.syms | 2 +- src/qemu/qemu_domain.c | 4 ++-- src/storage/storage_util.c | 32 ++++++++++++++++++++++++++++++++ src/util/virfile.c | 36 +++++++++++++++++++++++++++++++----- src/util/virfile.h | 12 +++++++++++- 5 files changed, 77 insertions(+), 9 deletions(-) -- 2.19.1

2 4

[libvirt] [PATCH V2 0/3] libxl: add support for soft reset
by Jim Fehlig 06 Nov '18

06 Nov '18

Patches 1 and 2 are new to V2 and make slight improvements to libxlDomainShutdownThread. Patch 3 is adjusted to work with Xen 4.6. Jim Fehlig (3): libxl: remove redundant calls to virObjectEventStateQueue libxl: Remove some goto labels in libxlDomainShutdownThread libxl: add support for soft reset src/libxl/libxl_domain.c | 99 ++++++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 30 deletions(-) -- 2.18.0

2 5

[libvirt] RFC: VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA is not functional?
by Han Han 06 Nov '18

06 Nov '18

Hello, I found snapshot APIs(like virDomainSnapshotNum) invoked with VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA will return 0 even there are internal no-metadata snapshots in the domain. Then I find the reason is in virDomainSnapshotObjListGetNames(): 937 /* If this common code is being used, we assume that all snapshots 938 ┆* have metadata, and thus can handle METADATA up front as an 939 ┆* all-or-none filter. XXX This might not always be true, if we 940 ┆* add the ability to track qcow2 internal snapshots without the 941 ┆* use of metadata. */ 942 if ((data.flags & VIR_DOMAIN_SNAPSHOT_FILTERS_METADATA) == 943 ┆ VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA) 944 ┆ return 0; So currently, with VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA, no snapshot will be returned. I checked the commit. It is really old(6478ec1673a Eric Blake 2012-08-13 18:09:12 -0600) . I guess it was initially designed for the function to list internal snapshots without metadata. However, now internal snapshot seems lower prioritized than external snapshot. Do we have plan to design this function? Since the APIs invoked with this flag don't return the **right** result, if the function for VIR_DOMAIN_SNAPSHOT_LIST_NO_METADATA will not be designed, how will we deal with the flag? Remove it from code OR update the docs to mark it unsupported? -- Best regards, ----------------------------------- Han Han Quality Engineer Redhat. Email: hhan(a)redhat.com Phone: +861065339333

2 2