October 2019 - Devel - libvirt List Archives

[libvirt] [PATCH RFC 00/40] qemu: Add support for incremental backups
by Peter Krempa 22 Nov '19

22 Nov '19

This series is my attempt to clean up and finish incremental backup as it was last posted in v10 here: https://www.redhat.com/archives/libvir-list/2019-August/msg01018.html Out of that series, patch 1 was dropped for now and I'll revisit it later. The patches adding docs, xml parsing and job plumbing were partially modified and stripped of currently unimplemented functionality. The qemu implementation was completely rewritten. Of the new series first 28 patches are cleanups and can be reviewed for pushing. The main incremental backup functionality is not complete yet and should not be pushed right now not even in experimental (disabled [1]) state due to possible API changes: - figure out whether we want our own event for the backup job or want to deal with the domain job event - if we go with the domain job event everything must be rewritten as a single domain job - otherwise plumbing for the event must be added - documentation must reflect the above after that will be sorted out the following needs to be done until we can finally enable it: - more testing. I merely verified that qemu generates files/exports NBD - blockjob support for bitmaps (for commit/pull/copy) - snapshot support for bitmaps - merging of bitmaps accross backing chain entries - figuring out a more elegant way to do the bitmap for the backup than creating two, one for the 'store' (for NBD server) and one for the disk itself. - ... Until then the following XML can be used to enable the functionality for testing without any guarantee [1]: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> ... <qemu:capabilities> <qemu:add capability='blockdev'/> <qemu:add capability='incremental-backup'/> </qemu:capabilities> </domain> Eric Blake (5): backup: Document new XML for backups backup: Introduce virDomainBackup APIs backup: Implement backup APIs for remote driver backup: Parse and output backup XML backup: Implement virsh support for backup Peter Krempa (35): util: hash: Add possibility to use simpler data free function in virHash util: hash: Add new constructor 'virHashNew' util: hash: Introduce virHashHasEntry qemu: domain: Split out setup of virStorageSource from qemu driver config qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData qemu: domain: clarify sematics of qemuDomainPrepareDiskSourceData qemu: domain: Tolerate NULL @disk in qemuDomainPrepareDiskSourceData conf: Remove virDomainDiskPathByName conf: Introduce virDomainDiskByTarget qemu: Replace use of virDomainDiskFindByBusAndDst with virDomainDiskByTarget conf: Remove unused virDomainDiskFindByBusAndDst Replace virDomainDiskByName by virDomainDiskByTarget in appropriate cases qemu: monitor: Introduce new interface to query-named-block-nodes qemu: block: Don't query monitor in qemuBlockStorageSourceCreateDetectSize qemu: checkpoint: Fix rollback and access to unlocked 'vm' when deleting checkpoints qemu: monitor: Remove non-transaction based dirty bitmap APIs conf: snapshot: Don't clear current snapshot when redefining an existing one conf: snapshot: Remove 'update_current' parameter from virDomainSnapshotRedefinePrep conf: Don't reuse variable for different object in virDomainCheckpointRedefinePrep conf: checkpoint: Don't clear current checkpoint when redefining an existing one conf: checkpoint: Don't clear current checkpoint when redefining qemu: checkpoint: Enforce that 'bitmap' name must match checkpoint name qemu: checkpoint: Split out checkpoint creation code qemu: checkpoint: Extract finalizing steps of checkpoint creation qemu: monitor: Add helper for generating data for block bitmap merging qemu: checkpoint: Use qemuMonitorTransactionBitmapMergeSourceAddBitmap qemu: blockjob: Refactor qemuBlockJobEventProcessConcludedTransition qemu: blockjob: Use 'g_free' in qemuBlockJobDataDispose Add 'backup' block job type qemu: monitor: Add support for blockdev-backup via 'transaction' qemu: domain: Track backup job data in the status XML qemu: blockjob: Track internal data for 'backup' blockjob tests: qemustatusxml2xml: Add test for 'pull' type backup job qemu: Implement backup job APIs and qemu handling qemu: blockjob: Implement concluded blockjob handler for backup blockjobs docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 168 ++++ docs/formatcheckpoint.html.in | 12 +- docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 219 +++++ examples/c/misc/event-test.c | 3 + include/libvirt/libvirt-domain.h | 29 +- libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + src/conf/Makefile.inc.am | 2 + src/conf/backup_conf.c | 518 ++++++++++ src/conf/backup_conf.h | 102 ++ src/conf/checkpoint_conf.c | 17 +- src/conf/domain_addr.c | 4 +- src/conf/domain_conf.c | 50 +- src/conf/domain_conf.h | 8 +- src/conf/snapshot_conf.c | 6 - src/conf/snapshot_conf.h | 1 - src/conf/virconftypes.h | 3 + src/driver-hypervisor.h | 20 + src/libvirt-domain-checkpoint.c | 7 +- src/libvirt-domain.c | 191 ++++ src/libvirt_private.syms | 13 +- src/libvirt_public.syms | 8 + src/libxl/libxl_driver.c | 2 +- src/qemu/Makefile.inc.am | 2 + src/qemu/qemu_backup.c | 895 ++++++++++++++++++ src/qemu/qemu_backup.h | 41 + src/qemu/qemu_block.c | 26 +- src/qemu/qemu_block.h | 5 +- src/qemu/qemu_blockjob.c | 171 ++-- src/qemu/qemu_blockjob.h | 19 + src/qemu/qemu_checkpoint.c | 232 +++-- src/qemu/qemu_checkpoint.h | 15 + src/qemu/qemu_domain.c | 161 +++- src/qemu/qemu_domain.h | 11 +- src/qemu/qemu_driver.c | 116 ++- src/qemu/qemu_hotplug.c | 2 +- src/qemu/qemu_migration.c | 3 +- src/qemu/qemu_monitor.c | 91 +- src/qemu/qemu_monitor.h | 49 +- src/qemu/qemu_monitor_json.c | 242 ++--- src/qemu/qemu_monitor_json.h | 33 +- src/remote/remote_driver.c | 3 + src/remote/remote_protocol.x | 53 +- src/remote_protocol-structs | 28 + src/test/test_driver.c | 2 +- src/util/vircgroup.c | 2 +- src/util/virhash.c | 85 +- src/util/virhash.h | 12 + tests/Makefile.am | 2 + tests/domainbackupxml2xmlin/backup-pull.xml | 9 + tests/domainbackupxml2xmlin/backup-push.xml | 9 + tests/domainbackupxml2xmlin/empty.xml | 1 + tests/domainbackupxml2xmlout/backup-pull.xml | 9 + tests/domainbackupxml2xmlout/backup-push.xml | 9 + tests/domainbackupxml2xmlout/empty.xml | 7 + tests/qemublocktest.c | 3 +- tests/qemumonitorjsontest.c | 53 +- .../qemustatusxml2xmldata/backup-pull-in.xml | 607 ++++++++++++ .../qemustatusxml2xmldata/backup-pull-out.xml | 1 + tests/qemuxml2xmltest.c | 2 + tests/virschematest.c | 2 + tools/Makefile.am | 1 + tools/virsh-backup.c | 209 ++++ tools/virsh-backup.h | 21 + tools/virsh-domain.c | 8 +- tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 37 + 71 files changed, 4133 insertions(+), 552 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/qemu/qemu_backup.c create mode 100644 src/qemu/qemu_backup.h create mode 100644 tests/domainbackupxml2xmlin/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlin/backup-push.xml create mode 100644 tests/domainbackupxml2xmlin/empty.xml create mode 100644 tests/domainbackupxml2xmlout/backup-pull.xml create mode 100644 tests/domainbackupxml2xmlout/backup-push.xml create mode 100644 tests/domainbackupxml2xmlout/empty.xml create mode 100644 tests/qemustatusxml2xmldata/backup-pull-in.xml create mode 120000 tests/qemustatusxml2xmldata/backup-pull-out.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 2.21.0

5 83

[libvirt] [PATCH 0/4] apparmor fixes triggered by multi disk snapshots
by Christian Ehrhardt 21 Nov '19

21 Nov '19

Hi, the bugs [1][2] that made me debug into this actually only need the last patch (one line), but while coming along I found several opportunities for minor improvements of the apparmor code in libvirt. But that way it became a 4 patch series around apparmor. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1746684 [2]: https://bugs.launchpad.net/libvirt/+bug/1845506 Christian Ehrhardt (4): virt-aa-helper: clarify command line options apparmor: drop useless call to get_profile_name apparmor: refactor AppArmorSetSecurityImageLabel apparmor: let AppArmorSetSecurityImageLabel append rules src/security/security_apparmor.c | 52 +++++++------------------------- src/security/virt-aa-helper.c | 14 +++++---- 2 files changed, 19 insertions(+), 47 deletions(-) -- 2.23.0

3 15

[libvirt] [PATCH 0/2] virt-aa-helper for shmem device
by Christian Ehrhardt 21 Nov '19

21 Nov '19

Cole was recently adding a few of the usual apparmor suspects to BZ 1761645 and I was taking a look at the low hanging fruits of it today. It isn't perfect, but would resolve the reported issue - so I'd appreciate a review. Limitations: - One could break the path creating elements in qemuBuildShmemBackendMemProps and qemuDomainPrepareShmemChardev into extra functions and then use those from virt-aa-helper. But I haven't done so yet and unless it is strictly required consider it too much for what we want/need to achieve here. - I haven't covered hotplug of shmem devices yet, it seems there is no infrastructure for their labels yet and I wasn't sure how important shmem-hotplug would even be to anyone. Christian Ehrhardt (2): virt-aa-helper: add rules for shmem devices virt-aa-helper: testcase for shmem devices src/security/virt-aa-helper.c | 35 +++++++++++++++++++++++++++++++++++ tests/virt-aa-helper-test | 15 +++++++++++++++ 2 files changed, 50 insertions(+) -- 2.23.0

3 18

[libvirt] [patch 1/1] virt-aa-helper: Add support for smartcard host-certificates
by Arnaud Patard 20 Nov '19

20 Nov '19

When emulating smartcard with host certificates, qemu needs to be able to read the certificates files, which is denied by apparmor. Add necessary code to add the smartcard certificates related directory to the apparmor profile. This code supports only this case smartcard 'host' and 'passthrough' settings are not supported, as I can't test them. Signed-off-by: Arnaud Patard <apatard(a)hupstream.com> Index: libvirt-5.0.0/src/security/virt-aa-helper.c =================================================================== --- libvirt-5.0.0.orig/src/security/virt-aa-helper.c +++ libvirt-5.0.0/src/security/virt-aa-helper.c @@ -1251,6 +1251,26 @@ get_files(vahControl * ctl) } } + for (i = 0; i < ctl->def->nsmartcards; i++) { + virDomainSmartcardDefPtr sc = ctl->def->smartcards[i]; + virDomainSmartcardType sc_type = sc->type; + char *sc_db = (char *)VIR_DOMAIN_SMARTCARD_DEFAULT_DATABASE; + if (sc->data.cert.database) + sc_db = sc->data.cert.database; + switch(sc_type) { + case VIR_DOMAIN_SMARTCARD_TYPE_HOST: + break; + case VIR_DOMAIN_SMARTCARD_TYPE_HOST_CERTIFICATES: + virBufferAsprintf(&buf, " \"%s/\" rk,\n", sc_db); + virBufferAsprintf(&buf, " \"%s/*\" rk,\n", sc_db); + break; + case VIR_DOMAIN_SMARTCARD_TYPE_PASSTHROUGH: + break; + case VIR_DOMAIN_SMARTCARD_TYPE_LAST: + break; + } + } + if (ctl->def->virtType == VIR_DOMAIN_VIRT_KVM) { for (i = 0; i < ctl->def->nnets; i++) { virDomainNetDefPtr net = ctl->def->nets[i];

3 3

[libvirt] [PATCH] lxc: Fix 'domblkstat' error with attached disk devices.
by jcfaracco＠gmail.com 19 Nov '19

19 Nov '19

From: Julio Faracco <jcfaracco(a)gmail.com> LXC was not working when you attached a new disk that points to block device. See: https://bugzilla.redhat.com/show_bug.cgi?id=1697115. Command line from virsh was showing problems with alias first (this feature is not supported) and after, problems to query block device. It was happening because extra disks were not being included into cgroup blkio.throttle properties and libvirt could not query this info. Applying those devices into 'allowed' list is not enough, libvirt should reset blkio.throttle as a workaround to include disks into container's cgroup directory. Before: virsh # domblkstat CentOS hda error: Failed to get block stats for domain 'CentOS' device 'hda' error: internal error: domain stats query failed Now: virsh # domblkstat CentOS hda hda rd_req 0 hda rd_bytes 0 hda wr_req 0 hda wr_bytes 0 Signed-off-by: Julio Faracco <jcfaracco(a)gmail.com> --- src/lxc/lxc_cgroup.c | 29 ++++++++++++++++++++++++++++- src/lxc/lxc_cgroup.h | 4 ++++ src/lxc/lxc_driver.c | 8 ++++---- 3 files changed, 36 insertions(+), 5 deletions(-) diff --git a/src/lxc/lxc_cgroup.c b/src/lxc/lxc_cgroup.c index 5efb495b56..de6d892521 100644 --- a/src/lxc/lxc_cgroup.c +++ b/src/lxc/lxc_cgroup.c @@ -302,6 +302,24 @@ virLXCTeardownHostUSBDeviceCgroup(virUSBDevicePtr dev G_GNUC_UNUSED, return 0; } +int +virLXCCgroupResetBlkioDeviceThrottle(virCgroupPtr cgroup, + const char *path) +{ + if (virCgroupSetBlkioDeviceReadBps(cgroup, path, 0) < 0) + return -1; + + if (virCgroupSetBlkioDeviceWriteBps(cgroup, path, 0) < 0) + return -1; + + if (virCgroupSetBlkioDeviceReadIops(cgroup, path, 0) < 0) + return -1; + + if (virCgroupSetBlkioDeviceWriteIops(cgroup, path, 0) < 0) + return -1; + + return 0; +} static int virLXCCgroupSetupDeviceACL(virDomainDefPtr def, virCgroupPtr cgroup) @@ -309,6 +327,7 @@ static int virLXCCgroupSetupDeviceACL(virDomainDefPtr def, int capMknod = def->caps_features[VIR_DOMAIN_CAPS_FEATURE_MKNOD]; int ret = -1; size_t i; + const char *src_path = NULL; static virLXCCgroupDevicePolicy devices[] = { {'c', LXC_DEV_MAJ_MEMORY, LXC_DEV_MIN_NULL}, {'c', LXC_DEV_MAJ_MEMORY, LXC_DEV_MIN_ZERO}, @@ -346,8 +365,16 @@ static int virLXCCgroupSetupDeviceACL(virDomainDefPtr def, !virStorageSourceIsBlockLocal(def->disks[i]->src)) continue; + /* Workaround to include disks into blkio.throttle. + * To include it, we need to reset any feature of + * blkio.throttle.* */ + src_path = virDomainDiskGetSource(def->disks[i]); + if (virLXCCgroupResetBlkioDeviceThrottle(cgroup, + src_path) < 0) + goto cleanup; + if (virCgroupAllowDevicePath(cgroup, - virDomainDiskGetSource(def->disks[i]), + src_path, (def->disks[i]->src->readonly ? VIR_CGROUP_DEVICE_READ : VIR_CGROUP_DEVICE_RW) | diff --git a/src/lxc/lxc_cgroup.h b/src/lxc/lxc_cgroup.h index 63e9e837b0..3d643a4fea 100644 --- a/src/lxc/lxc_cgroup.h +++ b/src/lxc/lxc_cgroup.h @@ -46,3 +46,7 @@ int virLXCTeardownHostUSBDeviceCgroup(virUSBDevicePtr dev, const char *path, void *opaque); + +int +virLXCCgroupResetBlkioDeviceThrottle(virCgroupPtr cgroup, + const char *path); diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 204a3ed522..87da55f308 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -2339,14 +2339,14 @@ lxcDomainBlockStats(virDomainPtr dom, goto endjob; } - if (!disk->info.alias) { + if (!disk->src->path) { virReportError(VIR_ERR_INTERNAL_ERROR, _("missing disk device alias name for %s"), disk->dst); goto endjob; } ret = virCgroupGetBlkioIoDeviceServiced(priv->cgroup, - disk->info.alias, + disk->src->path, &stats->rd_bytes, &stats->wr_bytes, &stats->rd_req, @@ -2424,14 +2424,14 @@ lxcDomainBlockStatsFlags(virDomainPtr dom, goto endjob; } - if (!disk->info.alias) { + if (!disk->src->path) { virReportError(VIR_ERR_INTERNAL_ERROR, _("missing disk device alias name for %s"), disk->dst); goto endjob; } if (virCgroupGetBlkioIoDeviceServiced(priv->cgroup, - disk->info.alias, + disk->src->path, &rd_bytes, &wr_bytes, &rd_req, -- 2.20.1

3 2

[libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
by Daniel P. Berrangé 18 Nov '19

18 Nov '19

This series is an effort to reduce the number of different languages we use by eliminating most use of perl in favour of python. This aligns with fact that the likely future build system we'll use (meson) is written in python, and that python is much more commonly used/understood by developers these days than perl. With this applied we use perl in a handful of places only: - src/rpc/gendispatch.pl - this is a horrendously large script and very hard to understand/follow. A straight syntax conversion to Python would still leave a hgue and hard to understand/follow script. It really needs a clean room rewrite from scratch, with better structure. - src/rpc/genprotocol.pl - fairly easy to convert, but might be obsolete depending on approach for rewriting gendispatch.pl, so ignored for now - tests/oomtrace.pl - will be purge by the patches that drop OOM handling anyway - tools/wireshark/util/genxdrstub.pl - a very large script, which I haven't got the courage to tackle yet. - cfg.mk/maint.mk - many syntax rules involve regexes which are fed to perl. Decision on what to do with syntax-check rules punted to another time. - build-aux/gitlog-to-changelog - build-aux/useless-if-before-free - Both pulled in from gnulib. Could be rewritten quite easily if desired, but given that we aren't maintainers of them right now, they're ignored as they don't really impact our developers. In v2: - Pulled in patch to hacking file - Converted many more scripts - Forced UTF-8 character set to avoid ascii codec on py3 < 3.7 Daniel P. Berrangé (24): docs: document that C & Python are the preferred languages cfg.mk: fix comment detection for python semicolon check build: force a UTF-8 locale for python build-aux: rewrite augest test generator in Python build-aux: rewrite po file minimizer in Python build-aux: rewrite duplicate header checker in Python build-aux: rewrite whitespace checker in Python build-aux: rewrite mock inline checker in Python build-aux: rewrite header ifdef checker in Python src: rewrite ACL permissions checker in Python src: rewrite symfile sorting checker in Python src: rewrite symfile library checker in Python src: rewrite systemtap probe generator in Python src: rewrite systemtap function generator in Python src: rewrite driver name checker in Python src: rewrite driver impl checker in Python src: rewrite ACL rule checker in Python src: rewrite polkit ACL generator in Python src: rewrite remote protocol checker in Python tests: rewrite test argv line wrapper in Python tests: rewrite qemu capability grouper in Python tests: rewrite file access checker in Python docs: rewrite hvsupport.html page generator in python docs: rewrite polkit docs generator in Python Makefile.am | 14 +- build-aux/augeas-gentest.pl | 60 ---- build-aux/augeas-gentest.py | 72 ++++ build-aux/check-spacing.pl | 198 ---------- build-aux/check-spacing.py | 204 +++++++++++ build-aux/header-ifdef.pl | 182 ---------- build-aux/header-ifdef.py | 206 +++++++++++ build-aux/minimize-po.pl | 37 -- build-aux/minimize-po.py | 60 ++++ build-aux/mock-noinline.pl | 75 ---- build-aux/mock-noinline.py | 88 +++++ build-aux/prohibit-duplicate-header.pl | 26 -- build-aux/prohibit-duplicate-header.py | 54 +++ cfg.mk | 32 +- configure.ac | 8 + docs/Makefile.am | 17 +- docs/genaclperms.pl | 125 ------- docs/genaclperms.py | 122 +++++++ docs/hacking.html.in | 30 ++ docs/hvsupport.pl | 458 ----------------------- docs/hvsupport.py | 479 +++++++++++++++++++++++++ po/Makefile.am | 2 +- src/Makefile.am | 156 +++----- src/access/Makefile.inc.am | 6 +- src/access/genpolkit.pl | 119 ------ src/access/genpolkit.py | 119 ++++++ src/bhyve/Makefile.inc.am | 4 +- src/check-aclperms.pl | 73 ---- src/check-aclperms.py | 77 ++++ src/check-aclrules.pl | 252 ------------- src/check-aclrules.py | 244 +++++++++++++ src/check-driverimpls.pl | 80 ----- src/check-driverimpls.py | 96 +++++ src/check-drivername.pl | 83 ----- src/check-drivername.py | 112 ++++++ src/check-remote-protocol.py | 131 +++++++ src/check-symfile.pl | 70 ---- src/check-symfile.py | 80 +++++ src/check-symsorting.pl | 106 ------ src/check-symsorting.py | 112 ++++++ src/dtrace2systemtap.pl | 130 ------- src/dtrace2systemtap.py | 140 ++++++++ src/esx/Makefile.inc.am | 2 +- src/hyperv/Makefile.inc.am | 2 +- src/interface/Makefile.inc.am | 2 +- src/libxl/Makefile.inc.am | 4 +- src/locking/Makefile.inc.am | 6 +- src/logging/Makefile.inc.am | 2 +- src/lxc/Makefile.inc.am | 4 +- src/network/Makefile.inc.am | 2 +- src/node_device/Makefile.inc.am | 2 +- src/nwfilter/Makefile.inc.am | 2 +- src/qemu/Makefile.inc.am | 4 +- src/remote/Makefile.inc.am | 4 +- src/rpc/Makefile.inc.am | 2 +- src/rpc/gensystemtap.pl | 193 ---------- src/rpc/gensystemtap.py | 174 +++++++++ src/secret/Makefile.inc.am | 2 +- src/storage/Makefile.inc.am | 2 +- src/util/Makefile.inc.am | 8 +- src/vbox/Makefile.inc.am | 2 +- src/vz/Makefile.inc.am | 2 +- tests/Makefile.am | 4 +- tests/check-file-access.pl | 126 ------- tests/check-file-access.py | 121 +++++++ tests/file_access_whitelist.txt | 2 +- tests/group-qemu-caps.pl | 124 ------- tests/group-qemu-caps.py | 115 ++++++ tests/test-wrap-argv.pl | 174 --------- tests/test-wrap-argv.py | 162 +++++++++ tests/testutils.c | 16 +- 71 files changed, 3132 insertions(+), 2872 deletions(-) delete mode 100755 build-aux/augeas-gentest.pl create mode 100755 build-aux/augeas-gentest.py delete mode 100755 build-aux/check-spacing.pl create mode 100755 build-aux/check-spacing.py delete mode 100644 build-aux/header-ifdef.pl create mode 100644 build-aux/header-ifdef.py delete mode 100755 build-aux/minimize-po.pl create mode 100755 build-aux/minimize-po.py delete mode 100644 build-aux/mock-noinline.pl create mode 100644 build-aux/mock-noinline.py delete mode 100644 build-aux/prohibit-duplicate-header.pl create mode 100644 build-aux/prohibit-duplicate-header.py delete mode 100755 docs/genaclperms.pl create mode 100755 docs/genaclperms.py delete mode 100755 docs/hvsupport.pl create mode 100755 docs/hvsupport.py delete mode 100755 src/access/genpolkit.pl create mode 100755 src/access/genpolkit.py delete mode 100755 src/check-aclperms.pl create mode 100755 src/check-aclperms.py delete mode 100755 src/check-aclrules.pl create mode 100755 src/check-aclrules.py delete mode 100755 src/check-driverimpls.pl create mode 100755 src/check-driverimpls.py delete mode 100755 src/check-drivername.pl create mode 100644 src/check-drivername.py create mode 100644 src/check-remote-protocol.py delete mode 100755 src/check-symfile.pl create mode 100755 src/check-symfile.py delete mode 100755 src/check-symsorting.pl create mode 100755 src/check-symsorting.py delete mode 100755 src/dtrace2systemtap.pl create mode 100755 src/dtrace2systemtap.py delete mode 100755 src/rpc/gensystemtap.pl create mode 100755 src/rpc/gensystemtap.py delete mode 100755 tests/check-file-access.pl create mode 100755 tests/check-file-access.py delete mode 100755 tests/group-qemu-caps.pl create mode 100755 tests/group-qemu-caps.py delete mode 100755 tests/test-wrap-argv.pl create mode 100755 tests/test-wrap-argv.py -- 2.21.0

4 42

[libvirt] [PATCH 0/7] net/qemu: move vlan/bandwidth validation out of network driver
by Laine Stump 15 Nov '19

15 Nov '19

As usual, the first 6 patches are just torquing stuff around to make the last patch simple (except that patch 1 makes interface validation errors more useful). 2-5 are just converting a bunch of accessor functions to tak/return const pointers, since that's what's available in the places where I wanted to use them. This does actually fix a documented bug: https://bugzilla.redhat.com/1741121 (that one is about vlan tag validation. There may also be one about bandwidth, but I didn't see it right away, so I gave up, as is my custom). Laine Stump (7): qemu: add mac address to error messages in qemuDomainValidateActualNetDef conf: make virDomainNetGetActualVlan arg/return val const conf: make virDomainNetGetActualBandwidth arg/return value const conf: return a const from virDomainNetGetActualVirtPortProfile conf: change args/return values of remaining virDomainNetGetActual*() to const conf: add hypervisor agnostic, domain start-time, validation function for NetDef net/qemu: move vlan/bandwidth validation out of network driver src/conf/domain_conf.c | 78 ++++++++++++++++++++++++---- src/conf/domain_conf.h | 21 ++++---- src/conf/netdev_bandwidth_conf.c | 2 +- src/conf/netdev_bandwidth_conf.h | 2 +- src/conf/netdev_vport_profile_conf.c | 2 +- src/conf/netdev_vport_profile_conf.h | 2 +- src/libvirt_private.syms | 1 + src/libxl/libxl_conf.c | 6 +-- src/libxl/libxl_domain.c | 4 ++ src/libxl/libxl_driver.c | 4 ++ src/libxl/xen_common.c | 4 +- src/lxc/lxc_driver.c | 8 ++- src/lxc/lxc_process.c | 16 +++--- src/network/bridge_driver.c | 37 ------------- src/qemu/qemu_command.c | 2 +- src/qemu/qemu_domain.c | 35 ++++++++----- src/qemu/qemu_hotplug.c | 6 +-- src/qemu/qemu_migration_cookie.c | 2 +- src/qemu/qemu_process.c | 2 +- src/util/virhostdev.c | 8 +-- src/util/virnetdev.c | 4 +- src/util/virnetdev.h | 2 +- src/util/virnetdevbandwidth.c | 6 +-- src/util/virnetdevbandwidth.h | 4 +- src/util/virnetdevmacvlan.c | 20 +++---- src/util/virnetdevmacvlan.h | 10 ++-- src/util/virnetdevmidonet.c | 4 +- src/util/virnetdevmidonet.h | 4 +- src/util/virnetdevopenvswitch.c | 8 +-- src/util/virnetdevopenvswitch.h | 6 +-- src/util/virnetdevtap.c | 12 ++--- src/util/virnetdevtap.h | 12 ++--- src/util/virnetdevvportprofile.c | 12 ++--- src/util/virnetdevvportprofile.h | 12 ++--- tests/bhyvexml2argvmock.c | 4 +- 35 files changed, 206 insertions(+), 156 deletions(-) -- 2.21.0

3 10

[libvirt] [PATCH v3 0/9] Enable ramfb parameter for mediated devices
by Jonathon Jongsma 14 Nov '19

14 Nov '19

This is the third version of a patch series that adds support for a boot display for mediated vgpu devices using the 'ramfb' parameter. This version fixes a bunch of test failures that I had inadvertently introduced in the last series. It also splits the 4th patch up into several separate patches as suggested by Cole to make things more readable. Jonathon Jongsma (9): qemu: fix domain device validation qemu: use g_autoptr in qemuDomainDeviceDefValidate() qemu: Set capabilities properly for tests qemu: set domain capability for ramfb device qemu: set domain capability for video type "none" qemu: validate vhost-user video backend in qemu_domain.c qemu: move validation of video accel to qemu_domain.c qemu: use domain caps to validate video device model qemu: add 'ramfb' attribute for mediated devices docs/formatdomain.html.in | 8 ++ docs/schemas/domaincommon.rng | 5 ++ src/conf/domain_capabilities.c | 17 +++- src/conf/domain_conf.c | 11 +++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 3 + src/qemu/qemu_command.c | 17 +++- src/qemu/qemu_domain.c | 61 ++++++------- src/qemu/qemu_process.c | 61 ------------- .../qemu_1.7.0.x86_64.xml | 1 + .../qemu_2.12.0-virt.aarch64.xml | 1 + .../qemu_2.12.0.ppc64.xml | 1 + .../qemu_2.12.0.s390x.xml | 1 + .../qemu_2.12.0.x86_64.xml | 1 + .../qemu_2.6.0-virt.aarch64.xml | 1 + .../qemu_2.6.0.aarch64.xml | 1 + .../domaincapsschemadata/qemu_2.6.0.ppc64.xml | 1 + .../qemu_2.6.0.x86_64.xml | 1 + .../domaincapsschemadata/qemu_2.7.0.s390x.xml | 1 + .../qemu_2.8.0-tcg.x86_64.xml | 1 + .../domaincapsschemadata/qemu_2.8.0.s390x.xml | 1 + .../qemu_2.8.0.x86_64.xml | 1 + .../qemu_2.9.0-q35.x86_64.xml | 1 + .../qemu_2.9.0-tcg.x86_64.xml | 1 + .../qemu_2.9.0.x86_64.xml | 1 + .../domaincapsschemadata/qemu_3.0.0.s390x.xml | 2 + .../qemu_3.1.0.x86_64.xml | 2 + .../domaincapsschemadata/qemu_4.0.0.s390x.xml | 1 + .../qemu_4.0.0.x86_64.xml | 2 + .../qemu_4.1.0.x86_64.xml | 2 + .../qemu_4.2.0.aarch64.xml | 2 + .../domaincapsschemadata/qemu_4.2.0.ppc64.xml | 1 + .../qemu_4.2.0.x86_64.xml | 2 + tests/qemuhotplugtest.c | 3 + ...tdev-mdev-display-ramfb.x86_64-latest.args | 37 ++++++++ .../hostdev-mdev-display-ramfb.xml | 33 +++++++ tests/qemuxml2argvtest.c | 2 + tests/qemuxml2xmltest.c | 89 +++++++++++-------- tests/securityselinuxlabeltest.c | 9 ++ 39 files changed, 254 insertions(+), 133 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-ramfb.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/hostdev-mdev-display-ramfb.xml -- 2.21.0

2 11

[libvirt] [PATCH v2 0/2] a few cleanups in qemu_hotplug.c
by Daniel Henrique Barboza 14 Nov '19

14 Nov '19

There is not much to do regarding cleanup and g_auto* in this file. Here's a couple of simple fixes I've found. changes from v1: - added a new patch to remove unused cleanup labels v1: https://www.redhat.com/archives/libvir-list/2019-September/msg01463.html Daniel Henrique Barboza (2): qemu_hotplug.c: use g_autoptr() with virConnectPtr qemu_process: remove unused cleanup labels src/qemu/qemu_hotplug.c | 72 +++++++++++++++++------------------------ 1 file changed, 29 insertions(+), 43 deletions(-) -- 2.21.0

2 3

[libvirt] [PATCH] libxl: Fix lock manager lock ordering
by Jim Fehlig 14 Nov '19

14 Nov '19

The ordering of lock manager locks in the libxl driver has a flaw that was uncovered by a migration error path. In the perform phase of migration, the source host calls virDomainLockProcessPause to release the lock before sending the VM to the destination host. If the send fails an attempt is made to reacquire the lock with virDomainLockProcessResume, but that too can fail if the destination host has not finished cleaning up the failed VM and releasing the lock it acquired when starting to receive the VM. This change delays calling virDomainLockProcessResume in libxlDomainStart until the VM is successfully created, but before it is unpaused. A similar approach is used by the qemu driver, avoiding the need to release the lock if VM creation fails. In the migration perform phase, releasing the lock with virDomainLockProcessPause is delayed until the VM is successfully sent to the destination, which avoids reacquiring the lock if the send fails. Signed-off-by: Jim Fehlig <jfehlig(a)suse.com> --- src/libxl/libxl_domain.c | 14 +++++++------- src/libxl/libxl_migration.c | 14 +++++--------- 2 files changed, 12 insertions(+), 16 deletions(-) diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c index 4073bf8d46..a830a19b99 100644 --- a/src/libxl/libxl_domain.c +++ b/src/libxl/libxl_domain.c @@ -1364,13 +1364,6 @@ libxlDomainStart(libxlDriverPrivatePtr driver, NULL) < 0) goto cleanup; - if (virDomainLockProcessResume(driver->lockManager, - "xen:///system", - vm, - priv->lockState) < 0) - goto cleanup; - VIR_FREE(priv->lockState); - if (libxlNetworkPrepareDevices(vm->def) < 0) goto cleanup_dom; @@ -1453,6 +1446,13 @@ libxlDomainStart(libxlDriverPrivatePtr driver, libxlLoggerOpenFile(cfg->logger, domid, vm->def->name, config_json); + if (virDomainLockProcessResume(driver->lockManager, + "xen:///system", + vm, + priv->lockState) < 0) + goto destroy_dom; + VIR_FREE(priv->lockState); + /* Always enable domain death events */ if (libxl_evenable_domain_death(cfg->ctx, vm->def->id, 0, &priv->deathW)) goto destroy_dom; diff --git a/src/libxl/libxl_migration.c b/src/libxl/libxl_migration.c index a1021d499b..8e64dc5d04 100644 --- a/src/libxl/libxl_migration.c +++ b/src/libxl/libxl_migration.c @@ -1253,20 +1253,16 @@ libxlDomainMigrationSrcPerform(libxlDriverPrivatePtr driver, sockfd = virNetSocketDupFD(sock, true); virObjectUnref(sock); - if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lockState) < 0) - VIR_WARN("Unable to release lease on %s", vm->def->name); - VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); - /* suspend vm and send saved data to dst through socket fd */ virObjectUnlock(vm); ret = libxlDoMigrateSrcSend(driver, vm, flags, sockfd); virObjectLock(vm); - if (ret < 0) { - virDomainLockProcessResume(driver->lockManager, - "xen:///system", - vm, - priv->lockState); + if (ret == 0) { + if (virDomainLockProcessPause(driver->lockManager, vm, &priv->lockState) < 0) + VIR_WARN("Unable to release lease on %s", vm->def->name); + VIR_DEBUG("Preserving lock state '%s'", NULLSTR(priv->lockState)); + } else { /* * Confirm phase will not be executed if perform fails. End the * job started in begin phase. -- 2.23.0

2 2