November 2019 - Devel - libvirt List Archives

[libvirt] [PATCH v3 0/6] PCI hostdev partial assignment support
by Daniel Henrique Barboza 13 Dec '19

13 Dec '19

changes from previous version [1]: - do not overload address type='none'. A new address type called 'unassigned' is introduced; - there is no unassigned' flag being created in virDomainHostdevDef. The logic added by new address type is enough; - do not allow function zero of multifunction devices to be unassigned. Nothing too special to discuss in this cover letter. More details can be found at the discussions of the previous version [1]. Commit messages of the patches have more background info as well. [1] https://www.redhat.com/archives/libvir-list/2019-November/msg01099.html Daniel Henrique Barboza (6): Introducing new address type='unassigned' for PCI hostdevs qemu: handle unassigned PCI hostdevs in command line and alias virhostdev.c: check all IOMMU devs in virHostdevPreparePCIDevices formatdomain.html.in: document <address type='unassigned'/> utils: PCI multifunction detection helpers domain_conf.c: don't allow function zero to be unassigned docs/formatdomain.html.in | 14 +++ docs/schemas/domaincommon.rng | 5 ++ src/conf/device_conf.c | 2 + src/conf/device_conf.h | 1 + src/conf/domain_conf.c | 20 ++++- src/libvirt_private.syms | 2 + src/qemu/qemu_alias.c | 6 ++ src/qemu/qemu_command.c | 5 ++ src/qemu/qemu_domain.c | 1 + src/qemu/qemu_domain_address.c | 5 ++ src/util/virhostdev.c | 88 +++++++++++++++++-- src/util/virhostdev.h | 3 + src/util/virpci.c | 17 ++++ src/util/virpci.h | 2 + .../hostdev-pci-address-unassigned.args | 31 +++++++ .../hostdev-pci-address-unassigned.xml | 42 +++++++++ ...pci-multifunction-zero-unassigned-fail.xml | 42 +++++++++ tests/qemuxml2argvtest.c | 8 ++ .../hostdev-pci-address-unassigned.xml | 58 ++++++++++++ tests/qemuxml2xmltest.c | 1 + tests/virpcimock.c | 9 +- 21 files changed, 351 insertions(+), 11 deletions(-) create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-unassigned.args create mode 100644 tests/qemuxml2argvdata/hostdev-pci-address-unassigned.xml create mode 100644 tests/qemuxml2argvdata/hostdev-pci-multifunction-zero-unassigned-fail.xml create mode 100644 tests/qemuxml2xmloutdata/hostdev-pci-address-unassigned.xml -- 2.23.0

2 9

[libvirt] [PATCH 0/2] configure: Provide OpenRC scripts for sub-daemons
by Michal Privoznik 13 Dec '19

13 Dec '19

Tested on my Gentoo and it works (TM). I've copied dependencies from systemd unit files. I was also thinking about providing a dummy init script that would do nothing but depend on the rest of sub-daemon init files for convenience: /etc/init.d/libvirt-dummy start (the name is still WIP) But I'm not sure if we want that - after we drop libvirtd we can use libvirtd script just for that. But if somebody wants it, I can write and post it as a follow up patch. Michal Prívozník (2): configure: Provide OpenRC scripts for sub-daemons news: Document init scripts docs/news.xml | 11 ++++++ m4/virt-init-script.m4 | 8 +++- src/Makefile.am | 56 +++++++++++++++++++++++++++- src/interface/Makefile.inc.am | 11 ++++++ src/interface/virtinterfaced.init.in | 26 +++++++++++++ src/libxl/Makefile.inc.am | 10 +++++ src/libxl/virtxend.init.in | 26 +++++++++++++ src/lxc/Makefile.inc.am | 10 +++++ src/lxc/virtlxcd.init.in | 26 +++++++++++++ src/network/Makefile.inc.am | 10 +++++ src/network/virtnetworkd.init.in | 26 +++++++++++++ src/node_device/Makefile.inc.am | 11 ++++++ src/node_device/virtnodedevd.init.in | 26 +++++++++++++ src/nwfilter/Makefile.inc.am | 11 ++++++ src/nwfilter/virtnwfilterd.init.in | 26 +++++++++++++ src/qemu/Makefile.inc.am | 10 +++++ src/qemu/virtqemud.init.in | 26 +++++++++++++ src/remote/Makefile.inc.am | 30 +++++++++++++++ src/remote/libvirtd.confd | 18 +++++++++ src/remote/libvirtd.init.in | 29 ++++++++++++++ src/remote/virtproxyd.confd | 10 +++++ src/remote/virtproxyd.init.in | 28 ++++++++++++++ src/secret/Makefile.inc.am | 10 +++++ src/secret/virtsecretd.init.in | 26 +++++++++++++ src/storage/Makefile.inc.am | 10 +++++ src/storage/virtstoraged.init.in | 26 +++++++++++++ src/vbox/Makefile.inc.am | 10 +++++ src/vbox/virtvboxd.init.in | 26 +++++++++++++ src/vz/Makefile.inc.am | 10 +++++ src/vz/virtvzd.init.in | 26 +++++++++++++ 30 files changed, 587 insertions(+), 2 deletions(-) create mode 100644 src/interface/virtinterfaced.init.in create mode 100644 src/libxl/virtxend.init.in create mode 100644 src/lxc/virtlxcd.init.in create mode 100644 src/network/virtnetworkd.init.in create mode 100644 src/node_device/virtnodedevd.init.in create mode 100644 src/nwfilter/virtnwfilterd.init.in create mode 100644 src/qemu/virtqemud.init.in create mode 100644 src/remote/libvirtd.confd create mode 100644 src/remote/libvirtd.init.in create mode 100644 src/remote/virtproxyd.confd create mode 100644 src/remote/virtproxyd.init.in create mode 100644 src/secret/virtsecretd.init.in create mode 100644 src/storage/virtstoraged.init.in create mode 100644 src/vbox/virtvboxd.init.in create mode 100644 src/vz/virtvzd.init.in -- 2.23.0

2 4

[libvirt] [RFC PATCH 0/9] add virtiofs support (virtio-fs epopee)
by Ján Tomko 12 Dec '19

12 Dec '19

https://bugzilla.redhat.com/show_bug.cgi?id=1694166 A work in progress, but I'm sending it now to get the XML bikesh^Wdiscussion out of the way. TODO: * documentation (due to the requirement of shared memory and therefore a <numa> topology specification, this probably deserves a separate article on https://libvirt.org/kbase.html ) * validation - refuse unsupported combinations and block migration * remove a few hardcoded values * figure out the binary path from qemu interop config * leave queue-size and cache-size on their default values or expose them in XML * use FD passing for vhost-user-fs device as well * clean up the code Tested with: commit 320000e72ec0613e164ce9608d865396fb2da278 Author: Linus Torvalds <torvalds(a)linux-foundation.org> Date: Wed Oct 30 14:17:18 2019 +0100 Merge tag 'iommu-fixes-v5.4-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu and commit d3d2efbb0e37494708dde359a9d0f492f84833e8 Author: Dr. David Alan Gilbert <dgilbert(a)redhat.com> CommitDate: 2019-10-24 11:50:55 +0100 virtio-fs: Allow mapping of meta data version table from https://gitlab.com/virtio-fs/qemu/tree/virtio-fs-dev git fetch https://gitlab.com/virtio-fs/qemu.git virtio-fs-dev (not yet merged upstream) Ján Tomko (9): qemu: address: take fsdriver type into account qemu: cmdline: take fsdriver into account qemu: add QEMU_CAPS_VHOST_USER_FS wip: tests: add xml2xml tests for virtio-fs wip: add XML -> ARGV tests for virtio-fs conf: qemu: add virtio-fs fsdriver type qemu: pass private data to qemuBuildFilesystemCommandLine wip: start virtiofsd wip: build vhost-user-fs device command line docs/schemas/domaincommon.rng | 1 + src/conf/domain_conf.c | 1 + src/conf/domain_conf.h | 2 + src/qemu/qemu_capabilities.c | 4 + src/qemu/qemu_capabilities.h | 3 + src/qemu/qemu_command.c | 67 +++++- src/qemu/qemu_domain.c | 4 + src/qemu/qemu_domain_address.c | 35 +++- src/qemu/qemu_extdevice.c | 191 ++++++++++++++++++ ...vhost-user-fs-fd-memory.x86_64-latest.args | 38 ++++ .../vhost-user-fs-fd-memory.xml | 38 ++++ ...vhost-user-fs-hugepages.x86_64-latest.args | 42 ++++ .../vhost-user-fs-hugepages.xml | 67 ++++++ tests/qemuxml2argvtest.c | 9 + .../vhost-user-fs-fd-memory.x86_64-latest.xml | 1 + .../vhost-user-fs-hugepages.x86_64-latest.xml | 1 + tests/qemuxml2xmltest.c | 3 + 17 files changed, 493 insertions(+), 14 deletions(-) create mode 100644 tests/qemuxml2argvdata/vhost-user-fs-fd-memory.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-fs-fd-memory.xml create mode 100644 tests/qemuxml2argvdata/vhost-user-fs-hugepages.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/vhost-user-fs-hugepages.xml create mode 120000 tests/qemuxml2xmloutdata/vhost-user-fs-fd-memory.x86_64-latest.xml create mode 120000 tests/qemuxml2xmloutdata/vhost-user-fs-hugepages.x86_64-latest.xml -- 2.21.0

5 19

[libvirt] [PATCH v2 00/14] drop usage of c-type gnulib module
by Pavel Hrdina 10 Dec '19

10 Dec '19

Changes in v2: - fixed missing parentheses in patch 01 - added patch 02 to cover c_isascii Pavel Hrdina (14): util: define IS_BLANK instead of using c_isblank from gnulib virkeyfile: define IS_ASCII instead c_isascii from gnulib use g_ascii_isalnum instead of c_isalnum from gnulib use g_ascii_isalpha instead of c_isalpha from gnulib use g_ascii_iscntrl instead of c_iscntrl from gnulib use g_ascii_isdigit instead of c_isdigit frum gnulib use g_ascii_islower instead of c_islower from gnulib use g_ascii_isprint instead of c_isprint from gnulib use g_ascii_isspace instead of c_isspace from gnulib use g_ascii_isxdigit instead of c_isxdigit from gnulib use g_ascii_tolower instead of c_tolower from gnulib use g_ascii_toupper instead of c_toupper from gnulib syntax-check: update c-type checks to refer to Glib bootstrap.conf: drop usage of c-type gnulib module bootstrap.conf | 1 - build-aux/syntax-check.mk | 11 ++--------- src/bhyve/bhyve_parse_command.c | 3 +-- src/conf/network_conf.c | 3 +-- src/conf/nwfilter_conf.c | 3 +-- src/interface/interface_backend_udev.c | 5 ++--- src/libxl/libxl_conf.c | 3 +-- src/node_device/node_device_udev.c | 3 +-- src/qemu/qemu_agent.c | 3 +-- src/qemu/qemu_domain.c | 3 +-- src/qemu/qemu_monitor.c | 3 +-- src/qemu/qemu_qapi.c | 4 +--- src/remote/remote_driver.c | 3 +-- src/rpc/virnetsocket.c | 3 +-- src/storage/parthelper.c | 5 ++--- src/util/virbitmap.c | 5 ++--- src/util/virconf.c | 18 +++++++++--------- src/util/virfile.c | 8 +++----- src/util/virhostcpu.c | 7 +++---- src/util/virkeyfile.c | 13 +++++++------ src/util/virmacaddr.c | 11 +++++------ src/util/virnetdevvportprofile.c | 3 +-- src/util/virpidfile.c | 3 +-- src/util/virstoragefile.c | 3 +-- src/util/virstring.c | 13 ++++++------- src/util/virutil.c | 11 +++++------ src/util/viruuid.c | 9 ++++----- src/vmx/vmx.c | 8 +++----- tools/virsh-console.c | 3 +-- tools/virsh-domain.c | 3 +-- tools/vsh-table.c | 5 ++--- tools/vsh.c | 5 ++--- 32 files changed, 73 insertions(+), 111 deletions(-) -- 2.23.0

2 18

[libvirt] libvirt mdev migration, mdevctl integration
by Alex Williamson 10 Dec '19

10 Dec '19

Hey folks, We had some discussions at KVM Forum around mdev live migration and what that might mean for libvirt handling of mdev devices and potential libvirt/mdevctl[1] flows. I believe the current situation is that libvirt knows nothing about an mdev beyond the UUID in the XML. It expects the mdev to exist on the system prior to starting the VM. The intention is for mdevctl to step in here by providing persistence for mdev devices such that these pre-defined mdevs are potentially not just ephemeral, for example, we can tag specific mdevs for automatic startup on each boot. It seems the next step in this journey is to figure out if libvirt can interact with mdevctl to "manage" a device. I believe we've avoided defining managed='yes' behavior for mdev hostdevs up to this point because creating an mdev device involves policy decisions. For example, which parent device hosts the mdev, are there optimal NUMA considerations, are there performance versus power considerations, what is the nature of the mdev, etc. mdevctl doesn't necessarily want to make placement decisions either, but it does understand how to create and remove an mdev, what it's type is, associate it to a fixed parent, apply attributes, etc. So would it be reasonable that for a manage='yes' mdev hostdev device, libvirt might attempt to use mdevctl to start an mdev by UUID and stop it when the VM is shutdown? This assumes the mdev referenced by the UUID is already defined and known to mdevct. I'd expect semantics much like managed='yes' around vfio-pci binding, ex. start/stop if it doesn't exist, leave it alone if it already exists. If that much seems reasonable, and someone is willing to invest some development time to support it, what are then the next steps to enable migration? AIUI, libvirt blindly assumes hostdev devices cannot be migrated. This may already be getting some work due to Jens' network failover support where the attached hostdev doesn't really migrate, but it allows the migration to proceed in a partially detached state so that it can jump back into action should the migration fail. Long term we expect that not only some mdev hostdevs might be migratable, but possibly some regular vfio-pci hostdevs as well. I think libvirt will need to remove any assumptions around hostdev migration and rather rely on introspection of the QEMU process to determine if any devices hold migration blockers (or simply try the migration and let QEMU fail quickly if there are blockers). So assuming we now have a VM with a managed='yes' mdev hostdev device, what do we need to do to reproduce that device at the migration target? mdevctl can dump a device in a json format, where libvirt could use this to define and start an equivalent device on the migration target (potentially this json is extended by mdevctl to include the migration compatibility vendor string). Part of our discussion at the Forum was around the extent to which libvirt would want to consider this json opaque. For instance, while libvirt doesn't currently support localhost migration, libvirt might want to use an alternate UUID for the mdev device on the migration target so as not to introduce additional barriers to such migrations. Potentially mdevctl could accept the json from the source system as a template and allow parameters such as UUID to be overwritten by commandline options. This might allow libvirt to consider the json as opaque. An issue here though is that the json will also include the parent device, which we obviously cannot assume is the same (particularly the bus address) on the migration target. We can allow commandline overrides for the parent just as we do above for the UUID when defining the mdev device from json, but it's an open issue who is going to be smart enough (perhaps dumb enough) to claim this responsibility. It would be interesting to understand how libvirt handles other host specific information during migration, for instance if node or processor affinities are part of the VM XML, how is that translated to the target? I could imagine that we could introduce a simple "first available" placement in mdevctl, but maybe there should minimally be a node allocation preference with optional enforcement (similar to numactl), or maybe something above libvirt needs to take this responsibility to prepare the target before we get ourselves into trouble. Anyway, I hope this captures some of what was discussed at KVM Forum and that we can continue that discussion here to map out the design and tasks to enable vfio/mdev hostdev migration in libvirt. Thanks, Alex [1] https://github.com/mdevctl/mdevctl

5 13

[libvirt] [PATCH] security: apparmor: Label externalDataStore
by Cole Robinson 10 Dec '19

10 Dec '19

Teach virt-aa-helper how to label a qcow2 data_file, tracked internally as externalDataStore. It should be treated the same as its sibling disk image Signed-off-by: Cole Robinson <crobinso(a)redhat.com> --- Compiled but not runtime tested, I don't have an apparmor setup src/security/virt-aa-helper.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/security/virt-aa-helper.c b/src/security/virt-aa-helper.c index 509187ac36..fe6fa12550 100644 --- a/src/security/virt-aa-helper.c +++ b/src/security/virt-aa-helper.c @@ -949,6 +949,10 @@ storage_source_add_files(virStorageSourcePtr src, if (add_file_path(tmp, depth, buf) < 0) return -1; + if (src->externalDataStore && + storage_source_add_files(src->externalDataStore, buf, depth) < 0) + return -1; + depth++; } -- 2.23.0

3 4

[libvirt] [PATCH v2] qemu: cold-unplug of sound
by Jidong Xia 09 Dec '19

09 Dec '19

With this patch users can cold unplug some sound devices. use "virsh detach-device vm sound.xml --config" command. Signed-off-by: Jidong Xia <xiajidong(a)cmss.chinamobile.com> --- src/conf/domain_conf.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ src/conf/domain_conf.h | 3 +++ src/libvirt_private.syms | 2 ++ src/qemu/qemu_driver.c | 9 ++++++++- 4 files changed, 61 insertions(+), 1 deletion(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 9580884..4b28502 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2753,6 +2753,14 @@ void virDomainSoundDefFree(virDomainSoundDefPtr def) VIR_FREE(def); } +virDomainSoundDefPtr +virDomainSoundDefRemove(virDomainDefPtr def, size_t idx) +{ + virDomainSoundDefPtr ret = def->sounds[idx]; + VIR_DELETE_ELEMENT(def->sounds, idx, def->nsounds); + return ret; +} + void virDomainMemballoonDefFree(virDomainMemballoonDefPtr def) { if (!def) @@ -14554,6 +14562,46 @@ virDomainSoundDefParseXML(virDomainXMLOptionPtr xmlopt, } +static bool +virDomainSoundDefEquals(const virDomainSoundDef *a, + const virDomainSoundDef *b) +{ + size_t i; + + if (a->model != b->model) + return false; + + if (a->ncodecs != b->ncodecs) + return false; + + for (i = 0; i < a->ncodecs; i++) { + if (a->codecs[i]->type != b->codecs[i]->type) + return false; + } + + if (a->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE && + !virDomainDeviceInfoAddressIsEqual(&a->info, &b->info)) + return false; + + return true; +} + + +ssize_t +virDomainSoundDefFind(const virDomainDef *def, + const virDomainSoundDef *sound) +{ + size_t i; + + for (i = 0; i < def->nsounds; i++) { + if (virDomainSoundDefEquals(sound, def->sounds[i])) + return i; + } + + return -1; +} + + static virDomainWatchdogDefPtr virDomainWatchdogDefParseXML(virDomainXMLOptionPtr xmlopt, xmlNodePtr node, diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 49cb697..cdb7b92 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -2883,7 +2883,10 @@ void virDomainChrDefFree(virDomainChrDefPtr def); int virDomainChrSourceDefCopy(virDomainChrSourceDefPtr dest, virDomainChrSourceDefPtr src); void virDomainSoundCodecDefFree(virDomainSoundCodecDefPtr def); +ssize_t virDomainSoundDefFind(const virDomainDef *def, + const virDomainSoundDef *sound); void virDomainSoundDefFree(virDomainSoundDefPtr def); +virDomainSoundDefPtr virDomainSoundDefRemove(virDomainDefPtr def, size_t idx); void virDomainMemballoonDefFree(virDomainMemballoonDefPtr def); void virDomainNVRAMDefFree(virDomainNVRAMDefPtr def); void virDomainWatchdogDefFree(virDomainWatchdogDefPtr def); diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index 8fe0bf9..cdbd64f 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -578,7 +578,9 @@ virDomainSmartcardTypeFromString; virDomainSmartcardTypeToString; virDomainSmbiosModeTypeFromString; virDomainSmbiosModeTypeToString; +virDomainSoundDefFind; virDomainSoundDefFree; +virDomainSoundDefRemove; virDomainSoundModelTypeFromString; virDomainSoundModelTypeToString; virDomainStartupPolicyTypeFromString; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 18bd010..8d91bb2 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -8302,6 +8302,7 @@ qemuDomainDetachDeviceConfig(virDomainDefPtr vmdef, { virDomainDiskDefPtr disk, det_disk; virDomainNetDefPtr net; + virDomainSoundDefPtr sound; virDomainHostdevDefPtr hostdev, det_hostdev; virDomainLeaseDefPtr lease, det_lease; virDomainControllerDefPtr cont, det_cont; @@ -8330,6 +8331,13 @@ qemuDomainDetachDeviceConfig(virDomainDefPtr vmdef, virDomainNetDefFree(virDomainNetRemove(vmdef, idx)); break; + case VIR_DOMAIN_DEVICE_SOUND: + sound = dev->data.sound; + if ((idx = virDomainSoundDefFind(vmdef, sound)) < 0) + return -1; + virDomainSoundDefFree(virDomainSoundDefRemove(vmdef, idx)); + break; + case VIR_DOMAIN_DEVICE_HOSTDEV: { hostdev = dev->data.hostdev; if ((idx = virDomainHostdevFind(vmdef, hostdev, &det_hostdev)) < 0) { @@ -8460,7 +8468,6 @@ qemuDomainDetachDeviceConfig(virDomainDefPtr vmdef, vmdef->vsock = NULL; break; - case VIR_DOMAIN_DEVICE_SOUND: case VIR_DOMAIN_DEVICE_VIDEO: case VIR_DOMAIN_DEVICE_GRAPHICS: case VIR_DOMAIN_DEVICE_HUB: -- 1.8.3.1

2 2

[libvirt] [PATCH 0/4] drop usage of c-strcase gnulib module
by Pavel Hrdina 09 Dec '19

09 Dec '19

Pavel Hrdina (4): use g_ascii_strcasecmp instead of c_strcasecmp from gnulib use g_ascii_strncasecmp instead of c_strncasecmp from gnulib syntax-check: update strcase check to refer to GLib bootstrap.conf: drop usage of c-strcase gnulib module bootstrap.conf | 1 - build-aux/syntax-check.mk | 2 +- src/internal.h | 11 +++++------ tests/virhashtest.c | 2 +- 4 files changed, 7 insertions(+), 9 deletions(-) -- 2.23.0

2 5

[libvirt] [PATCH v2 0/4] Update news for versions from 5.7 to 5.8
by Han Han 06 Dec '19

06 Dec '19

Remove news of unmanaged macvtap devices since Laine has update that. v1: https://www.redhat.com/archives/libvir-list/2019-November/msg00176.html Han Han (4): news: Support for kvm dedicated performance hint in 5.7 news: Support to run SLIRP in a separate process in 5.8 news: Support vhost-user-gpu in 5.8 news: Introduce virConnectSetIdentity API in 5.8 docs/news.xml | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) -- 2.23.0

2 10

[libvirt] [PATCH v5 00/23] scripts: convert most perl scripts to python
by Daniel P. Berrangé 04 Dec '19

04 Dec '19

This series is an effort to reduce the number of different languages we use by eliminating most use of perl in favour of python. This aligns with fact that the likely future build system we'll use (meson) is written in python, and that python is much more commonly used/understood by developers these days than perl. With this applied we use perl in a handful of places only: - src/rpc/gendispatch.pl - this is a horrendously large script and very hard to understand/follow. A straight syntax conversion to Python would still leave a hgue and hard to understand/follow script. It really needs a clean room rewrite from scratch, with better structure. - src/rpc/genprotocol.pl - fairly easy to convert, but might be obsolete depending on approach for rewriting gendispatch.pl, so ignored for now - tests/oomtrace.pl - will be purge by the patches that drop OOM handling anyway - tools/wireshark/util/genxdrstub.pl - a very large script, which I haven't got the courage to tackle yet. - cfg.mk/maint.mk - many syntax rules involve regexes which are fed to perl. Decision on what to do with syntax-check rules punted to another time. - build-aux/gitlog-to-changelog - build-aux/useless-if-before-free - Both pulled in from gnulib. Could be rewritten quite easily if desired, but given that we aren't maintainers of them right now, they're ignored as they don't really impact our developers. Note that the check-spacing.py script is significantly slower in Python than in Perl. After researching this it appears there is nothing that can be done. The Perl regex engine is simply much better optimized than the Python one. As previously discussed we need to loook at uncrustify or clang-format or some other tool to validate whitespace formatting. This is ongoing. We can either accept the slow down in the short term or keep the Perl version in the short term. In v5: - Rebased to cope with changes to require VPATH build - Merged the already-ACKd scripts In v4: - Moved all scripts into the scripts/ directory instead of having them scattered around source tree - Use re.search instead of re.match - Don't bother re-compiling regexes In v3: - All scripts comply with all flake8 style rules with exception of E129 visually indented line with same indent as next logical line In v2: - Pulled in patch to hacking file - Converted many more scripts - Forced UTF-8 character set to avoid ascii codec on py3 < 3.7 Daniel P. Berrangé (23): build-aux: rewrite duplicate header checker in Python build-aux: rewrite whitespace checker in Python build-aux: rewrite mock inline checker in Python build-aux: rewrite header ifdef checker in Python src: rewrite ACL permissions checker in Python src: rewrite symfile sorting checker in Python src: rewrite symfile library checker in Python src: rewrite systemtap probe generator in Python src: rewrite systemtap function generator in Python src: rewrite driver name checker in Python src: rewrite driver impl checker in Python src: rewrite ACL rule checker in Python src: rewrite polkit ACL generator in Python src: rewrite remote protocol checker in Python tests: rewrite test argv line wrapper in Python tests: rewrite qemu capability grouper in Python tests: rewrite file access checker in Python docs: rewrite hvsupport.html page generator in python docs: rewrite polkit docs generator in Python docs: move apibuild.py to the scripts/ directory docs: move reformat-news.py to the scripts/ directory docs: move esx_vi_generator.py to the scripts/ directory docs: move hyperv_wmi_generator.py to the scripts/ directory Makefile.am | 33 +- build-aux/check-spacing.pl | 198 ------- build-aux/header-ifdef.pl | 182 ------- build-aux/mock-noinline.pl | 75 --- build-aux/prohibit-duplicate-header.pl | 26 - build-aux/syntax-check.mk | 32 +- docs/Makefile.am | 16 +- docs/genaclperms.pl | 125 ----- docs/hvsupport.pl | 459 ---------------- {docs => scripts}/apibuild.py | 0 scripts/check-aclperms.py | 75 +++ scripts/check-aclrules.py | 263 +++++++++ scripts/check-driverimpls.py | 102 ++++ scripts/check-drivername.py | 114 ++++ scripts/check-file-access.py | 123 +++++ scripts/check-remote-protocol.py | 136 +++++ scripts/check-spacing.py | 229 ++++++++ scripts/check-symfile.py | 80 +++ scripts/check-symsorting.py | 117 ++++ scripts/dtrace2systemtap.py | 143 +++++ {src/esx => scripts}/esx_vi_generator.py | 0 scripts/genaclperms.py | 123 +++++ scripts/genpolkit.py | 122 +++++ scripts/gensystemtap.py | 184 +++++++ scripts/group-qemu-caps.py | 123 +++++ scripts/header-ifdef.py | 231 ++++++++ scripts/hvsupport.py | 504 ++++++++++++++++++ .../hyperv_wmi_generator.py | 0 scripts/mock-noinline.py | 85 +++ scripts/prohibit-duplicate-header.py | 56 ++ {docs => scripts}/reformat-news.py | 0 scripts/test-wrap-argv.py | 170 ++++++ src/Makefile.am | 146 ++--- src/access/Makefile.inc.am | 6 +- src/access/genpolkit.pl | 119 ----- src/admin/Makefile.inc.am | 6 +- src/check-aclperms.pl | 73 --- src/check-aclrules.pl | 252 --------- src/check-driverimpls.pl | 80 --- src/check-drivername.pl | 83 --- src/check-symfile.pl | 70 --- src/check-symsorting.pl | 106 ---- src/dtrace2systemtap.pl | 130 ----- src/esx/Makefile.inc.am | 6 +- src/hyperv/Makefile.inc.am | 5 +- src/rpc/Makefile.inc.am | 1 - src/rpc/gensystemtap.pl | 193 ------- tests/Makefile.am | 3 +- tests/check-file-access.pl | 126 ----- tests/file_access_whitelist.txt | 2 +- tests/group-qemu-caps.pl | 124 ----- tests/test-wrap-argv.pl | 174 ------ tests/testutils.c | 16 +- 53 files changed, 3092 insertions(+), 2755 deletions(-) delete mode 100755 build-aux/check-spacing.pl delete mode 100644 build-aux/header-ifdef.pl delete mode 100644 build-aux/mock-noinline.pl delete mode 100644 build-aux/prohibit-duplicate-header.pl delete mode 100755 docs/genaclperms.pl delete mode 100755 docs/hvsupport.pl rename {docs => scripts}/apibuild.py (100%) create mode 100755 scripts/check-aclperms.py create mode 100755 scripts/check-aclrules.py create mode 100755 scripts/check-driverimpls.py create mode 100644 scripts/check-drivername.py create mode 100755 scripts/check-file-access.py create mode 100644 scripts/check-remote-protocol.py create mode 100755 scripts/check-spacing.py create mode 100755 scripts/check-symfile.py create mode 100755 scripts/check-symsorting.py create mode 100755 scripts/dtrace2systemtap.py rename {src/esx => scripts}/esx_vi_generator.py (100%) create mode 100755 scripts/genaclperms.py create mode 100755 scripts/genpolkit.py create mode 100755 scripts/gensystemtap.py create mode 100755 scripts/group-qemu-caps.py create mode 100644 scripts/header-ifdef.py create mode 100755 scripts/hvsupport.py rename {src/hyperv => scripts}/hyperv_wmi_generator.py (100%) create mode 100644 scripts/mock-noinline.py create mode 100644 scripts/prohibit-duplicate-header.py rename {docs => scripts}/reformat-news.py (100%) create mode 100755 scripts/test-wrap-argv.py delete mode 100755 src/access/genpolkit.pl delete mode 100755 src/check-aclperms.pl delete mode 100755 src/check-aclrules.pl delete mode 100755 src/check-driverimpls.pl delete mode 100755 src/check-drivername.pl delete mode 100755 src/check-symfile.pl delete mode 100755 src/check-symsorting.pl delete mode 100755 src/dtrace2systemtap.pl delete mode 100755 src/rpc/gensystemtap.pl delete mode 100755 tests/check-file-access.pl delete mode 100755 tests/group-qemu-caps.pl delete mode 100755 tests/test-wrap-argv.pl -- 2.21.0

4 65