- Devel - libvirt List Archives

[libvirt] [jenkins-ci PATCH v2] libvirt-ocaml: add a simple check job
by Pino Toscano 03 Sep '19

03 Sep '19

Run 'make check-manifest' as build step, so we ensure that the MANIFEST file is up-to-date, and thus dist will (hopefully) work. Signed-off-by: Pino Toscano <ptoscano(a)redhat.com> --- Changes from v1: - run 'make check-manifest' as separate job guests/playbooks/build/projects/libvirt-ocaml.yml | 4 ++++ jenkins/projects/libvirt-ocaml.yaml | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/guests/playbooks/build/projects/libvirt-ocaml.yml b/guests/playbooks/build/projects/libvirt-ocaml.yml index 7a2e216..58f017b 100644 --- a/guests/playbooks/build/projects/libvirt-ocaml.yml +++ b/guests/playbooks/build/projects/libvirt-ocaml.yml @@ -25,3 +25,7 @@ ./configure --prefix=$VIRT_PREFIX $MAKE $MAKE opt +- include: '{{ playbook_base }}/jobs/generic-check-job.yml' + vars: + command: | + $MAKE check-manifest diff --git a/jenkins/projects/libvirt-ocaml.yaml b/jenkins/projects/libvirt-ocaml.yaml index 8eb03e3..0c819cc 100644 --- a/jenkins/projects/libvirt-ocaml.yaml +++ b/jenkins/projects/libvirt-ocaml.yaml @@ -21,3 +21,7 @@ ./configure --prefix=$VIRT_PREFIX $MAKE $MAKE opt + - generic-check-job: + parent_jobs: 'libvirt-ocaml-build' + command: | + $MAKE check-manifest -- 2.21.0

2 1

[libvirt] [PATCH] news: Update for 5.7.0 release
by Michal Privoznik 03 Sep '19

03 Sep '19

Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com> --- docs/news.xml | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index 0c79765fd0..f95af0f0e9 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -50,6 +50,30 @@ for Hyper-V guests. </description> </change> + <change> + <summary> + lib: Add virDomainGetGuestInfo() + </summary> + <description> + This API is intended to aggregate several guest agent information + queries and is inspired by stats API + <code>virDomainListGetStats()</code>. It is anticipated that this + information will be provided by a guest agent + running within the domain. It's exposed as <code>virsh + guestinfo</code>. + </description> + </change> + <change> + <summary> + Split libvirtd into separate daemons + </summary> + <description> + The big monolithic libvirtd daemon can now be replaced by smaller + per-driver daemons. Distributions can chose if they want the former + or the latter. The libvirtd is still kept around for backwards + compatibility. + </description> + </change> </section> <section title="Removed features"> <change> @@ -75,8 +99,117 @@ <code>--bandwidth</code> parameter. </description> </change> + <change> + <summary> + libxl: Implement domain metadata getter/setter + </summary> + <description> + The libxl driver now supports <code>virDomainGetMetadata()</code> and + <code>virDomainSetMetadata()</code> APIs. + </description> + </change> + <change> + <summary> + test driver: Expand API coverage + </summary> + <description> + Additional APIs have been implemented in the test driver. + </description> + </change> + <change> + <summary> + Report RNG device in domain capabilities XML + </summary> + <description> + Libvirt now reports if RNG device is supported by underlying + hypervisor in domain capabilities XML. + </description> + </change> + <change> + <summary> + Stop linking virt-login-shell and NSS plugins with libvirt.so + </summary> + <description> + In order to allow libvirt abort on out of memory, we need to stop + link libvirt.so to virt-login-shell or the NSS plugins where we don't + want to abort. This change also resulted in smaller binaries and + libraries. + </description> + </change> + <change> + <summary> + qemu: Allow migration with disk cache on + </summary> + <description> + When QEMU supports flushing caches at the end of migration, we can + safely allow migration even if <code>disk/driver/@cache</code> is not + <code>none</code> nor <code>directsync</code>. + </description> + </change> </section> <section title="Bug fixes"> + <change> + <summary> + Various security label remembering fixes + </summary> + <description> + In the previous release libvirt introduced remembering of original + owners and SELinux labels on files. However, the feature did not work + properly with snapshots, on migrations or on network filesystems. + This is now fixed. + </description> + </change> + <change> + <summary> + Allow greater PCI domain numbers + </summary> + <description> + Libvirt used to require PCI domain number to be not greater than + 0xFFFF. The code was changed to allow 32 bits long numbers. + </description> + </change> + <change> + <summary> + Various D-Bus fixes + </summary> + <description> + When D-Bus is not available, Libvirt was reporting random errors. + These are now gone. + </description> + </change> + <change> + <summary> + Prefer read only opening of PCI config files + </summary> + <description> + When enumerating PCI bus, Libvirt opens config files under + <code>sysfs</code> mount and parses them to learn various aspects of + the device (e.g. its capabilities). Only in a very limited number of + cases it is actually writing into the file. However, it used to open + the file also for writing even if it was only reading from it. + </description> + </change> + <change> + <summary> + Fix AppArmor profile + </summary> + <description> + Since the <code>5.6.0</code> release, Libvirt uses + <code>procfs</code> to learn the list of opened file descriptors when + spawning a command. However, our AppArmor profile was not allowing + such access. + </description> + </change> + <change> + <summary> + Don't block storage driver when starting or building a pool + </summary> + <description> + Starting or building a storage pool can take a long time to finish. + During this time the storage driver was blocked and thus no other API + involving the storage driver could run. This is now fixed. + </description> + </change> </section> </release> <release version="v5.6.0" date="2019-08-05"> -- 2.21.0

3 3

[libvirt] [jenkins-ci PATCH] libvirt-ocaml: run 'make check-manifest' as build step
by Pino Toscano 03 Sep '19

03 Sep '19

This ensures that the MANIFEST file is up-to-date, and thus dist will (hopefully) work. Signed-off-by: Pino Toscano <ptoscano(a)redhat.com> --- guests/playbooks/build/projects/libvirt-ocaml.yml | 1 + jenkins/projects/libvirt-ocaml.yaml | 1 + 2 files changed, 2 insertions(+) diff --git a/guests/playbooks/build/projects/libvirt-ocaml.yml b/guests/playbooks/build/projects/libvirt-ocaml.yml index 7a2e216..d293738 100644 --- a/guests/playbooks/build/projects/libvirt-ocaml.yml +++ b/guests/playbooks/build/projects/libvirt-ocaml.yml @@ -25,3 +25,4 @@ ./configure --prefix=$VIRT_PREFIX $MAKE $MAKE opt + $MAKE check-manifest diff --git a/jenkins/projects/libvirt-ocaml.yaml b/jenkins/projects/libvirt-ocaml.yaml index 8eb03e3..158dee1 100644 --- a/jenkins/projects/libvirt-ocaml.yaml +++ b/jenkins/projects/libvirt-ocaml.yaml @@ -21,3 +21,4 @@ ./configure --prefix=$VIRT_PREFIX $MAKE $MAKE opt + $MAKE check-manifest -- 2.21.0

2 3

[libvirt] [dockerfiles PATCH v2 0/3] Update libosinfo (& friends) dockerfiles
by Fabiano Fidêncio 03 Sep '19

03 Sep '19

As osinfo-db-tools has recently switched to using meson instead of autotools, let's update the libosinfo-dockerfiles accordingly. Together with this change, let's also drop the "cross" Dockerfiles, as those were never working due to some gobject dependency not being multi architecture friendly on Debian. https://gitlab.com/fidencio/libvirt-dockerfiles/tree/wip/osinfo_db_tools_me… Fabiano Fidêncio (3): refresh: Remove "cross" Dockerfiles refresh: Remove Dockerfiles from OSes not supported by osinfo-db-tools refresh: Update Dockerfiles due to new osinfo-db-tools build system buildenv-libosinfo-centos-7.zip | Bin 458 -> 0 bytes buildenv-libosinfo-debian-10-cross-aarch64.zip | Bin 683 -> 0 bytes buildenv-libosinfo-debian-10-cross-armv6l.zip | Bin 682 -> 0 bytes buildenv-libosinfo-debian-10-cross-armv7l.zip | Bin 686 -> 0 bytes buildenv-libosinfo-debian-10-cross-i686.zip | Bin 682 -> 0 bytes buildenv-libosinfo-debian-10-cross-mips.zip | Bin 678 -> 0 bytes buildenv-libosinfo-debian-10-cross-mips64el.zip | Bin 689 -> 0 bytes buildenv-libosinfo-debian-10-cross-mipsel.zip | Bin 681 -> 0 bytes buildenv-libosinfo-debian-10-cross-ppc64le.zip | Bin 687 -> 0 bytes buildenv-libosinfo-debian-10-cross-s390x.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-10.zip | Bin 555 -> 559 bytes buildenv-libosinfo-debian-9-cross-aarch64.zip | Bin 682 -> 0 bytes buildenv-libosinfo-debian-9-cross-armv6l.zip | Bin 681 -> 0 bytes buildenv-libosinfo-debian-9-cross-armv7l.zip | Bin 685 -> 0 bytes buildenv-libosinfo-debian-9-cross-mips.zip | Bin 678 -> 0 bytes buildenv-libosinfo-debian-9-cross-mips64el.zip | Bin 689 -> 0 bytes buildenv-libosinfo-debian-9-cross-mipsel.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-9-cross-ppc64le.zip | Bin 687 -> 0 bytes buildenv-libosinfo-debian-9-cross-s390x.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-9.zip | Bin 554 -> 0 bytes buildenv-libosinfo-debian-sid-cross-aarch64.zip | Bin 683 -> 0 bytes buildenv-libosinfo-debian-sid-cross-armv6l.zip | Bin 682 -> 0 bytes buildenv-libosinfo-debian-sid-cross-armv7l.zip | Bin 686 -> 0 bytes buildenv-libosinfo-debian-sid-cross-i686.zip | Bin 683 -> 0 bytes buildenv-libosinfo-debian-sid-cross-mips.zip | Bin 678 -> 0 bytes ...denv-libosinfo-debian-sid-cross-mips64el.zip | Bin 689 -> 0 bytes buildenv-libosinfo-debian-sid-cross-mipsel.zip | Bin 681 -> 0 bytes buildenv-libosinfo-debian-sid-cross-ppc64le.zip | Bin 687 -> 0 bytes buildenv-libosinfo-debian-sid-cross-s390x.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-sid.zip | Bin 555 -> 559 bytes buildenv-libosinfo-fedora-29.zip | Bin 488 -> 492 bytes buildenv-libosinfo-fedora-30.zip | Bin 551 -> 555 bytes buildenv-libosinfo-fedora-rawhide.zip | Bin 507 -> 512 bytes buildenv-libosinfo-ubuntu-16.zip | Bin 558 -> 0 bytes buildenv-libosinfo-ubuntu-18.zip | Bin 558 -> 0 bytes 35 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 buildenv-libosinfo-centos-7.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-aarch64.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-armv6l.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-armv7l.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-i686.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-mips.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-mips64el.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-mipsel.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-ppc64le.zip delete mode 100644 buildenv-libosinfo-debian-10-cross-s390x.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-aarch64.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-armv6l.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-armv7l.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mips.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mips64el.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mipsel.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-ppc64le.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-s390x.zip delete mode 100644 buildenv-libosinfo-debian-9.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-aarch64.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-armv6l.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-armv7l.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-i686.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-mips.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-mips64el.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-mipsel.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-ppc64le.zip delete mode 100644 buildenv-libosinfo-debian-sid-cross-s390x.zip delete mode 100644 buildenv-libosinfo-ubuntu-16.zip delete mode 100644 buildenv-libosinfo-ubuntu-18.zip -- 2.21.0

2 6

[libvirt] [ocaml PATCH 0/6] Split/reorganize sources
by Pino Toscano 03 Sep '19

03 Sep '19

This series splits the sources in the 3 parts built on their own (instead of a whole source including everything), so it is easier to rebuild on changes. Also, improve their file names. Pino Toscano (6): Rename libvirt_c_prologue.c to libvirt_c.h Turn libvirt_c.h as proper header Build C sources separately Rename the generated libvirt_c.c to libvirt_generated.c Build libvirt_generated.c with -Wmissing-prototypes Rename libvirt_c_epilogue.c to libvirt_c_common.c .gitignore | 2 +- MANIFEST | 4 +- Makefile.in | 2 +- libvirt/Makefile.in | 22 +++---- libvirt/README | 40 ++++--------- libvirt/generator.pl | 29 ++------- libvirt/libvirt.mli | 6 +- libvirt/{libvirt_c_prologue.c => libvirt_c.h} | 60 +++++++++++++------ ...ibvirt_c_epilogue.c => libvirt_c_common.c} | 40 +++++++------ libvirt/libvirt_c_oneoffs.c | 2 + 10 files changed, 99 insertions(+), 108 deletions(-) rename libvirt/{libvirt_c_prologue.c => libvirt_c.h} (79%) rename libvirt/{libvirt_c_epilogue.c => libvirt_c_common.c} (97%) -- 2.21.0

2 10

[libvirt] [PATCH 0/1] Rewrite virt-host-validate in Go & Go packaging complication
by Daniel P. Berrangé 03 Sep '19

03 Sep '19

This series contains code I mostly wrote some 6 months ago as an experiment. I've tidied it up and attempted to integrate it into the libvirt build system. Overall I much prefer the new impl of virt-host-validate, as explained in the commit message, I've wanted todo it this way for a long time, but was put off by C's XML handling verbosity. The main problem I've hit in integrating this work is the RPM build process integration. When building interactively, Go will dynamically download the 3rd party deps from git, similar to how we dynamically pull down gnulib. The difference is that this must happen for end users using tarballs too, not just devs usingn git. We could bundle the deps with our tarball the same way we do with gnulib, but Fedora would not accept using this, as it wants unbundled Go deps. I've done some things in the spec which makes this work against unbundled sources, but this still fails in mock as we don't get the build-id set in the resulting binary. I have attempted to fix that by passing extra flags to go build, but it still fails, so I've got some bug there. The resulting spec rules are very very very unpleasant, having to hardcode a bunch of build flags we should never touch. I've spoken with Nichlos Mailhot who is the Fedora Go packaging expert, who has basically confirmed my experiance that there is no nice solution to what I'm trying todo with the build process. He strongly recommends that we do *not* try to integrate Go into our existing build process, and that we instead keep any Go code as a completely seperate git repo that we build & install separately. This would make the RPM packaging much much simpler to deal with. Given what I've tried here, I'm inclined to agree with his recommendation. In the case of virt-host-validate, this is actually quite a straightforward thing to deal with, as the new impl of virt-host-validate has no direct dep wrt existing lkibvirt code in either direction. So we can trivially put it in a separate git repo. We would merely need an RPM dep from libvirt.spec.in to make the sure the external project gets pulled in on install of libvirt-daemon. Thus this patch is *NOT* something I'm proposing to merge as is. The code itself is fine, but it will need to go into a new git repo of its own. Daniel P. Berrangé (1): tools: rewrite virt-host-validate in Go to be data driven configure.ac | 1 + libvirt.spec.in | 23 + m4/virt-golang.m4 | 46 ++ m4/virt-host-validate.m4 | 8 +- po/POTFILES | 5 - tools/Makefile.am | 59 +- tools/host-validate/go.mod | 8 + tools/host-validate/go.sum | 4 + tools/host-validate/main.go | 98 +++ tools/host-validate/pkg/engine.go | 469 +++++++++++ tools/host-validate/pkg/facts.go | 784 +++++++++++++++++++ tools/host-validate/pkg/facts_test.go | 67 ++ tools/host-validate/pkg/xml_utils.go | 287 +++++++ tools/host-validate/rules/acpi.xml | 29 + tools/host-validate/rules/builtin.xml | 33 + tools/host-validate/rules/cgroups.xml | 403 ++++++++++ tools/host-validate/rules/cpu.xml | 148 ++++ tools/host-validate/rules/devices.xml | 54 ++ tools/host-validate/rules/freebsd-kernel.xml | 14 + tools/host-validate/rules/iommu.xml | 93 +++ tools/host-validate/rules/namespaces.xml | 94 +++ tools/host-validate/rules/pci.xml | 9 + tools/virt-host-validate-bhyve.c | 77 -- tools/virt-host-validate-bhyve.h | 24 - tools/virt-host-validate-common.c | 419 ---------- tools/virt-host-validate-common.h | 85 -- tools/virt-host-validate-lxc.c | 87 -- tools/virt-host-validate-lxc.h | 24 - tools/virt-host-validate-qemu.c | 116 --- tools/virt-host-validate-qemu.h | 24 - tools/virt-host-validate.c | 152 ---- tools/virt-host-validate.pod | 12 +- 32 files changed, 2693 insertions(+), 1063 deletions(-) create mode 100644 m4/virt-golang.m4 create mode 100644 tools/host-validate/go.mod create mode 100644 tools/host-validate/go.sum create mode 100644 tools/host-validate/main.go create mode 100644 tools/host-validate/pkg/engine.go create mode 100644 tools/host-validate/pkg/facts.go create mode 100644 tools/host-validate/pkg/facts_test.go create mode 100644 tools/host-validate/pkg/xml_utils.go create mode 100644 tools/host-validate/rules/acpi.xml create mode 100644 tools/host-validate/rules/builtin.xml create mode 100644 tools/host-validate/rules/cgroups.xml create mode 100644 tools/host-validate/rules/cpu.xml create mode 100644 tools/host-validate/rules/devices.xml create mode 100644 tools/host-validate/rules/freebsd-kernel.xml create mode 100644 tools/host-validate/rules/iommu.xml create mode 100644 tools/host-validate/rules/namespaces.xml create mode 100644 tools/host-validate/rules/pci.xml delete mode 100644 tools/virt-host-validate-bhyve.c delete mode 100644 tools/virt-host-validate-bhyve.h delete mode 100644 tools/virt-host-validate-common.c delete mode 100644 tools/virt-host-validate-common.h delete mode 100644 tools/virt-host-validate-lxc.c delete mode 100644 tools/virt-host-validate-lxc.h delete mode 100644 tools/virt-host-validate-qemu.c delete mode 100644 tools/virt-host-validate-qemu.h delete mode 100644 tools/virt-host-validate.c -- 2.21.0

3 4

[libvirt] [dockerfiles PATCH 0/2] Update libosinfo (& friends) dockerfiles
by Fabiano Fidêncio 03 Sep '19

03 Sep '19

As osinfo-db-tools has recently switched to using meson instead of autotools, let's update the libosinfo-dockerfiles accordingly. https://gitlab.com/fidencio/libvirt-dockerfiles/tree/wip/osinfo_db_tools_me… Fabiano Fidêncio (2): refresh: Remove Dockerfiles from OSes not supported by osinfo-db-tools refresh: Update Dockerfiles due to new osinfo-db-tools build system buildenv-libosinfo-centos-7.zip | Bin 458 -> 0 bytes buildenv-libosinfo-debian-10-cross-aarch64.zip | Bin 683 -> 688 bytes buildenv-libosinfo-debian-10-cross-armv6l.zip | Bin 682 -> 687 bytes buildenv-libosinfo-debian-10-cross-armv7l.zip | Bin 686 -> 690 bytes buildenv-libosinfo-debian-10-cross-i686.zip | Bin 682 -> 688 bytes buildenv-libosinfo-debian-10-cross-mips.zip | Bin 678 -> 683 bytes buildenv-libosinfo-debian-10-cross-mips64el.zip | Bin 689 -> 694 bytes buildenv-libosinfo-debian-10-cross-mipsel.zip | Bin 681 -> 685 bytes buildenv-libosinfo-debian-10-cross-ppc64le.zip | Bin 687 -> 692 bytes buildenv-libosinfo-debian-10-cross-s390x.zip | Bin 680 -> 686 bytes buildenv-libosinfo-debian-10.zip | Bin 555 -> 559 bytes buildenv-libosinfo-debian-9-cross-aarch64.zip | Bin 682 -> 0 bytes buildenv-libosinfo-debian-9-cross-armv6l.zip | Bin 681 -> 0 bytes buildenv-libosinfo-debian-9-cross-armv7l.zip | Bin 685 -> 0 bytes buildenv-libosinfo-debian-9-cross-mips.zip | Bin 678 -> 0 bytes buildenv-libosinfo-debian-9-cross-mips64el.zip | Bin 689 -> 0 bytes buildenv-libosinfo-debian-9-cross-mipsel.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-9-cross-ppc64le.zip | Bin 687 -> 0 bytes buildenv-libosinfo-debian-9-cross-s390x.zip | Bin 680 -> 0 bytes buildenv-libosinfo-debian-9.zip | Bin 554 -> 0 bytes buildenv-libosinfo-debian-sid-cross-aarch64.zip | Bin 683 -> 688 bytes buildenv-libosinfo-debian-sid-cross-armv6l.zip | Bin 682 -> 687 bytes buildenv-libosinfo-debian-sid-cross-armv7l.zip | Bin 686 -> 691 bytes buildenv-libosinfo-debian-sid-cross-i686.zip | Bin 683 -> 689 bytes buildenv-libosinfo-debian-sid-cross-mips.zip | Bin 678 -> 683 bytes ...denv-libosinfo-debian-sid-cross-mips64el.zip | Bin 689 -> 694 bytes buildenv-libosinfo-debian-sid-cross-mipsel.zip | Bin 681 -> 685 bytes buildenv-libosinfo-debian-sid-cross-ppc64le.zip | Bin 687 -> 692 bytes buildenv-libosinfo-debian-sid-cross-s390x.zip | Bin 680 -> 686 bytes buildenv-libosinfo-debian-sid.zip | Bin 555 -> 559 bytes buildenv-libosinfo-fedora-29.zip | Bin 488 -> 492 bytes buildenv-libosinfo-fedora-30.zip | Bin 551 -> 555 bytes buildenv-libosinfo-fedora-rawhide.zip | Bin 507 -> 512 bytes buildenv-libosinfo-ubuntu-16.zip | Bin 558 -> 0 bytes buildenv-libosinfo-ubuntu-18.zip | Bin 558 -> 0 bytes 35 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 buildenv-libosinfo-centos-7.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-aarch64.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-armv6l.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-armv7l.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mips.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mips64el.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-mipsel.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-ppc64le.zip delete mode 100644 buildenv-libosinfo-debian-9-cross-s390x.zip delete mode 100644 buildenv-libosinfo-debian-9.zip delete mode 100644 buildenv-libosinfo-ubuntu-16.zip delete mode 100644 buildenv-libosinfo-ubuntu-18.zip -- 2.21.0

1 2

[libvirt] [jenkins-ci PATCH] osinfo-db-tools: Don't try to build RPMs on CentOS 7
by Andrea Bolognani 03 Sep '19

03 Sep '19

It no longer is a supported target platform. Signed-off-by: Andrea Bolognani <abologna(a)redhat.com> --- guests/playbooks/build/projects/osinfo-db-tools.yml | 5 ++++- jenkins/projects/osinfo-db-tools.yaml | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/guests/playbooks/build/projects/osinfo-db-tools.yml b/guests/playbooks/build/projects/osinfo-db-tools.yml index 4bb6236..d142e0e 100644 --- a/guests/playbooks/build/projects/osinfo-db-tools.yml +++ b/guests/playbooks/build/projects/osinfo-db-tools.yml @@ -21,4 +21,7 @@ - include: '{{ playbook_base }}/jobs/meson-check-job.yml' - include: '{{ playbook_base }}/jobs/meson-rpm-job.yml' vars: - machines: '{{ rpm_machines }}' + machines: + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide diff --git a/jenkins/projects/osinfo-db-tools.yaml b/jenkins/projects/osinfo-db-tools.yaml index 15a95d3..2a9ea11 100644 --- a/jenkins/projects/osinfo-db-tools.yaml +++ b/jenkins/projects/osinfo-db-tools.yaml @@ -22,4 +22,7 @@ parent_jobs: 'osinfo-db-tools-syntax-check' - meson-rpm-job: parent_jobs: 'osinfo-db-tools-check' - machines: '{rpm_machines}' + machines: + - libvirt-fedora-29 + - libvirt-fedora-30 + - libvirt-fedora-rawhide -- 2.21.0

2 1

[libvirt] [PATCH 0/2] Remove xenapi driver
by Jim Fehlig 03 Sep '19

03 Sep '19

Remove the xenapi driver is unmaintained and has no known user base. We are stuck with references in include/libvirt/virterror.h and src/util/virerror.c but otherwise I think I've nuked xenapi everywhere. Jim Fehlig (2): xenapi: remove driver news: Mention removal of xenapi driver cfg.mk | 4 +- configure.ac | 4 - docs/aclpolkit.html.in | 4 - docs/auth.html.in | 2 - docs/news.xml | 10 + libvirt.spec.in | 4 +- m4/virt-driver-xenapi.m4 | 48 - po/POTFILES | 2 - src/Makefile.am | 1 - src/README | 1 - src/libvirt.c | 10 - src/util/virrandom.c | 3 +- src/xenapi/Makefile.inc.am | 29 - src/xenapi/xenapi_driver.c | 2125 ---------------------------- src/xenapi/xenapi_driver.h | 22 - src/xenapi/xenapi_driver_private.h | 58 - src/xenapi/xenapi_utils.c | 562 -------- src/xenapi/xenapi_utils.h | 76 - tools/virsh.c | 3 - 19 files changed, 13 insertions(+), 2955 deletions(-) delete mode 100644 m4/virt-driver-xenapi.m4 delete mode 100644 src/xenapi/Makefile.inc.am delete mode 100644 src/xenapi/xenapi_driver.c delete mode 100644 src/xenapi/xenapi_driver.h delete mode 100644 src/xenapi/xenapi_driver_private.h delete mode 100644 src/xenapi/xenapi_utils.c delete mode 100644 src/xenapi/xenapi_utils.h -- 2.22.0

2 3

[libvirt] [PATCH] docs: rewrite hvsupport.html page generator in python
by Daniel P. Berrangé 03 Sep '19

03 Sep '19

As part of an goal to eliminate Perl from libvirt build tools, rewrite the hvsupport.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. The new impl generates byte-for-byte identical output to the old impl. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- docs/Makefile.am | 6 +- docs/hvsupport.pl | 458 --------------------------------------------- docs/hvsupport.py | 463 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 466 insertions(+), 461 deletions(-) delete mode 100755 docs/hvsupport.pl create mode 100755 docs/hvsupport.py diff --git a/docs/Makefile.am b/docs/Makefile.am index 1cdb584b0b..f7aba5499d 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -194,7 +194,7 @@ EXTRA_DIST= \ $(internals_html_in) $(internals_html) $(fonts) \ $(kbase_html_in) $(kbase_html) \ aclperms.htmlinc \ - hvsupport.pl \ + hvsupport.py \ $(schema_DATA) acl_generated = aclperms.htmlinc @@ -230,11 +230,11 @@ web: $(dot_html) $(internals_html) $(kbase_html) \ hvsupport.html: $(srcdir)/hvsupport.html.in -$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ +$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.py $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ $(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \ $(top_srcdir)/src/driver.h - $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir)/src > $@ \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/hvsupport.py $(top_srcdir)/src > $@ \ || { rm $@ && exit 1; } news.html.in: \ diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl deleted file mode 100755 index 494b8a27ec..0000000000 --- a/docs/hvsupport.pl +++ /dev/null @@ -1,458 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; - -use File::Find; - -die "syntax: $0 SRCDIR\n" unless int(@ARGV) == 1; - -my $srcdir = shift @ARGV; - -my $symslibvirt = "$srcdir/libvirt_public.syms"; -my $symsqemu = "$srcdir/libvirt_qemu.syms"; -my $symslxc = "$srcdir/libvirt_lxc.syms"; -my @drivertable = ( - "$srcdir/driver-hypervisor.h", - "$srcdir/driver-interface.h", - "$srcdir/driver-network.h", - "$srcdir/driver-nodedev.h", - "$srcdir/driver-nwfilter.h", - "$srcdir/driver-secret.h", - "$srcdir/driver-state.h", - "$srcdir/driver-storage.h", - "$srcdir/driver-stream.h", - ); - -my %groupheaders = ( - "virHypervisorDriver" => "Hypervisor APIs", - "virNetworkDriver" => "Virtual Network APIs", - "virInterfaceDriver" => "Host Interface APIs", - "virNodeDeviceDriver" => "Host Device APIs", - "virStorageDriver" => "Storage Pool APIs", - "virSecretDriver" => "Secret APIs", - "virNWFilterDriver" => "Network Filter APIs", - ); - - -my @srcs; -find({ - wanted => sub { - if (m!$srcdir/.*/\w+_(driver|common|tmpl|monitor|hal|udev)\.c$!) { - push @srcs, $_ if $_ !~ /vbox_driver\.c/; - } - }, no_chdir => 1}, $srcdir); - -# Map API functions to the header and documentation files they're in -# so that we can generate proper hyperlinks to their documentation. -# -# The function names are grep'd from the XML output of apibuild.py. -sub getAPIFilenames { - my $filename = shift; - - my %files; - my $line; - - open FILE, "<", $filename or die "cannot read $filename: $!"; - - while (defined($line = <FILE>)) { - if ($line =~ /function name='([^']+)' file='([^']+)'/) { - $files{$1} = $2; - } - } - - close FILE; - - if (keys %files == 0) { - die "No functions found in $filename. Has the apibuild.py output changed?"; - } - return \%files; -} - -sub parseSymsFile { - my $apisref = shift; - my $prefix = shift; - my $filename = shift; - my $xmlfilename = shift; - - my $line; - my $vers; - my $prevvers; - - my $filenames = getAPIFilenames($xmlfilename); - - open FILE, "<$filename" - or die "cannot read $filename: $!"; - - while (defined($line = <FILE>)) { - chomp $line; - next if $line =~ /^\s*#/; - next if $line =~ /^\s*$/; - next if $line =~ /^\s*(global|local):/; - if ($line =~ /^\s*${prefix}_(\d+\.\d+\.\d+)\s*{\s*$/) { - if (defined $vers) { - die "malformed syms file"; - } - $vers = $1; - } elsif ($line =~ /\s*}\s*;\s*$/) { - if (defined $prevvers) { - die "malformed syms file"; - } - $prevvers = $vers; - $vers = undef; - } elsif ($line =~ /\s*}\s*${prefix}_(\d+\.\d+\.\d+)\s*;\s*$/) { - if ($1 ne $prevvers) { - die "malformed syms file $1 != $vers"; - } - $prevvers = $vers; - $vers = undef; - } elsif ($line =~ /\s*(\w+)\s*;\s*$/) { - $$apisref{$1} = {}; - $$apisref{$1}->{vers} = $vers; - $$apisref{$1}->{file} = $$filenames{$1}; - } else { - die "unexpected data $line\n"; - } - } - - close FILE; -} - -my %apis; -# Get the list of all public APIs and their corresponding version -parseSymsFile(\%apis, "LIBVIRT", $symslibvirt, "$srcdir/../docs/libvirt-api.xml"); - -# And the same for the QEMU specific APIs -parseSymsFile(\%apis, "LIBVIRT_QEMU", $symsqemu, "$srcdir/../docs/libvirt-qemu-api.xml"); - -# And the same for the LXC specific APIs -parseSymsFile(\%apis, "LIBVIRT_LXC", $symslxc, "$srcdir/../docs/libvirt-lxc-api.xml"); - - -# Some special things which aren't public APIs, -# but we want to report -$apis{virConnectSupportsFeature}->{vers} = "0.3.2"; -$apis{virDomainMigratePrepare}->{vers} = "0.3.2"; -$apis{virDomainMigratePerform}->{vers} = "0.3.2"; -$apis{virDomainMigrateFinish}->{vers} = "0.3.2"; -$apis{virDomainMigratePrepare2}->{vers} = "0.5.0"; -$apis{virDomainMigrateFinish2}->{vers} = "0.5.0"; -$apis{virDomainMigratePrepareTunnel}->{vers} = "0.7.2"; - -$apis{virDomainMigrateBegin3}->{vers} = "0.9.2"; -$apis{virDomainMigratePrepare3}->{vers} = "0.9.2"; -$apis{virDomainMigratePrepareTunnel3}->{vers} = "0.9.2"; -$apis{virDomainMigratePerform3}->{vers} = "0.9.2"; -$apis{virDomainMigrateFinish3}->{vers} = "0.9.2"; -$apis{virDomainMigrateConfirm3}->{vers} = "0.9.2"; - -$apis{virDomainMigrateBegin3Params}->{vers} = "1.1.0"; -$apis{virDomainMigratePrepare3Params}->{vers} = "1.1.0"; -$apis{virDomainMigratePrepareTunnel3Params}->{vers} = "1.1.0"; -$apis{virDomainMigratePerform3Params}->{vers} = "1.1.0"; -$apis{virDomainMigrateFinish3Params}->{vers} = "1.1.0"; -$apis{virDomainMigrateConfirm3Params}->{vers} = "1.1.0"; - - - -# Now we want to get the mapping between public APIs -# and driver struct fields. This lets us later match -# update the driver impls with the public APis. - -my $line; - -# Group name -> hash of APIs { fields -> api name } -my %groups; -my $ingrp; -foreach my $drivertable (@drivertable) { - open FILE, "<$drivertable" - or die "cannot read $drivertable: $!"; - - while (defined($line = <FILE>)) { - if ($line =~ /struct _(vir\w*Driver)/) { - my $grp = $1; - if ($grp ne "virStateDriver" && - $grp ne "virStreamDriver") { - $ingrp = $grp; - $groups{$ingrp} = { apis => {}, drivers => {} }; - } - } elsif ($ingrp) { - if ($line =~ /^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$/) { - my $field = $2; - my $name = $1; - - my $api; - if (exists $apis{"vir$name"}) { - $api = "vir$name"; - } elsif ($name =~ /\w+(Open|Close|URIProbe)/) { - next; - } else { - die "driver $name does not have a public API"; - } - $groups{$ingrp}->{apis}->{$field} = $api; - } elsif ($line =~ /};/) { - $ingrp = undef; - } - } - } - - close FILE; -} - - -# Finally, we read all the primary driver files and extract -# the driver API tables from each one. - -foreach my $src (@srcs) { - open FILE, "<$src" or - die "cannot read $src: $!"; - - my $groups_regex = join("|", keys %groups); - $ingrp = undef; - my $impl; - while (defined($line = <FILE>)) { - if (!$ingrp) { - # skip non-matching lines early to save time - next if not $line =~ /$groups_regex/; - - if ($line =~ /^\s*(?:static\s+)?($groups_regex)\s+(\w+)\s*=\s*{/ || - $line =~ /^\s*(?:static\s+)?($groups_regex)\s+NAME$\w+$\s*=\s*{/) { - $ingrp = $1; - $impl = $src; - - if ($impl =~ m,.*/node_device_(\w+)\.c,) { - $impl = $1; - } else { - $impl =~ s,.*/(\w+?)_((\w+)_)?(\w+)\.c,$1,; - } - - if ($groups{$ingrp}->{drivers}->{$impl}) { - die "Group $ingrp already contains $impl"; - } - - $groups{$ingrp}->{drivers}->{$impl} = {}; - } - - } else { - if ($line =~ m!\s*\.(\w+)\s*=\s*(\w+)\s*,?\s*(?:/\*\s*(\d+\.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$!) { - my $api = $1; - my $meth = $2; - my $vers = $3; - my $deleted = $4; - - next if $api eq "no" || $api eq "name"; - - if ($meth eq "NULL" && !defined $deleted) { - die "Method impl for $api is NULL, but no deleted version is provided"; - } - if ($meth ne "NULL" && defined $deleted) { - die "Method impl for $api is non-NULL, but deleted version is provided"; - } - - die "Method $meth in $src is missing version" unless defined $vers || $api eq "connectURIProbe"; - - if (!exists($groups{$ingrp}->{apis}->{$api})) { - next if $api =~ /\w(Open|Close|URIProbe)/; - - die "Found unexpected method $api in $ingrp\n"; - } - - $groups{$ingrp}->{drivers}->{$impl}->{$api} = {}; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{vers} = $vers; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{deleted} = $deleted; - if ($api eq "domainMigratePrepare" || - $api eq "domainMigratePrepare2" || - $api eq "domainMigratePrepare3") { - if (!$groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"}) { - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"} = {}; - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"}->{vers} = $vers; - } - } - - } elsif ($line =~ /}/) { - $ingrp = undef; - } - } - } - - close FILE; -} - - -# The '.open' driver method is used for 3 public APIs, so we -# have a bit of manual fixup todo with the per-driver versioning -# and support matrix - -$groups{virHypervisorDriver}->{apis}->{"openAuth"} = "virConnectOpenAuth"; -$groups{virHypervisorDriver}->{apis}->{"openReadOnly"} = "virConnectOpenReadOnly"; -$groups{virHypervisorDriver}->{apis}->{"domainMigrate"} = "virDomainMigrate"; - -my $openAuthVers = (0 * 1000 * 1000) + (4 * 1000) + 0; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $openVersStr = $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}->{vers}; - my $openVers; - if ($openVersStr =~ /(\d+)\.(\d+)\.(\d+)/) { - $openVers = ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - # virConnectOpenReadOnly always matches virConnectOpen version - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenReadOnly"} = - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}; - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"} = {}; - - # virConnectOpenAuth is always 0.4.0 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virConnectOpen entry - if ($openVersStr eq "Y" || - $openVers >= $openAuthVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}->{vers} = $openVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}->{vers} = "0.4.0"; - } -} - - -# Another special case for the virDomainCreateLinux which was replaced -# with virDomainCreateXML -$groups{virHypervisorDriver}->{apis}->{"domainCreateLinux"} = "virDomainCreateLinux"; - -my $createAPIVers = (0 * 1000 * 1000) + (0 * 1000) + 3; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $createVersStr = $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateXML"}->{vers}; - next unless defined $createVersStr; - my $createVers; - if ($createVersStr =~ /(\d+)\.(\d+)\.(\d+)/) { - $createVers = ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"} = {}; - - # virCreateLinux is always 0.0.3 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virCreateXML entry - if ($createVersStr eq "Y" || - $createVers >= $createAPIVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"}->{vers} = $createVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"}->{vers} = "0.0.3"; - } -} - - -# Finally we generate the HTML file with the tables - -print <<EOF; -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE html> -<html xmlns="http://www.w3.org/1999/xhtml"> -<body class="hvsupport"> -<h1>libvirt API support matrix</h1> - -<ul id="toc"></ul> - -<p> -This page documents which <a href="html/">libvirt calls</a> work on -which libvirt drivers / hypervisors, and which version the API appeared -in. If a hypervisor driver later dropped support for the API, the version -when it was removed is also mentioned (highlighted in -<span class="removedhv">dark red</span>). -</p> - -EOF - - foreach my $grp (sort { $a cmp $b } keys %groups) { - print "<h2><a id=\"$grp\">", $groupheaders{$grp}, "</a></h2>\n"; - print <<EOF; -<table class="top_table"> -<thead> -<tr> -<th>API</th> -<th>Version</th> -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print " <th>$drv</th>\n"; - } - - print <<EOF; -</tr> -</thead> -<tbody> -EOF - - my $row = 0; - foreach my $field (sort { - $groups{$grp}->{apis}->{$a} - cmp - $groups{$grp}->{apis}->{$b} - } keys %{$groups{$grp}->{apis}}) { - my $api = $groups{$grp}->{apis}->{$field}; - my $vers = $apis{$api}->{vers}; - my $htmlgrp = $apis{$api}->{file}; - print <<EOF; -<tr> -<td> -EOF - - if (defined $htmlgrp) { - print <<EOF; -<a href=\"html/libvirt-$htmlgrp.html#$api\">$api</a> -EOF - - } else { - print $api; - } - print <<EOF; -</td> -<td>$vers</td> -EOF - - foreach my $drv (sort {$a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print "<td>"; - if (exists $groups{$grp}->{drivers}->{$drv}->{$field}) { - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{vers}) { - print $groups{$grp}->{drivers}->{$drv}->{$field}->{vers}; - } - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{deleted}) { - print " - <span class=\"removedhv\">", $groups{$grp}->{drivers}->{$drv}->{$field}->{deleted}, "</span>"; - } - } - print "</td>\n"; - } - - print <<EOF; -</tr> -EOF - - $row++; - if (($row % 15) == 0) { - print <<EOF; -<tr> -<th>API</th> -<th>Version</th> -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print " <th>$drv</th>\n"; - } - - print <<EOF; -</tr> -EOF - } - - } - - print <<EOF; -</tbody> -</table> -EOF -} - -print <<EOF; -</body> -</html> -EOF diff --git a/docs/hvsupport.py b/docs/hvsupport.py new file mode 100755 index 0000000000..502389c25f --- /dev/null +++ b/docs/hvsupport.py @@ -0,0 +1,463 @@ +#!/usr/bin/env python + +from __future__ import print_function + +import sys +import os.path +import re + +if len(sys.argv) != 2: + print("syntax: %s SRCDIR\n" % sys.argv[0], fh=sys.stderr) + +srcdir = sys.argv[1] + +symslibvirt = os.path.join(srcdir, "libvirt_public.syms") +symsqemu = os.path.join(srcdir, "libvirt_qemu.syms") +symslxc = os.path.join(srcdir, "libvirt_lxc.syms") +drivertablefiles = [ + os.path.join(srcdir, "driver-hypervisor.h"), + os.path.join(srcdir, "driver-interface.h"), + os.path.join(srcdir, "driver-network.h"), + os.path.join(srcdir, "driver-nodedev.h"), + os.path.join(srcdir, "driver-nwfilter.h"), + os.path.join(srcdir, "driver-secret.h"), + os.path.join(srcdir, "driver-state.h"), + os.path.join(srcdir, "driver-storage.h"), + os.path.join(srcdir, "driver-stream.h"), +] + +groupheaders = { + "virHypervisorDriver": "Hypervisor APIs", + "virNetworkDriver": "Virtual Network APIs", + "virInterfaceDriver": "Host Interface APIs", + "virNodeDeviceDriver": "Host Device APIs", + "virStorageDriver": "Storage Pool APIs", + "virSecretDriver": "Secret APIs", + "virNWFilterDriver": "Network Filter APIs", +} + + +srcs = [] +for root, dirs, files in os.walk(srcdir): + for file in files: + if ((file.endswith("driver.c") and + not file.endswith("vbox_driver.c")) or + file.endswith("common.c") or + file.endswith("tmpl.c") or + file.endswith("monitor.c") or + file.endswith("hal.c") or + file.endswith("udev.c")): + srcs.append(os.path.join(root, file)) + + +# Map API functions to the header and documentation files they're in +# so that we can generate proper hyperlinks to their documentation. +# +# The function names are grep'd from the XML output of apibuild.py. +def getAPIFilenames(filename): + files = {} + + with open(filename) as fh: + prog = re.compile(r"\s*<function name='([^']+)' file='([^']+)'.*") + for line in fh: + res = prog.match(line) + if res is not None: + files[res.group(1)] = res.group(2) + + if len(files) == 0: + raise Exception("No functions found in %s. Has the apibuild.py output changed?" % filename) + + return files; + + +def parseSymsFile(apisref, prefix, filename, xmlfilename): + vers = None + prevvers = None + + filenames = getAPIFilenames(xmlfilename) + + with open(filename) as fh: + groupstartprog = re.compile(r"^\s*%s_(\d+\.\d+\.\d+)\s*{\s*$" % prefix) + groupendprog1 = re.compile(r"^\s*}\s*;\s*$") + groupendprog2 = re.compile(r"^\s*}\s*%s_(\d+\.\d+\.\d+)\s*;\s*$" % prefix) + symbolprog = re.compile(r"^\s*(\w+)\s*;\s*$") + for line in fh: + line = line.strip() + + if line == "": + continue + if line[0] == '#': + continue + if line.startswith("global:"): + continue + if line.startswith("local:"): + continue + + groupstartmatch = groupstartprog.match(line) + groupendmatch1 = groupendprog1.match(line) + groupendmatch2 = groupendprog2.match(line) + symbolmatch = symbolprog.match(line) + if groupstartmatch is not None: + if vers is not None: + raise Exception("malformed syms file when starting group") + + vers = groupstartmatch.group(1) + elif groupendmatch1 is not None: + if prevvers is not None: + raise Exception("malformed syms file when ending group") + + prevvers = vers + vers = None + elif groupendmatch2 is not None: + if groupendmatch2.group(1) != prevvers: + raise Exception("malformed syms file %s != %s when ending group" % ( + groupendmatch2.group(1), prevvers)) + + prevvers = vers + vers = None + elif symbolmatch is not None: + name = symbolmatch.group(1) + apisref[name] = { + "vers": vers, + "file": filenames.get(name), + } + else: + raise Exception("unexpected data %s" % line) + + +apis = {} +# Get the list of all public APIs and their corresponding version +parseSymsFile(apis, "LIBVIRT", symslibvirt, os.path.join(srcdir, "../docs/libvirt-api.xml")) + +# And the same for the QEMU specific APIs +parseSymsFile(apis, "LIBVIRT_QEMU", symsqemu, os.path.join(srcdir, "../docs/libvirt-qemu-api.xml")) + +# And the same for the LXC specific APIs +parseSymsFile(apis, "LIBVIRT_LXC", symslxc, os.path.join(srcdir, "../docs/libvirt-lxc-api.xml")) + + +# Some special things which aren't public APIs, +# but we want to report +apis["virConnectSupportsFeature"] = { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare"] = { + "vers": "0.3.2" +} +apis["virDomainMigratePerform"] = { + "vers": "0.3.2" +} +apis["virDomainMigrateFinish"] = { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare2"] = { + "vers": "0.5.0" +} +apis["virDomainMigrateFinish2"] = { + "vers": "0.5.0" +} +apis["virDomainMigratePrepareTunnel"] = { + "vers": "0.7.2" +} + +apis["virDomainMigrateBegin3"] = { + "vers": "0.9.2" +} +apis["virDomainMigratePrepare3"] = { + "vers": "0.9.2" +} +apis["virDomainMigratePrepareTunnel3"] = { + "vers": "0.9.2" +} +apis["virDomainMigratePerform3"] = { + "vers": "0.9.2" +} +apis["virDomainMigrateFinish3"] = { + "vers": "0.9.2" +} +apis["virDomainMigrateConfirm3"] = { + "vers": "0.9.2" +} + +apis["virDomainMigrateBegin3Params"] = { + "vers": "1.1.0" +} +apis["virDomainMigratePrepare3Params"] = { + "vers": "1.1.0" +} +apis["virDomainMigratePrepareTunnel3Params"] = { + "vers": "1.1.0" +} +apis["virDomainMigratePerform3Params"] = { + "vers": "1.1.0" +} +apis["virDomainMigrateFinish3Params"] = { + "vers": "1.1.0" +} +apis["virDomainMigrateConfirm3Params"] = { + "vers": "1.1.0" +} + + +# Now we want to get the mapping between public APIs +# and driver struct fields. This lets us later match +# update the driver impls with the public APis. + +# Group name -> hash of APIs { fields -> api name } +groups = {} +ingrp = None +for drivertablefile in drivertablefiles: + with open(drivertablefile) as fh: + starttableprog = re.compile(r"struct _(vir\w*Driver)") + endtableprog = re.compile(r"};") + callbackprog = re.compile(r"^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$") + ignoreapiprog = re.compile(r"\w+(Open|Close|URIProbe)") + for line in fh: + starttablematch = starttableprog.match(line) + if starttablematch is not None: + grp = starttablematch.group(1) + if grp != "virStateDriver" and grp != "virStreamDriver": + ingrp = grp + groups[ingrp] = { + "apis": {}, + "drivers": {} + } + elif ingrp != None: + callbackmatch = callbackprog.match(line) + if callbackmatch is not None: + name = callbackmatch.group(1) + field = callbackmatch.group(2) + + api = "vir" + name + if api in apis: + groups[ingrp]["apis"][field] = api + elif ignoreapiprog.match(api) != None: + continue + else: + raise Exception("driver %s does not have a public API" % name) + elif endtableprog.match(line): + ingrp = None + + +# Finally, we read all the primary driver files and extract +# the driver API tables from each one. + +for src in srcs: + with open(src) as fh: + groupsre = "|".join(groups.keys()) + groupsprog2 = re.compile(r"^\s*(static\s+)?(" + groupsre + r")\s+(\w+)\s*=\s*{") + groupsprog3 = re.compile(r"^\s*(static\s+)?(" + groupsre + r")\s+NAME$\w+$\s*=\s*{") + nodedevimplprog = re.compile(r".*/node_device_(\w+)\.c") + miscimplprog = re.compile(r".*/(\w+?)_((\w+)_)?(\w+)\.c") + callbackprog = re.compile(r"\s*\.(\w+)\s*=\s*(\w+)\s*,?\s*(?:/\*\s*(\d+\.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$") + skipapiprog = re.compile(r"\w+(Open|Close|URIProbe)") + + ingrp = None + impl = None + for line in fh: + if ingrp is None: + m = groupsprog2.match(line) + if m is None: + m = groupsprog3.match(line) + if m is not None: + ingrp = m.group(2) + impl = src + + implmatch = nodedevimplprog.match(impl) + if implmatch is None: + implmatch = miscimplprog.match(impl) + if implmatch is None: + raise Exception("Unexpected impl format '%s'" % impl) + impl = implmatch.group(1) + + if impl in groups[ingrp]["drivers"]: + raise Exception("Group %s already contains %s" % (ingrp, impl)) + + groups[ingrp]["drivers"][impl] = {} + else: + callbackmatch = callbackprog.match(line) + if callbackmatch is not None: + api = callbackmatch.group(1) + meth = callbackmatch.group(2) + vers = callbackmatch.group(3) + deleted = callbackmatch.group(4) + + if api == "no" or api == "name": + continue + + if meth == "NULL" and deleted is None: + raise Exception("Method impl for %s is NULL, but no deleted version is provided" % api) + + if meth != "NULL" and deleted is not None: + raise Exception("Method impl for %s is non-NULL, but deleted version is provided" % api) + + if vers is None and api != "connectURIProbe": + raise Exception("Method %s in %s is missing version" % (meth, src)) + + if api not in groups[ingrp]["apis"]: + if skipapiprog.match(api): + continue + + raise Exception("Found unexpected method %s in %s" % (api, ingrp)) + + groups[ingrp]["drivers"][impl][api] = { + "vers": vers, + "deleted": deleted, + } + + if (api == "domainMigratePrepare" or + api == "domainMigratePrepare2" or + api == "domainMigratePrepare3"): + if "domainMigrate" not in groups[ingrp]["drivers"][impl]: + groups[ingrp]["drivers"][impl]["domainMigrate"] = { + "vers": vers, + } + elif line.find("}") != -1: + ingrp = None + + +# The '.open' driver method is used for 3 public APIs, so we +# have a bit of manual fixup todo with the per-driver versioning +# and support matrix + +groups["virHypervisorDriver"]["apis"]["openAuth"] = "virConnectOpenAuth" +groups["virHypervisorDriver"]["apis"]["openReadOnly"] = "virConnectOpenReadOnly" +groups["virHypervisorDriver"]["apis"]["domainMigrate"] = "virDomainMigrate" + +openAuthVers = (0 * 1000 * 1000) + (4 * 1000) + 0; + +for drv in groups["virHypervisorDriver"]["drivers"].keys(): + openVersStr = groups["virHypervisorDriver"]["drivers"][drv]["connectOpen"]["vers"] + openVers = 0 + if openVersStr != "Y": + openVersBits = openVersStr.split(".") + if len(openVersBits) != 3: + raise Exception("Expected 3 digit version for %s" % openVersStr) + openVers = (int(openVersBits[0]) * 1000 * 1000) + (int(openVersBits[1]) * 1000) + int(openVersBits[2]) + + # virConnectOpenReadOnly always matches virConnectOpen version + groups["virHypervisorDriver"]["drivers"][drv]["connectOpenReadOnly"] = \ + groups["virHypervisorDriver"]["drivers"][drv]["connectOpen"] + + # virConnectOpenAuth is always 0.4.0 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virConnectOpen entry + if openVersStr == "Y" or openVers >= openAuthVers: + vers = openVersStr + else: + vers = "0.4.0" + groups["virHypervisorDriver"]["drivers"][drv]["connectOpenAuth"] = { + "vers": vers, + } + + +# Another special case for the virDomainCreateLinux which was replaced +# with virDomainCreateXML +groups["virHypervisorDriver"]["apis"]["domainCreateLinux"] = "virDomainCreateLinux"; + +createAPIVers = (0 * 1000 * 1000) + (0 * 1000) + 3; + +for drv in groups["virHypervisorDriver"]["drivers"].keys(): + if "domainCreateXML" not in groups["virHypervisorDriver"]["drivers"][drv]: + continue + createVersStr = groups["virHypervisorDriver"]["drivers"][drv]["domainCreateXML"]["vers"] + createVers = 0 + if createVersStr != "Y": + createVersBits = createVersStr.split(".") + if len(createVersBits) != 3: + raise Exception("Expected 3 digit version for %s" % createVersStr) + createVers = (int(createVersBits[0]) * 1000 * 1000) + (int(createVersBits[1]) * 1000) + int(createVersBits[2]) + + # virCreateLinux is always 0.0.3 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virCreateXML entry + if createVersStr == "Y" or createVers >= createAPIVers: + vers = createVersStr + else: + vers = "0.0.3" + + groups["virHypervisorDriver"]["drivers"][drv]["domainCreateLinux"] = { + "vers": vers, + } + + +# Finally we generate the HTML file with the tables + +print('''<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml"> +<body class="hvsupport"> +<h1>libvirt API support matrix</h1> + +<ul id="toc"></ul> + +<p> +This page documents which <a href="html/">libvirt calls</a> work on +which libvirt drivers / hypervisors, and which version the API appeared +in. If a hypervisor driver later dropped support for the API, the version +when it was removed is also mentioned (highlighted in +<span class="removedhv">dark red</span>). +</p> +''') + +for grp in sorted(groups.keys()): + print("<h2><a id=\"%s\">%s</a></h2>" % (grp, groupheaders[grp])) + print('''<table class="top_table"> +<thead> +<tr> +<th>API</th> +<th>Version</th>''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" <th>%s</th>" % drv) + + + print('''</tr> +</thead> +<tbody>''') + + row = 0 + def sortkey(field): + return groups[grp]["apis"][field] + for field in sorted(groups[grp]["apis"].keys(), key=sortkey): + api = groups[grp]["apis"][field] + vers = apis[api]["vers"] + htmlgrp = apis[api].get("file") + print("<tr>") + + if htmlgrp is not None: + print('''<td>\n<a href=\"html/libvirt-%s.html#%s\">%s</a>\n</td>''' % (htmlgrp, api, api)) + else: + print("<td>\n%s</td>" % api) + + print("<td>%s</td>" % vers) + + for drv in sorted(groups[grp]["drivers"].keys()): + info = "" + if field in groups[grp]["drivers"][drv]: + vers = groups[grp]["drivers"][drv][field]["vers"] + if vers is not None: + info = info + vers + + deleted = groups[grp]["drivers"][drv][field].get("deleted") + if deleted is not None: + info = info + (''' - <span class="removedhv">%s</span>''' % deleted) + + print("<td>%s</td>" % info) + + print("</tr>") + + row = row + 1 + if (row % 15) == 0: + print('''<tr> +<th>API</th> +<th>Version</th>''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" <th>%s</th>" % drv) + + print("</tr>") + + print("</tbody>\n</table>") + +print("</body>\n</html>") -- 2.21.0

2 1