[PATCH] ci: drop mipsel on Debian Sid
by Daniel P. Berrangé
This port was discontinued and purged from the archives:
https://lists.debian.org/debian-devel-announce/2023/09/msg00000.html
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
ci/buildenv/debian-sid-cross-mipsel.sh | 115 -----------------
.../debian-sid-cross-mipsel.Dockerfile | 121 ------------------
ci/gitlab/builds.yml | 22 ----
ci/gitlab/containers.yml | 8 --
ci/manifest.yml | 4 -
5 files changed, 270 deletions(-)
delete mode 100644 ci/buildenv/debian-sid-cross-mipsel.sh
delete mode 100644 ci/containers/debian-sid-cross-mipsel.Dockerfile
diff --git a/ci/buildenv/debian-sid-cross-mipsel.sh b/ci/buildenv/debian-sid-cross-mipsel.sh
deleted file mode 100644
index 90dac8c22a..0000000000
--- a/ci/buildenv/debian-sid-cross-mipsel.sh
+++ /dev/null
@@ -1,115 +0,0 @@
-# THIS FILE WAS AUTO-GENERATED
-#
-# $ lcitool manifest ci/manifest.yml
-#
-# https://gitlab.com/libvirt/libvirt-ci
-
-function install_buildenv() {
- export DEBIAN_FRONTEND=noninteractive
- apt-get update
- apt-get dist-upgrade -y
- apt-get install --no-install-recommends -y \
- augeas-lenses \
- augeas-tools \
- bash-completion \
- ca-certificates \
- ccache \
- codespell \
- cpp \
- diffutils \
- dwarves \
- ebtables \
- flake8 \
- gettext \
- git \
- grep \
- iproute2 \
- iptables \
- kmod \
- libc-dev-bin \
- libnbd-dev \
- libxml2-utils \
- locales \
- lvm2 \
- make \
- meson \
- nfs-common \
- ninja-build \
- numad \
- open-iscsi \
- perl-base \
- pkgconf \
- policykit-1 \
- python3 \
- python3-docutils \
- qemu-utils \
- scrub \
- sed \
- xsltproc
- sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen
- dpkg-reconfigure locales
- export DEBIAN_FRONTEND=noninteractive
- dpkg --add-architecture mipsel
- apt-get update
- apt-get dist-upgrade -y
- apt-get install --no-install-recommends -y dpkg-dev
- apt-get install --no-install-recommends -y \
- gcc-mipsel-linux-gnu \
- libacl1-dev:mipsel \
- libapparmor-dev:mipsel \
- libattr1-dev:mipsel \
- libaudit-dev:mipsel \
- libblkid-dev:mipsel \
- libc6-dev:mipsel \
- libcap-ng-dev:mipsel \
- libcurl4-gnutls-dev:mipsel \
- libdevmapper-dev:mipsel \
- libfuse-dev:mipsel \
- libglib2.0-dev:mipsel \
- libglusterfs-dev:mipsel \
- libgnutls28-dev:mipsel \
- libiscsi-dev:mipsel \
- libnl-3-dev:mipsel \
- libnl-route-3-dev:mipsel \
- libnuma-dev:mipsel \
- libparted-dev:mipsel \
- libpcap0.8-dev:mipsel \
- libpciaccess-dev:mipsel \
- librbd-dev:mipsel \
- libreadline-dev:mipsel \
- libsanlock-dev:mipsel \
- libsasl2-dev:mipsel \
- libselinux1-dev:mipsel \
- libssh-gcrypt-dev:mipsel \
- libssh2-1-dev:mipsel \
- libtirpc-dev:mipsel \
- libudev-dev:mipsel \
- libxml2-dev:mipsel \
- libyajl-dev:mipsel \
- systemtap-sdt-dev:mipsel
- mkdir -p /usr/local/share/meson/cross
- printf "[binaries]\n\
-c = '/usr/bin/mipsel-linux-gnu-gcc'\n\
-ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\
-strip = '/usr/bin/mipsel-linux-gnu-strip'\n\
-pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\
-\n\
-[host_machine]\n\
-system = 'linux'\n\
-cpu_family = 'mips'\n\
-cpu = 'mipsel'\n\
-endian = 'little'\n" > /usr/local/share/meson/cross/mipsel-linux-gnu
- dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt
- mkdir -p /usr/libexec/ccache-wrappers
- ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc
- ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc
-}
-
-export CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
-export LANG="en_US.UTF-8"
-export MAKE="/usr/bin/make"
-export NINJA="/usr/bin/ninja"
-export PYTHON="/usr/bin/python3"
-
-export ABI="mipsel-linux-gnu"
-export MESON_OPTS="--cross-file=mipsel-linux-gnu"
diff --git a/ci/containers/debian-sid-cross-mipsel.Dockerfile b/ci/containers/debian-sid-cross-mipsel.Dockerfile
deleted file mode 100644
index 81895b23dd..0000000000
--- a/ci/containers/debian-sid-cross-mipsel.Dockerfile
+++ /dev/null
@@ -1,121 +0,0 @@
-# THIS FILE WAS AUTO-GENERATED
-#
-# $ lcitool manifest ci/manifest.yml
-#
-# https://gitlab.com/libvirt/libvirt-ci
-
-FROM docker.io/library/debian:sid-slim
-
-RUN export DEBIAN_FRONTEND=noninteractive && \
- apt-get update && \
- apt-get install -y eatmydata && \
- eatmydata apt-get dist-upgrade -y && \
- eatmydata apt-get install --no-install-recommends -y \
- augeas-lenses \
- augeas-tools \
- bash-completion \
- ca-certificates \
- ccache \
- codespell \
- cpp \
- diffutils \
- dwarves \
- ebtables \
- flake8 \
- gettext \
- git \
- grep \
- iproute2 \
- iptables \
- kmod \
- libc-dev-bin \
- libnbd-dev \
- libxml2-utils \
- locales \
- lvm2 \
- make \
- meson \
- nfs-common \
- ninja-build \
- numad \
- open-iscsi \
- perl-base \
- pkgconf \
- policykit-1 \
- python3 \
- python3-docutils \
- qemu-utils \
- scrub \
- sed \
- xsltproc && \
- eatmydata apt-get autoremove -y && \
- eatmydata apt-get autoclean -y && \
- sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
- dpkg-reconfigure locales
-
-ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-ENV LANG "en_US.UTF-8"
-ENV MAKE "/usr/bin/make"
-ENV NINJA "/usr/bin/ninja"
-ENV PYTHON "/usr/bin/python3"
-
-RUN export DEBIAN_FRONTEND=noninteractive && \
- dpkg --add-architecture mipsel && \
- eatmydata apt-get update && \
- eatmydata apt-get dist-upgrade -y && \
- eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
- eatmydata apt-get install --no-install-recommends -y \
- gcc-mipsel-linux-gnu \
- libacl1-dev:mipsel \
- libapparmor-dev:mipsel \
- libattr1-dev:mipsel \
- libaudit-dev:mipsel \
- libblkid-dev:mipsel \
- libc6-dev:mipsel \
- libcap-ng-dev:mipsel \
- libcurl4-gnutls-dev:mipsel \
- libdevmapper-dev:mipsel \
- libfuse-dev:mipsel \
- libglib2.0-dev:mipsel \
- libglusterfs-dev:mipsel \
- libgnutls28-dev:mipsel \
- libiscsi-dev:mipsel \
- libnl-3-dev:mipsel \
- libnl-route-3-dev:mipsel \
- libnuma-dev:mipsel \
- libparted-dev:mipsel \
- libpcap0.8-dev:mipsel \
- libpciaccess-dev:mipsel \
- librbd-dev:mipsel \
- libreadline-dev:mipsel \
- libsanlock-dev:mipsel \
- libsasl2-dev:mipsel \
- libselinux1-dev:mipsel \
- libssh-gcrypt-dev:mipsel \
- libssh2-1-dev:mipsel \
- libtirpc-dev:mipsel \
- libudev-dev:mipsel \
- libxml2-dev:mipsel \
- libyajl-dev:mipsel \
- systemtap-sdt-dev:mipsel && \
- eatmydata apt-get autoremove -y && \
- eatmydata apt-get autoclean -y && \
- mkdir -p /usr/local/share/meson/cross && \
- printf "[binaries]\n\
-c = '/usr/bin/mipsel-linux-gnu-gcc'\n\
-ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\
-strip = '/usr/bin/mipsel-linux-gnu-strip'\n\
-pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\
-\n\
-[host_machine]\n\
-system = 'linux'\n\
-cpu_family = 'mips'\n\
-cpu = 'mipsel'\n\
-endian = 'little'\n" > /usr/local/share/meson/cross/mipsel-linux-gnu && \
- dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
- mkdir -p /usr/libexec/ccache-wrappers && \
- ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \
- ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc
-
-ENV ABI "mipsel-linux-gnu"
-ENV MESON_OPTS "--cross-file=mipsel-linux-gnu"
diff --git a/ci/gitlab/builds.yml b/ci/gitlab/builds.yml
index e1ea87db1b..2beca51767 100644
--- a/ci/gitlab/builds.yml
+++ b/ci/gitlab/builds.yml
@@ -898,28 +898,6 @@ mips64el-debian-sid-local-env:
NAME: debian-sid
-mipsel-debian-sid-prebuilt-env:
- extends: .cross_build_job_prebuilt_env
- needs:
- - job: mipsel-debian-sid-container
- optional: true
- allow_failure: false
- variables:
- CROSS: mipsel
- JOB_OPTIONAL: 1
- NAME: debian-sid
-
-mipsel-debian-sid-local-env:
- extends: .cross_build_job_local_env
- needs: []
- allow_failure: false
- variables:
- CROSS: mipsel
- IMAGE: docker.io/library/debian:sid-slim
- JOB_OPTIONAL: 1
- NAME: debian-sid
-
-
ppc64le-debian-sid-prebuilt-env:
extends: .cross_build_job_prebuilt_env
needs:
diff --git a/ci/gitlab/containers.yml b/ci/gitlab/containers.yml
index 05d22f7336..2463d1a5aa 100644
--- a/ci/gitlab/containers.yml
+++ b/ci/gitlab/containers.yml
@@ -283,14 +283,6 @@ mips64el-debian-sid-container:
NAME: debian-sid-cross-mips64el
-mipsel-debian-sid-container:
- extends: .container_job
- allow_failure: false
- variables:
- JOB_OPTIONAL: 1
- NAME: debian-sid-cross-mipsel
-
-
ppc64le-debian-sid-container:
extends: .container_job
allow_failure: false
diff --git a/ci/manifest.yml b/ci/manifest.yml
index 2af8e62f28..e115f45b2c 100644
--- a/ci/manifest.yml
+++ b/ci/manifest.yml
@@ -144,10 +144,6 @@ targets:
containers: false
builds: false
- - arch: mipsel
- containers: false
- builds: false
-
- arch: ppc64le
containers: false
builds: false
--
2.41.0
6 months, 2 weeks
[libvirt PATCH 01/15] util: properly deal with module vs. driver when
binding device to driver
by Laine Stump
(Sigh, resent because I forgot to update the list address in the reply :-/)
Sorry, I meant to respond to this earlier and forgot. Now I'm sending v2
of the patches, and wanted to make sure you didn't think I was just
ignoring your comments :-) (since this patch is still there in V2)
On 10/23/23 9:39 AM, Jason Gunthorpe wrote:
> On Mon, Oct 23, 2023 at 12:54:37AM -0400, Laine Stump wrote:
>
>> When we recently gained the ability to manually specify a driver to
>> bind to with virsh nodedev-detach, the fragility of this system became
>> apparent - if a user gives the driver name as "vfio_pci", then we
>> would modprobe the module, but then erroneously believe it hadn't been
>> loaded because /sys/bus/pci/drivers/vfio_pci didn't exist. For manual
>> specification of the driver name, this could be dealt with by telling
>> the user "always use the correct name for the driver, don't assume
>> that it is the same as the modulename", but it would still end up
>> confusing people, especially since some drivers do use underscore in
>> their name (e.g. the mlx5_vfio_pci driver/module).
>
> I think you are creating more confusion by allowing this. The driver
> name from module.alias should be used consistently in its exact
> format.
But the name in the modules.alias file *is not* the name of the driver.
It is the name of the module, and the name of the driver may or may not
be the same - that's the entire reason this issue came to light.
And in the end I don't do anything more than what modprobe itself does.
(Well, actually that's not true, since modprobe never has to actually
*use* the drivers that are implemented by the modules that it loads, it
doesn't need to understand that the name of the driver may be different
from the name of the module, and so it doesn't have to look at the link
in /sys/modules/$modulename/drivers/pci:$drivername to get the proper
name that must be written to sysfs to get the driver bound to the
device. It just unilaterally decides that all horizontal lines in a
module name are "_", and so it replaces all "-" with "_")
Really it is modprobe's practice of auto-translating all "-" to "_",
combined with the inconsistency of naming between module names and
driver names that is causing the confusion; what this patch is doing is
helping to clear up that confusion, not make it worse. Essentially,
whether someone uses the module name or the driver name, libvirt will
get the correct driver bound to the device. (and we do have to allow
both module name and driver name; the former in cases where the module
hasn't been loaded yet and so the driver isn't available, and the latter
in cases where the driver is statically linked into the kernel, as
discussed below).
>
> Log a 'did you mean XYZ" or something if it doesn't work out.
>
> Pass the "driver name" directly to modprobe and let it sort it out
In the cases (e.g. vfio_pci/vfio-pci) where the module name and driver
name mismatch only due to -/_, what you say will work as long as the
*driver* name is used (i.e. "vfio-pci"), but if we use the name
discovered from modules.alias ("vfio_pci") then the modprobe will
succeed, but the bind will fail.
And we can't just assume that the driver name will always have "-" in
place of "_" and do an auto-replace in the name prior to binding the
device to the driver, because some drivers (in particular,
mlx5_vfio_pci/mlx5_vfio_pci) *don't* use "-" in the driver name. libvirt
has to do *something* in order to work in both of these cases, and I
think anything less than what I've done would be "half baked" (well,
half *something*, but I'd rather not swear on a public forum :-))
> Forget about modules entirely in the libvirt level, don't even check
> it.
Yes, if we're manually specifying the driver, then (at least in the case
of vfio-pci and mlx5_vfio_pci) it *does* work to always use the driver
name (because modprobe will change vfio-pci to vfio_pci, and leave
mlx5_vfio_pci alone). But when we're discovering the variant driver via
modules.alias (which is the official method of finding the best VFIO
variant driver for a device), we don't start out with the driver name,
we start out with (a modalias string, that leads us to) the *module*
name, not a driver name. And while modprobe will of course accept the
name of the module to load, you can't use the module name to bind the
driver to the kernel - you have to do *something* to derive the driver
name from the module name. That's what this code does.
(I haven't seen an example where module name and driver name have more
differences than just "-" vs "_", and it appears to be convention to
"not do that", however it theoretically is completely doable - the
driver name is just a string in a C file. I'd rather not have my code
break at some random time in the future when somebody writes such a driver.)
>
>> c) All of this is conveniently ignoring the possibility of a VFIO
>> variant driver that is statically linked in the kernel. The entire
>> design of variant driver auto-detection is based on doing a lookup
>> in modules.alias, and that only lists *loadable modules* (not
>> drivers), so unless I'm missing something, it would be impossible
>> to auto-detect a VFIO variant driver that was statically
>> linked. This is beyond libvirt's ability to fix; the best that
>> could be done would be to manually specify the driver name in the
>> libvirt config, which I suppose is better than nothing :-)
>
> Why? This seems wrong. Statically linked drivers appear in
> /sys/bus/drivers/XX too
Yes, they appear in /sys/bus/drivers. But there isn't (as far as I know
so far) a way to programmatically associate a statically linked variant
driver with a device using the information available from the running
kernel.
What I've been told until now is that the only way to find the proper
VFIO variant driver for a particular device is to take the modalias
string from that device's sysfs directory, and search for a line in the
modules.alias file that starts with "alias vfio_pci:" and has the "most
specific match" for the device's modalias - *that* is the name of the
*module* that contains the best VFIO variant driver for the device.
If a driver is statically linked into the kernel, then it doesn't have a
module, so it doesn't get a line in modules.alias (does it? I suppose I
should try that to verify, but it doesn't make sense), and so there is
no way to identify that driver as being a match for a device.
Or am I missing something?
In this case, as I've said in the commit logs somewhere, it's still
possible to use a statically linked variant driver for a device, you
just have to manually specify the driver name rather than having code
that discovers it (which is what you have to do for *all* variant
drivers until the final patch of this series anyway :-)
6 months, 2 weeks
[PATCH] build: suppress "ignoring duplicate libraries" warning on macOS
by Laine Stump
Xcode 15, which provides the compiler toolchain for building libvirt
on macOS has switched to a new linker that warns about duplicated
"-lblah" options on the ld commandline. In practice this is impossible
to prevent in a large project, and also harmless.
Fortunately the new ld command also has an option,
-no_warn_duplicate_libraries, that supresses this harmless/pointless
warning, meson has a simple way to check if that option is supported,
and libvirt's meson.build files already have examples of adding an
option to the ld commandline if it's available.
Signed-off-by: Laine Stump <laine(a)redhat.com>
---
meson.build | 4 ++++
src/meson.build | 1 +
tests/meson.build | 1 +
tools/meson.build | 1 +
4 files changed, 7 insertions(+)
diff --git a/meson.build b/meson.build
index dc0969abcc..a39f942617 100644
--- a/meson.build
+++ b/meson.build
@@ -542,6 +542,10 @@ libvirt_no_indirect = cc.get_supported_link_arguments([
'-Wl,--no-copy-dt-needed-entries',
])
+libvirt_no_warn_duplicate_libraries = cc.get_supported_link_arguments([
+ '-Wl,-no_warn_duplicate_libraries',
+])
+
if host_machine.system() == 'windows'
version_script_flags = '-Wl,'
elif host_machine.system() == 'darwin'
diff --git a/src/meson.build b/src/meson.build
index 5fc4d03b4a..6538c43628 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -18,6 +18,7 @@ src_dep = declare_dependency(
link_args: (
libvirt_relro
+ libvirt_no_indirect
+ + libvirt_no_warn_duplicate_libraries
+ coverage_flags
+ driver_modules_flags
+ win32_link_flags
diff --git a/tests/meson.build b/tests/meson.build
index b235c5f4dd..e1cd57654a 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -28,6 +28,7 @@ tests_dep = declare_dependency(
],
link_args: (
libvirt_export_dynamic
+ + libvirt_no_warn_duplicate_libraries
+ libvirt_flat_namespace
+ coverage_flags
),
diff --git a/tools/meson.build b/tools/meson.build
index f2d58cfdcb..c72f760a1d 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -16,6 +16,7 @@ tools_dep = declare_dependency(
libvirt_relro
+ libvirt_no_indirect
+ libvirt_no_undefined
+ + libvirt_no_warn_duplicate_libraries
),
)
--
2.41.0
6 months, 2 weeks
[PATCH] tests: ignore $__CF_USER_TEXT_ENCODING in env during commandtest
by Laine Stump
This environment variable is supposedly set according to the contents
of ~/.CFUserTextEncoding, and certainly on MacOS 14 (Sonoma) it shows
up in the environment, causing commandtest to fail. However, the value
that is shown in $__CF_USER_TEXT_ENCODING during the test 1) is not in
the environment of the shell the test is run from, and 2) doesn't
match the contents of ~/.CFUserTextEncoding.
It is true, though, that filtering out this environment setting from
the test results permits commandtest to pass on MacOS 14.
Signed-off-by: Laine Stump <laine(a)redhat.com>
---
[*] There may be a better way to suppress this environment setting
(maybe something done to prevent it from ever being added to the
environment in the first place?), and that would be fine too. This
patch does work though.
[*] Andrea's patches to force rpcgen to generate ANSI C code are also
required for the test suite to pass on MacOS 14.
tests/commandhelper.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/tests/commandhelper.c b/tests/commandhelper.c
index 9b56feb120..08ee48a3a8 100644
--- a/tests/commandhelper.c
+++ b/tests/commandhelper.c
@@ -169,9 +169,12 @@ static int printEnvironment(FILE *log)
for (i = 0; i < length; i++) {
/* Ignore the variables used to instruct the loader into
- * behaving differently, as they could throw the tests off. */
- if (!STRPREFIX(newenv[i], "LD_"))
+ * behaving differently, as they could throw the tests off.
+ * Also ignore __CF_USER_TEXT_ENCODING, which is set by MacOS. */
+ if (!STRPREFIX(newenv[i], "LD_") &&
+ !STRPREFIX(newenv[i], "__CF_USER_TEXT_ENCODING=")) {
fprintf(log, "ENV:%s\n", newenv[i]);
+ }
}
return 0;
--
2.41.0
6 months, 2 weeks
[libvirt PATCH] qemu_process: fix crash in qemuSaveImageDecompressionStart
by Pavel Hrdina
Commit changing the code to allow passing NULL as @data into
qemuSaveImageDecompressionStart() was not correct as it left the
original call into the function as well.
Introduced-by: 2f3e582a1ac1008eba8d43c751cdba8712dd1614
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2247754
Signed-off-by: Pavel Hrdina <phrdina(a)redhat.com>
---
src/qemu/qemu_process.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 1ef032dbd2..b9267d8699 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -8208,9 +8208,6 @@ qemuProcessStartWithMemoryState(virConnectPtr conn,
}
}
- if (qemuSaveImageDecompressionStart(data, fd, &intermediatefd, &errbuf, &cmd) < 0)
- return -1;
-
/* No cookie means libvirt which saved the domain was too old to mess up
* the CPU definitions.
*/
--
2.41.0
6 months, 2 weeks
[PATCH 00/16] Farewell rpcgen
by Daniel P. Berrangé
This series something I was hacking on a little while back in an
attempt to make our RPC layer more maintainable. There are many
aspects I'm unhappy about with current code
* When serializing a message we have no clue how big
it will be, but xdrmem_create wants a fixed size,
so we have to keep trying to serialize in a loop
making it bigger each time
* We don't control memory allocation/free'ing directly
so we can't do a virSecureErase on fields inside the
RPC message struct that handle secrets.
* The XDR API is generally unpleasant to use as it is
outside our virNetMessage object. Ideally we would
be reading/writing directly from/to the virNetMessage
buffer with APIs on virNetMessage,instead of indirectly
via a XDR object.
* We want more from XDR than it actually gives us. Our
XDR protocol files have annotations to express what
we want our code generator todo, or for ACLs. The
relationship between the structs and the message
numbers is implicit. Essentially we've defined our
own language indirectly via comments, and then
parse this with regexes which is horrid.
* The code rpcgen creates is poor quality which we have
to post-process to fix bugs/problems. It also lacks
support for modern features like g_auto.
Anyway, in a fit of rage I looked at the XDR RFC and thought..
This language is trivial, why do we need to outsource to
rpcgen and libtirpc instead of dealing with it directly.
This small series moves in that direction. It creates an
XDR language lexer and parser, and then a code generator
which emits code that is (nearly) identical to what rpcgen
would emit. This is sufficient to eliminate rpcgen usage
and support g_auto. Since we're still using libtirpc
at this stage we can be confident we're still doing the
same thing on the wire. I've got some unit tests too
with the rpcgen generation to validate stuff.
The next step is to change the code generator so that
instead of generating code for libtirpc APIs, it will
instead directly speak virNetMessage APIs. That would
give us full control over our RPC stack guaranteed to
be platform portable instead of fighting slight differences
in RPC libraries (eg xdr_quad vs xdr_int64 madness).
I was going to wait until I had written such code before
sending this series, but I've got diverted onto other more
important tasks. So here at least is what I have so far.
After that foundation is done, we are in a place where
we can actually do more innovative things. For example
we can directly extend the XDR protocol language if we
like, turning our magic comments into properly parsable
constructs.
With this, we would be in a position to replace our
Perl RPC client/server dispatch code generators with
something that is more supportable in Python. The python
code would work with properly represented objects and
formal parsers and not regexes and anonymous complex
perl data structures.
Daniel P. Berrangé (16):
rpcgen: drop type-puning workarounds
build-aux: skip E203 and W503 flake8 checks
build-aux: introduce 'black' tool for python formatting
rpcgen: add an XDR protocol lexer
rpcgen: add an XDR protocol abstract syntax tree
rpcgen: add an XDR protocol parser
rpcgen: define a visitor API for XDR protocol specs
rpcgen: add a C code generator for XDR protocol specs
rpcgen: add test case for XDR serialization
rpcgen: define entrypoint for running new rpcgen impl
build: switch over to new rpc generator code
rpcgen: add g_auto function support
rpc: use g_auto for client RPC return parameters
admin: use g_auto for client RPC return parameters
remote: use g_auto for client RPC return parameters
rpc: add helpers for XDR type serialization
build-aux/Makefile.in | 1 +
build-aux/meson.build | 5 +
build-aux/syntax-check.mk | 42 +-
libvirt.spec.in | 2 +-
meson.build | 13 +-
scripts/meson.build | 2 +
scripts/rpcgen/main.py | 90 ++
scripts/rpcgen/meson.build | 16 +
scripts/rpcgen/rpcgen/ast.py | 270 ++++++
scripts/rpcgen/rpcgen/generator.py | 509 ++++++++++++
scripts/rpcgen/rpcgen/lexer.py | 213 +++++
scripts/rpcgen/rpcgen/meson.build | 7 +
scripts/rpcgen/rpcgen/parser.py | 497 +++++++++++
scripts/rpcgen/rpcgen/visitor.py | 156 ++++
scripts/rpcgen/tests/demo.c | 495 +++++++++++
scripts/rpcgen/tests/demo.h | 264 ++++++
scripts/rpcgen/tests/demo.x | 127 +++
scripts/rpcgen/tests/meson.build | 20 +
scripts/rpcgen/tests/simple.x | 35 +
scripts/rpcgen/tests/test_demo.c | 782 ++++++++++++++++++
scripts/rpcgen/tests/test_demo_enum.bin | Bin 0 -> 4 bytes
.../tests/test_demo_enum_fixed_array.bin | Bin 0 -> 52 bytes
.../tests/test_demo_enum_pointer_null.bin | Bin 0 -> 4 bytes
.../tests/test_demo_enum_pointer_set.bin | Bin 0 -> 8 bytes
.../rpcgen/tests/test_demo_enum_scalar.bin | Bin 0 -> 4 bytes
.../test_demo_enum_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_enum_variable_array_set.bin | Bin 0 -> 16 bytes
.../tests/test_demo_int_fixed_array.bin | Bin 0 -> 12 bytes
.../tests/test_demo_int_pointer_null.bin | Bin 0 -> 4 bytes
.../tests/test_demo_int_pointer_set.bin | Bin 0 -> 8 bytes
scripts/rpcgen/tests/test_demo_int_scalar.bin | Bin 0 -> 4 bytes
.../test_demo_int_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_int_variable_array_set.bin | Bin 0 -> 16 bytes
.../tests/test_demo_opaque_fixed_array.bin | Bin 0 -> 12 bytes
.../test_demo_opaque_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_opaque_variable_array_set.bin | Bin 0 -> 8 bytes
.../test_demo_string_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_string_variable_array_set.bin | Bin 0 -> 12 bytes
scripts/rpcgen/tests/test_demo_struct.bin | Bin 0 -> 8 bytes
.../tests/test_demo_struct_fixed_array.bin | Bin 0 -> 136 bytes
.../tests/test_demo_struct_pointer_null.bin | Bin 0 -> 4 bytes
.../tests/test_demo_struct_pointer_set.bin | Bin 0 -> 12 bytes
.../rpcgen/tests/test_demo_struct_scalar.bin | 1 +
.../test_demo_struct_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_struct_variable_array_set.bin | Bin 0 -> 28 bytes
.../tests/test_demo_test_struct_all_types.bin | Bin 0 -> 1752 bytes
scripts/rpcgen/tests/test_demo_union_case.bin | Bin 0 -> 8 bytes
.../rpcgen/tests/test_demo_union_default.bin | Bin 0 -> 8 bytes
.../tests/test_demo_union_fixed_array.bin | Bin 0 -> 168 bytes
.../tests/test_demo_union_no_default_case.bin | Bin 0 -> 8 bytes
.../tests/test_demo_union_pointer_null.bin | Bin 0 -> 4 bytes
.../tests/test_demo_union_pointer_set.bin | Bin 0 -> 12 bytes
.../rpcgen/tests/test_demo_union_scalar.bin | Bin 0 -> 8 bytes
.../test_demo_union_variable_array_empty.bin | Bin 0 -> 4 bytes
.../test_demo_union_variable_array_set.bin | Bin 0 -> 28 bytes
.../test_demo_union_void_default_case.bin | Bin 0 -> 8 bytes
.../test_demo_union_void_default_default.bin | 1 +
scripts/rpcgen/tests/test_generator.py | 60 ++
scripts/rpcgen/tests/test_lexer.py | 116 +++
scripts/rpcgen/tests/test_parser.py | 91 ++
src/admin/admin_remote.c | 50 +-
src/admin/meson.build | 8 +-
src/locking/meson.build | 8 +-
src/logging/meson.build | 8 +-
src/lxc/meson.build | 12 +-
src/remote/meson.build | 8 +-
src/remote/remote_driver.c | 754 ++++++-----------
src/rpc/gendispatch.pl | 60 +-
src/rpc/genprotocol.pl | 144 ----
src/rpc/meson.build | 9 +-
src/rpc/virnetmessage.c | 704 ++++++++++++++++
src/rpc/virnetmessage.h | 88 ++
72 files changed, 4897 insertions(+), 771 deletions(-)
create mode 100755 scripts/rpcgen/main.py
create mode 100644 scripts/rpcgen/meson.build
create mode 100644 scripts/rpcgen/rpcgen/ast.py
create mode 100644 scripts/rpcgen/rpcgen/generator.py
create mode 100644 scripts/rpcgen/rpcgen/lexer.py
create mode 100644 scripts/rpcgen/rpcgen/meson.build
create mode 100644 scripts/rpcgen/rpcgen/parser.py
create mode 100644 scripts/rpcgen/rpcgen/visitor.py
create mode 100644 scripts/rpcgen/tests/demo.c
create mode 100644 scripts/rpcgen/tests/demo.h
create mode 100644 scripts/rpcgen/tests/demo.x
create mode 100644 scripts/rpcgen/tests/meson.build
create mode 100644 scripts/rpcgen/tests/simple.x
create mode 100644 scripts/rpcgen/tests/test_demo.c
create mode 100644 scripts/rpcgen/tests/test_demo_enum.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_fixed_array.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_pointer_null.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_pointer_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_scalar.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_enum_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_fixed_array.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_pointer_null.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_pointer_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_scalar.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_int_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_opaque_fixed_array.bin
create mode 100644 scripts/rpcgen/tests/test_demo_opaque_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_opaque_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_string_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_string_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_fixed_array.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_pointer_null.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_pointer_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_scalar.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_struct_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_test_struct_all_types.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_case.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_default.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_fixed_array.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_no_default_case.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_pointer_null.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_pointer_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_scalar.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_variable_array_empty.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_variable_array_set.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_void_default_case.bin
create mode 100644 scripts/rpcgen/tests/test_demo_union_void_default_default.bin
create mode 100644 scripts/rpcgen/tests/test_generator.py
create mode 100644 scripts/rpcgen/tests/test_lexer.py
create mode 100644 scripts/rpcgen/tests/test_parser.py
delete mode 100755 src/rpc/genprotocol.pl
--
2.39.1
6 months, 2 weeks
[libvirt PATCH 00/11] ci: Build RPMs on MinGW
by Andrea Bolognani
Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/1059447352
The openSUSE Leap 15 job failed, but that's caused by an unrelated
packaging issue:
$ pkg-config --cflags libtirpc
-I/usr/include/tirpc
$ rpm -ql libtirpc-devel | grep usr/include
/usr/include/netconfig.h
/usr/include/rpc
/usr/include/rpc/auth.h
/usr/include/rpc/auth_des.h
...
/usr/include/rpc/types.h
/usr/include/rpc/xdr.h
/usr/include/rpcsvc
/usr/include/rpcsvc/crypt.h
/usr/include/rpcsvc/crypt.x
The CI updates towards the end depend on the following unmerged
libvirt-ci changes:
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/440
Andrea Bolognani (11):
rpm: Disable expensive tests for MinGW builds
rpm: Explicitly enable NLS support
rpm: Rename module-init-tools -> kmod
rpm: Explain a couple of BuildRequires
rpm: Add libxml2 BuildRequires for xmllint
rpm: Shuffle BuildRequires around
rpm: Split call to mingw_debug_package
rpm: Introduce with_mingw32/with_mingw64
rpm: Introduce with_native
ci: Refresh generated files
ci: Build RPMs on MinGW
.gitlab-ci.yml | 11 +-
ci/buildenv/fedora-38-cross-mingw32.sh | 1 +
ci/buildenv/fedora-38-cross-mingw64.sh | 1 +
ci/buildenv/fedora-rawhide-cross-mingw32.sh | 1 +
ci/buildenv/fedora-rawhide-cross-mingw64.sh | 1 +
.../fedora-38-cross-mingw32.Dockerfile | 1 +
.../fedora-38-cross-mingw64.Dockerfile | 1 +
.../fedora-rawhide-cross-mingw32.Dockerfile | 1 +
.../fedora-rawhide-cross-mingw64.Dockerfile | 1 +
ci/jobs.sh | 19 +-
libvirt.spec.in | 470 ++++++++++--------
11 files changed, 296 insertions(+), 212 deletions(-)
--
2.41.0
6 months, 2 weeks
[libvirt PATCH 0/2] rpc: Make rpcgen produce ANSI C code
by Andrea Bolognani
Test pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/1059558331
Should make it possible to build libvirt on macOS 14, where it
currently fails with
src/rpc/virnetprotocol.c:13:1: error: a function definition without
a prototype is deprecated in all versions of C and is not supported
in C2x [-Werror,-Wdeprecated-non-prototype] xdr_virNetMessageType(xdrs, objp)
Andrea Bolognani (2):
rpc: Drop support for portable-rpcgen
rpc: Make rpcgen produce ANSI C code
meson.build | 12 +-----------
src/rpc/genprotocol.pl | 5 ++---
2 files changed, 3 insertions(+), 14 deletions(-)
--
2.41.0
6 months, 2 weeks
Release of libvirt-9.9.0
by Jiri Denemark
The 9.9.0 release of both libvirt and libvirt-python is tagged and
signed tarballs and source RPMs are available at
https://download.libvirt.org/
https://download.libvirt.org/python/
Thanks everybody who helped with this release by sending patches,
reviewing, testing, or providing feedback. Your work is greatly
appreciated.
* New features
* QEMU: implement reverting external snapshots
Reverting external snapshots is now possible using the existing API
``virDomainSnapshotRevert()``. Management application can check host
capabilities for ``<externalSnapshot/>`` element within the list of
guest features to see if the current libvirt supports both deleting
and reverting external snapshots.
* virsh: add ``console --resume`` support
The ``virsh console`` subcommand now accepts a ``--resume`` option. This
will resume a paused guest after connecting to the console.
* Improvements
* virsh: Improve ``virsh start --console`` behavior
The ``virsh start --console`` now tries to connect to the guest console
before starting the vCPUs.
* virsh: Improve ``virsh create --console`` behavior
The ``virsh create --console`` now tries to connect to the guest console
before starting the vCPUs.
Enjoy.
Jirka
6 months, 2 weeks
