[libvirt PATCH v2] port libvirt to Meson build system
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository. Patches for v2 are available from my gitlab repository: git clone -b meson https://gitlab.com/phrdina/libvirt.git and link to gitlab pipeline: https://gitlab.com/phrdina/libvirt/-/pipelines/172042918 Patches for v1 are available from different branch 'meson-v1' in the same repository. You can get the attached diff by running: git diff meson-v1 meson During review we discovered some areas that should be improved once the meson rewrite is merged into libvirt master: - As of June 2020 we can drop library check for yal2 and use pkg-confg only check because openSUSE Leap 42.3 is no longer supported by libvirt and latest SLES 15 was released on 2018-07-16. - It was suggested to drop README file which is just a link to README.rst and with meson no longer required as it was with autotools. - There is a warning while building libvirt: ../tests/qemuxml2xmltest.c: In function ‘mymain’: ../tests/qemuxml2xmltest.c:132:1: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without 132 | mymain(void) | ^~~~~~ - Review discovered that with meson we no longer link with libnl-route-3. Meson links only with libraries that are actually required and we never used any symbol from this library so it can be dropped. - During review it was pointed out that we can refactor how meson generates html files by removing meson-html-gen.py and moving the logic directly to meson. - There was a question why we install augeas test files so we should figure out if we need to install them or it can be dropped. Here is the list of main changes done in v2, I will not list all of them as it can be easily checked in the diff: - I've reordered and changed the patches to not break git bisect. Now patch 'meson: remove automake specific directives' doesn't remove BUILD_SOURCES because that would break running 'autogen.sh'. Patch 'meson: m4: drop not relevant m4 files' was renamed to 'meson: drop remaining m4 helper files' and moved at the end of the series as it would again break running 'autogen.sh'. Patch 'meson: tests: remove '.libs' from all relevant paths' was moved in the middle of the series before adding 'src' directory into meson because it would break running 'make check'. To not break git bisect we agreed that meson will fail with error until the whole rewrite is done and we will have a temporary option 'force_incomplete_build' that will skip the error and force to run meson setup. I've added a new patch 'meson: add driver_remote option only into meson_options.txt' to break circular dependency between driver_remote and libssh, libssh2, sasl and xdr options to make meson build -Dforce_incomplete_build work properly. - We started running tests on FreeBSD which discovered an issue with meson build. Meson defines _FILE_OFFSET_BITS=64 all the time so I had to introduce a patch that handles this issue: 'meson: tests: virmockstathelpers: replace check for _FILE_OFFSET_BITS' - All scripts except for meson-python.sh wrapper which sets environment variables were converted from shell to python. - Removed autoconf options debug_logs, default_editor and test_suite as we will not convert them to meson. - Introduced git_werror meson option to copy the behavior of --werror option from autotools. It's not the same as with autotools but it is a close compromise. - Fixed virt-aa-helper-test which had hard-coded path to compiled virt-aa-helper binary and that failed with meson. - Correctly replace all SYSCONFDIR and RUNSTATEDIR in man pages files. - Changed the way how we run syntax-check. Instead of having single target it is now part of meson test with suite label 'syntax-checl'. We will not define sytax-check tests if building from tarball, only if we are building from git. We use git to get list of files. - Fixed all issues discovered during review with linking binaries, missing docs and cpu_map files, typos, etc. - Meson cross compile configuration files were moved to libvirt-ci. Pavel
On 7/29/20 4:38 PM, Pavel Hrdina wrote:
Maybe this was discussed already, but if not: do we want to have a script that would bundle meson + ninja commands, so that we could just "./build_and_dont_bother_me.sh". The advantage would be that we could set some defaults, just like we're doing for autogen.sh with "--system". Michal
On Wed, Jul 29, 2020 at 05:32:45PM +0200, Michal Privoznik wrote:
On 7/29/20 4:38 PM, Pavel Hrdina wrote:
Maybe this was discussed already, but if not: do we want to have a script that would bundle meson + ninja commands, so that we could just "./build_and_dont_bother_me.sh". The advantage would be that we could set some defaults, just like we're doing for autogen.sh with "--system".
When I was looking into this topic I found that some projects have configure which is a shell script that calls meson setup and Makefile which calls ninja -C build. Other projects switched completely and removed all files related to autotools, automake, shell, make. I don't think there is a need for such script as developers usually have their own scripts to build projects. And for the --system option that one is actually ported to meson, you can use it like this: meson build -Dsystem=true ninja -C build ./build/src/libvirtd ./build/tools/virsh Pavel
On Wed, Jul 29, 2020 at 16:38:20 +0200, Pavel Hrdina wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Looks like that everything I've required is dealt with. Since the version in the repository now correctly generates kbase files which I've reported offline yesterday and I don't see any problem myself: Series Reviewed-by: Peter Krempa <pkrempa@redhat.com>
On Thu, Jul 30, 2020 at 01:19:54PM +0200, Peter Krempa wrote:
On Wed, Jul 29, 2020 at 16:38:20 +0200, Pavel Hrdina wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Looks like that everything I've required is dealt with.
Since the version in the repository now correctly generates kbase files which I've reported offline yesterday and I don't see any problem myself:
Series
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Thanks a lot for the review, I take that the series is now somehow ready to be pushed. Since I'll be pushing it right after the release is done there is still some time for others to take a look and check that everything is OK. With the fix for generated kbase files the gitlab ci discovered a race-condition with come generated header files. libvirt_driver.a static library includes "rpc/virnetprotocol.h" in libvirt-stream.c but that header file is generated during build time so we need to add a 'rpc_dep' dependency to make sure that the header file is generated before building this library. There was a similar issue with virt_daemons where we indirectly include "admin/admin_protocol.h" which is also generated at build time so we need to add 'admin_dep'. I've already pushed that fix into gitlab and the CI works correctly now. Looking forward to finally have this huge task done \o/.
On Wed, Jul 29, 2020 at 10:38 AM Pavel Hrdina <phrdina@redhat.com> wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Patches for v2 are available from my gitlab repository:
git clone -b meson https://gitlab.com/phrdina/libvirt.git
and link to gitlab pipeline:
https://gitlab.com/phrdina/libvirt/-/pipelines/172042918
Patches for v1 are available from different branch 'meson-v1' in the same repository. You can get the attached diff by running:
git diff meson-v1 meson
During review we discovered some areas that should be improved once the meson rewrite is merged into libvirt master:
- As of June 2020 we can drop library check for yal2 and use pkg-confg only check because openSUSE Leap 42.3 is no longer supported by libvirt and latest SLES 15 was released on 2018-07-16.
- It was suggested to drop README file which is just a link to README.rst and with meson no longer required as it was with autotools.
- There is a warning while building libvirt:
../tests/qemuxml2xmltest.c: In function ‘mymain’: ../tests/qemuxml2xmltest.c:132:1: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without 132 | mymain(void) | ^~~~~~
- Review discovered that with meson we no longer link with libnl-route-3. Meson links only with libraries that are actually required and we never used any symbol from this library so it can be dropped.
- During review it was pointed out that we can refactor how meson generates html files by removing meson-html-gen.py and moving the logic directly to meson.
- There was a question why we install augeas test files so we should figure out if we need to install them or it can be dropped.
Here is the list of main changes done in v2, I will not list all of them as it can be easily checked in the diff:
- I've reordered and changed the patches to not break git bisect. Now patch 'meson: remove automake specific directives' doesn't remove BUILD_SOURCES because that would break running 'autogen.sh'.
Patch 'meson: m4: drop not relevant m4 files' was renamed to 'meson: drop remaining m4 helper files' and moved at the end of the series as it would again break running 'autogen.sh'.
Patch 'meson: tests: remove '.libs' from all relevant paths' was moved in the middle of the series before adding 'src' directory into meson because it would break running 'make check'.
To not break git bisect we agreed that meson will fail with error until the whole rewrite is done and we will have a temporary option 'force_incomplete_build' that will skip the error and force to run meson setup.
I've added a new patch 'meson: add driver_remote option only into meson_options.txt' to break circular dependency between driver_remote and libssh, libssh2, sasl and xdr options to make meson build -Dforce_incomplete_build work properly.
- We started running tests on FreeBSD which discovered an issue with meson build. Meson defines _FILE_OFFSET_BITS=64 all the time so I had to introduce a patch that handles this issue: 'meson: tests: virmockstathelpers: replace check for _FILE_OFFSET_BITS'
- All scripts except for meson-python.sh wrapper which sets environment variables were converted from shell to python.
- Removed autoconf options debug_logs, default_editor and test_suite as we will not convert them to meson.
- Introduced git_werror meson option to copy the behavior of --werror option from autotools. It's not the same as with autotools but it is a close compromise.
- Fixed virt-aa-helper-test which had hard-coded path to compiled virt-aa-helper binary and that failed with meson.
- Correctly replace all SYSCONFDIR and RUNSTATEDIR in man pages files.
- Changed the way how we run syntax-check. Instead of having single target it is now part of meson test with suite label 'syntax-checl'. We will not define sytax-check tests if building from tarball, only if we are building from git. We use git to get list of files.
- Fixed all issues discovered during review with linking binaries, missing docs and cpu_map files, typos, etc.
- Meson cross compile configuration files were moved to libvirt-ci.
Pavel
Series LGTM. Reviewed-by: Neal Gompa <ngompa13@gmail.com> -- 真実はいつも一つ！/ Always, there's only one truth!
On 7/29/20 8:38 AM, Pavel Hrdina wrote:
- As of June 2020 we can drop library check for yal2 and use pkg-confg only check because openSUSE Leap 42.3 is no longer supported by libvirt and latest SLES 15 was released on 2018-07-16.
openSUSE Leap 42.3 was EOL on July 1, 2019. As for SLES15, in the meantime SP1 and SP2 have been released and both contain pkg-config files in libyajl-devel. Same for openSUSE Leap 15.1 and 15.2. Regards, Jim
On Wed, Jul 29, 2020 at 04:38:20PM +0200, Pavel Hrdina wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Patches for v2 are available from my gitlab repository:
git clone -b meson https://gitlab.com/phrdina/libvirt.git
and link to gitlab pipeline:
https://gitlab.com/phrdina/libvirt/-/pipelines/172042918
Patches for v1 are available from different branch 'meson-v1' in the same repository. You can get the attached diff by running:
git diff meson-v1 meson
Thanks for the review and all the feedback. I just pushed the patches to master. Pavel
On Mon, Aug 03, 2020 at 09:37:17AM +0200, Pavel Hrdina wrote:
On Wed, Jul 29, 2020 at 04:38:20PM +0200, Pavel Hrdina wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Patches for v2 are available from my gitlab repository:
git clone -b meson https://gitlab.com/phrdina/libvirt.git
and link to gitlab pipeline:
https://gitlab.com/phrdina/libvirt/-/pipelines/172042918
Patches for v1 are available from different branch 'meson-v1' in the same repository. You can get the attached diff by running:
git diff meson-v1 meson
Thanks for the review and all the feedback. I just pushed the patches to master.
I noticed a problem in a minimal build root, doesn't auto-disable ESX for some reason: meson.build:1607:2: ERROR: Problem encountered: Curl is required for the ESX driver Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On a Monday in 2020, Pavel Hrdina wrote:
On Wed, Jul 29, 2020 at 04:38:20PM +0200, Pavel Hrdina wrote:
The second time around I will not post the complete series as it doesn't make that much sense. Instead I'll post a diff between the two series to provide the changes between these series together with links to both versions pushed to gitlab repository.
Patches for v2 are available from my gitlab repository:
git clone -b meson https://gitlab.com/phrdina/libvirt.git
and link to gitlab pipeline:
https://gitlab.com/phrdina/libvirt/-/pipelines/172042918
Patches for v1 are available from different branch 'meson-v1' in the same repository. You can get the attached diff by running:
git diff meson-v1 meson
Thanks for the review and all the feedback. I just pushed the patches to master.
https://cirrus-ci.com/task/6453169632837632?command=build#L1177 In file included from ../tests/qemuhotplugmock.c:21: In file included from ../src/qemu/qemu_hotplug.h:25: In file included from ../src/qemu/qemu_domain.h:42: ../src/logging/log_manager.h:25:10: fatal error: 'logging/log_protocol.h' file not found #include "logging/log_protocol.h" ^~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. There might be some dependency issue. Jano
participants (7)
-
Daniel P. Berrangé -
Jim Fehlig -
Ján Tomko -
Michal Privoznik -
Neal Gompa -
Pavel Hrdina -
Peter Krempa