[libvirt] [jenkins-ci PATCH 0/5] guests: Enable ccache transparently
This allows us to drop the ugly workaround from the Go and MinGW templates; as a nice side-effect, we also remove the need to hardcode most OS-dependent paths in the inventory. Andrea Bolognani (5): guests: Reorder tasks guests: Figure out OS-dependent paths automatically guests: Figure out more OS-dependent paths ahead of time guests: Create ccache symlink farm jobs: Remove $CC override guests/group_vars/all/main.yml | 7 +-- guests/host_vars/libvirt-freebsd-10/main.yml | 5 -- guests/host_vars/libvirt-freebsd-11/main.yml | 5 -- guests/host_vars/libvirt-freebsd-current/main.yml | 5 -- guests/site.yml | 17 +++--- guests/tasks/paths.yml | 69 +++++++++++++++++++++++ guests/tasks/users.yml | 33 ++++++++++- guests/templates/bashrc | 5 +- jobs/defaults.yaml | 6 -- jobs/go.yaml | 8 --- 10 files changed, 111 insertions(+), 49 deletions(-) create mode 100644 guests/tasks/paths.yml -- 2.14.3
We're going to introduce more customization steps that need to be performed after installing some packages. To avoid hacking too much around the core issue, just move all environment customization after package installation and be done with it. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/site.yml | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/guests/site.yml b/guests/site.yml index a13e0e9..46d2db6 100644 --- a/guests/site.yml +++ b/guests/site.yml @@ -31,12 +31,6 @@ project: blacklist state: absent - # Configure environment. Needs to happen after installing base packages - - include: tasks/bootloader.yml - - include: tasks/services.yml - - include: tasks/kludges.yml - - include: tasks/users.yml - # Install build dependencies for each project - include: tasks/packages.yml with_items: @@ -53,11 +47,13 @@ when: - flavor == "jenkins" + # Configure environment. Needs to happen after installing packages + - include: tasks/bootloader.yml + - include: tasks/services.yml + - include: tasks/kludges.yml + - include: tasks/users.yml + # Configure the Jenkins agent - include: tasks/jenkins.yml when: - flavor == 'jenkins' - - # Some of the kludges involve tweaking files that are included in - # the packages we just installed, so go through them again here - - include: tasks/kludges.yml -- 2.14.3
When configuring a guest, we need to know the absolute path for some commands and files; unfortunately, while most of them are the same regardless of the OS, some of them don't. Right now we hardcode such paths in the inventory, but it would be better if we didn't have to do that: this commit introduces a way to let Ansible figure out the paths for itself instead. We're going to use it for more paths later on. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/group_vars/all/main.yml | 5 --- guests/host_vars/libvirt-freebsd-10/main.yml | 5 --- guests/host_vars/libvirt-freebsd-11/main.yml | 5 --- guests/host_vars/libvirt-freebsd-current/main.yml | 5 --- guests/site.yml | 1 + guests/tasks/paths.yml | 37 +++++++++++++++++++++++ 6 files changed, 38 insertions(+), 20 deletions(-) create mode 100644 guests/tasks/paths.yml diff --git a/guests/group_vars/all/main.yml b/guests/group_vars/all/main.yml index 4e05c66..b364acf 100644 --- a/guests/group_vars/all/main.yml +++ b/guests/group_vars/all/main.yml @@ -18,8 +18,3 @@ GI_TYPELIB_PATH: $VIRT_PREFIX/lib/girepository-1.0 OSINFO_SYSTEM_DIR: $VIRT_PREFIX/share/osinfo PERL5LIB: PYTHONPATH: - -bash: /bin/bash -java: /usr/bin/java -su: /bin/su -sudoers: /etc/sudoers diff --git a/guests/host_vars/libvirt-freebsd-10/main.yml b/guests/host_vars/libvirt-freebsd-10/main.yml index 487b3e8..ab5e367 100644 --- a/guests/host_vars/libvirt-freebsd-10/main.yml +++ b/guests/host_vars/libvirt-freebsd-10/main.yml @@ -4,11 +4,6 @@ ansible_python_interpreter: /usr/local/bin/python2 PERL5LIB: $VIRT_PREFIX/lib/perl5/site_perl/mach/5.26 PYTHONPATH: $VIRT_PREFIX/lib/python3.6/site-packages -bash: /usr/local/bin/bash -java: /usr/local/bin/java -su: /usr/bin/su -sudoers: /usr/local/etc/sudoers - projects: - libosinfo - libvirt diff --git a/guests/host_vars/libvirt-freebsd-11/main.yml b/guests/host_vars/libvirt-freebsd-11/main.yml index 487b3e8..ab5e367 100644 --- a/guests/host_vars/libvirt-freebsd-11/main.yml +++ b/guests/host_vars/libvirt-freebsd-11/main.yml @@ -4,11 +4,6 @@ ansible_python_interpreter: /usr/local/bin/python2 PERL5LIB: $VIRT_PREFIX/lib/perl5/site_perl/mach/5.26 PYTHONPATH: $VIRT_PREFIX/lib/python3.6/site-packages -bash: /usr/local/bin/bash -java: /usr/local/bin/java -su: /usr/bin/su -sudoers: /usr/local/etc/sudoers - projects: - libosinfo - libvirt diff --git a/guests/host_vars/libvirt-freebsd-current/main.yml b/guests/host_vars/libvirt-freebsd-current/main.yml index 487b3e8..ab5e367 100644 --- a/guests/host_vars/libvirt-freebsd-current/main.yml +++ b/guests/host_vars/libvirt-freebsd-current/main.yml @@ -4,11 +4,6 @@ ansible_python_interpreter: /usr/local/bin/python2 PERL5LIB: $VIRT_PREFIX/lib/perl5/site_perl/mach/5.26 PYTHONPATH: $VIRT_PREFIX/lib/python3.6/site-packages -bash: /usr/local/bin/bash -java: /usr/local/bin/java -su: /usr/bin/su -sudoers: /usr/local/etc/sudoers - projects: - libosinfo - libvirt diff --git a/guests/site.yml b/guests/site.yml index 46d2db6..063b0c6 100644 --- a/guests/site.yml +++ b/guests/site.yml @@ -48,6 +48,7 @@ - flavor == "jenkins" # Configure environment. Needs to happen after installing packages + - include: tasks/paths.yml - include: tasks/bootloader.yml - include: tasks/services.yml - include: tasks/kludges.yml diff --git a/guests/tasks/paths.yml b/guests/tasks/paths.yml new file mode 100644 index 0000000..36b699f --- /dev/null +++ b/guests/tasks/paths.yml @@ -0,0 +1,37 @@ +--- +- set_fact: + commands: {} + files: {} + +- name: 'Look for commands' + shell: 'which {{ item }} 2>/dev/null || true' + register: which + with_items: + - bash + - java + - su + +- name: 'Look for files' + shell: 'find /etc /usr/local/etc -name {{ item }} 2>/dev/null' + register: find + with_items: + - sudoers + +- set_fact: + commands: '{{ commands|combine({ item.item: item.stdout }) }}' + no_log: True + with_items: + '{{ which.results }}' + +- set_fact: + files: '{{ files|combine({ item.item: item.stdout }) }}' + no_log: True + with_items: + '{{ find.results }}' + +- name: 'Export paths' + set_fact: + bash: '{{ commands["bash"] }}' + java: '{{ commands["java"] }}' + su: '{{ commands["su"] }}' + sudoers: '{{ files["sudoers"] }}' -- 2.14.3
We will need to know ccache's absolute path at the playbook level later on, so while we're at it we might as well treat all OS-dependent paths the same way and figure them out when configuring the guest rather than at login time. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/tasks/paths.yml | 32 ++++++++++++++++++++++++++++++++ guests/templates/bashrc | 8 +++++--- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/guests/tasks/paths.yml b/guests/tasks/paths.yml index 36b699f..00decc9 100644 --- a/guests/tasks/paths.yml +++ b/guests/tasks/paths.yml @@ -8,7 +8,12 @@ register: which with_items: - bash + - ccache + - gmake - java + - make + - python2 + - python3 - su - name: 'Look for files' @@ -32,6 +37,33 @@ - name: 'Export paths' set_fact: bash: '{{ commands["bash"] }}' + ccache: '{{ commands["ccache"] }}' java: '{{ commands["java"] }}' su: '{{ commands["su"] }}' sudoers: '{{ files["sudoers"] }}' + +# Prefer gmake, fall back to make +- name: 'Export paths' + set_fact: + make: '{{ commands["gmake"] }}' + when: + - commands["gmake"] != '' + +- name: 'Export paths' + set_fact: + make: '{{ commands["make"] }}' + when: + - make is undefined + +# Prefer python3, fall back to python2 +- name: 'Export paths' + set_fact: + python: '{{ commands["python3"] }}' + when: + - commands["python3"] != '' + +- name: 'Export paths' + set_fact: + python: '{{ commands["python2"] }}' + when: + - python is undefined diff --git a/guests/templates/bashrc b/guests/templates/bashrc index e925cd9..65c82cd 100644 --- a/guests/templates/bashrc +++ b/guests/templates/bashrc @@ -11,9 +11,11 @@ export OSINFO_SYSTEM_DIR="{{ OSINFO_SYSTEM_DIR }}" export PERL5LIB="{{ PERL5LIB }}" export PYTHONPATH="{{ PYTHONPATH }}" -which python3 >/dev/null 2>&1 && export PYTHON="python3" || export PYTHON="python2" -which gmake >/dev/null 2>&1 && export MAKE="gmake" || export MAKE="make" -which ccache >/dev/null 2>&1 && export CC="ccache cc" +export CCACHE="{{ ccache }}" +export MAKE="{{ make }}" +export PYTHON="{{ python }}" + +test "$CCACHE" && export CC="$CCACHE cc" # Enable bash completion. Only needed on FreeBSD, the system-wide # shell profile will take care of it for us everywhere else -- 2.14.3
ccache can detect the name it's been called with and transparently invoke the compiler if it matches. The Fedora package uses this trick to enable ccache out of the box, but other operating systems and distributions leave the task up to the user. Replicate the setup in the user's ~/.ccache directory so that it works across all guests. With the linke farm in place, setting $CC in the user's shell profile is no longer necessary. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/group_vars/all/main.yml | 2 +- guests/tasks/users.yml | 33 ++++++++++++++++++++++++++++++++- guests/templates/bashrc | 3 --- 3 files changed, 33 insertions(+), 5 deletions(-) diff --git a/guests/group_vars/all/main.yml b/guests/group_vars/all/main.yml index b364acf..cd8ef25 100644 --- a/guests/group_vars/all/main.yml +++ b/guests/group_vars/all/main.yml @@ -9,7 +9,7 @@ jenkins_url: https://ci.centos.org/computer/{{ inventory_hostname }}/slave-agent # Environment variables and paths to commands/files that might be # OS-dependent. Can be overridden on a per-host basis VIRT_PREFIX: $HOME/build/libvirt -PATH: $VIRT_PREFIX/bin:$PATH +PATH: $VIRT_PREFIX/bin:$HOME/.ccache/bin:$PATH C_INCLUDE_PATH: $VIRT_PREFIX/include LD_LIBRARY_PATH: $VIRT_PREFIX/lib PKG_CONFIG_PATH: $VIRT_PREFIX/lib/pkgconfig diff --git a/guests/tasks/users.yml b/guests/tasks/users.yml index e691b32..033ca2a 100644 --- a/guests/tasks/users.yml +++ b/guests/tasks/users.yml @@ -53,10 +53,13 @@ - name: '{{ flavor }}: Configure ccache' file: - path: /home/{{ flavor }}/.ccache + path: /home/{{ flavor }}/.{{ item }} state: directory owner: '{{ flavor }}' group: '{{ flavor }}' + with_items: + - ccache + - ccache/bin - name: '{{ flavor }}: Configure ccache' copy: @@ -65,6 +68,34 @@ owner: '{{ flavor }}' group: '{{ flavor }}' +- name: '{{ flavor }}: Create ccache wrappers' + file: + src: '{{ ccache }}' + dest: /home/{{ flavor }}/.ccache/bin/{{ item }} + state: link + owner: '{{ flavor }}' + group: '{{ flavor }}' + with_items: + - cc + - clang + when: + - ccache != '' + - os_name == 'FreeBSD' + +- name: '{{ flavor }}: Create ccache wrappers' + file: + src: '{{ ccache }}' + dest: /home/{{ flavor }}/.ccache/bin/{{ item }} + state: link + owner: '{{ flavor }}' + group: '{{ flavor }}' + with_items: + - cc + - gcc + when: + - ccache != '' + - os_name != 'FreeBSD' + - name: '{{ flavor }}: Create shell profile' template: src: templates/{{ item }} diff --git a/guests/templates/bashrc b/guests/templates/bashrc index 65c82cd..866087e 100644 --- a/guests/templates/bashrc +++ b/guests/templates/bashrc @@ -11,12 +11,9 @@ export OSINFO_SYSTEM_DIR="{{ OSINFO_SYSTEM_DIR }}" export PERL5LIB="{{ PERL5LIB }}" export PYTHONPATH="{{ PYTHONPATH }}" -export CCACHE="{{ ccache }}" export MAKE="{{ make }}" export PYTHON="{{ python }}" -test "$CCACHE" && export CC="$CCACHE cc" - # Enable bash completion. Only needed on FreeBSD, the system-wide # shell profile will take care of it for us everywhere else bash_completion=/usr/local/share/bash-completion/bash_completion.sh -- 2.14.3
Disclaimer: I don't feel like I can review the patches properly. Not only because I never heard the term "symlink farm" before, but mostly because jenkins-ci. However I noticed two nits here and there. I hope it won't discourage others from review. I apologize in advance if it does. On Wed, Apr 18, 2018 at 05:25:21PM +0200, Andrea Bolognani wrote:
ccache can detect the name it's been called with and transparently invoke the compiler if it matches. The Fedora package uses this trick to enable ccache out of the box, but other operating systems and distributions leave the task up to the user.
Replicate the setup in the user's ~/.ccache directory so that it works across all guests. With the linke farm in
s/linke/link/ ?
place, setting $CC in the user's shell profile is no longer necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/group_vars/all/main.yml | 2 +- guests/tasks/users.yml | 33 ++++++++++++++++++++++++++++++++- guests/templates/bashrc | 3 --- 3 files changed, 33 insertions(+), 5 deletions(-)
diff --git a/guests/group_vars/all/main.yml b/guests/group_vars/all/main.yml index b364acf..cd8ef25 100644 --- a/guests/group_vars/all/main.yml +++ b/guests/group_vars/all/main.yml @@ -9,7 +9,7 @@ jenkins_url: https://ci.centos.org/computer/{{ inventory_hostname }}/slave-agent # Environment variables and paths to commands/files that might be # OS-dependent. Can be overridden on a per-host basis VIRT_PREFIX: $HOME/build/libvirt -PATH: $VIRT_PREFIX/bin:$PATH +PATH: $VIRT_PREFIX/bin:$HOME/.ccache/bin:$PATH C_INCLUDE_PATH: $VIRT_PREFIX/include LD_LIBRARY_PATH: $VIRT_PREFIX/lib PKG_CONFIG_PATH: $VIRT_PREFIX/lib/pkgconfig diff --git a/guests/tasks/users.yml b/guests/tasks/users.yml index e691b32..033ca2a 100644 --- a/guests/tasks/users.yml +++ b/guests/tasks/users.yml @@ -53,10 +53,13 @@
- name: '{{ flavor }}: Configure ccache' file: - path: /home/{{ flavor }}/.ccache + path: /home/{{ flavor }}/.{{ item }} state: directory owner: '{{ flavor }}' group: '{{ flavor }}' + with_items: + - ccache + - ccache/bin
With new enough ansible, you don't need the 'ccache' entry as this behaves similarly to `mkdir -p`: "If directory, all immediate subdirectories will be created if they do not exist, since 1.7 they will be created with the supplied permissions." [1] You can also use 'recurse' if doing more than just creating them. [1] https://docs.ansible.com/ansible/latest/modules/file_module.html Have a nice day Martin
On Thu, 2018-04-19 at 17:36 +0200, Martin Kletzander wrote:
Replicate the setup in the user's ~/.ccache directory so that it works across all guests. With the linke farm in
s/linke/link/ ?
Close: I actually meant to write Linkle[1] :P
- name: '{{ flavor }}: Configure ccache' file: - path: /home/{{ flavor }}/.ccache + path: /home/{{ flavor }}/.{{ item }} state: directory owner: '{{ flavor }}' group: '{{ flavor }}' + with_items: + - ccache + - ccache/bin
With new enough ansible, you don't need the 'ccache' entry as this behaves similarly to `mkdir -p`:
"If directory, all immediate subdirectories will be created if they do not exist, since 1.7 they will be created with the supplied permissions." [1]
If you dig through the playbooks, you'll certainly find no lack of suboptimal or roundabout approaches to the problem at hand, mostly as a consequence of me pretty much figuring out Ansible as I went. I'll be make to get rid of this specific instance before pushing. More patches in the spirit would be very much welcome O:-) [1] https://zelda.gamepedia.com/Linkle -- Andrea Bolognani / Red Hat / Virtualization
Neither Go nor MinGW builds were able to deal with our previous way of enabling ccache correctly, which forced us to work around the issue at the job template level. Now that ccache is enabled in a completely transparent fashion through a symlink farm, we can drop the workaround. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- jobs/defaults.yaml | 6 ------ jobs/go.yaml | 8 -------- 2 files changed, 14 deletions(-) diff --git a/jobs/defaults.yaml b/jobs/defaults.yaml index 99e8b62..43686c5 100644 --- a/jobs/defaults.yaml +++ b/jobs/defaults.yaml @@ -23,17 +23,11 @@ global_env: | local_env: | mingw32_local_env: | - # The MinGW build needs to use the MinGW compiler toolchain, - # while $CC is pointing to the native toolchain, so we have - # to unset it here. - export CC= export VIRT_PREFIX="$VIRT_PREFIX/i686-w64-mingw32/sys-root/mingw" export PKG_CONFIG_PATH="$VIRT_PREFIX/lib/pkgconfig" export PKG_CONFIG_LIBDIR="/usr/i686-w64-mingw32/sys-root/mingw/lib/pkgconfig:/usr/i686-w64-mingw32/sys-root/mingw/share/pkgconfig" mingw32_autogen_args: --host=i686-w64-mingw32 mingw64_local_env: | - # See above - export CC= export VIRT_PREFIX="$VIRT_PREFIX/x86_64-w64-mingw32/sys-root/mingw" export PKG_CONFIG_PATH="$VIRT_PREFIX/lib/pkgconfig" export PKG_CONFIG_LIBDIR="/usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig:/usr/x86_64-w64-mingw32/sys-root/mingw/share/pkgconfig" diff --git a/jobs/go.yaml b/jobs/go.yaml index 9a349ca..bffe56e 100644 --- a/jobs/go.yaml +++ b/jobs/go.yaml @@ -42,11 +42,6 @@ - shell: | {global_env} {local_env} - # go doesn't handle the $CC variable containing whitespace - # correctly, so we have to make sure it's unset when building. - # See https://github.com/golang/go/issues/11685 - export CC= - go build -v publishers: - email: @@ -85,9 +80,6 @@ - shell: | {global_env} {local_env} - # See above - export CC= - go test $TEST_ARGS publishers: - email: -- 2.14.3
On Wed, Apr 18, 2018 at 05:25:17PM +0200, Andrea Bolognani wrote:
This allows us to drop the ugly workaround from the Go and MinGW templates; as a nice side-effect, we also remove the need to hardcode most OS-dependent paths in the inventory.
Andrea Bolognani (5): guests: Reorder tasks guests: Figure out OS-dependent paths automatically guests: Figure out more OS-dependent paths ahead of time guests: Create ccache symlink farm jobs: Remove $CC override
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
On Thu, 2018-04-26 at 16:23 +0200, Pavel Hrdina wrote:
On Wed, Apr 18, 2018 at 05:25:17PM +0200, Andrea Bolognani wrote:
This allows us to drop the ugly workaround from the Go and MinGW templates; as a nice side-effect, we also remove the need to hardcode most OS-dependent paths in the inventory.
Andrea Bolognani (5): guests: Reorder tasks guests: Figure out OS-dependent paths automatically guests: Figure out more OS-dependent paths ahead of time guests: Create ccache symlink farm jobs: Remove $CC override
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Thanks, I've pushed them now. Feel free to deploy the changes to the CI environment whenever it's convenient for you :) -- Andrea Bolognani / Red Hat / Virtualization
participants (3)
-
Andrea Bolognani -
Martin Kletzander -
Pavel Hrdina