[libvirt] [jenkins-ci PATCH 0/3] lcitool: Some pip-related fixes
Andrea Bolognani (3): lcitool: Rename temp -> resolved lcitool: Fix flattening of pip mappings lcitool: Fix pip use on CentOS 7 guests/playbooks/update/tasks/packages.yml | 61 ++++++++++++---------- 1 file changed, 32 insertions(+), 29 deletions(-) -- 2.24.1
Since we're using the dictionary to resolve the mappings for the specific target OS, this is a much more suitable name. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 46 +++++++++++----------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index bd55e34..116c46d 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -10,7 +10,7 @@ file: '{{ base }}/vars/projects/{{ project }}.yml' - set_fact: - temp: {} + resolved: {} - name: '{{ project }}: Verify mappings' fail: @@ -22,7 +22,7 @@ - name: '{{ project }}: Look up mappings (default)' set_fact: - temp: '{{ temp|combine({ item: mappings[item]["default"] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item]["default"] }) }}' with_items: '{{ packages }}' when: @@ -30,7 +30,7 @@ - name: '{{ project }}: Look up mappings (package format)' set_fact: - temp: '{{ temp|combine({ item: mappings[item][package_format] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item][package_format] }) }}' with_items: '{{ packages }}' when: @@ -38,7 +38,7 @@ - name: '{{ project }}: Look up mappings (OS name)' set_fact: - temp: '{{ temp|combine({ item: mappings[item][os_name] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item][os_name] }) }}' with_items: '{{ packages }}' when: @@ -46,7 +46,7 @@ - name: '{{ project }}: Look up mappings (OS version)' set_fact: - temp: '{{ temp|combine({ item: mappings[item][os_name + os_version] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item][os_name + os_version] }) }}' with_items: '{{ packages }}' when: @@ -54,7 +54,7 @@ - name: '{{ project }}: Look up mappings (arch with default)' set_fact: - temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + "default"] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item]["x86_64" + "-" + "default"] }) }}' with_items: '{{ packages }}' when: @@ -62,7 +62,7 @@ - name: '{{ project }}: Look up mappings (arch with package format)' set_fact: - temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + package_format] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item]["x86_64" + "-" + package_format] }) }}' with_items: '{{ packages }}' when: @@ -70,7 +70,7 @@ - name: '{{ project }}: Look up mappings (arch with OS name)' set_fact: - temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item]["x86_64" + "-" + os_name] }) }}' with_items: '{{ packages }}' when: @@ -78,7 +78,7 @@ - name: '{{ project }}: Look up mappings (arch with OS version)' set_fact: - temp: '{{ temp|combine({ item: mappings[item]["x86_64" + "-" + os_name + os_version] }) }}' + resolved: '{{ resolved|combine({ item: mappings[item]["x86_64" + "-" + os_name + os_version] }) }}' with_items: '{{ packages }}' when: @@ -89,12 +89,12 @@ - name: '{{ project }}: Flatten package list' set_fact: - flattened: '{{ flattened }} + [ "{{ temp[item] }}" ]' + flattened: '{{ flattened }} + [ "{{ resolved[item] }}" ]' with_items: - '{{ temp }}' + '{{ resolved }}' when: - - temp[item] != None - - temp[item] not in flattened + - resolved[item] != None + - resolved[item] not in flattened - name: '{{ project }}: Install/remove packages (state={{ state }})' package: @@ -102,7 +102,7 @@ state: '{{ state }}' - set_fact: - pip_temp: {} + pip_resolved: {} - name: '{{ project }}: Verify pip mappings' fail: @@ -115,7 +115,7 @@ - name: '{{ project }}: Look up pip mappings (default)' set_fact: - pip_temp: '{{ pip_temp|combine({ item: pip_mappings[item]["default"] }) }}' + pip_resolved: '{{ pip_resolved|combine({ item: pip_mappings[item]["default"] }) }}' with_items: '{{ packages }}' when: @@ -126,12 +126,12 @@ - name: '{{ project }}: Flatten pip package list' set_fact: - pip_flattened: '{{ pip_flattened }} + [ "{{ pip_temp[item] }}" ]' + pip_flattened: '{{ pip_flattened }} + [ "{{ pip_resolved[item] }}" ]' with_items: - '{{ pip_temp }}' + '{{ pip_resolved }}' when: - - pip_temp[item] != None - - pip_temp[item] not in pip_flattened + - pip_resolved[item] != None + - pip_resolved[item] not in pip_flattened - name: '{{ project }}: Install packages from pip (state={{ state }})' pip: @@ -141,7 +141,7 @@ with_items: '{{ packages }}' when: - - temp[item] is defined - - temp[item] == None - - pip_temp[item] is defined - - pip_temp[item] != None + - resolved[item] is defined + - resolved[item] == None + - pip_resolved[item] is defined + - pip_resolved[item] != None -- 2.24.1
On Tue, Jan 07, 2020 at 04:50:18PM +0100, Andrea Bolognani wrote:
Since we're using the dictionary to resolve the mappings for the specific target OS, this is a much more suitable name.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 46 +++++++++++----------- 1 file changed, 23 insertions(+), 23 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Right now we're looping in the wrong place: instead of unconditionally adding to the flattened list of pip packages and then calling pip multiple times with that same list, we should conditionally add items to the flattened list and then call pip a single time with the result. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index 116c46d..fc8a3d1 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -121,6 +121,15 @@ when: - pip_mappings[item]["default"] is defined +- name: '{{ project }}: Remove unwanted pip mappings' + set_fact: + pip_resolved: '{{ pip_resolved|combine({ item: None }) }}' + with_items: + '{{ packages }}' + when: + - resolved[item] is defined + - resolved[item] != None + - set_fact: pip_flattened: [] @@ -138,10 +147,3 @@ name: '{{ pip_flattened|sort }}' executable: pip3 state: '{{ state }}' - with_items: - '{{ packages }}' - when: - - resolved[item] is defined - - resolved[item] == None - - pip_resolved[item] is defined - - pip_resolved[item] != None -- 2.24.1
On Tue, Jan 07, 2020 at 04:50:19PM +0100, Andrea Bolognani wrote:
Right now we're looping in the wrong place: instead of unconditionally adding to the flattened list of pip packages and then calling pip multiple times with that same list, we should conditionally add items to the flattened list and then call pip a single time with the result.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
The 'pip' Ansible module doesn't work correctly when there is a mismatch between the Python version we're adding packages to and the one used to run Ansible itself: this is the case on CentOS 7, where we have Python 3 for the former but Python 2 for the latter. Call pip3 manually using the 'command' module instead to work around this issue. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/guests/playbooks/update/tasks/packages.yml b/guests/playbooks/update/tasks/packages.yml index fc8a3d1..4e7d5f6 100644 --- a/guests/playbooks/update/tasks/packages.yml +++ b/guests/playbooks/update/tasks/packages.yml @@ -143,7 +143,8 @@ - pip_resolved[item] not in pip_flattened - name: '{{ project }}: Install packages from pip (state={{ state }})' - pip: - name: '{{ pip_flattened|sort }}' - executable: pip3 - state: '{{ state }}' + command: 'pip3 install {{ item }}' + args: + warn: no + with_items: + '{{ pip_flattened|sort }}' -- 2.24.1
On Tue, Jan 07, 2020 at 04:50:20PM +0100, Andrea Bolognani wrote:
The 'pip' Ansible module doesn't work correctly when there is a mismatch between the Python version we're adding packages to and the one used to run Ansible itself: this is the case on CentOS 7, where we have Python 3 for the former but Python 2 for the latter.
Call pip3 manually using the 'command' module instead to work around this issue.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- guests/playbooks/update/tasks/packages.yml | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Andrea Bolognani -
Daniel P. Berrangé