[libvirt PATCH 0/2] coverity patches

Pavel Hrdina (2): src: rework static analysis detection gitlab-ci: add coverity job .gitlab-ci.yml | 20 ++++++++++++++++++++ ci/containers/README.rst | 22 ++++++++++++++++++++++ meson.build | 14 -------------- src/internal.h | 4 ++++ 4 files changed, 46 insertions(+), 14 deletions(-) -- 2.26.2

Inspired by QEMU code. Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-) diff --git a/meson.build b/meson.build index cecaad199d..dbbc9632f1 100644 --- a/meson.build +++ b/meson.build @@ -142,20 +142,6 @@ if get_option('test_coverage') endif -# Detect when running under the clang static analyzer's scan-build driver -# or Coverity-prevent's cov-build. Define STATIC_ANALYSIS accordingly. - -rc = run_command( - 'sh', '-c', - 'test -n "${CCC_ANALYZER_HTML}"' + - ' -o -n "${CCC_ANALYZER_ANALYSIS+set}"' + - ' -o -n "$COVERITY_BUILD_COMMAND$COVERITY_LD_PRELOAD"', -) -if rc.returncode() == 0 - conf.set('STATIC_ANALYSIS', 1) -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so diff --git a/src/internal.h b/src/internal.h index d167e56b48..5226667d3d 100644 --- a/src/internal.h +++ b/src/internal.h @@ -29,6 +29,10 @@ #include <stdlib.h> #include "glibcompat.h" +#if defined __clang_analyzer__ || defined __COVERITY__ +#define STATIC_ANALYSIS 1 +#endif + #if STATIC_ANALYSIS # undef NDEBUG /* Don't let a prior NDEBUG definition cause trouble. */ # include <assert.h> -- 2.26.2

On Mon, Nov 16, 2020 at 04:36:05PM +0100, Pavel Hrdina wrote:
Inspired by QEMU code.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 11/16/20 10:36 AM, Pavel Hrdina wrote:
Inspired by QEMU code.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/meson.build b/meson.build index cecaad199d..dbbc9632f1 100644 --- a/meson.build +++ b/meson.build @@ -142,20 +142,6 @@ if get_option('test_coverage') endif
-# Detect when running under the clang static analyzer's scan-build driver -# or Coverity-prevent's cov-build. Define STATIC_ANALYSIS accordingly. - -rc = run_command( - 'sh', '-c', - 'test -n "${CCC_ANALYZER_HTML}"' + - ' -o -n "${CCC_ANALYZER_ANALYSIS+set}"' + - ' -o -n "$COVERITY_BUILD_COMMAND$COVERITY_LD_PRELOAD"', -) -if rc.returncode() == 0 - conf.set('STATIC_ANALYSIS', 1) -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so
diff --git a/src/internal.h b/src/internal.h index d167e56b48..5226667d3d 100644 --- a/src/internal.h +++ b/src/internal.h @@ -29,6 +29,10 @@ #include <stdlib.h> #include "glibcompat.h"
+#if defined __clang_analyzer__ || defined __COVERITY__
^^ Bah humbug ... what defines __COVERITY__ then? In my Coverity environment, nothing... OK, sure I can add it, no problem but something in QEMU's coverity build environment must do that as it's not predefined as far as I can tell. John
+#define STATIC_ANALYSIS 1 +#endif + #if STATIC_ANALYSIS # undef NDEBUG /* Don't let a prior NDEBUG definition cause trouble. */ # include <assert.h>

On Fri, Nov 20, 2020 at 06:57:10AM -0500, John Ferlan wrote:
On 11/16/20 10:36 AM, Pavel Hrdina wrote:
Inspired by QEMU code.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/meson.build b/meson.build index cecaad199d..dbbc9632f1 100644 --- a/meson.build +++ b/meson.build @@ -142,20 +142,6 @@ if get_option('test_coverage') endif
-# Detect when running under the clang static analyzer's scan-build driver -# or Coverity-prevent's cov-build. Define STATIC_ANALYSIS accordingly. - -rc = run_command( - 'sh', '-c', - 'test -n "${CCC_ANALYZER_HTML}"' + - ' -o -n "${CCC_ANALYZER_ANALYSIS+set}"' + - ' -o -n "$COVERITY_BUILD_COMMAND$COVERITY_LD_PRELOAD"', -) -if rc.returncode() == 0 - conf.set('STATIC_ANALYSIS', 1) -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so
diff --git a/src/internal.h b/src/internal.h index d167e56b48..5226667d3d 100644 --- a/src/internal.h +++ b/src/internal.h @@ -29,6 +29,10 @@ #include <stdlib.h> #include "glibcompat.h"
+#if defined __clang_analyzer__ || defined __COVERITY__
^^ Bah humbug ... what defines __COVERITY__ then?
In my Coverity environment, nothing... OK, sure I can add it, no problem but something in QEMU's coverity build environment must do that as it's not predefined as far as I can tell.
Hi John, There is no need to add it anywhere. When building something using coverity it is indeed not defined so GCC will ignore any code guarded with __COVERITY__. The __COVERITY__ is defined by cov-emit binary which is executed by cov-build internally which creates the files that are later analyzed. This is directly from cov-emit --help: Description The cov-emit command parses a source file and outputs it into a directory (emit repository) that can later be analyzed with cov-analyze. The cov-emit command is typically called by cov-translate, which is in turn typically called by cov-build (cov-emit is a low-level command and is not normally called directly). The cov-emit command defines the __COVERITY__ preprocessor symbol. Pavel

On 11/20/20 8:39 AM, Pavel Hrdina wrote:
On Fri, Nov 20, 2020 at 06:57:10AM -0500, John Ferlan wrote:
On 11/16/20 10:36 AM, Pavel Hrdina wrote:
Inspired by QEMU code.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/meson.build b/meson.build index cecaad199d..dbbc9632f1 100644 --- a/meson.build +++ b/meson.build @@ -142,20 +142,6 @@ if get_option('test_coverage') endif
-# Detect when running under the clang static analyzer's scan-build driver -# or Coverity-prevent's cov-build. Define STATIC_ANALYSIS accordingly. - -rc = run_command( - 'sh', '-c', - 'test -n "${CCC_ANALYZER_HTML}"' + - ' -o -n "${CCC_ANALYZER_ANALYSIS+set}"' + - ' -o -n "$COVERITY_BUILD_COMMAND$COVERITY_LD_PRELOAD"', -) -if rc.returncode() == 0 - conf.set('STATIC_ANALYSIS', 1) -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so
diff --git a/src/internal.h b/src/internal.h index d167e56b48..5226667d3d 100644 --- a/src/internal.h +++ b/src/internal.h @@ -29,6 +29,10 @@ #include <stdlib.h> #include "glibcompat.h"
+#if defined __clang_analyzer__ || defined __COVERITY__
^^ Bah humbug ... what defines __COVERITY__ then?
In my Coverity environment, nothing... OK, sure I can add it, no problem but something in QEMU's coverity build environment must do that as it's not predefined as far as I can tell.
Hi John,
There is no need to add it anywhere. When building something using coverity it is indeed not defined so GCC will ignore any code guarded with __COVERITY__.
The __COVERITY__ is defined by cov-emit binary which is executed by cov-build internally which creates the files that are later analyzed.
This is directly from cov-emit --help:
Description
The cov-emit command parses a source file and outputs it into a directory (emit repository) that can later be analyzed with cov-analyze. The cov-emit command is typically called by cov-translate, which is in turn typically called by cov-build (cov-emit is a low-level command and is not normally called directly). The cov-emit command defines the __COVERITY__ preprocessor symbol.
Pavel
Well it didn't seem to get defined in my case; otherwise, I wouldn't have noted it here. ;-) Of course more research ends up figuring this out <read on if truly interested>. Prior to this change, STATIC_ANALYSIS was defined in my <build-dir>/ meson-config.h file because COVERITY_LD_PRELOAD is defined. After this change it is not defined there. After this change, I got a build error because I have some extra sa_assert's in my local sources to avoid various Coverity warnings. One of those is in qemu_agent.c in the qemuAgentCommandFull where I had to add: + if (ret == 0) + sa_assert(*reply); just prior to the return. This resulted in the build error: ../src/qemu/qemu_agent.c: In function ‘qemuAgentCommandFull’: ../src/qemu/qemu_agent.c:1137:26: error: suggest braces around empty body in an ‘if’ statement [-Werror=empty-body] 1137 | sa_assert(*reply); | ^ cc1: all warnings being treated as errors [10/305] Compiling C object src/qemu/l..._driver_qemu_impl.a.p/qemu_hotplug.c.o ninja: build stopped: subcommand failed. which implied to me that sa_assert wasn't defined as a result of a coding error if sa_assert wasn't defined. FWIW: This was done to avoid callers complaining because *reply was set to NULL before a goto cleanup; and before being set to msg.rxObject. Coverity has a "hard time" deciding when one variable (in this case @ret) controls two conditions (in this case @*reply would be filled in). So in the caller it runs a pass where the return value would 0 and *reply = NULL - it's an impossible condition when you read the code. Hence I added the sa_assert, which interestingly enough in this case tripped because of missing { ... }, but that ended up being showing me that STATIC_ANALYSIS was not defined in my build env any more. What got a bit more confusing to me on this though is that if I fix my local patch to have the { }, then I don't get any Coverity warnings like I was getting when the sa_assert wasn't there. So that perhaps implies there's a pass running with *and* without STATIC_ANALYSIS being defined. If I look at the Coverity created build-log.txt I see two passes being made: ... [1182705] EXECUTING: /bin/sh -c "gcc ... qemu_agent.c" ... [1182707] COMPILING: /opt/cov-analysis-linux64-2020.09/bin/cov-translate gcc ... qemu_agent.c ... So I believe I have my answer <sigh>... John

On Fri, Nov 20, 2020 at 10:31:54AM -0500, John Ferlan wrote:
On 11/20/20 8:39 AM, Pavel Hrdina wrote:
On Fri, Nov 20, 2020 at 06:57:10AM -0500, John Ferlan wrote:
On 11/16/20 10:36 AM, Pavel Hrdina wrote:
Inspired by QEMU code.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- meson.build | 14 -------------- src/internal.h | 4 ++++ 2 files changed, 4 insertions(+), 14 deletions(-)
diff --git a/meson.build b/meson.build index cecaad199d..dbbc9632f1 100644 --- a/meson.build +++ b/meson.build @@ -142,20 +142,6 @@ if get_option('test_coverage') endif
-# Detect when running under the clang static analyzer's scan-build driver -# or Coverity-prevent's cov-build. Define STATIC_ANALYSIS accordingly. - -rc = run_command( - 'sh', '-c', - 'test -n "${CCC_ANALYZER_HTML}"' + - ' -o -n "${CCC_ANALYZER_ANALYSIS+set}"' + - ' -o -n "$COVERITY_BUILD_COMMAND$COVERITY_LD_PRELOAD"', -) -if rc.returncode() == 0 - conf.set('STATIC_ANALYSIS', 1) -endif - - # Add RPATH information when building for a non-standard prefix, or # when explicitly requested to do so
diff --git a/src/internal.h b/src/internal.h index d167e56b48..5226667d3d 100644 --- a/src/internal.h +++ b/src/internal.h @@ -29,6 +29,10 @@ #include <stdlib.h> #include "glibcompat.h"
+#if defined __clang_analyzer__ || defined __COVERITY__
^^ Bah humbug ... what defines __COVERITY__ then?
In my Coverity environment, nothing... OK, sure I can add it, no problem but something in QEMU's coverity build environment must do that as it's not predefined as far as I can tell.
Hi John,
There is no need to add it anywhere. When building something using coverity it is indeed not defined so GCC will ignore any code guarded with __COVERITY__.
The __COVERITY__ is defined by cov-emit binary which is executed by cov-build internally which creates the files that are later analyzed.
This is directly from cov-emit --help:
Description
The cov-emit command parses a source file and outputs it into a directory (emit repository) that can later be analyzed with cov-analyze. The cov-emit command is typically called by cov-translate, which is in turn typically called by cov-build (cov-emit is a low-level command and is not normally called directly). The cov-emit command defines the __COVERITY__ preprocessor symbol.
Pavel
Well it didn't seem to get defined in my case; otherwise, I wouldn't have noted it here. ;-) Of course more research ends up figuring this out <read on if truly interested>.
Prior to this change, STATIC_ANALYSIS was defined in my <build-dir>/ meson-config.h file because COVERITY_LD_PRELOAD is defined. After this change it is not defined there.
After this change, I got a build error because I have some extra sa_assert's in my local sources to avoid various Coverity warnings. One of those is in qemu_agent.c in the qemuAgentCommandFull where I had to add:
+ if (ret == 0) + sa_assert(*reply);
just prior to the return. This resulted in the build error:
../src/qemu/qemu_agent.c: In function ‘qemuAgentCommandFull’: ../src/qemu/qemu_agent.c:1137:26: error: suggest braces around empty body in an ‘if’ statement [-Werror=empty-body] 1137 | sa_assert(*reply); | ^ cc1: all warnings being treated as errors [10/305] Compiling C object src/qemu/l..._driver_qemu_impl.a.p/qemu_hotplug.c.o ninja: build stopped: subcommand failed.
which implied to me that sa_assert wasn't defined as a result of a coding error if sa_assert wasn't defined.
FWIW: This was done to avoid callers complaining because *reply was set to NULL before a goto cleanup; and before being set to msg.rxObject. Coverity has a "hard time" deciding when one variable (in this case @ret) controls two conditions (in this case @*reply would be filled in). So in the caller it runs a pass where the return value would 0 and *reply = NULL - it's an impossible condition when you read the code. Hence I added the sa_assert, which interestingly enough in this case tripped because of missing { ... }, but that ended up being showing me that STATIC_ANALYSIS was not defined in my build env any more.
What got a bit more confusing to me on this though is that if I fix my local patch to have the { }, then I don't get any Coverity warnings like I was getting when the sa_assert wasn't there. So that perhaps implies there's a pass running with *and* without STATIC_ANALYSIS being defined. If I look at the Coverity created build-log.txt I see two passes being made:
... [1182705] EXECUTING: /bin/sh -c "gcc ... qemu_agent.c" ... [1182707] COMPILING: /opt/cov-analysis-linux64-2020.09/bin/cov-translate gcc ... qemu_agent.c ...
So I believe I have my answer <sigh>...
Well, that's exactly what I have wrote in my previous reply:
There is no need to add it anywhere. When building something using coverity it is indeed not defined so GCC will ignore any code guarded with __COVERITY__.
The compilation using GCC doesn't have __COVERITY__ so it doesn't have STATIC_ANALYSIS as well. And that is correct, there is no need to have it defined at this point. Before this change it was mandatory to run "meson build" using "cov-build" as well in order to get the COVERITY_LD_PRELOAD defined to detect running under coverity. After this change this is no longer necessary. I understand that this change probably complicates all the workarounds and hacks that you have locally to silence coverity, but overall it simplifies the static analysis detection and speeds up the coverity build process as running "meson build" using "cov-build" is slow. Pavel

Introduce new job to make a coverity build and upload coverity data to scan.coverity.com where the analysis is then executed. Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- .gitlab-ci.yml | 20 ++++++++++++++++++++ ci/containers/README.rst | 22 ++++++++++++++++++++++ 2 files changed, 42 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 725c76e9ee..6792accf8f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -585,3 +585,23 @@ check-dco: - $CI_PROJECT_NAMESPACE == 'libvirt' variables: GIT_DEPTH: 1000 + + +# Coverity job that is run only by schedules +coverity: + image: $CI_REGISTRY_IMAGE/ci-centos-8:latest + needs: + - x64-centos-8-container + stage: builds + script: + - curl https://scan.coverity.com/download/linux64 --form project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN -o /tmp/cov-analysis-linux64.tgz + - tar xfz /tmp/cov-analysis-linux64.tgz + - meson build + - cov-analysis-linux64-*/bin/cov-build --dir cov-int ninja -C build + - tar cfz cov-int.tar.gz cov-int + - curl https://scan.coverity.com/builds?project=$COVERITY_SCAN_PROJECT_NAME --form token=$COVERITY_SCAN_TOKEN --form email=$GITLAB_USER_EMAIL --form file=@cov-int.tar.gz --form version="$(git describe --tags)" --form description="$(git describe --tags) / $CI_COMMIT_TITLE / $CI_COMMIT_REF_NAME:$CI_PIPELINE_ID" + only: + refs: + - schedules + variables: + - $COVERITY_SCAN_PROJECT_NAME && $COVERITY_SCAN_TOKEN diff --git a/ci/containers/README.rst b/ci/containers/README.rst index 530897e311..f2ee132613 100644 --- a/ci/containers/README.rst +++ b/ci/containers/README.rst @@ -12,3 +12,25 @@ https://gitlab.com/libvirt/libvirt-ci The containers are built during the CI process and cached in the GitLab container registry of the project doing the build. The cached containers can be deleted at any time and will be correctly rebuilt. + + +Coverity scan integration +========================= + +This will be used only by the main repository for master branch by running +scheduled pipeline in GitLab. + +The service is proved by `Coverity Scan`_ and requires that the project is +registered there to get free coverity analysis which we already have for +`libvirt project`_. + +To run the coverity job it requires two new variables: + + * ``COVERITY_SCAN_PROJECT_NAME``, containing the `libvirt project`_ + name. + + * ``COVERITY_SCAN_TOKEN``, token visible to admins of `libvirt project`_ + + +.. _Coverity Scan: https://scan.coverity.com/ +.. _libvirt project: https://scan.coverity.com/projects/libvirt -- 2.26.2

On Mon, Nov 16, 2020 at 04:36:06PM +0100, Pavel Hrdina wrote:
Introduce new job to make a coverity build and upload coverity data to scan.coverity.com where the analysis is then executed.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com> --- .gitlab-ci.yml | 20 ++++++++++++++++++++ ci/containers/README.rst | 22 ++++++++++++++++++++++ 2 files changed, 42 insertions(+)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (3)
-
Daniel P. Berrangé
-
John Ferlan
-
Pavel Hrdina