[libvirt] [PATCH 0/1] Enable code coverage on CI
From: Claudio André <claudioandre.br@gmail.com> *** *** NOTE: In order to add links to demonstrate it working, I'm using a cover letter for only 1 patch. Sorry about this! It is possible to see how it behaves, just access: - https://gist.github.com/claudioandre/60b262ff8c110133c0a94d5f0ce2362c - https://coveralls.io/builds/11681156 ==== Claudio André (1): CI: add code coverage analysis .travis.yml | 11 ++++++++++- README.md | 1 + 2 files changed, 11 insertions(+), 1 deletion(-) -- 2.11.0
From: Claudio André <claudioandre.br@gmail.com> It builds the code coverage report and uploads the coverage data to a web service in order to allow to track libvirt's code coverage over time. --- .travis.yml | 11 ++++++++++- README.md | 1 + 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 266ffff..24b8d6c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -39,6 +39,7 @@ addons: - dnsmasq-base - librbd-dev - w3c-dtd-xhtml + - lcov notifications: irc: @@ -64,7 +65,8 @@ before_install: # the custom PATH is just to pick up OS-X homebrew & its harmless on Linux before_script: - - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh + - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh "$COVERAGE" + - gem install coveralls-lcov script: - VIR_TEST_DEBUG=1 make -j3 && make -j3 syntax-check && make -j3 check @@ -77,9 +79,16 @@ matrix: dist: trusty - compiler: gcc dist: trusty + env: COVERAGE="--enable-test-coverage" - compiler: clang os: osx script: # many unit tests fail & so does syntax-check, so skip for now # one day we must fix it though.... - make -j3 + +after_success: + - 'if [ -n "${COVERAGE}" ]; then + make -j3 cov; + coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage/libvirt.info; + fi' diff --git a/README.md b/README.md index 1035bcf..021f71b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ [](https://travis-ci.org/libvirt/libvirt) +[](https://coveralls.io/github/libvirt/libvirt) Libvirt API for virtualization ============================== -- 2.11.0
On Thu, May 25, 2017 at 01:14:36AM -0300, claudioandre.br@gmail.com wrote:
From: Claudio André <claudioandre.br@gmail.com>
It builds the code coverage report and uploads the coverage data to a web service in order to allow to track libvirt's code coverage over time. ---
You can add the links from the cover letter right here. It is below the '---' marker, so it won't be part of the commit message, but it is also before the first 'diff --git', so it won't be considered part of the patch either. Just like the stats below.
.travis.yml | 11 ++++++++++- README.md | 1 + 2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml index 266ffff..24b8d6c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -39,6 +39,7 @@ addons: - dnsmasq-base - librbd-dev - w3c-dtd-xhtml + - lcov
notifications: irc: @@ -64,7 +65,8 @@ before_install:
# the custom PATH is just to pick up OS-X homebrew & its harmless on Linux before_script: - - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh + - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh "$COVERAGE" + - gem install coveralls-lcov script: - VIR_TEST_DEBUG=1 make -j3 && make -j3 syntax-check && make -j3 check
@@ -77,9 +79,16 @@ matrix: dist: trusty - compiler: gcc dist: trusty + env: COVERAGE="--enable-test-coverage" - compiler: clang os: osx script: # many unit tests fail & so does syntax-check, so skip for now # one day we must fix it though.... - make -j3 + +after_success: + - 'if [ -n "${COVERAGE}" ]; then + make -j3 cov; + coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage/libvirt.info;
I like this, but I have no idea where COVERALLS_TOKEN comes from. How do we make sure that no user branches will post stuff to the coveralls.io site? I think I can safely guess how the rest works, and that seems OK.
+ fi' diff --git a/README.md b/README.md index 1035bcf..021f71b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,5 @@ [](https://travis-ci.org/libvirt/libvirt) +[](https://coveralls.io/github/libvirt/libvirt)
Libvirt API for virtualization ============================== -- 2.11.0
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Em 25/05/2017 03:49, Martin Kletzander escreveu:
On Thu, May 25, 2017 at 01:14:36AM -0300, claudioandre.br@gmail.com wrote:
From: Claudio André <claudioandre.br@gmail.com>
It builds the code coverage report and uploads the coverage data to a web service in order to allow to track libvirt's code coverage over time. ---
You can add the links from the cover letter right here. It is below the '---' marker, so it won't be part of the commit message, but it is also before the first 'diff --git', so it won't be considered part of the patch either. Just like the stats below.
Thanks for the information
+after_success: + - 'if [ -n "${COVERAGE}" ]; then + make -j3 cov; + coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage/libvirt.info;
I like this, but I have no idea where COVERALLS_TOKEN comes from.
The Token comes from coveralls.io site. 1. create your/libvirt account on coveralls (using GitHub credentials) 2. enable the libvirt repository and you will be able to see a field named: REPO TOKEN. 3. then, using Travis website (or Travis CLI tool, I prefer the first for this case) you save the token ciphered. For Coveralls, click here and there and you will find your way to enable the repository and get the token (probably https://coveralls.io/github/YOUR_NICK/libvirt/settings) For Travis: https://travis-ci.org/YOUR_NICK/libvirt/settings, you'll see "Environment Variables" and the add button.
How do we make sure that no user branches will post stuff to the coveralls.io site?
Because Travis save the token ciphered, nobody else can see the token. In fact, if you save it using the site, it does not exist even in the .travis.yml file. And if Travis, Coveralls, ..., are compromised, you can easily regenerate the Token fast enough to avoid damage. Claudio Sorry if the message sounds, umm, pedantic. Thanks
On Thu, May 25, 2017 at 08:44:48PM -0300, Claudio André wrote:
Em 25/05/2017 03:49, Martin Kletzander escreveu:
On Thu, May 25, 2017 at 01:14:36AM -0300, claudioandre.br@gmail.com wrote:
From: Claudio André <claudioandre.br@gmail.com>
It builds the code coverage report and uploads the coverage data to a web service in order to allow to track libvirt's code coverage over time. ---
You can add the links from the cover letter right here. It is below the '---' marker, so it won't be part of the commit message, but it is also before the first 'diff --git', so it won't be considered part of the patch either. Just like the stats below.
Thanks for the information
+after_success: + - 'if [ -n "${COVERAGE}" ]; then + make -j3 cov; + coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage/libvirt.info;
I like this, but I have no idea where COVERALLS_TOKEN comes from.
The Token comes from coveralls.io site. 1. create your/libvirt account on coveralls (using GitHub credentials) 2. enable the libvirt repository and you will be able to see a field named: REPO TOKEN. 3. then, using Travis website (or Travis CLI tool, I prefer the first for this case) you save the token ciphered.
Oh, so it's saved in your profile on the travis website. That makes sense. I was just wondering where this environment variable gets defined, that's it. And I didn't check the documentation because I didn't like the fact that coveralls.io wanted _write_ access to my commit messages and repositories, so I didn't sign up. And their documentation seems pretty sparse when you're not logged in.
For Coveralls, click here and there and you will find your way to enable the repository and get the token (probably https://coveralls.io/github/YOUR_NICK/libvirt/settings) For Travis: https://travis-ci.org/YOUR_NICK/libvirt/settings, you'll see "Environment Variables" and the add button.
How do we make sure that no user branches will post stuff to the coveralls.io site?
Because Travis save the token ciphered, nobody else can see the token. In fact, if you save it using the site, it does not exist even in the .travis.yml file. And if Travis, Coveralls, ..., are compromised, you can easily regenerate the Token fast enough to avoid damage.
Claudio Sorry if the message sounds, umm, pedantic.
Thanks
Em 26/05/2017 04:12, Martin Kletzander escreveu:
Oh, so it's saved in your profile on the travis website. That makes sense. I was just wondering where this environment variable gets defined, that's it. And I didn't check the documentation because I didn't like the fact that coveralls.io wanted _write_ access to my commit messages and repositories, so I didn't sign up. And their documentation seems pretty sparse when you're not logged in.
Agreed. coveralls.io is a nice tool with a list of respectable clients, but it is far from the maturity of, e.g., Travis. Claudio
Sorry, I missed the list in CC. Em 26/05/2017 04:12, Martin Kletzander escreveu:
And I didn't check the documentation because I didn't like the fact that coveralls.io wanted _write_ access to my commit messages and repositories, so I didn't sign up.
I'm afraid I disagree. It requires _write_ access ONLY to commit statuses: - This application will be able to read and write commit statuses (no direct code access). From GitHub help site: Commit status You can grant access for an app to report your commit status. Commit status access allows apps to determine if a build is a successful against a specific commit. Apps won't have access to your code, but they can read and write status information against a specific commit. Claudio
On Thu, May 25, 2017 at 01:14:36AM -0300, claudioandre.br@gmail.com wrote:
From: Claudio André <claudioandre.br@gmail.com>
It builds the code coverage report and uploads the coverage data to a web service in order to allow to track libvirt's code coverage over time. --- .travis.yml | 11 ++++++++++- README.md | 1 + 2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml index 266ffff..24b8d6c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -39,6 +39,7 @@ addons: - dnsmasq-base - librbd-dev - w3c-dtd-xhtml + - lcov
notifications: irc: @@ -64,7 +65,8 @@ before_install:
# the custom PATH is just to pick up OS-X homebrew & its harmless on Linux before_script: - - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh + - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh "$COVERAGE" + - gem install coveralls-lcov
Does this actually work on OS-X ? The fact that you only enabled testing on Trusty suggests not, and if so you shouldn't run this in the common path. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Em 25/05/2017 05:21, Daniel P. Berrange escreveu:
On Thu, May 25, 2017 at 01:14:36AM -0300, claudioandre.br@gmail.com wrote:
@@ -64,7 +65,8 @@ before_install:
# the custom PATH is just to pick up OS-X homebrew & its harmless on Linux before_script: - - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh + - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh "$COVERAGE" + - gem install coveralls-lcov Does this actually work on OS-X ? The fact that you only enabled testing on Trusty suggests not, and if so you shouldn't run this in the common path.
Yes, it does work on OS-X (I tested on 10.11). I enabled only on Trusty because I prefer to use the latest LTS to it (I mean, the latest version and LTS). That said, do you want any change? Claudio
participants (4)
-
Claudio André -
claudioandre.br＠gmail.com
-
Daniel P. Berrange -
Martin Kletzander