[libvirt] [PATCH 0/5] Enable CI build testing with Travis

Since we already sync our repo to github, we can easily take advantage of the free Travis CI system to improve our testing coverage. Our current CentOS CI hardware doesn't have scope for running many more VMs than it already does and even if it did, that imposes an administrative burden to manage the OS. So this uses Travis to get us automated testing of Ubuntu Precise & Trusty, with GCC and CLang, and OS-X with CLang. Daniel P. Berrange (5): annotate all mocked functions with noinline Prevent test failures with ebtables/iptables/ip6tables are missing Fix detection of required RBD version Ignore SASL deprecation warnings on OS-X Add travis build configuration .travis.yml | 85 ++++++++++++++++++++++++++++++++++++++++ build-aux/mock-noinline.pl | 72 ++++++++++++++++++++++++++++++++++ cfg.mk | 6 ++- m4/virt-storage-rbd.m4 | 16 +++++++- src/internal.h | 22 +++++++++++ src/qemu/qemu_capspriv.h | 2 +- src/rpc/virnetsaslcontext.c | 12 ++++++ src/rpc/virnetsocket.h | 6 ++- src/util/vircommand.h | 2 +- src/util/vircrypto.h | 2 +- src/util/virfile.h | 2 +- src/util/virhostcpu.h | 4 +- src/util/virmacaddr.h | 2 +- src/util/virnetdev.h | 9 +++-- src/util/virnetdevip.h | 2 +- src/util/virnetdevopenvswitch.h | 2 +- src/util/virnetdevtap.h | 6 +-- src/util/virnuma.h | 16 ++++---- src/util/virrandom.h | 6 +-- src/util/virscsi.h | 2 +- src/util/virscsivhost.h | 2 +- src/util/virtpm.h | 2 +- src/util/virutil.h | 10 ++--- src/util/viruuid.h | 2 +- tests/networkxml2firewalltest.c | 13 ++++++ tests/nwfilterebiptablestest.c | 13 ++++++ tests/nwfilterxml2firewalltest.c | 12 ++++++ tests/virfirewalltest.c | 13 ++++++ 28 files changed, 302 insertions(+), 41 deletions(-) create mode 100644 .travis.yml create mode 100644 build-aux/mock-noinline.pl -- 2.9.3

CLang's optimizer is more aggressive at inlining functions than gcc and so will often inline functions that our tests want to mock-override. This causes the test to fail in bizarre ways. We don't want to disable inlining completely, but we must at least prevent inlining of mocked functions. Fortunately there is a 'noinline' attribute that lets us control this per function. A syntax check rule is added that parses tests/*mock.c to extract the list of functions that are mocked (restricted to names starting with 'vir' prefix). It then checks that src/*.h header file to ensure it has a 'ATTRIBUTE_NOINLINE' annotation. This should prevent use from bit-rotting in future. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- build-aux/mock-noinline.pl | 72 +++++++++++++++++++++++++++++++++++++++++ cfg.mk | 6 +++- src/internal.h | 17 ++++++++++ src/qemu/qemu_capspriv.h | 2 +- src/rpc/virnetsocket.h | 6 ++-- src/util/vircommand.h | 2 +- src/util/vircrypto.h | 2 +- src/util/virfile.h | 2 +- src/util/virhostcpu.h | 4 +-- src/util/virmacaddr.h | 2 +- src/util/virnetdev.h | 9 +++--- src/util/virnetdevip.h | 2 +- src/util/virnetdevopenvswitch.h | 2 +- src/util/virnetdevtap.h | 6 ++-- src/util/virnuma.h | 16 ++++----- src/util/virrandom.h | 6 ++-- src/util/virscsi.h | 2 +- src/util/virscsivhost.h | 2 +- src/util/virtpm.h | 2 +- src/util/virutil.h | 10 +++--- src/util/viruuid.h | 2 +- 21 files changed, 135 insertions(+), 39 deletions(-) create mode 100644 build-aux/mock-noinline.pl diff --git a/build-aux/mock-noinline.pl b/build-aux/mock-noinline.pl new file mode 100644 index 0000000..eafe20d --- /dev/null +++ b/build-aux/mock-noinline.pl @@ -0,0 +1,72 @@ +#!/usr/bin/perl + +my %noninlined; +my %mocked; + +# Functions in public header don't get the noinline annotation +# so whitelist them here +$noninlined{"virEventAddTimeout"} = 1; + +foreach my $arg (@ARGV) { + if ($arg =~ /\.h$/) { + #print "Scan header $arg\n"; + &scan_annotations($arg); + } elsif ($arg =~ /mock\.c$/) { + #print "Scan mock $arg\n"; + &scan_overrides($arg); + } +} + +my $warned = 0; +foreach my $func (keys %mocked) { + next if exists $noninlined{$func}; + + $warned++; + print STDERR "$func is mocked at $mocked{$func} but missing noinline annotation\n"; +} + +exit $warned ? 1 : 0; + + +sub scan_annotations { + my $file = shift; + + open FH, $file or die "cannot read $file: $!"; + + my $func; + while (<FH>) { + if (/^\s*(\w+)\(/ || /^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(/) { + my $name = $1; + if ($name !~ /ATTRIBUTE/) { + $func = $name; + } + } elsif (/^\s*$/) { + $func = undef; + } + if (/ATTRIBUTE_NOINLINE/) { + if (defined $func) { + $noninlined{$func} = 1; + } + } + } + + close FH +} + +sub scan_overrides { + my $file = shift; + + open FH, $file or die "cannot read $file: $!"; + + my $func; + while (<FH>) { + if (/^(\w+)\(/ || /^\w+\s*(?:\*\s*)?(\w+)\(/) { + my $name = $1; + if ($name =~ /^vir/) { + $mocked{$name} = "$file:$."; + } + } + } + + close FH +} diff --git a/cfg.mk b/cfg.mk index cc174a3..241510e 100644 --- a/cfg.mk +++ b/cfg.mk @@ -1062,7 +1062,7 @@ _autogen: # regenerate HACKING as part of the syntax-check ifneq ($(_gl-Makefile),) syntax-check: $(top_srcdir)/HACKING spacing-check test-wrap-argv \ - prohibit-duplicate-header + prohibit-duplicate-header mock-noinline endif # Don't include duplicate header in the source (either *.c or *.h) @@ -1076,6 +1076,10 @@ spacing-check: { echo '$(ME): incorrect formatting, see HACKING for rules' 1>&2; \ exit 1; } +mock-noinline: + $(AM_V_GEN)files=`$(VC_LIST) | grep '\.[ch]$$'`; \ + $(PERL) $(top_srcdir)/build-aux/mock-noinline.pl $$files + test-wrap-argv: $(AM_V_GEN)files=`$(VC_LIST) | grep -E '\.(ldargs|args)'`; \ $(PERL) $(top_srcdir)/tests/test-wrap-argv.pl --check $$files diff --git a/src/internal.h b/src/internal.h index d64be93..b1a0556 100644 --- a/src/internal.h +++ b/src/internal.h @@ -153,6 +153,20 @@ # endif /** + * ATTRIBUTE_NOINLINE: + * + * Force compiler not to inline a method. Should be used if + * the method need to be overridable by test mocks. + */ +# ifndef ATTRIBUTE_NOINLINE +# if __GNUC_PREREQ (4, 0) +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# else +# define ATTRIBUTE_NOINLINE +# endif +# endif + +/** * ATTRIBUTE_FMT_PRINTF * * Macro used to check printf like functions, if compiling @@ -236,6 +250,9 @@ # ifndef ATTRIBUTE_RETURN_CHECK # define ATTRIBUTE_RETURN_CHECK # endif +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE +# endif # # ifndef ATTRIBUTE_FALLTHROUGH # define ATTRIBUTE_FALLTHROUGH do {} while(0) diff --git a/src/qemu/qemu_capspriv.h b/src/qemu/qemu_capspriv.h index a7c92fd..94fa75b 100644 --- a/src/qemu/qemu_capspriv.h +++ b/src/qemu/qemu_capspriv.h @@ -95,7 +95,7 @@ virQEMUCapsSetCPUModelInfo(virQEMUCapsPtr qemuCaps, virCPUDefPtr virQEMUCapsProbeHostCPUForEmulator(virCapsPtr caps, virQEMUCapsPtr qemuCaps, - virDomainVirtType type); + virDomainVirtType type) ATTRIBUTE_NOINLINE; void virQEMUCapsSetGICCapabilities(virQEMUCapsPtr qemuCaps, diff --git a/src/rpc/virnetsocket.h b/src/rpc/virnetsocket.h index 1e75ee6..de795af 100644 --- a/src/rpc/virnetsocket.h +++ b/src/rpc/virnetsocket.h @@ -136,9 +136,11 @@ int virNetSocketGetUNIXIdentity(virNetSocketPtr sock, uid_t *uid, gid_t *gid, pid_t *pid, - unsigned long long *timestamp); + unsigned long long *timestamp) + ATTRIBUTE_NOINLINE; int virNetSocketGetSELinuxContext(virNetSocketPtr sock, - char **context); + char **context) + ATTRIBUTE_NOINLINE; int virNetSocketSetBlocking(virNetSocketPtr sock, bool blocking); diff --git a/src/util/vircommand.h b/src/util/vircommand.h index 99dcdeb..e7c2e51 100644 --- a/src/util/vircommand.h +++ b/src/util/vircommand.h @@ -58,7 +58,7 @@ enum { void virCommandPassFD(virCommandPtr cmd, int fd, - unsigned int flags); + unsigned int flags) ATTRIBUTE_NOINLINE; void virCommandPassListenFDs(virCommandPtr cmd); diff --git a/src/util/vircrypto.h b/src/util/vircrypto.h index 52ba3b3..068602f 100644 --- a/src/util/vircrypto.h +++ b/src/util/vircrypto.h @@ -55,6 +55,6 @@ int virCryptoEncryptData(virCryptoCipher algorithm, ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(6) ATTRIBUTE_NONNULL(8) ATTRIBUTE_NONNULL(9) ATTRIBUTE_RETURN_CHECK; -uint8_t *virCryptoGenerateRandom(size_t nbytes); +uint8_t *virCryptoGenerateRandom(size_t nbytes) ATTRIBUTE_NOINLINE; #endif /* __VIR_CRYPTO_H__ */ diff --git a/src/util/virfile.h b/src/util/virfile.h index ba1c57c..41c25a2 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -188,7 +188,7 @@ void virFileActivateDirOverride(const char *argv0) off_t virFileLength(const char *path, int fd) ATTRIBUTE_NONNULL(1); bool virFileIsDir (const char *file) ATTRIBUTE_NONNULL(1); -bool virFileExists(const char *file) ATTRIBUTE_NONNULL(1); +bool virFileExists(const char *file) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NOINLINE; bool virFileIsExecutable(const char *file) ATTRIBUTE_NONNULL(1); enum { diff --git a/src/util/virhostcpu.h b/src/util/virhostcpu.h index e9c22ee..67033de 100644 --- a/src/util/virhostcpu.h +++ b/src/util/virhostcpu.h @@ -38,7 +38,7 @@ bool virHostCPUHasBitmap(void); virBitmapPtr virHostCPUGetPresentBitmap(void); virBitmapPtr virHostCPUGetOnlineBitmap(void); int virHostCPUGetCount(void); -int virHostCPUGetThreadsPerSubcore(virArch arch); +int virHostCPUGetThreadsPerSubcore(virArch arch) ATTRIBUTE_NOINLINE; int virHostCPUGetMap(unsigned char **cpumap, unsigned int *online, @@ -51,7 +51,7 @@ int virHostCPUGetInfo(virArch hostarch, unsigned int *cores, unsigned int *threads); -int virHostCPUGetKVMMaxVCPUs(void); +int virHostCPUGetKVMMaxVCPUs(void) ATTRIBUTE_NOINLINE; int virHostCPUStatsAssign(virNodeCPUStatsPtr param, const char *name, diff --git a/src/util/virmacaddr.h b/src/util/virmacaddr.h index ae26867..f4f5e2c 100644 --- a/src/util/virmacaddr.h +++ b/src/util/virmacaddr.h @@ -48,7 +48,7 @@ void virMacAddrGetRaw(const virMacAddr *src, unsigned char dst[VIR_MAC_BUFLEN]); const char *virMacAddrFormat(const virMacAddr *addr, char *str); void virMacAddrGenerate(const unsigned char prefix[VIR_MAC_PREFIX_BUFLEN], - virMacAddrPtr addr); + virMacAddrPtr addr) ATTRIBUTE_NOINLINE; int virMacAddrParse(const char* str, virMacAddrPtr addr) ATTRIBUTE_RETURN_CHECK; int virMacAddrParseHex(const char* str, diff --git a/src/util/virnetdev.h b/src/util/virnetdev.h index 437a776..12a3123 100644 --- a/src/util/virnetdev.h +++ b/src/util/virnetdev.h @@ -121,7 +121,7 @@ int virNetDevExists(const char *brname) int virNetDevSetOnline(const char *ifname, bool online) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevGetOnline(const char *ifname, bool *online) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; @@ -129,7 +129,7 @@ int virNetDevGetOnline(const char *ifname, int virNetDevSetMAC(const char *ifname, const virMacAddr *macaddr) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevGetMAC(const char *ifname, virMacAddrPtr macaddr) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; @@ -263,7 +263,8 @@ int virNetDevSysfsFile(char **pf_sysfs_device_link, const char *ifname, const char *file) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) - ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; -int virNetDevRunEthernetScript(const char *ifname, const char *script); +int virNetDevRunEthernetScript(const char *ifname, const char *script) + ATTRIBUTE_NOINLINE; #endif /* __VIR_NETDEV_H__ */ diff --git a/src/util/virnetdevip.h b/src/util/virnetdevip.h index cc466ca..6b509ea 100644 --- a/src/util/virnetdevip.h +++ b/src/util/virnetdevip.h @@ -67,7 +67,7 @@ int virNetDevIPAddrAdd(const char *ifname, virSocketAddr *addr, virSocketAddr *peer, unsigned int prefix) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevIPRouteAdd(const char *ifname, virSocketAddrPtr addr, unsigned int prefix, diff --git a/src/util/virnetdevopenvswitch.h b/src/util/virnetdevopenvswitch.h index 7380a2d..51bb1dd 100644 --- a/src/util/virnetdevopenvswitch.h +++ b/src/util/virnetdevopenvswitch.h @@ -59,6 +59,6 @@ int virNetDevOpenvswitchInterfaceStats(const char *ifname, int virNetDevOpenvswitchGetVhostuserIfname(const char *path, char **ifname) - ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; #endif /* __VIR_NETDEV_OPENVSWITCH_H__ */ diff --git a/src/util/virnetdevtap.h b/src/util/virnetdevtap.h index 6bb3b88..311f069 100644 --- a/src/util/virnetdevtap.h +++ b/src/util/virnetdevtap.h @@ -38,7 +38,7 @@ int virNetDevTapCreate(char **ifname, int *tapfd, size_t tapfdSize, unsigned int flags) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevTapDelete(const char *ifname, const char *tunpath) @@ -48,7 +48,7 @@ int virNetDevTapGetName(int tapfd, char **ifname) ATTRIBUTE_NONNULL(2) ATTRIBUTE_RETURN_CHECK; char* virNetDevTapGetRealDeviceName(char *ifname) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; typedef enum { VIR_NETDEV_TAP_CREATE_NONE = 0, @@ -87,7 +87,7 @@ int virNetDevTapCreateInBridgePort(const char *brname, unsigned int *actualMTU, unsigned int flags) ATTRIBUTE_NONNULL(1) ATTRIBUTE_NONNULL(2) ATTRIBUTE_NONNULL(3) - ATTRIBUTE_RETURN_CHECK; + ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; int virNetDevTapInterfaceStats(const char *ifname, virDomainInterfaceStatsPtr stats) diff --git a/src/util/virnuma.h b/src/util/virnuma.h index f3eef32..e4e1fd0 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -34,20 +34,20 @@ int virNumaSetupMemoryPolicy(virDomainNumatuneMemMode mode, virBitmapPtr nodeset); virBitmapPtr virNumaGetHostMemoryNodeset(void); -bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); -bool virNumaIsAvailable(void); -int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodesetIsAvailable(virBitmapPtr nodeset) ATTRIBUTE_NOINLINE; +bool virNumaIsAvailable(void) ATTRIBUTE_NOINLINE; +int virNumaGetMaxNode(void) ATTRIBUTE_NOINLINE; +bool virNumaNodeIsAvailable(int node) ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, - int *ndistances); + int *ndistances) ATTRIBUTE_NOINLINE; int virNumaGetNodeMemory(int node, unsigned long long *memsize, - unsigned long long *memfree); + unsigned long long *memfree) ATTRIBUTE_NOINLINE; unsigned int virNumaGetMaxCPUs(void); -int virNumaGetNodeCPUs(int node, virBitmapPtr *cpus); +int virNumaGetNodeCPUs(int node, virBitmapPtr *cpus) ATTRIBUTE_NOINLINE; int virNumaGetPageInfo(int node, unsigned int page_size, @@ -59,7 +59,7 @@ int virNumaGetPages(int node, unsigned int **pages_avail, unsigned int **pages_free, size_t *npages) - ATTRIBUTE_NONNULL(5); + ATTRIBUTE_NONNULL(5) ATTRIBUTE_NOINLINE; int virNumaSetPagePoolSize(int node, unsigned int page_size, unsigned long long page_count, diff --git a/src/util/virrandom.h b/src/util/virrandom.h index f457d2d..7a984ee 100644 --- a/src/util/virrandom.h +++ b/src/util/virrandom.h @@ -24,11 +24,11 @@ # include "internal.h" -uint64_t virRandomBits(int nbits); +uint64_t virRandomBits(int nbits) ATTRIBUTE_NOINLINE; double virRandom(void); uint32_t virRandomInt(uint32_t max); int virRandomBytes(unsigned char *buf, size_t buflen) - ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK; -int virRandomGenerateWWN(char **wwn, const char *virt_type); + ATTRIBUTE_NONNULL(1) ATTRIBUTE_RETURN_CHECK ATTRIBUTE_NOINLINE; +int virRandomGenerateWWN(char **wwn, const char *virt_type) ATTRIBUTE_NOINLINE; #endif /* __VIR_RANDOM_H__ */ diff --git a/src/util/virscsi.h b/src/util/virscsi.h index 7d88d4e..9f8b3ec 100644 --- a/src/util/virscsi.h +++ b/src/util/virscsi.h @@ -37,7 +37,7 @@ char *virSCSIDeviceGetSgName(const char *sysfs_prefix, const char *adapter, unsigned int bus, unsigned int target, - unsigned long long unit); + unsigned long long unit) ATTRIBUTE_NOINLINE; char *virSCSIDeviceGetDevName(const char *sysfs_prefix, const char *adapter, unsigned int bus, diff --git a/src/util/virscsivhost.h b/src/util/virscsivhost.h index 6018b83..21887dd 100644 --- a/src/util/virscsivhost.h +++ b/src/util/virscsivhost.h @@ -61,6 +61,6 @@ void virSCSIVHostDeviceGetUsedBy(virSCSIVHostDevicePtr dev, const char **drv_name, const char **dom_name); void virSCSIVHostDeviceFree(virSCSIVHostDevicePtr dev); -int virSCSIVHostOpenVhostSCSI(int *vhostfd); +int virSCSIVHostOpenVhostSCSI(int *vhostfd) ATTRIBUTE_NOINLINE; #endif /* __VIR_SCSIHOST_H__ */ diff --git a/src/util/virtpm.h b/src/util/virtpm.h index fe71307..b21fc05 100644 --- a/src/util/virtpm.h +++ b/src/util/virtpm.h @@ -22,6 +22,6 @@ #ifndef __VIR_TPM_H__ # define __VIR_TPM_H__ -char *virTPMCreateCancelPath(const char *devpath); +char *virTPMCreateCancelPath(const char *devpath) ATTRIBUTE_NOINLINE; #endif /* __VIR_TPM_H__ */ diff --git a/src/util/virutil.h b/src/util/virutil.h index e097b77..86e9051 100644 --- a/src/util/virutil.h +++ b/src/util/virutil.h @@ -142,8 +142,8 @@ char *virGetUserConfigDirectory(void); char *virGetUserCacheDirectory(void); char *virGetUserRuntimeDirectory(void); char *virGetUserShell(uid_t uid); -char *virGetUserName(uid_t uid); -char *virGetGroupName(gid_t gid); +char *virGetUserName(uid_t uid) ATTRIBUTE_NOINLINE; +char *virGetGroupName(gid_t gid) ATTRIBUTE_NOINLINE; int virGetGroupList(uid_t uid, gid_t group, gid_t **groups) ATTRIBUTE_NONNULL(3); int virGetUserID(const char *name, @@ -204,12 +204,12 @@ verify((int)VIR_TRISTATE_BOOL_ABSENT == (int)VIR_TRISTATE_SWITCH_ABSENT); unsigned int virGetListenFDs(void); -long virGetSystemPageSize(void); -long virGetSystemPageSizeKB(void); +long virGetSystemPageSize(void) ATTRIBUTE_NOINLINE; +long virGetSystemPageSizeKB(void) ATTRIBUTE_NOINLINE; unsigned long long virMemoryLimitTruncate(unsigned long long value); bool virMemoryLimitIsSet(unsigned long long value); -unsigned long long virMemoryMaxValue(bool ulong); +unsigned long long virMemoryMaxValue(bool ulong) ATTRIBUTE_NOINLINE; /** * VIR_ASSIGN_IS_OVERFLOW: diff --git a/src/util/viruuid.h b/src/util/viruuid.h index 5790a17..1d67e9e 100644 --- a/src/util/viruuid.h +++ b/src/util/viruuid.h @@ -49,7 +49,7 @@ int virGetHostUUID(unsigned char *host_uuid) ATTRIBUTE_NONNULL(1); int virUUIDIsValid(unsigned char *uuid); -int virUUIDGenerate(unsigned char *uuid); +int virUUIDGenerate(unsigned char *uuid) ATTRIBUTE_NOINLINE; int virUUIDParse(const char *uuidstr, unsigned char *uuid) -- 2.9.3

When running tests in a restricted container (as opposed to a full OS install), we can't assume ebtables/iptbles/ip6tables are going to be installed. We must check this and mark the tests as skipped. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- tests/networkxml2firewalltest.c | 13 +++++++++++++ tests/nwfilterebiptablestest.c | 13 +++++++++++++ tests/nwfilterxml2firewalltest.c | 12 ++++++++++++ tests/virfirewalltest.c | 13 +++++++++++++ 4 files changed, 51 insertions(+) diff --git a/tests/networkxml2firewalltest.c b/tests/networkxml2firewalltest.c index 58a9516..fed0e66 100644 --- a/tests/networkxml2firewalltest.c +++ b/tests/networkxml2firewalltest.c @@ -108,6 +108,14 @@ testCompareXMLToIPTablesHelper(const void *data) return result; } +static bool +hasNetfilterTools(void) +{ + return virFileIsExecutable(IPTABLES_PATH) && + virFileIsExecutable(IP6TABLES_PATH) && + virFileIsExecutable(EBTABLES_PATH); +} + static int mymain(void) @@ -131,6 +139,11 @@ mymain(void) virFirewallSetLockOverride(true); if (virFirewallSetBackend(VIR_FIREWALL_BACKEND_DIRECT) < 0) { + if (!hasNetfilterTools()) { + fprintf(stderr, "iptables/ip6tables/ebtables tools not present"); + return EXIT_AM_SKIP; + } + ret = -1; goto cleanup; } diff --git a/tests/nwfilterebiptablestest.c b/tests/nwfilterebiptablestest.c index 4357133..6073423 100644 --- a/tests/nwfilterebiptablestest.c +++ b/tests/nwfilterebiptablestest.c @@ -517,6 +517,14 @@ testNWFilterEBIPTablesApplyDropAllRules(const void *opaque ATTRIBUTE_UNUSED) return ret; } +static bool +hasNetfilterTools(void) +{ + return virFileIsExecutable(IPTABLES_PATH) && + virFileIsExecutable(IP6TABLES_PATH) && + virFileIsExecutable(EBTABLES_PATH); +} + static int mymain(void) @@ -526,6 +534,11 @@ mymain(void) virFirewallSetLockOverride(true); if (virFirewallSetBackend(VIR_FIREWALL_BACKEND_DIRECT) < 0) { + if (!hasNetfilterTools()) { + fprintf(stderr, "iptables/ip6tables/ebtables tools not present"); + return EXIT_AM_SKIP; + } + ret = -1; goto cleanup; } diff --git a/tests/nwfilterxml2firewalltest.c b/tests/nwfilterxml2firewalltest.c index 95ab46e..3d6e792 100644 --- a/tests/nwfilterxml2firewalltest.c +++ b/tests/nwfilterxml2firewalltest.c @@ -445,6 +445,14 @@ testCompareXMLToIPTablesHelper(const void *data) return result; } +static bool +hasNetfilterTools(void) +{ + return virFileIsExecutable(IPTABLES_PATH) && + virFileIsExecutable(IP6TABLES_PATH) && + virFileIsExecutable(EBTABLES_PATH); +} + static int mymain(void) @@ -468,6 +476,10 @@ mymain(void) virFirewallSetLockOverride(true); if (virFirewallSetBackend(VIR_FIREWALL_BACKEND_DIRECT) < 0) { + if (!hasNetfilterTools()) { + fprintf(stderr, "iptables/ip6tables/ebtables tools not present"); + return EXIT_AM_SKIP; + } ret = -1; goto cleanup; } diff --git a/tests/virfirewalltest.c b/tests/virfirewalltest.c index 0f703a6..ac99b82 100644 --- a/tests/virfirewalltest.c +++ b/tests/virfirewalltest.c @@ -1123,11 +1123,24 @@ testFirewallQuery(const void *opaque ATTRIBUTE_UNUSED) return ret; } +static bool +hasNetfilterTools(void) +{ + return virFileIsExecutable(IPTABLES_PATH) && + virFileIsExecutable(IP6TABLES_PATH) && + virFileIsExecutable(EBTABLES_PATH); +} + static int mymain(void) { int ret = 0; + if (!hasNetfilterTools()) { + fprintf(stderr, "iptables/ip6tables/ebtables tools not present"); + return EXIT_AM_SKIP; + } + # define RUN_TEST_DIRECT(name, method) \ do { \ struct testFirewallData data; \ -- 2.9.3

If building libvirt against Ubuntu precise, the librbd.h is lacking many functions that libvirt expects. We have no version check, so we were enabling RBD even though it cannot compile. This configure check uses existance of 'rbd_get_features' as an identifier for the min required version. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- m4/virt-storage-rbd.m4 | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/m4/virt-storage-rbd.m4 b/m4/virt-storage-rbd.m4 index 48522a6..727bd71 100644 --- a/m4/virt-storage-rbd.m4 +++ b/m4/virt-storage-rbd.m4 @@ -28,12 +28,24 @@ AC_DEFUN([LIBVIRT_STORAGE_CHECK_RBD], [ AC_CHECK_HEADER([rbd/librbd.h], [LIBRBD_FOUND=yes; break;]) if test "$LIBRBD_FOUND" = "yes"; then - with_storage_rbd=yes LIBRBD_LIBS="-lrbd -lrados" + + old_LIBS="$LIBS" + LIBS="$LIBS $LIBRBD_LIBS" + AC_CHECK_FUNCS([rbd_get_features],[],[LIBRBD_FOUND=no]) + LIBS="$old_LIBS" + fi + + if test "$LIBRDB_FOUND" = "yes"; then + with_storage_rbd=yes AC_DEFINE_UNQUOTED([WITH_STORAGE_RBD], [1], [whether RBD backend for storage driver is enabled]) else - with_storage_rbd=no + if test "$with_storage_rbd" = "yes"; then + AC_MSG_ERROR([You must install the librbd library & headers to compile libvirt]) + else + with_storage_rbd=no + fi fi fi AM_CONDITIONAL([WITH_STORAGE_RBD], [test "$with_storage_rbd" = "yes"]) -- 2.9.3

Apple have annotated all SASL functions as deprecated for unknown reasons. Since they still work, lets just ignore the warnings. If Apple finally delete the SASL functions our configure check should already catch that Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- src/internal.h | 5 +++++ src/rpc/virnetsaslcontext.c | 12 ++++++++++++ 2 files changed, 17 insertions(+) diff --git a/src/internal.h b/src/internal.h index b1a0556..713734c 100644 --- a/src/internal.h +++ b/src/internal.h @@ -265,6 +265,10 @@ _Pragma ("GCC diagnostic push") \ _Pragma ("GCC diagnostic ignored \"-Wcast-align\"") +# define VIR_WARNINGS_NO_DEPRECATED \ + _Pragma ("GCC diagnostic push") \ + _Pragma ("GCC diagnostic ignored \"-Wdeprecated-declarations\"") + # if HAVE_SUGGEST_ATTRIBUTE_FORMAT # define VIR_WARNINGS_NO_PRINTF \ _Pragma ("GCC diagnostic push") \ @@ -289,6 +293,7 @@ _Pragma ("GCC diagnostic pop") # else # define VIR_WARNINGS_NO_CAST_ALIGN +# define VIR_WARNINGS_NO_DEPRECATED # define VIR_WARNINGS_NO_PRINTF # define VIR_WARNINGS_NO_WLOGICALOP_EQUAL_EXPR # define VIR_WARNINGS_RESET diff --git a/src/rpc/virnetsaslcontext.c b/src/rpc/virnetsaslcontext.c index c4492ec..a7b891f 100644 --- a/src/rpc/virnetsaslcontext.c +++ b/src/rpc/virnetsaslcontext.c @@ -73,6 +73,14 @@ static int virNetSASLContextOnceInit(void) VIR_ONCE_GLOBAL_INIT(virNetSASLContext) +/* Apple have annotated all SASL functions as deprecated for + * unknown reasons. Since they still work, lets just ignore + * the warnings. If Apple finally delete the SASL functions + * our configure check should already catch that + */ +#ifdef __APPLE__ +VIR_WARNINGS_NO_DEPRECATED +#endif virNetSASLContextPtr virNetSASLContextNewClient(void) { @@ -686,3 +694,7 @@ void virNetSASLSessionDispose(void *obj) sasl_dispose(&sasl->conn); VIR_FREE(sasl->callbacks); } + +#ifdef __APPLE__ +VIR_WARNINGS_RESET +#endif -- 2.9.3

This travis configuration tests libvirt builds on 5 platforms that we don't exercise in the CentOS CI system. - Ubuntu Trusty with GCC - Ubuntu Trusty with CLang - Ubuntu Precise with GCC - Ubuntu Precise with CLang - OS-X with CLang NB, syntax-check fails on OS-X with errors like: /bin/sh: /usr/bin/grep: Argument list too long Presumably their grep impl isn't as good as the GNU one, so this test config skips syntax-check on OS-X for now. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> --- .travis.yml | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 .travis.yml diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..613bd2d --- /dev/null +++ b/.travis.yml @@ -0,0 +1,85 @@ +sudo: false +language: c +dist: precise +compiler: + - gcc +cache: ccache +env: + - VIR_TEST_DEBUG=1 CONFIGURE_ARGS="" +addons: + apt: + packages: + - xsltproc + - autopoint + - libxml2-dev + - libncurses5-dev + - libreadline-dev + - zlib1g-dev + - libgnutls-dev + - libgcrypt11-dev + - libavahi-client-dev + - libsasl2-dev + - libxen-dev + - lvm2 + - libgcrypt11-dev + - libparted0-dev + - libdevmapper-dev + - uuid-dev + - libudev-dev + - libpciaccess-dev + - libcap-ng-dev + - libnl-3-dev + - libnl-route-3-dev + - libyajl-dev + - libpcap0.8-dev + - libnuma-dev + - libnetcf-dev + - libaudit-dev +# - dwarves + - libxml2-utils + - libapparmor-dev + - dnsmasq-base + - librbd-dev + - w3c-dtd-xhtml + +notifications: + irc: + # The channel name "irc.oftc.net#virt" is encrypted against libvirt/libvirt + # to prevent IRC notifications from github forks. This was created using: + # $ travis encrypt -r "libvirt/libvirt" "irc.oftc.net#virt" + channels: + - secure: "hUPdkLxX7nh75+clpnk4U0XLExLfV9DFKSvQSAUtf5JtDNMslj7AeOCf2wcbkNsEhkiF557odTAnov1s5m1w/yaa56zbjFAh5agzqRKya3QjqsrvlBKw/WuN+l82iMNLLeebTgCPAXrbAbGWH8YmYssp/7+eMsnKaVh84EQQNbMCHlLg6ovE26Fs18mZ6J5RC3OPa1vbv+xkdCHvGg/Oyp4K8bpU7RYyimA56jdxI/OfdTH9HxntHYSzykR7hDbyzZhdIlAUyRKReQVjcV5+R8fdDL/1imyGA/88KTztMeKXpZ5Rf+Ss3vYLZb6qsLLegCZ4AU/q0vvbWxjpZGJZoeyrVpfBTZdYGIzmLTMl9GYXXa/gDwFlbvRDiPDG4TIy6GlMUROinj7KRKEHu1fWRYu012ife5OjidxcwrTnz21vYaCv3AKWPpMPxwIzQPkY1hex9uLLX6z+TrAxxDLF+7UzRT9w2RLFBkLYlj2aDVrLAVb/ynRsxDz5CGzC61FSQVft2e308SkGjdn8YxvguCuXv+N70Fu1cvFyh5XYeHb4fbBRo0Ctzaec78leHlQvRGWKJxXDXRkE2lvvBc7YbBNSAYh7Fs8Y+zY7l7rMxvXdrt3nuaNQhe74V3yhxPDAld66qmAn9TYMmaZW2f5/KKKILLbCa0t2MxiAc6L2OI8=" + on_success: change + on_failure: always + email: + # The list name 'libvirt-ci@redhat.com" is encrypted against libvirt/libvirt + # to prevent IRC notifications from github forks. This was created using: + # $ travis encrypt -r "libvirt/libvirt" "libvirt-ci@redhat.com" + recipients: + - secure: "QcU9eP96P0RlDNzVRZl/4sxyydPStGzECrpgJhr2IPB/7pHk23yaBrmUsq9S830tB+jwLGma1IscNB8uf7Sf7WY+cYIpfR8v030OffWnaipo/Gcs0dpnlfURWHjOFQI3RJzGEihsqvbwUFOwsM+3IDyO3qdWaiT6cN2Tj9ROlwYCySSX5YWzLyX7arBZ4lp8ESs7ohQaEwp2cegnMP2oGPJJe4SebvlCDjHZbjkU5aEradwUWnRQDJZWTKknpNLArVFxN2/ixp6f/MGY4DmkHoDweio6mHIPN5zTs5Jt32aiX6wDBa+bBa4v8TCRqzhYkQ63ZZhNV8bY5Uf9ufTdyvt96yIANyakd85b1QpMdAX76IyJi1l0/Uub6DTQZAcq3vK7iPjGeTVSpyoXrqTfGy4JxMjqDoocpWvv8ALX1wrYI/HfN2R2Aepw9jModTimOsebYhJ1yMhSt8qnh5AQNftGKL2JBKoA1LWdU2YJ5fO1bGjKNiVEkGFQTPYFWrYCUY5JcT+s5WCzNeMNm8s9na8liYhGl3WtS3rPr5M8bof+BMsBhG2hQ0loduc94x2GkvyhQZUgRbqrwNR+y4hn+rWFC3hBzzyiAULs43vY/PJ+eBdKEf3VAc0MkhQ8GgXGSA61fR6aXYonroI/WnBVItwDmUnnMfSziZXxk09GLl4=" + +git: + submodules: true + +before_install: + - if [ "$TRAVIS_OS_NAME" == "osx" ]; then brew update && brew install gnutls libgcrypt yajl gettext rpcgen ; fi + +# the custom PATH is just to pick up OS-X homebrew & its harmless on Linux +before_script: + - PATH="/usr/local/opt/gettext/bin:/usr/local/opt/rpcgen/bin:$PATH" ./autogen.sh $CONFIGURE_ARGS +script: + - make -j3 && make -j3 syntax-check && make -j3 check + +# Environments here are run in addition to the main environment defined above +matrix: + include: + - compiler: clang + dist: precise + - compiler: clang + dist: trusty + - compiler: gcc + dist: trusty + - compiler: clang + os: osx + script: + - make -j3 && make -j3 check -- 2.9.3

Daniel P. Berrange wrote:
This travis configuration tests libvirt builds on 5 platforms that we don't exercise in the CentOS CI system.
- Ubuntu Trusty with GCC - Ubuntu Trusty with CLang - Ubuntu Precise with GCC - Ubuntu Precise with CLang - OS-X with CLang
NB, syntax-check fails on OS-X with errors like:
/bin/sh: /usr/bin/grep: Argument list too long
Presumably their grep impl isn't as good as the GNU one, so this test config skips syntax-check on OS-X for now.
This looks like the same problem I'm having on FreeBSD. Its ARG_MAX value is too low and not tunable, and the command line for syntax-check does not fit. I work around that by applying a patch to make ARG_MAX tunable on FreeBSD (haven't yet upstreamed that though), but not sure how to tweak that on macOS. Probably a better way would be to somehow reduce length of the command line. Roman Bogorodskiy

On 8 Apr 2017, at 16:28, Roman Bogorodskiy wrote:
This looks like the same problem I'm having on FreeBSD. Its ARG_MAX value is too low and not tunable, and the command line for syntax-check does not fit.
I work around that by applying a patch to make ARG_MAX tunable on FreeBSD (haven't yet upstreamed that though), but not sure how to tweak that on macOS.
ARG_MAX on macOS is set to 256*1024 on macOS [1], and kern.argmax is also a readonly sysctl. Briefly looking over the xnu source tree leads me to believe that making ARG_MAX unable there is even more tricky than on FreeBSD. Just increasing ARG_MAX and recompiling xnu might work though. Then again, most apple boxes tend to not run self-compiled kernels, so I would just go with your second suggestion.
Probably a better way would be to somehow reduce length of the command line.
Fabian Freyer [1] https://github.com/opensource-apple/xnu/blob/53c5e2e62fc4182595609153d4b9964...

On 04/07/2017 11:39 AM, Daniel P. Berrange wrote:
Since we already sync our repo to github, we can easily take advantage of the free Travis CI system to improve our testing coverage. Our current CentOS CI hardware doesn't have scope for running many more VMs than it already does and even if it did, that imposes an administrative burden to manage the OS.
So this uses Travis to get us automated testing of Ubuntu Precise & Trusty, with GCC and CLang, and OS-X with CLang.
Daniel P. Berrange (5): annotate all mocked functions with noinline Prevent test failures with ebtables/iptables/ip6tables are missing Fix detection of required RBD version Ignore SASL deprecation warnings on OS-X Add travis build configuration
.travis.yml | 85 ++++++++++++++++++++++++++++++++++++++++ build-aux/mock-noinline.pl | 72 ++++++++++++++++++++++++++++++++++ cfg.mk | 6 ++- m4/virt-storage-rbd.m4 | 16 +++++++- src/internal.h | 22 +++++++++++ src/qemu/qemu_capspriv.h | 2 +- src/rpc/virnetsaslcontext.c | 12 ++++++ src/rpc/virnetsocket.h | 6 ++- src/util/vircommand.h | 2 +- src/util/vircrypto.h | 2 +- src/util/virfile.h | 2 +- src/util/virhostcpu.h | 4 +- src/util/virmacaddr.h | 2 +- src/util/virnetdev.h | 9 +++-- src/util/virnetdevip.h | 2 +- src/util/virnetdevopenvswitch.h | 2 +- src/util/virnetdevtap.h | 6 +-- src/util/virnuma.h | 16 ++++---- src/util/virrandom.h | 6 +-- src/util/virscsi.h | 2 +- src/util/virscsivhost.h | 2 +- src/util/virtpm.h | 2 +- src/util/virutil.h | 10 ++--- src/util/viruuid.h | 2 +- tests/networkxml2firewalltest.c | 13 ++++++ tests/nwfilterebiptablestest.c | 13 ++++++ tests/nwfilterxml2firewalltest.c | 12 ++++++ tests/virfirewalltest.c | 13 ++++++ 28 files changed, 302 insertions(+), 41 deletions(-) create mode 100644 .travis.yml create mode 100644 build-aux/mock-noinline.pl
ACK series. Thanks, Yash
participants (4)
-
Daniel P. Berrange
-
Fabian Freyer
-
Roman Bogorodskiy
-
Yash Mankad