[libvirt] [PATCH v3 0/9] tests: Fix 'make check' on FreeBSD

Changes from [v2]: * introduce a wrapper around canonicalize_file_name() instead of moving to realpath(); * fix more usages of mode_t with va_arg(). Changes from [v1]: * drop the virnetsockettest changes, since the relationship between the errors seen on FreeBSD and IPv4 clients connecting to IPv6 sockets has turned out to be just a red herring; * fix the mocking instead of dropping test cases that don't work without it. [v2] https://www.redhat.com/archives/libvir-list/2018-April/msg02685.html [v1] https://www.redhat.com/archives/libvir-list/2018-April/msg02611.html Andrea Bolognani (9): util: Introduce virFileCanonicalizePath() tests: Mock virFileCanonicalizePath() all: Use virFileCanonicalizePath() instead of canonicalize_file_name() tests: Stop mocking canonicalize_file_name() syntax-check: Prohibit canonicalize_file_name() tests: Fix mode_t usage with va_arg() tests: Build virpcimock on non-Linux tests: Extend NUMA mocking tests: Drop DO_TEST_LINUX() cfg.mk | 10 ++++ src/libvirt_private.syms | 1 + src/storage/storage_backend_fs.c | 2 +- src/util/virfile.c | 15 +++++- src/util/virfile.h | 1 + src/util/virpci.c | 2 +- tests/nssmock.c | 2 +- tests/qemuxml2argvmock.c | 38 ++++++++++---- tests/qemuxml2argvtest.c | 46 ++++++----------- tests/vircgroupmock.c | 2 +- tests/virpcimock.c | 104 ++++++++++++++++++++++----------------- tests/virstoragetest.c | 10 ++-- tests/virtestmock.c | 8 +-- 13 files changed, 141 insertions(+), 100 deletions(-) -- 2.14.3

It's a trivial wrapper around canonicalize_file_name(), which we need in order to fully mock file access on non-Linux platforms. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/libvirt_private.syms | 1 + src/util/virfile.c | 13 +++++++++++++ src/util/virfile.h | 1 + 3 files changed, 15 insertions(+) diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index d2728749fb..cbf27ffa05 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1761,6 +1761,7 @@ virFileAccessibleAs; virFileActivateDirOverride; virFileBindMountDevice; virFileBuildPath; +virFileCanonicalizePath; virFileClose; virFileComparePaths; virFileCopyACLs; diff --git a/src/util/virfile.c b/src/util/virfile.c index e12a584ca1..2a16b01ae5 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -3299,6 +3299,19 @@ virFileSanitizePath(const char *path) return cleanpath; } +/** + * virFileCanonicalizePath: + * + * Returns the canonical representation of @path. + * + * The returned string must be freed after use. + */ +char * +virFileCanonicalizePath(const char *path) +{ + return canonicalize_file_name(path); +} + /** * virFileRemoveLastComponent: * diff --git a/src/util/virfile.h b/src/util/virfile.h index cd2a3867c2..341320b3d3 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -218,6 +218,7 @@ int virFileGetMountReverseSubtree(const char *mtabpath, size_t *nmountsret) ATTRIBUTE_RETURN_CHECK; char *virFileSanitizePath(const char *path); +char *virFileCanonicalizePath(const char *path) ATTRIBUTE_NOINLINE; enum { VIR_FILE_OPEN_NONE = 0, -- 2.14.3

On Thu, May 03, 2018 at 12:54:15PM +0200, Andrea Bolognani wrote:
It's a trivial wrapper around canonicalize_file_name(), which we need in order to fully mock file access on non-Linux platforms.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/libvirt_private.syms | 1 + src/util/virfile.c | 13 +++++++++++++ src/util/virfile.h | 1 + 3 files changed, 15 insertions(+)
diff --git a/src/libvirt_private.syms b/src/libvirt_private.syms index d2728749fb..cbf27ffa05 100644 --- a/src/libvirt_private.syms +++ b/src/libvirt_private.syms @@ -1761,6 +1761,7 @@ virFileAccessibleAs; virFileActivateDirOverride; virFileBindMountDevice; virFileBuildPath; +virFileCanonicalizePath; virFileClose; virFileComparePaths; virFileCopyACLs; diff --git a/src/util/virfile.c b/src/util/virfile.c index e12a584ca1..2a16b01ae5 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -3299,6 +3299,19 @@ virFileSanitizePath(const char *path) return cleanpath; }
+/** + * virFileCanonicalizePath: + * + * Returns the canonical representation of @path. + * + * The returned string must be freed after use. + */ +char * +virFileCanonicalizePath(const char *path) +{ + return canonicalize_file_name(path); +} + /** * virFileRemoveLastComponent: * diff --git a/src/util/virfile.h b/src/util/virfile.h index cd2a3867c2..341320b3d3 100644 --- a/src/util/virfile.h +++ b/src/util/virfile.h @@ -218,6 +218,7 @@ int virFileGetMountReverseSubtree(const char *mtabpath, size_t *nmountsret) ATTRIBUTE_RETURN_CHECK;
char *virFileSanitizePath(const char *path); +char *virFileCanonicalizePath(const char *path) ATTRIBUTE_NOINLINE;
ATTRIBUTE_NOINLINE feels wierd, but I guess it is needed in virfile.c uses the method Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

We're going to need this later on. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/tests/virpcimock.c b/tests/virpcimock.c index 176c64d654..3d3a4b4a19 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -43,6 +43,7 @@ static char *(*real_canonicalize_file_name)(const char *path); static int (*real_open)(const char *path, int flags, ...); static int (*real_close)(int fd); static DIR * (*real_opendir)(const char *name); +static char *(*real_virFileCanonicalizePath)(const char *path); /* Don't make static, since it causes problems with clang * when passed as an arg to virAsprintf() @@ -814,6 +815,7 @@ init_syms(void) VIR_MOCK_REAL_INIT(open); VIR_MOCK_REAL_INIT(close); VIR_MOCK_REAL_INIT(opendir); + VIR_MOCK_REAL_INIT(virFileCanonicalizePath); } static void @@ -1046,6 +1048,28 @@ close(int fd) return -1; return real_close(fd); } + +char * +virFileCanonicalizePath(const char *path) +{ + char *ret; + + init_syms(); + + if (STRPREFIX(path, SYSFS_PCI_PREFIX)) { + char *newpath; + + if (getrealpath(&newpath, path) < 0) + return NULL; + + ret = real_virFileCanonicalizePath(newpath); + VIR_FREE(newpath); + } else { + ret = real_virFileCanonicalizePath(path); + } + + return ret; +} #else /* Nothing to override on non-__linux__ platforms */ #endif -- 2.14.3

On Thu, May 03, 2018 at 12:54:16PM +0200, Andrea Bolognani wrote:
We're going to need this later on.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

The latter is impossible to mock on platforms that use the gnulib implementation, such as FreeBSD, while the former doesn't suffer from this limitation. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/storage/storage_backend_fs.c | 2 +- src/util/virfile.c | 2 +- src/util/virpci.c | 2 +- tests/virstoragetest.c | 10 +++++----- tests/virtestmock.c | 6 +++--- 5 files changed, 11 insertions(+), 11 deletions(-) diff --git a/src/storage/storage_backend_fs.c b/src/storage/storage_backend_fs.c index 9b0fcf92c5..09b452db4e 100644 --- a/src/storage/storage_backend_fs.c +++ b/src/storage/storage_backend_fs.c @@ -823,7 +823,7 @@ virStorageFileBackendFileGetUniqueIdentifier(virStorageSourcePtr src) virStorageFileBackendFsPrivPtr priv = src->drv->priv; if (!priv->canonpath) { - if (!(priv->canonpath = canonicalize_file_name(src->path))) { + if (!(priv->canonpath = virFileCanonicalizePath(src->path))) { virReportSystemError(errno, _("can't canonicalize path '%s'"), src->path); return NULL; diff --git a/src/util/virfile.c b/src/util/virfile.c index 2a16b01ae5..41dda410cb 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -1597,7 +1597,7 @@ virFileResolveLinkHelper(const char *linkpath, return VIR_STRDUP_QUIET(*resultpath, linkpath) < 0 ? -1 : 0; } - *resultpath = canonicalize_file_name(linkpath); + *resultpath = virFileCanonicalizePath(linkpath); return *resultpath == NULL ? -1 : 0; } diff --git a/src/util/virpci.c b/src/util/virpci.c index 23932558e9..8d02366664 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -2622,7 +2622,7 @@ virPCIGetDeviceAddressFromSysfsLink(const char *device_link) return NULL; } - device_path = canonicalize_file_name(device_link); + device_path = virFileCanonicalizePath(device_link); if (device_path == NULL) { virReportSystemError(errno, _("Failed to resolve device link '%s'"), diff --git a/tests/virstoragetest.c b/tests/virstoragetest.c index b032d8b93f..0e11602e84 100644 --- a/tests/virstoragetest.c +++ b/tests/virstoragetest.c @@ -171,7 +171,7 @@ testPrepImages(void) fprintf(stderr, "unable to create directory %s\n", datadir "/dir"); goto cleanup; } - if (!(canondir = canonicalize_file_name(absdir))) { + if (!(canondir = virFileCanonicalizePath(absdir))) { virReportOOMError(); goto cleanup; } @@ -186,7 +186,7 @@ testPrepImages(void) fprintf(stderr, "unable to create raw file\n"); goto cleanup; } - if (!(canonraw = canonicalize_file_name(absraw))) { + if (!(canonraw = virFileCanonicalizePath(absraw))) { virReportOOMError(); goto cleanup; } @@ -206,7 +206,7 @@ testPrepImages(void) "-F", "raw", "-b", "raw", "qcow2", NULL); if (virCommandRun(cmd, NULL) < 0) goto skip; - if (!(canonqcow2 = canonicalize_file_name(absqcow2))) { + if (!(canonqcow2 = virFileCanonicalizePath(absqcow2))) { virReportOOMError(); goto cleanup; } @@ -220,7 +220,7 @@ testPrepImages(void) virCommandAddArg(cmd, "wrap"); if (virCommandRun(cmd, NULL) < 0) goto skip; - if (!(canonwrap = canonicalize_file_name(abswrap))) { + if (!(canonwrap = virFileCanonicalizePath(abswrap))) { virReportOOMError(); goto cleanup; } @@ -233,7 +233,7 @@ testPrepImages(void) virCommandAddArg(cmd, "qed"); if (virCommandRun(cmd, NULL) < 0) goto skip; - if (!(canonqed = canonicalize_file_name(absqed))) { + if (!(canonqed = virFileCanonicalizePath(absqed))) { virReportOOMError(); goto cleanup; } diff --git a/tests/virtestmock.c b/tests/virtestmock.c index 688945d805..4e679f35c0 100644 --- a/tests/virtestmock.c +++ b/tests/virtestmock.c @@ -133,12 +133,12 @@ checkPath(const char *path) virAsprintfQuiet(&relPath, "./%s", path) < 0) goto error; - /* Le sigh. Both canonicalize_file_name() and realpath() + /* Le sigh. Both anonicalize_file_name() and realpath() * expect @path to exist otherwise they return an error. So * if we are called over an non-existent file, this could * return an error. In that case do our best and hope we will * catch possible error. */ - if ((fullPath = canonicalize_file_name(relPath ? relPath : path))) { + if ((fullPath = virFileCanonicalizePath(relPath ? relPath : path))) { path = fullPath; } else { /* Yeah, our worst nightmares just became true. Path does @@ -148,7 +148,7 @@ checkPath(const char *path) virFileRemoveLastComponent(crippledPath); - if ((fullPath = canonicalize_file_name(crippledPath))) + if ((fullPath = virFileCanonicalizePath(crippledPath))) path = fullPath; } -- 2.14.3

On Thu, May 03, 2018 at 12:54:17PM +0200, Andrea Bolognani wrote:
The latter is impossible to mock on platforms that use the gnulib implementation, such as FreeBSD, while the former doesn't suffer from this limitation.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- src/storage/storage_backend_fs.c | 2 +- src/util/virfile.c | 2 +- src/util/virpci.c | 2 +- tests/virstoragetest.c | 10 +++++----- tests/virtestmock.c | 6 +++--- 5 files changed, 11 insertions(+), 11 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

We're using virFileCanonicalizePath() everywhere now, so mocking this function has become entirely pointless. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 21 --------------------- 1 file changed, 21 deletions(-) diff --git a/tests/virpcimock.c b/tests/virpcimock.c index 3d3a4b4a19..79352175de 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -39,7 +39,6 @@ static int (*real_lstat)(const char *path, struct stat *sb); static int (*real___lxstat)(int ver, const char *path, struct stat *sb); static int (*real_stat)(const char *path, struct stat *sb); static int (*real___xstat)(int ver, const char *path, struct stat *sb); -static char *(*real_canonicalize_file_name)(const char *path); static int (*real_open)(const char *path, int flags, ...); static int (*real_close)(int fd); static DIR * (*real_opendir)(const char *name); @@ -811,7 +810,6 @@ init_syms(void) VIR_MOCK_REAL_INIT(access); VIR_MOCK_REAL_INIT_ALT(lstat, __lxstat); VIR_MOCK_REAL_INIT_ALT(stat, __xstat); - VIR_MOCK_REAL_INIT(canonicalize_file_name); VIR_MOCK_REAL_INIT(open); VIR_MOCK_REAL_INIT(close); VIR_MOCK_REAL_INIT(opendir); @@ -969,25 +967,6 @@ stat(const char *path, struct stat *sb) return ret; } -char * -canonicalize_file_name(const char *path) -{ - char *ret; - - init_syms(); - - if (STRPREFIX(path, SYSFS_PCI_PREFIX)) { - char *newpath; - if (getrealpath(&newpath, path) < 0) - return NULL; - ret = real_canonicalize_file_name(newpath); - VIR_FREE(newpath); - } else { - ret = real_canonicalize_file_name(path); - } - return ret; -} - int open(const char *path, int flags, ...) { -- 2.14.3

On Thu, May 03, 2018 at 12:54:18PM +0200, Andrea Bolognani wrote:
We're using virFileCanonicalizePath() everywhere now, so mocking this function has become entirely pointless.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 21 --------------------- 1 file changed, 21 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

We want to make sure our wrapper is used instead in order to keep the test suite working. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- cfg.mk | 10 ++++++++++ src/util/virfile.c | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/cfg.mk b/cfg.mk index c3d18279ce..6bebd0ad9f 100644 --- a/cfg.mk +++ b/cfg.mk @@ -462,6 +462,13 @@ sc_prohibit_ctype_h: halt='use c-ctype.h instead of ctype.h' \ $(_sc_search_regexp) +# We have our own wrapper for mocking purposes +sc_prohibit_canonicalize_file_name: + @prohibit='\<canonicalize_file_name\(' \ + exclude='exempt from syntax-check' \ + halt='use virFileCanonicalizePath() instead of canonicalize_file_name()' \ + $(_sc_search_regexp) + # Insist on correct types for [pug]id. sc_correct_id_types: @prohibit='\<(int|long) *[pug]id\>' \ @@ -1208,6 +1215,9 @@ exclude_file_name_regexp--sc_prohibit_nonreentrant = \ exclude_file_name_regexp--sc_prohibit_select = \ ^cfg\.mk$$ +exclude_file_name_regexp--sc_prohibit_canonicalize_file_name = \ + ^cfg\.mk$$ + exclude_file_name_regexp--sc_prohibit_raw_allocation = \ ^(docs/hacking\.html\.in|src/util/viralloc\.[ch]|examples/.*|tests/(securityselinuxhelper|(vircgroup|nss)mock|commandhelper)\.c|tools/wireshark/src/packet-libvirt\.c)$$ diff --git a/src/util/virfile.c b/src/util/virfile.c index 41dda410cb..40f106d01d 100644 --- a/src/util/virfile.c +++ b/src/util/virfile.c @@ -3309,7 +3309,7 @@ virFileSanitizePath(const char *path) char * virFileCanonicalizePath(const char *path) { - return canonicalize_file_name(path); + return canonicalize_file_name(path); /* exempt from syntax-check */ } /** -- 2.14.3

On Thu, May 03, 2018 at 12:54:19PM +0200, Andrea Bolognani wrote:
We want to make sure our wrapper is used instead in order to keep the test suite working.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- cfg.mk | 10 ++++++++++ src/util/virfile.c | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Clang complains about it: error: second argument to 'va_arg' is of promotable type 'mode_t' (aka 'unsigned short'); this va_arg has undefined behavior because arguments will be promoted to 'int' [-Werror,-Wvarargs] mode = va_arg(ap, mode_t); ^~~~~~ Work around the issue by passing int to va_arg() and casting its return value to mode_t afterwards. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/nssmock.c | 2 +- tests/vircgroupmock.c | 2 +- tests/virpcimock.c | 2 +- tests/virtestmock.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/nssmock.c b/tests/nssmock.c index 7929c3061c..fc8e6b22f0 100644 --- a/tests/nssmock.c +++ b/tests/nssmock.c @@ -87,7 +87,7 @@ open(const char *path, int flags, ...) va_list ap; mode_t mode; va_start(ap, flags); - mode = va_arg(ap, int); + mode = (mode_t) va_arg(ap, int); va_end(ap); ret = real_open(newpath ? newpath : path, flags, mode); } else { diff --git a/tests/vircgroupmock.c b/tests/vircgroupmock.c index 56337c2f3e..51861be38e 100644 --- a/tests/vircgroupmock.c +++ b/tests/vircgroupmock.c @@ -731,7 +731,7 @@ int open(const char *path, int flags, ...) va_list ap; mode_t mode; va_start(ap, flags); - mode = va_arg(ap, mode_t); + mode = (mode_t) va_arg(ap, int); va_end(ap); ret = real_open(newpath ? newpath : path, flags, mode); } else { diff --git a/tests/virpcimock.c b/tests/virpcimock.c index 79352175de..7e4ac191d0 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -983,7 +983,7 @@ open(const char *path, int flags, ...) va_list ap; mode_t mode; va_start(ap, flags); - mode = va_arg(ap, mode_t); + mode = (mode_t) va_arg(ap, int); va_end(ap); ret = real_open(newpath ? newpath : path, flags, mode); } else { diff --git a/tests/virtestmock.c b/tests/virtestmock.c index 4e679f35c0..c3b0aa2d9c 100644 --- a/tests/virtestmock.c +++ b/tests/virtestmock.c @@ -181,7 +181,7 @@ int open(const char *path, int flags, ...) va_list ap; mode_t mode; va_start(ap, flags); - mode = va_arg(ap, mode_t); + mode = (mode_t) va_arg(ap, int); va_end(ap); ret = real_open(path, flags, mode); } else { -- 2.14.3

On Thu, May 03, 2018 at 12:54:20PM +0200, Andrea Bolognani wrote:
Clang complains about it:
error: second argument to 'va_arg' is of promotable type 'mode_t' (aka 'unsigned short'); this va_arg has undefined behavior because arguments will be promoted to 'int' [-Werror,-Wvarargs]
mode = va_arg(ap, mode_t); ^~~~~~
Work around the issue by passing int to va_arg() and casting its return value to mode_t afterwards.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/nssmock.c | 2 +- tests/vircgroupmock.c | 2 +- tests/virpcimock.c | 2 +- tests/virtestmock.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

There are only a couple issues preventing it from working on other platform such as FreeBSD. Let's fix them. With the mocking in place, qemumemlocktest and qemuxml2xmltest can finally succeed on FreeBSD. This commit is best viewed with 'git show -w'. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 57 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 24 deletions(-) diff --git a/tests/virpcimock.c b/tests/virpcimock.c index 7e4ac191d0..09cb26567c 100644 --- a/tests/virpcimock.c +++ b/tests/virpcimock.c @@ -20,19 +20,18 @@ #include <config.h> -#ifdef __linux__ -# include "virmock.h" -# include <stdio.h> -# include <stdlib.h> -# include <unistd.h> -# include <fcntl.h> -# include <sys/stat.h> -# include <stdarg.h> -# include <dirent.h> -# include "viralloc.h" -# include "virstring.h" -# include "virfile.h" -# include "dirname.h" +#include "virmock.h" +#include <stdio.h> +#include <stdlib.h> +#include <unistd.h> +#include <fcntl.h> +#include <sys/stat.h> +#include <stdarg.h> +#include <dirent.h> +#include "viralloc.h" +#include "virstring.h" +#include "virfile.h" +#include "dirname.h" static int (*real_access)(const char *path, int mode); static int (*real_lstat)(const char *path, struct stat *sb); @@ -51,20 +50,20 @@ static char *(*real_virFileCanonicalizePath)(const char *path); char *fakerootdir; char *fakesysfspcidir; -# define SYSFS_PCI_PREFIX "/sys/bus/pci/" +#define SYSFS_PCI_PREFIX "/sys/bus/pci/" -# define STDERR(...) \ +#define STDERR(...) \ fprintf(stderr, "%s %zu: ", __FUNCTION__, (size_t) __LINE__); \ fprintf(stderr, __VA_ARGS__); \ fprintf(stderr, "\n"); \ -# define ABORT(...) \ +#define ABORT(...) \ do { \ STDERR(__VA_ARGS__); \ abort(); \ } while (0) -# define ABORT_OOM() \ +#define ABORT_OOM() \ ABORT("Out of memory") /* * The plan: @@ -341,6 +340,7 @@ pci_device_new_from_stub(const struct pciDevice *data) char *configSrc; char tmp[256]; struct stat sb; + bool configSrcExists = false; if (VIR_STRDUP_QUIET(id, data->id) < 0) ABORT_OOM(); @@ -368,10 +368,18 @@ pci_device_new_from_stub(const struct pciDevice *data) if (virFileMakePath(devpath) < 0) ABORT("Unable to create: %s", devpath); + if (real_stat && real_stat(configSrc, &sb) == 0) + configSrcExists = true; + +#ifdef HAVE___XSTAT + if (!configSrcExists && + real___xstat && real___xstat(_STAT_VER, configSrc, &sb) == 0) + configSrcExists = true; +#endif + /* If there is a config file for the device within virpcitestdata dir, * symlink it. Otherwise create a dummy config file. */ - if ((real_stat && real_stat(configSrc, &sb) == 0) || - (real___xstat && real___xstat(_STAT_VER, configSrc, &sb) == 0)) { + if (configSrcExists) { /* On success, copy @configSrc into the destination (a copy, * rather than a symlink, is required since we write into the * file, and parallel VPATH builds must not stomp on the @@ -834,7 +842,7 @@ init_env(void) make_file(fakesysfspcidir, "drivers_probe", NULL, -1); -# define MAKE_PCI_DRIVER(name, ...) \ +#define MAKE_PCI_DRIVER(name, ...) \ pci_driver_new(name, 0, __VA_ARGS__, -1, -1) MAKE_PCI_DRIVER("iwlwifi", 0x8086, 0x0044); @@ -842,7 +850,7 @@ init_env(void) MAKE_PCI_DRIVER("pci-stub", -1, -1); pci_driver_new("vfio-pci", PCI_ACTION_BIND, -1, -1); -# define MAKE_PCI_DEVICE(Id, Vendor, Device, ...) \ +#define MAKE_PCI_DEVICE(Id, Vendor, Device, ...) \ do { \ struct pciDevice dev = {.id = (char *)Id, .vendor = Vendor, \ .device = Device, __VA_ARGS__}; \ @@ -891,6 +899,7 @@ access(const char *path, int mode) return ret; } +#ifdef HAVE___LXSTAT int __lxstat(int ver, const char *path, struct stat *sb) { @@ -909,6 +918,7 @@ __lxstat(int ver, const char *path, struct stat *sb) } return ret; } +#endif /* HAVE___LXSTAT */ int lstat(const char *path, struct stat *sb) @@ -929,6 +939,7 @@ lstat(const char *path, struct stat *sb) return ret; } +#ifdef HAVE___XSTAT int __xstat(int ver, const char *path, struct stat *sb) { @@ -947,6 +958,7 @@ __xstat(int ver, const char *path, struct stat *sb) } return ret; } +#endif /* HAVE___XSTAT */ int stat(const char *path, struct stat *sb) @@ -1049,6 +1061,3 @@ virFileCanonicalizePath(const char *path) return ret; } -#else -/* Nothing to override on non-__linux__ platforms */ -#endif -- 2.14.3

On Thu, 2018-05-03 at 12:54 +0200, Andrea Bolognani wrote:
There are only a couple issues preventing it from working on other platform such as FreeBSD. Let's fix them.
With the mocking in place, qemumemlocktest and qemuxml2xmltest can finally succeed on FreeBSD.
Too bad it also happens to break compilation on macOS duplicate symbol _rpl_lstat in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(lstat.o) duplicate symbol _rpl_open in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(open.o) duplicate symbol _rpl_stat in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(stat.o) ld: 3 duplicate symbols for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [virpcimock.la] Error 1 and on MinGW ../../tests/virpcimock.c: In function 'make_symlink': ../../tests/virpcimock.c:203:9: error: implicit declaration of function 'symlink'; did you mean 'unlink'? [-Werror=implicit-function-declaration] if (symlink(target, filepath) < 0) ^~~~~~~ unlink ../../tests/virpcimock.c:203:9: error: nested extern declaration of 'symlink' [-Werror=nested-externs] ../../tests/virpcimock.c: In function 'pci_read_file': ../../tests/virpcimock.c:228:5: error: implicit declaration of function 'bzero' [-Werror=implicit-function-declaration] bzero(buf, buf_size); ^~~~~ ../../tests/virpcimock.c:228:5: error: incompatible implicit declaration of built-in function 'bzero' [-Werror] In file included from ../gnulib/lib/fcntl.h:58:0, from ../../tests/virpcimock.c:27: ../../tests/virpcimock.c: At top level: ../../tests/virpcimock.c:964:1: error: redefinition of 'rpl_stat' stat(const char *path, struct stat *sb) ^ ../../tests/virpcimock.c:924:1: note: previous definition of 'rpl_stat' was here lstat(const char *path, struct stat *sb) ^ cc1: all warnings being treated as errors gmake[2]: *** [Makefile:5505: virpcimock.lo] Error 1 I'll look into solving the latter, but I have no intention of spending time on the former because I don't have access to macOS machines and debugging this kind of failure through Travis would be just too painful and time consuming. In the meantime, and for the purpose of reviewing the series, consider the attached patch, which merely adds FreeBSD to the list of platforms virpcimock should be compiled on, rather than getting rid of such a list altogether, squashed in. -- Andrea Bolognani / Red Hat / Virtualization

On Thu, May 03, 2018 at 02:28:48PM +0200, Andrea Bolognani wrote:
On Thu, 2018-05-03 at 12:54 +0200, Andrea Bolognani wrote:
There are only a couple issues preventing it from working on other platform such as FreeBSD. Let's fix them.
With the mocking in place, qemumemlocktest and qemuxml2xmltest can finally succeed on FreeBSD.
Too bad it also happens to break compilation on macOS
duplicate symbol _rpl_lstat in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(lstat.o) duplicate symbol _rpl_open in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(open.o) duplicate symbol _rpl_stat in: .libs/virpcimock.o ../gnulib/lib/.libs/libgnu.a(stat.o) ld: 3 duplicate symbols for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [virpcimock.la] Error 1
and on MinGW
../../tests/virpcimock.c: In function 'make_symlink': ../../tests/virpcimock.c:203:9: error: implicit declaration of function 'symlink'; did you mean 'unlink'? [-Werror=implicit-function-declaration] if (symlink(target, filepath) < 0) ^~~~~~~ unlink ../../tests/virpcimock.c:203:9: error: nested extern declaration of 'symlink' [-Werror=nested-externs] ../../tests/virpcimock.c: In function 'pci_read_file': ../../tests/virpcimock.c:228:5: error: implicit declaration of function 'bzero' [-Werror=implicit-function-declaration] bzero(buf, buf_size); ^~~~~ ../../tests/virpcimock.c:228:5: error: incompatible implicit declaration of built-in function 'bzero' [-Werror] In file included from ../gnulib/lib/fcntl.h:58:0, from ../../tests/virpcimock.c:27: ../../tests/virpcimock.c: At top level: ../../tests/virpcimock.c:964:1: error: redefinition of 'rpl_stat' stat(const char *path, struct stat *sb) ^ ../../tests/virpcimock.c:924:1: note: previous definition of 'rpl_stat' was here lstat(const char *path, struct stat *sb) ^ cc1: all warnings being treated as errors gmake[2]: *** [Makefile:5505: virpcimock.lo] Error 1
I'll look into solving the latter, but I have no intention of spending time on the former because I don't have access to macOS machines and debugging this kind of failure through Travis would be just too painful and time consuming.
That's ok - FreeBSD testing is already giving us better confidence in our macOS support as there's some overlapping heritage. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Thu, May 03, 2018 at 12:54:21PM +0200, Andrea Bolognani wrote:
There are only a couple issues preventing it from working on other platform such as FreeBSD. Let's fix them.
With the mocking in place, qemumemlocktest and qemuxml2xmltest can finally succeed on FreeBSD.
This commit is best viewed with 'git show -w'.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/virpcimock.c | 57 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 24 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (on basis that your followup fix is squashed in) Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

While the current amount of mocking works just fine on most of our target platforms, it somehow causes issues when using Clang on FreeBSD. Work around the issue by mocking a couple more functions. It's not pretty, but it makes qemuxml2argvtest pass on FreeBSD at long last. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/qemuxml2argvmock.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/tests/qemuxml2argvmock.c b/tests/qemuxml2argvmock.c index adab5c9111..6d78063f00 100644 --- a/tests/qemuxml2argvmock.c +++ b/tests/qemuxml2argvmock.c @@ -55,25 +55,45 @@ time_t time(time_t *t) return ret; } +bool +virNumaIsAvailable(void) +{ + return true; +} + int virNumaGetMaxNode(void) { - const int maxnodesNum = 7; - - return maxnodesNum; + return 7; } -#if WITH_NUMACTL && HAVE_NUMA_BITMASK_ISBITSET -/* - * In case libvirt is compiled with full NUMA support, we need to mock - * this function in order to fake what numa nodes are available. - */ +/* We shouldn't need to mock virNumaNodeIsAvailable() and *definitely* not + * virNumaNodesetIsAvailable(), but it seems to be the only way to get + * mocking to work with Clang on FreeBSD, so keep these duplicates around + * until we figure out a cleaner solution */ bool virNumaNodeIsAvailable(int node) { return node >= 0 && node <= virNumaGetMaxNode(); } -#endif /* WITH_NUMACTL && HAVE_NUMA_BITMASK_ISBITSET */ + +bool +virNumaNodesetIsAvailable(virBitmapPtr nodeset) +{ + ssize_t bit = -1; + + if (!nodeset) + return true; + + while ((bit = virBitmapNextSetBit(nodeset, bit)) >= 0) { + if (virNumaNodeIsAvailable(bit)) + continue; + + return false; + } + + return true; +} char * virTPMCreateCancelPath(const char *devpath) -- 2.14.3

On Thu, May 03, 2018 at 12:54:22PM +0200, Andrea Bolognani wrote:
While the current amount of mocking works just fine on most of our target platforms, it somehow causes issues when using Clang on FreeBSD.
Work around the issue by mocking a couple more functions. It's not pretty, but it makes qemuxml2argvtest pass on FreeBSD at long last.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/qemuxml2argvmock.c | 38 +++++++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 9 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Now that mocking NUMA information works on FreeBSD, there are no longer any test cases that need to be restricted to Linux only. Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/qemuxml2argvtest.c | 46 +++++++++++++++------------------------------- 1 file changed, 15 insertions(+), 31 deletions(-) diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 5b3bd4a996..5d8aedb2c6 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -767,24 +767,6 @@ mymain(void) FLAG_EXPECT_PARSE_ERROR | FLAG_EXPECT_FAILURE, \ parseFlags, GIC_NONE, __VA_ARGS__) -# define DO_TEST_LINUX(name, ...) \ - DO_TEST_LINUX_FULL(name, NULL, -1, 0, 0, GIC_NONE, __VA_ARGS__) - -# ifdef __linux__ - /* This is a macro that invokes test only on Linux. It's - * meant to be called in those cases where qemuxml2argvmock - * cooperation is expected (e.g. we need a fixed time, - * predictable NUMA topology and so on). On non-Linux - * platforms the macro just consume its argument. */ -# define DO_TEST_LINUX_FULL(name, ...) \ - DO_TEST_FULL(name, __VA_ARGS__) -# else /* __linux__ */ -# define DO_TEST_LINUX_FULL(name, ...) \ - do { \ - const char *tmp ATTRIBUTE_UNUSED = name; \ - } while (0) -# endif /* __linux__ */ - # define NONE QEMU_CAPS_LAST /* Unset or set all envvars here that are copied in qemudBuildCommandLine @@ -935,16 +917,16 @@ mymain(void) QEMU_CAPS_HDA_DUPLEX, QEMU_CAPS_USB_REDIR, QEMU_CAPS_DEVICE_PC_DIMM, QEMU_CAPS_OBJECT_MEMORY_FILE); - DO_TEST_LINUX("hugepages-pages", - QEMU_CAPS_OBJECT_MEMORY_RAM, - QEMU_CAPS_OBJECT_MEMORY_FILE); + DO_TEST("hugepages-pages", + QEMU_CAPS_OBJECT_MEMORY_RAM, + QEMU_CAPS_OBJECT_MEMORY_FILE); DO_TEST("hugepages-pages2", QEMU_CAPS_OBJECT_MEMORY_RAM, QEMU_CAPS_OBJECT_MEMORY_FILE); DO_TEST("hugepages-pages3", QEMU_CAPS_OBJECT_MEMORY_RAM, QEMU_CAPS_OBJECT_MEMORY_FILE); - DO_TEST_LINUX("hugepages-shared", - QEMU_CAPS_OBJECT_MEMORY_RAM, - QEMU_CAPS_OBJECT_MEMORY_FILE); + DO_TEST("hugepages-shared", + QEMU_CAPS_OBJECT_MEMORY_RAM, + QEMU_CAPS_OBJECT_MEMORY_FILE); DO_TEST_PARSE_ERROR("hugepages-memaccess-invalid", NONE); DO_TEST_FAILURE("hugepages-pages4", QEMU_CAPS_OBJECT_MEMORY_RAM, QEMU_CAPS_OBJECT_MEMORY_FILE); @@ -1562,9 +1544,9 @@ mymain(void) DO_TEST_FULL("restore-v2-fd", "fd:7", 7, 0, 0, GIC_NONE, NONE); DO_TEST_FULL("migrate", "tcp:10.0.0.1:5000", -1, 0, 0, GIC_NONE, NONE); - DO_TEST_LINUX_FULL("migrate-numa-unaligned", "stdio", 7, 0, 0, GIC_NONE, - QEMU_CAPS_NUMA, - QEMU_CAPS_OBJECT_MEMORY_RAM); + DO_TEST_FULL("migrate-numa-unaligned", "stdio", 7, 0, 0, GIC_NONE, + QEMU_CAPS_NUMA, + QEMU_CAPS_OBJECT_MEMORY_RAM); DO_TEST("qemu-ns", NONE); DO_TEST("qemu-ns-no-env", NONE); @@ -1662,12 +1644,14 @@ mymain(void) DO_TEST("numatune-memory", NONE); DO_TEST_PARSE_ERROR("numatune-memory-invalid-nodeset", NONE); - DO_TEST_LINUX("numatune-memnode", QEMU_CAPS_NUMA, - QEMU_CAPS_OBJECT_MEMORY_RAM); + DO_TEST("numatune-memnode", + QEMU_CAPS_NUMA, + QEMU_CAPS_OBJECT_MEMORY_RAM); DO_TEST_FAILURE("numatune-memnode", NONE); - DO_TEST_LINUX("numatune-memnode-no-memory", QEMU_CAPS_NUMA, - QEMU_CAPS_OBJECT_MEMORY_RAM); + DO_TEST("numatune-memnode-no-memory", + QEMU_CAPS_NUMA, + QEMU_CAPS_OBJECT_MEMORY_RAM); DO_TEST_FAILURE("numatune-memnode-no-memory", NONE); DO_TEST("numatune-distances", QEMU_CAPS_NUMA, QEMU_CAPS_NUMA_DIST); -- 2.14.3

On Thu, May 03, 2018 at 12:54:23PM +0200, Andrea Bolognani wrote:
Now that mocking NUMA information works on FreeBSD, there are no longer any test cases that need to be restricted to Linux only.
Signed-off-by: Andrea Bolognani <abologna@redhat.com> --- tests/qemuxml2argvtest.c | 46 +++++++++++++++------------------------------- 1 file changed, 15 insertions(+), 31 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Andrea Bolognani
-
Daniel P. Berrangé