[libvirt] [PATCH] Do not inline virNumaNodeIsAvailable
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1). --- This only leaves the mysterious check-protocol failure. And too large stack frame size when building the tests with -O0. src/internal.h | 10 ++++++++++ src/util/virnuma.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/src/internal.h b/src/internal.h index 4d473af..84aa330 100644 --- a/src/internal.h +++ b/src/internal.h @@ -139,6 +139,16 @@ # endif /** + * ATTRIBUTE_NOINLINE: + * + * Macro to indicate a function that cannot be inlined + * (e.g. a function that is mocked in the test suite) + */ +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# endif + +/** * ATTRIBUTE_SENTINEL: * * Macro to check for NULL-terminated varargs lists diff --git a/src/util/virnuma.h b/src/util/virnuma.h index 1f3c0ad..4ddcc5a 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -37,7 +37,8 @@ virBitmapPtr virNumaGetHostNodeset(void); bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); bool virNumaIsAvailable(void); int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodeIsAvailable(int node) + ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, int *ndistances); -- 2.0.5
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
--- This only leaves the mysterious check-protocol failure. And too large stack frame size when building the tests with -O0.
src/internal.h | 10 ++++++++++ src/util/virnuma.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/internal.h b/src/internal.h index 4d473af..84aa330 100644 --- a/src/internal.h +++ b/src/internal.h @@ -139,6 +139,16 @@ # endif
/** + * ATTRIBUTE_NOINLINE: + * + * Macro to indicate a function that cannot be inlined + * (e.g. a function that is mocked in the test suite) + */ +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# endif + +/** * ATTRIBUTE_SENTINEL: * * Macro to check for NULL-terminated varargs lists diff --git a/src/util/virnuma.h b/src/util/virnuma.h index 1f3c0ad..4ddcc5a 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -37,7 +37,8 @@ virBitmapPtr virNumaGetHostNodeset(void); bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); bool virNumaIsAvailable(void); int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodeIsAvailable(int node) + ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, int *ndistances);
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc. Excerpt from gcc online docs [1]: When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced. Excerpt from gcc online docs [1]: By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0. However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined. Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference. [1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
--- This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
And too large stack frame size when building the tests with -O0.
I'm using CFLAGS='-O0 -ggdb -Wno-frame-larger-than=' for this as a parameter for autogen.sh O:-)
src/internal.h | 10 ++++++++++ src/util/virnuma.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/internal.h b/src/internal.h index 4d473af..84aa330 100644 --- a/src/internal.h +++ b/src/internal.h @@ -139,6 +139,16 @@ # endif
/** + * ATTRIBUTE_NOINLINE: + * + * Macro to indicate a function that cannot be inlined + * (e.g. a function that is mocked in the test suite) + */ +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# endif + +/** * ATTRIBUTE_SENTINEL: * * Macro to check for NULL-terminated varargs lists diff --git a/src/util/virnuma.h b/src/util/virnuma.h index 1f3c0ad..4ddcc5a 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -37,7 +37,8 @@ virBitmapPtr virNumaGetHostNodeset(void); bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); bool virNumaIsAvailable(void); int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodeIsAvailable(int node) + ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, int *ndistances);
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Mon, Mar 09, 2015 at 11:01:31AM +0100, Martin Kletzander wrote:
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc.
Excerpt from gcc online docs [1]:
When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced.
Excerpt from gcc online docs [1]:
By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0.
However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined.
But we haven't added the 'inline' keyword to this function at all - it is just a normal function marked for export in the .so file, so I'm puzzelled why it is getting inlined.
Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
--- This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
And too large stack frame size when building the tests with -O0.
I'm using CFLAGS='-O0 -ggdb -Wno-frame-larger-than=' for this as a parameter for autogen.sh O:-)
src/internal.h | 10 ++++++++++ src/util/virnuma.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/internal.h b/src/internal.h index 4d473af..84aa330 100644 --- a/src/internal.h +++ b/src/internal.h @@ -139,6 +139,16 @@ # endif
/** + * ATTRIBUTE_NOINLINE: + * + * Macro to indicate a function that cannot be inlined + * (e.g. a function that is mocked in the test suite) + */ +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# endif + +/** * ATTRIBUTE_SENTINEL: * * Macro to check for NULL-terminated varargs lists diff --git a/src/util/virnuma.h b/src/util/virnuma.h index 1f3c0ad..4ddcc5a 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -37,7 +37,8 @@ virBitmapPtr virNumaGetHostNodeset(void); bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); bool virNumaIsAvailable(void); int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodeIsAvailable(int node) + ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, int *ndistances);
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Mar 09, 2015 at 10:09:17AM +0000, Daniel P. Berrange wrote:
On Mon, Mar 09, 2015 at 11:01:31AM +0100, Martin Kletzander wrote:
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc.
Excerpt from gcc online docs [1]:
When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced.
Excerpt from gcc online docs [1]:
By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0.
However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined.
But we haven't added the 'inline' keyword to this function at all - it is just a normal function marked for export in the .so file, so I'm puzzelled why it is getting inlined.
Exactly, that's what I'm trying to find out as well.
Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference.
After updating clang and llvm from 3.5 to 3.6, I still don't get this error. And I have only 4 (fake) nodes available, so it _is_ rewriting that function.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
--- This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
And too large stack frame size when building the tests with -O0.
I'm using CFLAGS='-O0 -ggdb -Wno-frame-larger-than=' for this as a parameter for autogen.sh O:-)
src/internal.h | 10 ++++++++++ src/util/virnuma.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/src/internal.h b/src/internal.h index 4d473af..84aa330 100644 --- a/src/internal.h +++ b/src/internal.h @@ -139,6 +139,16 @@ # endif
/** + * ATTRIBUTE_NOINLINE: + * + * Macro to indicate a function that cannot be inlined + * (e.g. a function that is mocked in the test suite) + */ +# ifndef ATTRIBUTE_NOINLINE +# define ATTRIBUTE_NOINLINE __attribute__((__noinline__)) +# endif + +/** * ATTRIBUTE_SENTINEL: * * Macro to check for NULL-terminated varargs lists diff --git a/src/util/virnuma.h b/src/util/virnuma.h index 1f3c0ad..4ddcc5a 100644 --- a/src/util/virnuma.h +++ b/src/util/virnuma.h @@ -37,7 +37,8 @@ virBitmapPtr virNumaGetHostNodeset(void); bool virNumaNodesetIsAvailable(virBitmapPtr nodeset); bool virNumaIsAvailable(void); int virNumaGetMaxNode(void); -bool virNumaNodeIsAvailable(int node); +bool virNumaNodeIsAvailable(int node) + ATTRIBUTE_NOINLINE; int virNumaGetDistances(int node, int **distances, int *ndistances);
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Mar 09, 2015 at 02:03:16PM +0100, Martin Kletzander wrote:
On Mon, Mar 09, 2015 at 10:09:17AM +0000, Daniel P. Berrange wrote:
On Mon, Mar 09, 2015 at 11:01:31AM +0100, Martin Kletzander wrote:
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc.
Excerpt from gcc online docs [1]:
When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced.
Excerpt from gcc online docs [1]:
By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0.
However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined.
But we haven't added the 'inline' keyword to this function at all - it is just a normal function marked for export in the .so file, so I'm puzzelled why it is getting inlined.
Exactly, that's what I'm trying to find out as well.
Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference.
After updating clang and llvm from 3.5 to 3.6, I still don't get this error. And I have only 4 (fake) nodes available, so it _is_ rewriting that function.
I'm getting the error with 3.5.1, as I said in the commit message. These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
--- This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently. It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again). Jan
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
On Mon, Mar 09, 2015 at 02:03:16PM +0100, Martin Kletzander wrote:
On Mon, Mar 09, 2015 at 10:09:17AM +0000, Daniel P. Berrange wrote:
On Mon, Mar 09, 2015 at 11:01:31AM +0100, Martin Kletzander wrote:
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote:
Explicitly request that virNumaNodeIsAvailable not be inlined. This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc.
Excerpt from gcc online docs [1]:
When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced.
Excerpt from gcc online docs [1]:
By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0.
However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined.
But we haven't added the 'inline' keyword to this function at all - it is just a normal function marked for export in the .so file, so I'm puzzelled why it is getting inlined.
Exactly, that's what I'm trying to find out as well.
Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference.
After updating clang and llvm from 3.5 to 3.6, I still don't get this error. And I have only 4 (fake) nodes available, so it _is_ rewriting that function.
I'm getting the error with 3.5.1, as I said in the commit message.
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
--- This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently. It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this CC=clang ./autogen.sh && make && make check Or is there more to it than that ? Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Mar 10, 2015 at 12:24:03PM +0000, Daniel P. Berrange wrote:
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
On Mon, Mar 09, 2015 at 02:03:16PM +0100, Martin Kletzander wrote:
On Mon, Mar 09, 2015 at 10:09:17AM +0000, Daniel P. Berrange wrote:
On Mon, Mar 09, 2015 at 11:01:31AM +0100, Martin Kletzander wrote:
On Thu, Mar 05, 2015 at 11:09:41AM +0000, Daniel P. Berrange wrote:
On Thu, Mar 05, 2015 at 12:05:52PM +0100, Ján Tomko wrote: >Explicitly request that virNumaNodeIsAvailable not be inlined. >This fixes the test suite when building with clang (3.5.1).
Huh, so clang will inline functions, even if they are exported in the .so library ? Is there some clang compiler flag we can use to stop that ? I'd only expect it to inline stuff which was declared static, or whose impl body was in the header file
If I understand it correctly, that means that clang is not "compatible" with gcc.
Excerpt from gcc online docs [1]:
When a function is both inline and static, if all calls to the function are integrated into the caller, and the function's address is never used, then the function's own assembler code is never referenced.
Excerpt from gcc online docs [1]:
By default, Clang builds C code in GNU C11 mode, so it uses standard C99 semantics for the inline keyword. These semantics are different from those in GNU C89 mode, which is the default mode in versions of GCC prior to 5.0.
However further reading of the second documentation and c89 semantics it doesn't say anything about the fact that such function should be inlined.
But we haven't added the 'inline' keyword to this function at all - it is just a normal function marked for export in the .so file, so I'm puzzelled why it is getting inlined.
Exactly, that's what I'm trying to find out as well.
Anyway, is this clang 3.6 specific? I don't have this problem when compiling with 3.5. Nor does this show with gcc -std=gnu11. I'm getting 3.6 to check whether that's the difference.
After updating clang and llvm from 3.5 to 3.6, I still don't get this error. And I have only 4 (fake) nodes available, so it _is_ rewriting that function.
I'm getting the error with 3.5.1, as I said in the commit message.
I missed that, sorry.
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
I tried changing that, it fails. I tried adjusting the tests to more nodes, it fails. After adjusting the mock function again, it works. So it gets mocked all the time, but I know where the difference is, probably. Try building with -O0, it will probably disable the inlining. However, if that's the case, I believe it's still clang's fault since they don't document inlining functions without the "inline" keyword just because you optimize.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
>--- >This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently.
Oh, it does fail all the time for me.
It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this
CC=clang ./autogen.sh && make && make check
Or is there more to it than that ?
I have a script that does "export CC=clang" and then ./autogen.sh and make (everything).
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Mar 10, 2015 at 01:45:38PM +0100, Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 12:24:03PM +0000, Daniel P. Berrange wrote:
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
I tried changing that, it fails. I tried adjusting the tests to more nodes, it fails. After adjusting the mock function again, it works. So it gets mocked all the time, but I know where the difference is, probably. Try building with -O0, it will probably disable the inlining. However, if that's the case, I believe it's still clang's fault since they don't document inlining functions without the "inline" keyword just because you optimize.
-O0 does disable the inlining, but introduces the stack size warning.
[1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html [2] http://clang.llvm.org/compatibility.html
>>--- >>This only leaves the mysterious check-protocol failure.
That's not that mysterious, it's just that we check the order and clang sorts enums before structs, but gcc doesn't. Also clang adds "public:" to structs, so it probably treats it as a C++ or C# structs or something?
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently.
Oh, it does fail all the time for me.
Did I mistakenly build with gcc to fix it?
It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this
CC=clang ./autogen.sh && make && make check
Or is there more to it than that ?
./autogen.sh --system --without-driver-modules CC="clang" && make -j5 check Now it failed two times in a row, and once without parallel make. I have no idea what I did back then that fixed it. Jan
On Tue, Mar 10, 2015 at 02:27:44PM +0100, Ján Tomko wrote:
On Tue, Mar 10, 2015 at 01:45:38PM +0100, Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 12:24:03PM +0000, Daniel P. Berrange wrote:
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
I tried changing that, it fails. I tried adjusting the tests to more nodes, it fails. After adjusting the mock function again, it works. So it gets mocked all the time, but I know where the difference is, probably. Try building with -O0, it will probably disable the inlining. However, if that's the case, I believe it's still clang's fault since they don't document inlining functions without the "inline" keyword just because you optimize.
-O0 does disable the inlining, but introduces the stack size warning.
> [1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html > [2] http://clang.llvm.org/compatibility.html > > >>--- > >>This only leaves the mysterious check-protocol failure. > > That's not that mysterious, it's just that we check the order and > clang sorts enums before structs, but gcc doesn't. Also clang adds > "public:" to structs, so it probably treats it as a C++ or C# structs > or something? >
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently.
Oh, it does fail all the time for me.
Did I mistakenly build with gcc to fix it?
It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this
CC=clang ./autogen.sh && make && make check
Or is there more to it than that ?
./autogen.sh --system --without-driver-modules CC="clang" && make -j5 check
Now it failed two times in a row, and once without parallel make. I have no idea what I did back then that fixed it.
I managed to reproduce your issue! I used -O2, so that's really it. I guess that even though clang leaves that code without inlining, llvm's internal optimizer just screws that up for us. Anyway, we might fix future problems by disabling inlining for binaries used for testing, but that would be an overkill when building. On the other hand, adding "noinline" to any (possibly) mocked function seems like the other extreme.
Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 02:27:44PM +0100, Ján Tomko wrote:
On Tue, Mar 10, 2015 at 01:45:38PM +0100, Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 12:24:03PM +0000, Daniel P. Berrange wrote:
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
I tried changing that, it fails. I tried adjusting the tests to more nodes, it fails. After adjusting the mock function again, it works. So it gets mocked all the time, but I know where the difference is, probably. Try building with -O0, it will probably disable the inlining. However, if that's the case, I believe it's still clang's fault since they don't document inlining functions without the "inline" keyword just because you optimize.
-O0 does disable the inlining, but introduces the stack size warning.
>> [1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html >> [2] http://clang.llvm.org/compatibility.html >> >> >>--- >> >>This only leaves the mysterious check-protocol failure. >> >> That's not that mysterious, it's just that we check the order and >> clang sorts enums before structs, but gcc doesn't. Also clang adds >> "public:" to structs, so it probably treats it as a C++ or C# structs >> or something? >>
By the way if I compile with clang with -std=gnu11 or -std=gnu99, the "public:" stuff is gone :)
It is mysterious, because it doesn't fail consistently.
Oh, it does fail all the time for me.
Did I mistakenly build with gcc to fix it?
It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this
CC=clang ./autogen.sh && make && make check
Or is there more to it than that ?
./autogen.sh --system --without-driver-modules CC="clang" && make -j5 check
Now it failed two times in a row, and once without parallel make. I have no idea what I did back then that fixed it.
I managed to reproduce your issue! I used -O2, so that's really it. I guess that even though clang leaves that code without inlining, llvm's internal optimizer just screws that up for us.
Anyway, we might fix future problems by disabling inlining for binaries used for testing, but that would be an overkill when building. On the other hand, adding "noinline" to any (possibly) mocked function seems like the other extreme.
Funny that I didn't notice this thread initially and spotted it while googling how to debug clang's function inlining while debugging the very same problem described here. I have: $ clang -v FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Target: x86_64-unknown-freebsd10.1 Thread model: posix Selected GCC installation: $ qemuxml2argvtest fails are 100% reproducible, there are no fails other than that though. The patch proposed by Ján fixes the problem for me. Also, it goes away when I build with CFLAGS=-fno-inline.
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Roman Bogorodskiy
On Sat, Apr 25, 2015 at 07:30:21PM +0300, Roman Bogorodskiy wrote:
Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 02:27:44PM +0100, Ján Tomko wrote:
On Tue, Mar 10, 2015 at 01:45:38PM +0100, Martin Kletzander wrote:
On Tue, Mar 10, 2015 at 12:24:03PM +0000, Daniel P. Berrange wrote:
On Tue, Mar 10, 2015 at 01:05:22PM +0100, Ján Tomko wrote:
These are the failing qemuxml2argvtest cases: 60) QEMU XML-2-ARGV hugepages-pages ... libvirt: error : internal error: NUMA node 1 is unavailable 63) QEMU XML-2-ARGV hugepages-shared ... libvirt: error : internal error: NUMA node 1 is unavailable 324) QEMU XML-2-ARGV numatune-memnode ... libvirt: error : internal error: NUMA node 1 is unavailable 326) QEMU XML-2-ARGV numatune-memnode-no-memory ... libvirt: error : internal error: NUMA node 3 is unavailable 329) QEMU XML-2-ARGV numatune-auto-prefer ... libvirt: error : internal error: NUMA node 1 is unavailable
So with 4 fake nodes, the tests could still pass even if the function is not mocked. Try changing the nodeset in #326 to 4 if it fails.
I tried changing that, it fails. I tried adjusting the tests to more nodes, it fails. After adjusting the mock function again, it works. So it gets mocked all the time, but I know where the difference is, probably. Try building with -O0, it will probably disable the inlining. However, if that's the case, I believe it's still clang's fault since they don't document inlining functions without the "inline" keyword just because you optimize.
-O0 does disable the inlining, but introduces the stack size warning.
> > >> [1] https://gcc.gnu.org/onlinedocs/gcc/Inline.html > >> [2] http://clang.llvm.org/compatibility.html > >> > >> >>--- > >> >>This only leaves the mysterious check-protocol failure. > >> > >> That's not that mysterious, it's just that we check the order and > >> clang sorts enums before structs, but gcc doesn't. Also clang adds > >> "public:" to structs, so it probably treats it as a C++ or C# structs > >> or something? > >> > > By the way if I compile with clang with -std=gnu11 or -std=gnu99, the > "public:" stuff is gone :) >
It is mysterious, because it doesn't fail consistently.
Oh, it does fail all the time for me.
Did I mistakenly build with gcc to fix it?
It was working for me after I tried it again after 'git clean -fxd', today it failed again (though I don't remember if I ran autogen again).
How exactly are you running the build with clang ? Are you just doing this
CC=clang ./autogen.sh && make && make check
Or is there more to it than that ?
./autogen.sh --system --without-driver-modules CC="clang" && make -j5 check
Now it failed two times in a row, and once without parallel make. I have no idea what I did back then that fixed it.
I managed to reproduce your issue! I used -O2, so that's really it. I guess that even though clang leaves that code without inlining, llvm's internal optimizer just screws that up for us.
Anyway, we might fix future problems by disabling inlining for binaries used for testing, but that would be an overkill when building. On the other hand, adding "noinline" to any (possibly) mocked function seems like the other extreme.
Funny that I didn't notice this thread initially and spotted it while googling how to debug clang's function inlining while debugging the very same problem described here.
I have:
$ clang -v FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Target: x86_64-unknown-freebsd10.1 Thread model: posix Selected GCC installation: $
qemuxml2argvtest fails are 100% reproducible, there are no fails other than that though.
The patch proposed by Ján fixes the problem for me. Also, it goes away when I build with CFLAGS=-fno-inline.
Have you (or anyone else in this discussion) found out what else we might do to fix this? If not, then we might probably try moving somewhere with this. How about using -fno-inline for clang build?
participants (4)
-
Daniel P. Berrange -
Ján Tomko -
Martin Kletzander -
Roman Bogorodskiy