[libvirt] Problems with pdwtags on Ubuntu 10.04
The "help avoid accidental remote_protocol.x changes" commit 180d4b2b added a make check rule that tried using pdwtags from the dwarves package to protect against accidental remote_protocol.x changes. I installed dwarves package on Ubuntu 10.04 and make check fails for me now. The temporary file remote_protocol-structs-t is empty for me. It seems that pdwtags doesn't output the expected format for the embedded perl script. pdwtags output doesn't contain /* DD */ comments between the structs. A snippet from the pdwtags output looks like this: struct remote_nonnull_network { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */ /* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ }; typedef struct remote_nonnull_network remote_nonnull_network; struct remote_nonnull_nwfilter { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */ /* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ }; typedef struct remote_nonnull_nwfilter remote_nonnull_nwfilter; If i run pdwtags with the --verbose flag then the output looks like this: /* <44e> remote/remote_protocol.h:71 */ typedef struct remote_nonnull_domain remote_nonnull_domain; /* <459> remote/remote_protocol.h:73 */ struct remote_nonnull_network { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */ /* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ }; /* <482> remote/remote_protocol.h:77 */ typedef struct remote_nonnull_network remote_nonnull_network; /* <48d> remote/remote_protocol.h:79 */ struct remote_nonnull_nwfilter { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */ /* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ }; /* <4b6> remote/remote_protocol.h:83 */ typedef struct remote_nonnull_nwfilter remote_nonnull_nwfilter; Now there are "markers" that can be used for splitting, but they still differ from the expected format. The attached patch fixes this issue for me on Ubuntu, but I doubt that this can be the general solution. Matthias
On 05/18/2010 11:57 AM, Matthias Bolte wrote:
The "help avoid accidental remote_protocol.x changes" commit 180d4b2b added a make check rule that tried using pdwtags from the dwarves package to protect against accidental remote_protocol.x changes.
I installed dwarves package on Ubuntu 10.04 and make check fails for me now.
What is your 'pdwtags --version'? On Fedora 13, it's v1.8pre1. Very likely, it's just a matter of slightly varying output styles between the two versions that needs some accommodation.
struct remote_nonnull_network { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */
/* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ };
typedef struct remote_nonnull_network remote_nonnull_network;
On F-13, the same chunk looks like: /* 80 */ struct remote_nonnull_network { remote_nonnull_string name; /* 0 4 */ remote_uuid uuid; /* 4 16 */ /* size: 20, cachelines: 1, members: 2 */ /* last cacheline: 20 bytes */ }; /* size: 20 */ /* 81 */ typedef struct remote_nonnull_network remote_nonnull_network; /* size: 20 */
If i run pdwtags with the --verbose flag then the output looks like this:
/* <44e> remote/remote_protocol.h:71 */ typedef struct remote_nonnull_domain remote_nonnull_domain;
/* <459> remote/remote_protocol.h:73 */ struct remote_nonnull_network { remote_nonnull_string name; /* 0 8 */ remote_uuid uuid; /* 8 16 */
/* size: 24, cachelines: 1 */ /* last cacheline: 24 bytes */ };
And here: /* 80 */ /* <0> (null):0 */ struct remote_nonnull_network { remote_nonnull_string name; /* 0 4 */ remote_uuid uuid; /* 4 16 */ /* size: 20, cachelines: 1, members: 2 */ /* last cacheline: 20 bytes */ }; /* size: 20 */ /* 81 */ /* <0> (null):0 */ typedef struct remote_nonnull_network remote_nonnull_network; /* size: 20 */
Now there are "markers" that can be used for splitting, but they still differ from the expected format.
The attached patch fixes this issue for me on Ubuntu, but I doubt that this can be the general solution.
Jim, any ideas? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
2010/5/18 Eric Blake <eblake@redhat.com>:
On 05/18/2010 11:57 AM, Matthias Bolte wrote:
The "help avoid accidental remote_protocol.x changes" commit 180d4b2b added a make check rule that tried using pdwtags from the dwarves package to protect against accidental remote_protocol.x changes.
I installed dwarves package on Ubuntu 10.04 and make check fails for me now.
What is your 'pdwtags --version'? On Fedora 13, it's v1.8pre1. Very likely, it's just a matter of slightly varying output styles between the two versions that needs some accommodation.
$ pdwtags --version pdwtags: --version: No such file or directory The dwarves package has version 1.3. Matthias
Matthias Bolte wrote:
The "help avoid accidental remote_protocol.x changes" commit 180d4b2b added a make check rule that tried using pdwtags from the dwarves package to protect against accidental remote_protocol.x changes.
I installed dwarves package on Ubuntu 10.04 and make check fails for me now.
The temporary file remote_protocol-structs-t is empty for me. It seems that pdwtags doesn't output the expected format for the embedded perl script. pdwtags output doesn't contain /* DD */ comments between the structs. A snippet from the pdwtags output looks like this:
Thanks for the report. At first I thought it might be worthwhile to adjust the splitting code to accommodate 1.3 with --verbose: -e 'foreach my $$p (split m!\n\n/\* (?:\d+|<\S+> \S+) \*/\n!) {'\ That works with pdwtags --verbose when it's 1.3, but with 1.8.x, as Eric noted, we get yet another variant: /* 93 */ /* <0> (null):0 */ which is not matched by the above. I could match only the lines with the "<hex-digit> ...:\d+" comments, but prefer to use a tighter regexp (albeit more involved) so that I can continue to require a blank line (the "\n\n") just before the separator. Matthias, Would you please verify that this solves the problem when using your older pdwtags program?
From a8d8ff6ba4791972483093215291eef5fa87cf5d Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering@redhat.com> Date: Wed, 19 May 2010 15:36:27 +0200 Subject: [PATCH] tests: the remote_protocol check also accommodates older pdwtags
This test was failing on systems using pdwtags from dwarves-1.3. Reported by Matthias Bolte. Two-pronged fix: - use --verbose to work also with dwarves-1.3; adapt regular expressions to handle now-varying separators - require a minimum number of post-split clauses, in order to skip upon any future format change. Currently there are 318; if there are 300 or fewer, give a warning similar to when pdwtags is missing. * src/Makefile.am (remote_protocol-structs): Use pdwtags' --verbose option to make 1.3 emit member sizes and offsets. Consistently output WARNING messages to stderr. --- src/Makefile.am | 40 +++++++++++++++++++++++++++++++--------- 1 files changed, 31 insertions(+), 9 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 889de8e..7ddf6aa 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -167,22 +167,44 @@ EXTRA_DIST += remote/remote_protocol.x remote/rpcgen_fix.pl # * remove comments and preceding TAB throughout # * remove empty lines throughout # * remove white space at end of buffer + +# With pdwtags 1.8, --verbose output includes separators like these: +# /* 93 */ +# /* <0> (null):0 */ +# whereas with pdwtags 1.3, they look like this: +# /* <2d2> /usr/include/libio.h:180 */ +# The concatenation of the following regexps matches both cases. +r1 = (?:/\* \d+ \*/\n)? +r2 = /\* <[[:xdigit:]]+> \S+:\d+ \*/ + .PHONY: remote_protocol-structs remote_protocol-structs: $(AM_V_GEN)if (pdwtags --help) > /dev/null 2>&1; then \ - pdwtags libvirt_driver_remote_la-remote_protocol.$(OBJEXT) \ + pdwtags --verbose libvirt_driver_remote_la-remote_protocol.$(OBJEXT) \ | perl -0777 -n \ - -e 'foreach my $$p (split m!\n\n/\* \d+ \*/\n!)' \ - -e ' { if ($$p =~ /^struct remote_/) {' \ - -e ' $$p =~ s!\t*/\*.*?\*/!!sg;' \ - -e ' $$p =~ s!\s+\n!\n!sg;' \ - -e ' $$p =~ s!\s+$$!!;' \ - -e ' print "$$p\n" } }' \ + -e 'foreach my $$p (split m!\n\n$(r1)$(r2)\n!) {' \ + -e ' if ($$p =~ /^struct remote_/) {' \ + -e ' $$p =~ s!\t*/\*.*?\*/!!sg;' \ + -e ' $$p =~ s!\s+\n!\n!sg;' \ + -e ' $$p =~ s!\s+$$!!;' \ + -e ' print "$$p\n";' \ + -e ' $$n++;' \ + -e ' }' \ + -e '}' \ + -e 'END {' \ + -e ' if ($$n < 300) {' \ + -e ' warn "WARNING: your pdwtags program is too old\n";' \ + -e ' warn "WARNING: skipping the $@ test\n";' \ + -e ' warn "WARNING: install dwarves-1.8 or newer\n";' \ + -e ' exit 8;' \ + -e ' }' \ + -e '}' \ > $@-t; \ + case $$? in 8) exit 0;; 0) ;; *) exit 1;; esac; \ diff -u $@-t $(srcdir)/$@; st=$$?; rm -f $@-t; exit $$st; \ else \ - echo 'WARNING: you lack pdwtags; skipping the $@ test'; \ - echo 'WARNING: install the dwarves package to get pdwtags'; \ + echo 'WARNING: you lack pdwtags; skipping the $@ test' >&2; \ + echo 'WARNING: install the dwarves package to get pdwtags' >&2; \ fi EXTRA_DIST += remote_protocol-structs check-local: remote_protocol-structs -- 1.7.1.259.g3aef8
2010/5/19 Jim Meyering <jim@meyering.net>:
Matthias Bolte wrote:
The "help avoid accidental remote_protocol.x changes" commit 180d4b2b added a make check rule that tried using pdwtags from the dwarves package to protect against accidental remote_protocol.x changes.
I installed dwarves package on Ubuntu 10.04 and make check fails for me now.
The temporary file remote_protocol-structs-t is empty for me. It seems that pdwtags doesn't output the expected format for the embedded perl script. pdwtags output doesn't contain /* DD */ comments between the structs. A snippet from the pdwtags output looks like this:
Thanks for the report. At first I thought it might be worthwhile to adjust the splitting code to accommodate 1.3 with --verbose:
-e 'foreach my $$p (split m!\n\n/\* (?:\d+|<\S+> \S+) \*/\n!) {'\
That works with pdwtags --verbose when it's 1.3, but with 1.8.x, as Eric noted, we get yet another variant:
/* 93 */ /* <0> (null):0 */
which is not matched by the above. I could match only the lines with the "<hex-digit> ...:\d+" comments, but prefer to use a tighter regexp (albeit more involved) so that I can continue to require a blank line (the "\n\n") just before the separator.
Matthias, Would you please verify that this solves the problem when using your older pdwtags program?
Yes, this patch solves the problem.
From a8d8ff6ba4791972483093215291eef5fa87cf5d Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering@redhat.com> Date: Wed, 19 May 2010 15:36:27 +0200 Subject: [PATCH] tests: the remote_protocol check also accommodates older pdwtags
This test was failing on systems using pdwtags from dwarves-1.3. Reported by Matthias Bolte. Two-pronged fix: - use --verbose to work also with dwarves-1.3; adapt regular expressions to handle now-varying separators - require a minimum number of post-split clauses, in order to skip upon any future format change. Currently there are 318; if there are 300 or fewer, give a warning similar to when pdwtags is missing. * src/Makefile.am (remote_protocol-structs): Use pdwtags' --verbose option to make 1.3 emit member sizes and offsets. Consistently output WARNING messages to stderr. --- src/Makefile.am | 40 +++++++++++++++++++++++++++++++--------- 1 files changed, 31 insertions(+), 9 deletions(-)
diff --git a/src/Makefile.am b/src/Makefile.am index 889de8e..7ddf6aa 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -167,22 +167,44 @@ EXTRA_DIST += remote/remote_protocol.x remote/rpcgen_fix.pl # * remove comments and preceding TAB throughout # * remove empty lines throughout # * remove white space at end of buffer + +# With pdwtags 1.8, --verbose output includes separators like these: +# /* 93 */ +# /* <0> (null):0 */ +# whereas with pdwtags 1.3, they look like this: +# /* <2d2> /usr/include/libio.h:180 */ +# The concatenation of the following regexps matches both cases. +r1 = (?:/\* \d+ \*/\n)? +r2 = /\* <[[:xdigit:]]+> \S+:\d+ \*/ + .PHONY: remote_protocol-structs remote_protocol-structs: $(AM_V_GEN)if (pdwtags --help) > /dev/null 2>&1; then \ - pdwtags libvirt_driver_remote_la-remote_protocol.$(OBJEXT) \ + pdwtags --verbose libvirt_driver_remote_la-remote_protocol.$(OBJEXT) \ | perl -0777 -n \ - -e 'foreach my $$p (split m!\n\n/\* \d+ \*/\n!)' \ - -e ' { if ($$p =~ /^struct remote_/) {' \ - -e ' $$p =~ s!\t*/\*.*?\*/!!sg;' \ - -e ' $$p =~ s!\s+\n!\n!sg;' \ - -e ' $$p =~ s!\s+$$!!;' \ - -e ' print "$$p\n" } }' \ + -e 'foreach my $$p (split m!\n\n$(r1)$(r2)\n!) {' \ + -e ' if ($$p =~ /^struct remote_/) {' \ + -e ' $$p =~ s!\t*/\*.*?\*/!!sg;' \ + -e ' $$p =~ s!\s+\n!\n!sg;' \ + -e ' $$p =~ s!\s+$$!!;' \ + -e ' print "$$p\n";' \ + -e ' $$n++;' \ + -e ' }' \ + -e '}' \ + -e 'END {' \ + -e ' if ($$n < 300) {' \ + -e ' warn "WARNING: your pdwtags program is too old\n";' \ + -e ' warn "WARNING: skipping the $@ test\n";' \ + -e ' warn "WARNING: install dwarves-1.8 or newer\n";' \
Maybe the warning should suggest dwarves-1.3 as minimum version, because this patch makes it work with dwarves-1.3.
+ -e ' exit 8;' \ + -e ' }' \ + -e '}' \ > $@-t; \ + case $$? in 8) exit 0;; 0) ;; *) exit 1;; esac; \ diff -u $@-t $(srcdir)/$@; st=$$?; rm -f $@-t; exit $$st; \ else \ - echo 'WARNING: you lack pdwtags; skipping the $@ test'; \ - echo 'WARNING: install the dwarves package to get pdwtags'; \ + echo 'WARNING: you lack pdwtags; skipping the $@ test' >&2; \ + echo 'WARNING: install the dwarves package to get pdwtags' >&2; \ fi EXTRA_DIST += remote_protocol-structs check-local: remote_protocol-structs -- 1.7.1.259.g3aef8
ACK. Matthias
Matthias Bolte wrote: ...
Yes, this patch solves the problem.
...
+ -e ' warn "WARNING: your pdwtags program is too old\n";' \ + -e ' warn "WARNING: skipping the $@ test\n";' \ + -e ' warn "WARNING: install dwarves-1.8 or newer\n";' \
Maybe the warning should suggest dwarves-1.3 as minimum version, because this patch makes it work with dwarves-1.3.
Good point. That was from an earlier (abandoned) version that required a version of pdwtags that accepts --version. Adjusted and pushed.
participants (3)
-
Eric Blake -
Jim Meyering -
Matthias Bolte