5:21 a.m.
Less of
use re 'eval';
''=~('(?{'.(
('`')| '%').('['^'-').
('`'|'!').
('`'|',').'"\\$~='
.('['^'+') .('`'|
'/').('['^'+').'||'.
"'"."'".';'.('`'|'/'
).('['^'+').('`'|'%').
('`'|'.').('\\$%;').(
'['^'"').(',!-~,#,,').(
'['^'(').',(.).,\\'
.'$+,'.('`'|"'").('['^'(')
.',\\$~&&(\\$'
.'_='.('['^')').('`'|('%')).(
'['^'-').('`'|
'%').('['^')').('['^'(').(('`')|
'%').')'.("\`"|
'&').('`'|'/').('['^"\)").'\\(a)~='.(
'`'|"'").("\["^
')').('`'|'%').('['^'+').('\\$|--,(').
"'"."'".(')').(
'['^'#').('^'^('`'|'/')).(':'&'=').',<'.
('^'^('`'|'.')
).'>;\\(a);='.('`'|'-').('`'|'!').('['^'+')
.'~~'.('['^')'
).('`'|'%').('['^'-').('`'|'%').('['^')').
('['^'(').('`'|'%').','.('['^')').('`'|'%').('['^'-').('`'
|'%').('['^')').('['^'(').('`'|'%').'\\(a)~;'.('`'|'-').('`'|
'!').('['^'+').'\\{'.('['^'(').('['^'"').('['^'(').(('[')^
'/').('`'|'%').('`'|'-').'\\$^'.('`'^'/').'=~'.('{'^"\,").(
'`'|')').('`'|'.').'?'.('`'^'#').('`'^',').('{'^'(').(':').
"'".('`'|'#').('`'|',').('`'|'%').('`'|'!').('['^')')."'".
';(\\$-=\\$_%'.('^'^('`'|'-')).')||(--\\$|,'.('`'|'-'
).(
'`'|'!').('['^'+').'\\$_='.('['^')').('`'|'%').('['
^((
'-'))).('`'|'%').('['^')').('['^'(').('`'
|('%')). ','
.'\\@~,\\(a););'.('['^'+').('['^(')')).(
'`'|')' ).(
"\`"|
'.').('['^'/').'\\$\\"'.("\["^
('#')). '('
.'\\$=/'.('^'^('`'|'-')).'*'.
(('`')| '!'
).("\`"| '"').('['^ "\(").
'\\$|' .+
('*').( '^'^('`' |','))
.'-\\' .+
'$-),'. '\\$_,'. '\\$'
.'/'. (
('`')| ('&')).( '`'|
'/')
.('['^ ')').'\\' .'$'
.'-'
.'&'. (('^')^( '`'|
'/')
).'?' .'\\@;' .':'
.''.
'\\' .'@~;' .''.
('['
^'(' ).( '`'| ',')
.''. ((( '`' ))|
'%' ).( '`' |((
'%' ))) .+( '['
^(( '+' ))) .+
(( '!')). ((
(( '\\') ))
). '$%\\}'. ((
((( '\\' ))))) .+
'$' .'%..' .''. (((
'^') )^("\`"| '/' )).(
"\^"^( ('`')|
('/'))). '"})');
More
######
## #########
## ###### #####
####################
###############
######## #
####### #
######## #
####### ##
######## ###
####### ####
####### ###### #####
####### ############ #####
######## ################## ######
###################### #################
################# #############
########### #######
Daniel P. Berrangé (11):
cfg.mk: fix comment detection for python semicolon check
docs: rewrite hvsupport.html page generator in python
docs: rewrite ACL permissions checker in Python
docs: rewrite symfile sorting checker in Python
docs: rewrite symfile library checker in Python
docs: rewrite augest test generator in Python
docs: rewrite po file minimizer in Python
docs: rewrite duplicate header checker in Python
docs: rewrite whitespace checker in Python
docs: rewrite mock inline checker in Python
docs: rewrite header ifdef checker in Python
Makefile.am | 12 +-
build-aux/augeas-gentest.pl | 60 ----
build-aux/augeas-gentest.py | 72 ++++
build-aux/check-spacing.pl | 198 ----------
build-aux/check-spacing.py | 204 +++++++++++
build-aux/header-ifdef.pl | 182 ----------
build-aux/header-ifdef.py | 206 +++++++++++
build-aux/minimize-po.pl | 37 --
build-aux/minimize-po.py | 60 ++++
build-aux/mock-noinline.pl | 75 ----
build-aux/mock-noinline.py | 88 +++++
build-aux/prohibit-duplicate-header.pl | 26 --
build-aux/prohibit-duplicate-header.py | 54 +++
cfg.mk | 10 +-
docs/Makefile.am | 6 +-
docs/hvsupport.pl | 458 -----------------------
docs/hvsupport.py | 479 +++++++++++++++++++++++++
po/Makefile.am | 2 +-
src/Makefile.am | 16 +-
src/check-aclperms.pl | 73 ----
src/check-aclperms.py | 77 ++++
src/check-symfile.pl | 70 ----
src/check-symfile.py | 80 +++++
src/check-symsorting.pl | 106 ------
src/check-symsorting.py | 112 ++++++
25 files changed, 1455 insertions(+), 1308 deletions(-)
delete mode 100755 build-aux/augeas-gentest.pl
create mode 100755 build-aux/augeas-gentest.py
delete mode 100755 build-aux/check-spacing.pl
create mode 100755 build-aux/check-spacing.py
delete mode 100644 build-aux/header-ifdef.pl
create mode 100644 build-aux/header-ifdef.py
delete mode 100755 build-aux/minimize-po.pl
create mode 100755 build-aux/minimize-po.py
delete mode 100644 build-aux/mock-noinline.pl
create mode 100644 build-aux/mock-noinline.py
delete mode 100644 build-aux/prohibit-duplicate-header.pl
create mode 100644 build-aux/prohibit-duplicate-header.py
delete mode 100755 docs/hvsupport.pl
create mode 100755 docs/hvsupport.py
delete mode 100755 src/check-aclperms.pl
create mode 100755 src/check-aclperms.py
delete mode 100755 src/check-symfile.pl
create mode 100755 src/check-symfile.py
delete mode 100755 src/check-symsorting.pl
create mode 100755 src/check-symsorting.py
--
2.21.0
5:21 a.m.
New subject: [libvirt] [PATCH 01/11] cfg.mk: fix comment detection for python semicolon check
The pattern
^[^#].*\;$$
Was attempting to detect any trailing ';' in python code
which was not in a comment. This does not allow for the
comment '#' character to be indented with whitespace.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
cfg.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/cfg.mk b/cfg.mk
index 1f29729949..d10dfa16a1 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -815,7 +815,7 @@ sc_require_enum_last_marker:
# In Python files we don't want to end lines with a semicolon like in C
sc_prohibit_semicolon_at_eol_in_python:
- @prohibit='^[^#].*\;$$' \
+ @prohibit='^[^#]*\;$$' \
in_vc_files='\.py$$' \
halt='python does not require to end lines with a semicolon' \
$(_sc_search_regexp)
--
2.21.0
8:57 a.m.
New subject: [libvirt] [PATCH 01/11] cfg.mk: fix comment detection for python semicolon check
On 9/5/19 5:21 AM, Daniel P. Berrangé wrote:
The pattern
^[^#].*\;$$
Was attempting to detect any trailing ';' in python code
which was not in a comment. This does not allow for the
comment '#' character to be indented with whitespace.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
cfg.mk | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/cfg.mk b/cfg.mk
index 1f29729949..d10dfa16a1 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -815,7 +815,7 @@ sc_require_enum_last_marker:
# In Python files we don't want to end lines with a semicolon like in C
sc_prohibit_semicolon_at_eol_in_python:
- @prohibit='^[^#].*\;$$' \
+ @prohibit='^[^#]*\;$$' \
But this new pattern does not prohibit:
ch = '#';
I think you want:
'^[ \t]*[^#].*\;$$'
to flag all lines that have any amount of leading whitespace, where the
first non-whitespace is not #, and which end in ;.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
9:05 a.m.
New subject: [libvirt] [PATCH 01/11] cfg.mk: fix comment detection for python semicolon check
On Thu, Sep 05, 2019 at 08:57:43AM -0500, Eric Blake wrote:
On 9/5/19 5:21 AM, Daniel P. Berrangé wrote:
> The pattern
>
> ^[^#].*\;$$
>
> Was attempting to detect any trailing ';' in python code
> which was not in a comment. This does not allow for the
> comment '#' character to be indented with whitespace.
>
> Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
> ---
> cfg.mk | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/cfg.mk b/cfg.mk
> index 1f29729949..d10dfa16a1 100644
> --- a/cfg.mk
> +++ b/cfg.mk
> @@ -815,7 +815,7 @@ sc_require_enum_last_marker:
>
> # In Python files we don't want to end lines with a semicolon like in C
> sc_prohibit_semicolon_at_eol_in_python:
> - @prohibit='^[^#].*\;$$' \
> + @prohibit='^[^#]*\;$$' \
But this new pattern does not prohibit:
ch = '#';
I think you want:
'^[ \t]*[^#].*\;$$'
to flag all lines that have any amount of leading whitespace, where the
first non-whitespace is not #, and which end in ;.
Unfortunately this doesn't work when face with a line
# foo();
The '[ \t]*' matches 0 times. '[^#]' matches on the first leading
whitespace character. '.*' then matches ' # foo()'. So we get a
bogus warning.
This does make me realize how to fix it though - we need to force
the '[ \t]*' to always be matched against leading space with:
'^[ \t]*[^# \t].*\;$$'
This will still give a bogus warning though if the comment is
following after a valid statement eg
some call # foo();
Luckily we don't have any examples of this pattern, so I could
leave it to some other sucker to fix when they need that.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
5:21 a.m.
New subject: [libvirt] [PATCH 02/11] docs: rewrite hvsupport.html page generator in python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the hvsupport.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
The new impl generates byte-for-byte identical output to the
old impl.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
docs/Makefile.am | 6 +-
docs/hvsupport.pl | 458 --------------------------------------------
docs/hvsupport.py | 479 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 482 insertions(+), 461 deletions(-)
delete mode 100755 docs/hvsupport.pl
create mode 100755 docs/hvsupport.py
diff --git a/docs/Makefile.am b/docs/Makefile.am
index 1cdb584b0b..f7aba5499d 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -194,7 +194,7 @@ EXTRA_DIST= \
$(internals_html_in) $(internals_html) $(fonts) \
$(kbase_html_in) $(kbase_html) \
aclperms.htmlinc \
- hvsupport.pl \
+ hvsupport.py \
$(schema_DATA)
acl_generated = aclperms.htmlinc
@@ -230,11 +230,11 @@ web: $(dot_html) $(internals_html) $(kbase_html) \
hvsupport.html: $(srcdir)/hvsupport.html.in
-$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \
+$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.py $(api_DATA) \
$(top_srcdir)/src/libvirt_public.syms \
$(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \
$(top_srcdir)/src/driver.h
- $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir)/src > $@ \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/hvsupport.py $(top_srcdir)/src > $@ \
|| { rm $@ && exit 1; }
news.html.in: \
diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl
deleted file mode 100755
index 494b8a27ec..0000000000
--- a/docs/hvsupport.pl
+++ /dev/null
@@ -1,458 +0,0 @@
-#!/usr/bin/env perl
-
-use strict;
-use warnings;
-
-use File::Find;
-
-die "syntax: $0 SRCDIR\n" unless int(@ARGV) == 1;
-
-my $srcdir = shift @ARGV;
-
-my $symslibvirt = "$srcdir/libvirt_public.syms";
-my $symsqemu = "$srcdir/libvirt_qemu.syms";
-my $symslxc = "$srcdir/libvirt_lxc.syms";
-my @drivertable = (
- "$srcdir/driver-hypervisor.h",
- "$srcdir/driver-interface.h",
- "$srcdir/driver-network.h",
- "$srcdir/driver-nodedev.h",
- "$srcdir/driver-nwfilter.h",
- "$srcdir/driver-secret.h",
- "$srcdir/driver-state.h",
- "$srcdir/driver-storage.h",
- "$srcdir/driver-stream.h",
- );
-
-my %groupheaders = (
- "virHypervisorDriver" => "Hypervisor APIs",
- "virNetworkDriver" => "Virtual Network APIs",
- "virInterfaceDriver" => "Host Interface APIs",
- "virNodeDeviceDriver" => "Host Device APIs",
- "virStorageDriver" => "Storage Pool APIs",
- "virSecretDriver" => "Secret APIs",
- "virNWFilterDriver" => "Network Filter APIs",
- );
-
-
-my @srcs;
-find({
- wanted => sub {
- if (m!$srcdir/.*/\w+_(driver|common|tmpl|monitor|hal|udev)\.c$!) {
- push @srcs, $_ if $_ !~ /vbox_driver\.c/;
- }
- }, no_chdir => 1}, $srcdir);
-
-# Map API functions to the header and documentation files they're in
-# so that we can generate proper hyperlinks to their documentation.
-#
-# The function names are grep'd from the XML output of apibuild.py.
-sub getAPIFilenames {
- my $filename = shift;
-
- my %files;
- my $line;
-
- open FILE, "<", $filename or die "cannot read $filename: $!";
-
- while (defined($line = <FILE>)) {
- if ($line =~ /function name='([^']+)' file='([^']+)'/) {
- $files{$1} = $2;
- }
- }
-
- close FILE;
-
- if (keys %files == 0) {
- die "No functions found in $filename. Has the apibuild.py output
changed?";
- }
- return \%files;
-}
-
-sub parseSymsFile {
- my $apisref = shift;
- my $prefix = shift;
- my $filename = shift;
- my $xmlfilename = shift;
-
- my $line;
- my $vers;
- my $prevvers;
-
- my $filenames = getAPIFilenames($xmlfilename);
-
- open FILE, "<$filename"
- or die "cannot read $filename: $!";
-
- while (defined($line = <FILE>)) {
- chomp $line;
- next if $line =~ /^\s*#/;
- next if $line =~ /^\s*$/;
- next if $line =~ /^\s*(global|local):/;
- if ($line =~ /^\s*${prefix}_(\d+\.\d+\.\d+)\s*{\s*$/) {
- if (defined $vers) {
- die "malformed syms file";
- }
- $vers = $1;
- } elsif ($line =~ /\s*}\s*;\s*$/) {
- if (defined $prevvers) {
- die "malformed syms file";
- }
- $prevvers = $vers;
- $vers = undef;
- } elsif ($line =~ /\s*}\s*${prefix}_(\d+\.\d+\.\d+)\s*;\s*$/) {
- if ($1 ne $prevvers) {
- die "malformed syms file $1 != $vers";
- }
- $prevvers = $vers;
- $vers = undef;
- } elsif ($line =~ /\s*(\w+)\s*;\s*$/) {
- $$apisref{$1} = {};
- $$apisref{$1}->{vers} = $vers;
- $$apisref{$1}->{file} = $$filenames{$1};
- } else {
- die "unexpected data $line\n";
- }
- }
-
- close FILE;
-}
-
-my %apis;
-# Get the list of all public APIs and their corresponding version
-parseSymsFile(\%apis, "LIBVIRT", $symslibvirt,
"$srcdir/../docs/libvirt-api.xml");
-
-# And the same for the QEMU specific APIs
-parseSymsFile(\%apis, "LIBVIRT_QEMU", $symsqemu,
"$srcdir/../docs/libvirt-qemu-api.xml");
-
-# And the same for the LXC specific APIs
-parseSymsFile(\%apis, "LIBVIRT_LXC", $symslxc,
"$srcdir/../docs/libvirt-lxc-api.xml");
-
-
-# Some special things which aren't public APIs,
-# but we want to report
-$apis{virConnectSupportsFeature}->{vers} = "0.3.2";
-$apis{virDomainMigratePrepare}->{vers} = "0.3.2";
-$apis{virDomainMigratePerform}->{vers} = "0.3.2";
-$apis{virDomainMigrateFinish}->{vers} = "0.3.2";
-$apis{virDomainMigratePrepare2}->{vers} = "0.5.0";
-$apis{virDomainMigrateFinish2}->{vers} = "0.5.0";
-$apis{virDomainMigratePrepareTunnel}->{vers} = "0.7.2";
-
-$apis{virDomainMigrateBegin3}->{vers} = "0.9.2";
-$apis{virDomainMigratePrepare3}->{vers} = "0.9.2";
-$apis{virDomainMigratePrepareTunnel3}->{vers} = "0.9.2";
-$apis{virDomainMigratePerform3}->{vers} = "0.9.2";
-$apis{virDomainMigrateFinish3}->{vers} = "0.9.2";
-$apis{virDomainMigrateConfirm3}->{vers} = "0.9.2";
-
-$apis{virDomainMigrateBegin3Params}->{vers} = "1.1.0";
-$apis{virDomainMigratePrepare3Params}->{vers} = "1.1.0";
-$apis{virDomainMigratePrepareTunnel3Params}->{vers} = "1.1.0";
-$apis{virDomainMigratePerform3Params}->{vers} = "1.1.0";
-$apis{virDomainMigrateFinish3Params}->{vers} = "1.1.0";
-$apis{virDomainMigrateConfirm3Params}->{vers} = "1.1.0";
-
-
-
-# Now we want to get the mapping between public APIs
-# and driver struct fields. This lets us later match
-# update the driver impls with the public APis.
-
-my $line;
-
-# Group name -> hash of APIs { fields -> api name }
-my %groups;
-my $ingrp;
-foreach my $drivertable (@drivertable) {
- open FILE, "<$drivertable"
- or die "cannot read $drivertable: $!";
-
- while (defined($line = <FILE>)) {
- if ($line =~ /struct _(vir\w*Driver)/) {
- my $grp = $1;
- if ($grp ne "virStateDriver" &&
- $grp ne "virStreamDriver") {
- $ingrp = $grp;
- $groups{$ingrp} = { apis => {}, drivers => {} };
- }
- } elsif ($ingrp) {
- if ($line =~ /^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$/) {
- my $field = $2;
- my $name = $1;
-
- my $api;
- if (exists $apis{"vir$name"}) {
- $api = "vir$name";
- } elsif ($name =~ /\w+(Open|Close|URIProbe)/) {
- next;
- } else {
- die "driver $name does not have a public API";
- }
- $groups{$ingrp}->{apis}->{$field} = $api;
- } elsif ($line =~ /};/) {
- $ingrp = undef;
- }
- }
- }
-
- close FILE;
-}
-
-
-# Finally, we read all the primary driver files and extract
-# the driver API tables from each one.
-
-foreach my $src (@srcs) {
- open FILE, "<$src" or
- die "cannot read $src: $!";
-
- my $groups_regex = join("|", keys %groups);
- $ingrp = undef;
- my $impl;
- while (defined($line = <FILE>)) {
- if (!$ingrp) {
- # skip non-matching lines early to save time
- next if not $line =~ /$groups_regex/;
-
- if ($line =~ /^\s*(?:static\s+)?($groups_regex)\s+(\w+)\s*=\s*{/ ||
- $line =~ /^\s*(?:static\s+)?($groups_regex)\s+NAME\(\w+\)\s*=\s*{/) {
- $ingrp = $1;
- $impl = $src;
-
- if ($impl =~ m,.*/node_device_(\w+)\.c,) {
- $impl = $1;
- } else {
- $impl =~ s,.*/(\w+?)_((\w+)_)?(\w+)\.c,$1,;
- }
-
- if ($groups{$ingrp}->{drivers}->{$impl}) {
- die "Group $ingrp already contains $impl";
- }
-
- $groups{$ingrp}->{drivers}->{$impl} = {};
- }
-
- } else {
- if ($line =~
m!\s*\.(\w+)\s*=\s*(\w+)\s*,?\s*(?:/\*\s*(\d+\.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$!)
{
- my $api = $1;
- my $meth = $2;
- my $vers = $3;
- my $deleted = $4;
-
- next if $api eq "no" || $api eq "name";
-
- if ($meth eq "NULL" && !defined $deleted) {
- die "Method impl for $api is NULL, but no deleted version is
provided";
- }
- if ($meth ne "NULL" && defined $deleted) {
- die "Method impl for $api is non-NULL, but deleted version is
provided";
- }
-
- die "Method $meth in $src is missing version" unless defined
$vers || $api eq "connectURIProbe";
-
- if (!exists($groups{$ingrp}->{apis}->{$api})) {
- next if $api =~ /\w(Open|Close|URIProbe)/;
-
- die "Found unexpected method $api in $ingrp\n";
- }
-
- $groups{$ingrp}->{drivers}->{$impl}->{$api} = {};
- $groups{$ingrp}->{drivers}->{$impl}->{$api}->{vers} = $vers;
- $groups{$ingrp}->{drivers}->{$impl}->{$api}->{deleted} =
$deleted;
- if ($api eq "domainMigratePrepare" ||
- $api eq "domainMigratePrepare2" ||
- $api eq "domainMigratePrepare3") {
- if
(!$groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"}) {
-
$groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"} = {};
-
$groups{$ingrp}->{drivers}->{$impl}->{"domainMigrate"}->{vers} =
$vers;
- }
- }
-
- } elsif ($line =~ /}/) {
- $ingrp = undef;
- }
- }
- }
-
- close FILE;
-}
-
-
-# The '.open' driver method is used for 3 public APIs, so we
-# have a bit of manual fixup todo with the per-driver versioning
-# and support matrix
-
-$groups{virHypervisorDriver}->{apis}->{"openAuth"} =
"virConnectOpenAuth";
-$groups{virHypervisorDriver}->{apis}->{"openReadOnly"} =
"virConnectOpenReadOnly";
-$groups{virHypervisorDriver}->{apis}->{"domainMigrate"} =
"virDomainMigrate";
-
-my $openAuthVers = (0 * 1000 * 1000) + (4 * 1000) + 0;
-
-foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) {
- my $openVersStr =
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}->{vers};
- my $openVers;
- if ($openVersStr =~ /(\d+)\.(\d+)\.(\d+)/) {
- $openVers = ($1 * 1000 * 1000) + ($2 * 1000) + $3;
- }
-
- # virConnectOpenReadOnly always matches virConnectOpen version
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenReadOnly"}
=
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"};
-
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}
= {};
-
- # virConnectOpenAuth is always 0.4.0 if the driver existed
- # before this time, otherwise it matches the version of
- # the driver's virConnectOpen entry
- if ($openVersStr eq "Y" ||
- $openVers >= $openAuthVers) {
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}->{vers}
= $openVersStr;
- } else {
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}->{vers}
= "0.4.0";
- }
-}
-
-
-# Another special case for the virDomainCreateLinux which was replaced
-# with virDomainCreateXML
-$groups{virHypervisorDriver}->{apis}->{"domainCreateLinux"} =
"virDomainCreateLinux";
-
-my $createAPIVers = (0 * 1000 * 1000) + (0 * 1000) + 3;
-
-foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) {
- my $createVersStr =
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateXML"}->{vers};
- next unless defined $createVersStr;
- my $createVers;
- if ($createVersStr =~ /(\d+)\.(\d+)\.(\d+)/) {
- $createVers = ($1 * 1000 * 1000) + ($2 * 1000) + $3;
- }
-
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"}
= {};
-
- # virCreateLinux is always 0.0.3 if the driver existed
- # before this time, otherwise it matches the version of
- # the driver's virCreateXML entry
- if ($createVersStr eq "Y" ||
- $createVers >= $createAPIVers) {
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"}->{vers}
= $createVersStr;
- } else {
-
$groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux"}->{vers}
= "0.0.3";
- }
-}
-
-
-# Finally we generate the HTML file with the tables
-
-print <<EOF;
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE html>
-<html xmlns="http://www.w3.org/1999/xhtml">
-<body class="hvsupport">
-<h1>libvirt API support matrix</h1>
-
-<ul id="toc"></ul>
-
-<p>
-This page documents which <a href="html/">libvirt calls</a> work
on
-which libvirt drivers / hypervisors, and which version the API appeared
-in. If a hypervisor driver later dropped support for the API, the version
-when it was removed is also mentioned (highlighted in
-<span class="removedhv">dark red</span>).
-</p>
-
-EOF
-
- foreach my $grp (sort { $a cmp $b } keys %groups) {
- print "<h2><a id=\"$grp\">", $groupheaders{$grp},
"</a></h2>\n";
- print <<EOF;
-<table class="top_table">
-<thead>
-<tr>
-<th>API</th>
-<th>Version</th>
-EOF
-
- foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) {
- print " <th>$drv</th>\n";
- }
-
- print <<EOF;
-</tr>
-</thead>
-<tbody>
-EOF
-
- my $row = 0;
- foreach my $field (sort {
- $groups{$grp}->{apis}->{$a}
- cmp
- $groups{$grp}->{apis}->{$b}
- } keys %{$groups{$grp}->{apis}}) {
- my $api = $groups{$grp}->{apis}->{$field};
- my $vers = $apis{$api}->{vers};
- my $htmlgrp = $apis{$api}->{file};
- print <<EOF;
-<tr>
-<td>
-EOF
-
- if (defined $htmlgrp) {
- print <<EOF;
-<a href=\"html/libvirt-$htmlgrp.html#$api\">$api</a>
-EOF
-
- } else {
- print $api;
- }
- print <<EOF;
-</td>
-<td>$vers</td>
-EOF
-
- foreach my $drv (sort {$a cmp $b } keys %{$groups{$grp}->{drivers}}) {
- print "<td>";
- if (exists $groups{$grp}->{drivers}->{$drv}->{$field}) {
- if ($groups{$grp}->{drivers}->{$drv}->{$field}->{vers}) {
- print
$groups{$grp}->{drivers}->{$drv}->{$field}->{vers};
- }
- if ($groups{$grp}->{drivers}->{$drv}->{$field}->{deleted}) {
- print " - <span class=\"removedhv\">",
$groups{$grp}->{drivers}->{$drv}->{$field}->{deleted},
"</span>";
- }
- }
- print "</td>\n";
- }
-
- print <<EOF;
-</tr>
-EOF
-
- $row++;
- if (($row % 15) == 0) {
- print <<EOF;
-<tr>
-<th>API</th>
-<th>Version</th>
-EOF
-
- foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) {
- print " <th>$drv</th>\n";
- }
-
- print <<EOF;
-</tr>
-EOF
- }
-
- }
-
- print <<EOF;
-</tbody>
-</table>
-EOF
-}
-
-print <<EOF;
-</body>
-</html>
-EOF
diff --git a/docs/hvsupport.py b/docs/hvsupport.py
new file mode 100755
index 0000000000..cfbecd82ae
--- /dev/null
+++ b/docs/hvsupport.py
@@ -0,0 +1,479 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2011-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import sys
+import os.path
+import re
+
+if len(sys.argv) != 2:
+ print("syntax: %s SRCDIR\n" % sys.argv[0], file=sys.stderr)
+
+srcdir = sys.argv[1]
+
+symslibvirt = os.path.join(srcdir, "libvirt_public.syms")
+symsqemu = os.path.join(srcdir, "libvirt_qemu.syms")
+symslxc = os.path.join(srcdir, "libvirt_lxc.syms")
+drivertablefiles = [
+ os.path.join(srcdir, "driver-hypervisor.h"),
+ os.path.join(srcdir, "driver-interface.h"),
+ os.path.join(srcdir, "driver-network.h"),
+ os.path.join(srcdir, "driver-nodedev.h"),
+ os.path.join(srcdir, "driver-nwfilter.h"),
+ os.path.join(srcdir, "driver-secret.h"),
+ os.path.join(srcdir, "driver-state.h"),
+ os.path.join(srcdir, "driver-storage.h"),
+ os.path.join(srcdir, "driver-stream.h"),
+]
+
+groupheaders = {
+ "virHypervisorDriver": "Hypervisor APIs",
+ "virNetworkDriver": "Virtual Network APIs",
+ "virInterfaceDriver": "Host Interface APIs",
+ "virNodeDeviceDriver": "Host Device APIs",
+ "virStorageDriver": "Storage Pool APIs",
+ "virSecretDriver": "Secret APIs",
+ "virNWFilterDriver": "Network Filter APIs",
+}
+
+
+srcs = []
+for root, dirs, files in os.walk(srcdir):
+ for file in files:
+ if ((file.endswith("driver.c") and
+ not file.endswith("vbox_driver.c")) or
+ file.endswith("common.c") or
+ file.endswith("tmpl.c") or
+ file.endswith("monitor.c") or
+ file.endswith("hal.c") or
+ file.endswith("udev.c")):
+ srcs.append(os.path.join(root, file))
+
+
+# Map API functions to the header and documentation files they're in
+# so that we can generate proper hyperlinks to their documentation.
+#
+# The function names are grep'd from the XML output of apibuild.py.
+def getAPIFilenames(filename):
+ files = {}
+
+ with open(filename) as fh:
+ prog = re.compile(r"\s*<function name='([^']+)'
file='([^']+)'.*")
+ for line in fh:
+ res = prog.match(line)
+ if res is not None:
+ files[res.group(1)] = res.group(2)
+
+ if len(files) == 0:
+ raise Exception("No functions found in %s. Has the apibuild.py output
changed?" % filename)
+
+ return files
+
+
+def parseSymsFile(apisref, prefix, filename, xmlfilename):
+ vers = None
+ prevvers = None
+
+ filenames = getAPIFilenames(xmlfilename)
+
+ with open(filename) as fh:
+ groupstartprog = re.compile(r"^\s*%s_(\d+\.\d+\.\d+)\s*{\s*$" %
prefix)
+ groupendprog1 = re.compile(r"^\s*}\s*;\s*$")
+ groupendprog2 = re.compile(r"^\s*}\s*%s_(\d+\.\d+\.\d+)\s*;\s*$" %
prefix)
+ symbolprog = re.compile(r"^\s*(\w+)\s*;\s*$")
+ for line in fh:
+ line = line.strip()
+
+ if line == "":
+ continue
+ if line[0] == '#':
+ continue
+ if line.startswith("global:"):
+ continue
+ if line.startswith("local:"):
+ continue
+
+ groupstartmatch = groupstartprog.match(line)
+ groupendmatch1 = groupendprog1.match(line)
+ groupendmatch2 = groupendprog2.match(line)
+ symbolmatch = symbolprog.match(line)
+ if groupstartmatch is not None:
+ if vers is not None:
+ raise Exception("malformed syms file when starting group")
+
+ vers = groupstartmatch.group(1)
+ elif groupendmatch1 is not None:
+ if prevvers is not None:
+ raise Exception("malformed syms file when ending group")
+
+ prevvers = vers
+ vers = None
+ elif groupendmatch2 is not None:
+ if groupendmatch2.group(1) != prevvers:
+ raise Exception("malformed syms file %s != %s when ending
group" % (
+ groupendmatch2.group(1), prevvers))
+
+ prevvers = vers
+ vers = None
+ elif symbolmatch is not None:
+ name = symbolmatch.group(1)
+ apisref[name] = {
+ "vers": vers,
+ "file": filenames.get(name),
+ }
+ else:
+ raise Exception("unexpected data %s" % line)
+
+
+apis = {}
+# Get the list of all public APIs and their corresponding version
+parseSymsFile(apis, "LIBVIRT", symslibvirt, os.path.join(srcdir,
"../docs/libvirt-api.xml"))
+
+# And the same for the QEMU specific APIs
+parseSymsFile(apis, "LIBVIRT_QEMU", symsqemu, os.path.join(srcdir,
"../docs/libvirt-qemu-api.xml"))
+
+# And the same for the LXC specific APIs
+parseSymsFile(apis, "LIBVIRT_LXC", symslxc, os.path.join(srcdir,
"../docs/libvirt-lxc-api.xml"))
+
+
+# Some special things which aren't public APIs,
+# but we want to report
+apis["virConnectSupportsFeature"] = {
+ "vers": "0.3.2"
+}
+apis["virDomainMigratePrepare"] = {
+ "vers": "0.3.2"
+}
+apis["virDomainMigratePerform"] = {
+ "vers": "0.3.2"
+}
+apis["virDomainMigrateFinish"] = {
+ "vers": "0.3.2"
+}
+apis["virDomainMigratePrepare2"] = {
+ "vers": "0.5.0"
+}
+apis["virDomainMigrateFinish2"] = {
+ "vers": "0.5.0"
+}
+apis["virDomainMigratePrepareTunnel"] = {
+ "vers": "0.7.2"
+}
+
+apis["virDomainMigrateBegin3"] = {
+ "vers": "0.9.2"
+}
+apis["virDomainMigratePrepare3"] = {
+ "vers": "0.9.2"
+}
+apis["virDomainMigratePrepareTunnel3"] = {
+ "vers": "0.9.2"
+}
+apis["virDomainMigratePerform3"] = {
+ "vers": "0.9.2"
+}
+apis["virDomainMigrateFinish3"] = {
+ "vers": "0.9.2"
+}
+apis["virDomainMigrateConfirm3"] = {
+ "vers": "0.9.2"
+}
+
+apis["virDomainMigrateBegin3Params"] = {
+ "vers": "1.1.0"
+}
+apis["virDomainMigratePrepare3Params"] = {
+ "vers": "1.1.0"
+}
+apis["virDomainMigratePrepareTunnel3Params"] = {
+ "vers": "1.1.0"
+}
+apis["virDomainMigratePerform3Params"] = {
+ "vers": "1.1.0"
+}
+apis["virDomainMigrateFinish3Params"] = {
+ "vers": "1.1.0"
+}
+apis["virDomainMigrateConfirm3Params"] = {
+ "vers": "1.1.0"
+}
+
+
+# Now we want to get the mapping between public APIs
+# and driver struct fields. This lets us later match
+# update the driver impls with the public APis.
+
+# Group name -> hash of APIs { fields -> api name }
+groups = {}
+ingrp = None
+for drivertablefile in drivertablefiles:
+ with open(drivertablefile) as fh:
+ starttableprog = re.compile(r"struct _(vir\w*Driver)")
+ endtableprog = re.compile(r"};")
+ callbackprog = re.compile(r"^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$")
+ ignoreapiprog = re.compile(r"\w+(Open|Close|URIProbe)")
+ for line in fh:
+ starttablematch = starttableprog.match(line)
+ if starttablematch is not None:
+ grp = starttablematch.group(1)
+ if grp != "virStateDriver" and grp !=
"virStreamDriver":
+ ingrp = grp
+ groups[ingrp] = {
+ "apis": {},
+ "drivers": {}
+ }
+ elif ingrp != None:
+ callbackmatch = callbackprog.match(line)
+ if callbackmatch is not None:
+ name = callbackmatch.group(1)
+ field = callbackmatch.group(2)
+
+ api = "vir" + name
+ if api in apis:
+ groups[ingrp]["apis"][field] = api
+ elif ignoreapiprog.match(api) != None:
+ continue
+ else:
+ raise Exception("driver %s does not have a public API"
% name)
+ elif endtableprog.match(line):
+ ingrp = None
+
+
+# Finally, we read all the primary driver files and extract
+# the driver API tables from each one.
+
+for src in srcs:
+ with open(src) as fh:
+ groupsre = "|".join(groups.keys())
+ groupsprog2 = re.compile(r"^\s*(static\s+)?(" + groupsre +
r")\s+(\w+)\s*=\s*{")
+ groupsprog3 = re.compile(r"^\s*(static\s+)?(" + groupsre +
r")\s+NAME\(\w+\)\s*=\s*{")
+ nodedevimplprog = re.compile(r".*/node_device_(\w+)\.c")
+ miscimplprog = re.compile(r".*/(\w+?)_((\w+)_)?(\w+)\.c")
+ callbackprog =
re.compile(r"\s*\.(\w+)\s*=\s*(\w+)\s*,?\s*(?:/\*\s*(\d+\.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$")
+ skipapiprog = re.compile(r"\w+(Open|Close|URIProbe)")
+
+ ingrp = None
+ impl = None
+ for line in fh:
+ if ingrp is None:
+ m = groupsprog2.match(line)
+ if m is None:
+ m = groupsprog3.match(line)
+ if m is not None:
+ ingrp = m.group(2)
+ impl = src
+
+ implmatch = nodedevimplprog.match(impl)
+ if implmatch is None:
+ implmatch = miscimplprog.match(impl)
+ if implmatch is None:
+ raise Exception("Unexpected impl format '%s'" %
impl)
+ impl = implmatch.group(1)
+
+ if impl in groups[ingrp]["drivers"]:
+ raise Exception("Group %s already contains %s" %
(ingrp, impl))
+
+ groups[ingrp]["drivers"][impl] = {}
+ else:
+ callbackmatch = callbackprog.match(line)
+ if callbackmatch is not None:
+ api = callbackmatch.group(1)
+ meth = callbackmatch.group(2)
+ vers = callbackmatch.group(3)
+ deleted = callbackmatch.group(4)
+
+ if api == "no" or api == "name":
+ continue
+
+ if meth == "NULL" and deleted is None:
+ raise Exception("Method impl for %s is NULL, but no deleted
version is provided" % api)
+
+ if meth != "NULL" and deleted is not None:
+ raise Exception("Method impl for %s is non-NULL, but deleted
version is provided" % api)
+
+ if vers is None and api != "connectURIProbe":
+ raise Exception("Method %s in %s is missing version" %
(meth, src))
+
+ if api not in groups[ingrp]["apis"]:
+ if skipapiprog.match(api):
+ continue
+
+ raise Exception("Found unexpected method %s in %s" %
(api, ingrp))
+
+ groups[ingrp]["drivers"][impl][api] = {
+ "vers": vers,
+ "deleted": deleted,
+ }
+
+ if (api == "domainMigratePrepare" or
+ api == "domainMigratePrepare2" or
+ api == "domainMigratePrepare3"):
+ if "domainMigrate" not in
groups[ingrp]["drivers"][impl]:
+
groups[ingrp]["drivers"][impl]["domainMigrate"] = {
+ "vers": vers,
+ }
+ elif line.find("}") != -1:
+ ingrp = None
+
+
+# The '.open' driver method is used for 3 public APIs, so we
+# have a bit of manual fixup todo with the per-driver versioning
+# and support matrix
+
+groups["virHypervisorDriver"]["apis"]["openAuth"] =
"virConnectOpenAuth"
+groups["virHypervisorDriver"]["apis"]["openReadOnly"] =
"virConnectOpenReadOnly"
+groups["virHypervisorDriver"]["apis"]["domainMigrate"] =
"virDomainMigrate"
+
+openAuthVers = (0 * 1000 * 1000) + (4 * 1000) + 0
+
+for drv in groups["virHypervisorDriver"]["drivers"].keys():
+ openVersStr =
groups["virHypervisorDriver"]["drivers"][drv]["connectOpen"]["vers"]
+ openVers = 0
+ if openVersStr != "Y":
+ openVersBits = openVersStr.split(".")
+ if len(openVersBits) != 3:
+ raise Exception("Expected 3 digit version for %s" % openVersStr)
+ openVers = (int(openVersBits[0]) * 1000 * 1000) + (int(openVersBits[1]) * 1000) +
int(openVersBits[2])
+
+ # virConnectOpenReadOnly always matches virConnectOpen version
+
groups["virHypervisorDriver"]["drivers"][drv]["connectOpenReadOnly"]
= \
+
groups["virHypervisorDriver"]["drivers"][drv]["connectOpen"]
+
+ # virConnectOpenAuth is always 0.4.0 if the driver existed
+ # before this time, otherwise it matches the version of
+ # the driver's virConnectOpen entry
+ if openVersStr == "Y" or openVers >= openAuthVers:
+ vers = openVersStr
+ else:
+ vers = "0.4.0"
+
groups["virHypervisorDriver"]["drivers"][drv]["connectOpenAuth"]
= {
+ "vers": vers,
+ }
+
+
+# Another special case for the virDomainCreateLinux which was replaced
+# with virDomainCreateXML
+groups["virHypervisorDriver"]["apis"]["domainCreateLinux"]
= "virDomainCreateLinux"
+
+createAPIVers = (0 * 1000 * 1000) + (0 * 1000) + 3
+
+for drv in groups["virHypervisorDriver"]["drivers"].keys():
+ if "domainCreateXML" not in
groups["virHypervisorDriver"]["drivers"][drv]:
+ continue
+ createVersStr =
groups["virHypervisorDriver"]["drivers"][drv]["domainCreateXML"]["vers"]
+ createVers = 0
+ if createVersStr != "Y":
+ createVersBits = createVersStr.split(".")
+ if len(createVersBits) != 3:
+ raise Exception("Expected 3 digit version for %s" % createVersStr)
+ createVers = (int(createVersBits[0]) * 1000 * 1000) + (int(createVersBits[1]) *
1000) + int(createVersBits[2])
+
+ # virCreateLinux is always 0.0.3 if the driver existed
+ # before this time, otherwise it matches the version of
+ # the driver's virCreateXML entry
+ if createVersStr == "Y" or createVers >= createAPIVers:
+ vers = createVersStr
+ else:
+ vers = "0.0.3"
+
+
groups["virHypervisorDriver"]["drivers"][drv]["domainCreateLinux"]
= {
+ "vers": vers,
+ }
+
+
+# Finally we generate the HTML file with the tables
+
+print('''<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml">
+<body class="hvsupport">
+<h1>libvirt API support matrix</h1>
+
+<ul id="toc"></ul>
+
+<p>
+This page documents which <a href="html/">libvirt calls</a> work
on
+which libvirt drivers / hypervisors, and which version the API appeared
+in. If a hypervisor driver later dropped support for the API, the version
+when it was removed is also mentioned (highlighted in
+<span class="removedhv">dark red</span>).
+</p>
+''')
+
+for grp in sorted(groups.keys()):
+ print("<h2><a id=\"%s\">%s</a></h2>" %
(grp, groupheaders[grp]))
+ print('''<table class="top_table">
+<thead>
+<tr>
+<th>API</th>
+<th>Version</th>''')
+
+ for drv in sorted(groups[grp]["drivers"].keys()):
+ print(" <th>%s</th>" % drv)
+
+
+ print('''</tr>
+</thead>
+<tbody>''')
+
+ row = 0
+ def sortkey(field):
+ return groups[grp]["apis"][field]
+ for field in sorted(groups[grp]["apis"].keys(), key=sortkey):
+ api = groups[grp]["apis"][field]
+ vers = apis[api]["vers"]
+ htmlgrp = apis[api].get("file")
+ print("<tr>")
+
+ if htmlgrp is not None:
+ print('''<td>\n<a
href=\"html/libvirt-%s.html#%s\">%s</a>\n</td>''' %
(htmlgrp, api, api))
+ else:
+ print("<td>\n%s</td>" % api)
+
+ print("<td>%s</td>" % vers)
+
+ for drv in sorted(groups[grp]["drivers"].keys()):
+ info = ""
+ if field in groups[grp]["drivers"][drv]:
+ vers = groups[grp]["drivers"][drv][field]["vers"]
+ if vers is not None:
+ info = info + vers
+
+ deleted =
groups[grp]["drivers"][drv][field].get("deleted")
+ if deleted is not None:
+ info = info + (''' - <span
class="removedhv">%s</span>''' % deleted)
+
+ print("<td>%s</td>" % info)
+
+ print("</tr>")
+
+ row = row + 1
+ if (row % 15) == 0:
+ print('''<tr>
+<th>API</th>
+<th>Version</th>''')
+
+ for drv in sorted(groups[grp]["drivers"].keys()):
+ print(" <th>%s</th>" % drv)
+
+ print("</tr>")
+
+ print("</tbody>\n</table>")
+
+print("</body>\n</html>")
--
2.21.0
5:21 a.m.
New subject: [libvirt] [PATCH 03/11] docs: rewrite ACL permissions checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-aclperms.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
src/Makefile.am | 4 +--
src/check-aclperms.pl | 73 ----------------------------------------
src/check-aclperms.py | 77 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 79 insertions(+), 75 deletions(-)
delete mode 100755 src/check-aclperms.pl
create mode 100755 src/check-aclperms.py
diff --git a/src/Makefile.am b/src/Makefile.am
index f5093b9c90..2eb4387231 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -359,11 +359,11 @@ check-aclrules:
$(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES)))
check-aclperms:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-aclperms.pl \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/check-aclperms.py \
$(srcdir)/access/viraccessperm.h \
$(srcdir)/access/viraccessperm.c
-EXTRA_DIST += check-driverimpls.pl check-aclrules.pl check-aclperms.pl
+EXTRA_DIST += check-driverimpls.pl check-aclrules.pl check-aclperms.py
check-local: check-protocol check-symfile check-symsorting \
check-drivername check-driverimpls check-aclrules \
diff --git a/src/check-aclperms.pl b/src/check-aclperms.pl
deleted file mode 100755
index 55b6598313..0000000000
--- a/src/check-aclperms.pl
+++ /dev/null
@@ -1,73 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# <http://www.gnu.org/licenses/>;.
-#
-# This script just validates that the stringified version of
-# a virAccessPerm enum matches the enum constant name. We do
-# a lot of auto-generation of code, so when these don't match
-# problems occur, preventing auth from succeeding at all.
-
-my $hdr = shift;
-my $impl = shift;
-
-my %perms;
-
-my @perms;
-
-open HDR, $hdr or die "cannot read $hdr: $!";
-
-while (<HDR>) {
- if (/^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$)/) {
- my $perm = $1;
-
- $perms{$perm} = 1 unless ($perm =~ /_LAST$/);
- }
-}
-
-close HDR;
-
-
-open IMPL, $impl or die "cannot read $impl: $!";
-
-my $group;
-my $warned = 0;
-
-while (defined (my $line = <IMPL>)) {
- if ($line =~ /VIR_ACCESS_PERM_([_A-Z]+)_LAST/) {
- $group = $1;
- } elsif ($line =~ /"[_a-z]+"/) {
- my @bits = split /,/, $line;
- foreach my $bit (@bits) {
- if ($bit =~ /"([_a-z]+)"/) {
- my $perm = uc($group . "_" . $1);
- if (!exists $perms{$perm}) {
- print STDERR "Unknown perm string $1 for group $group\n";
- $warned = 1;
- }
- delete $perms{$perm};
- }
- }
- }
-}
-close IMPL;
-
-foreach my $perm (keys %perms) {
- print STDERR "Perm $perm had not string form\n";
- $warned = 1;
-}
-
-exit $warned;
diff --git a/src/check-aclperms.py b/src/check-aclperms.py
new file mode 100755
index 0000000000..ace6ab6757
--- /dev/null
+++ b/src/check-aclperms.py
@@ -0,0 +1,77 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2013-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+#
+# This script just validates that the stringified version of
+# a virAccessPerm enum matches the enum constant name. We do
+# a lot of auto-generation of code, so when these don't match
+# problems occur, preventing auth from succeeding at all.
+
+from __future__ import print_function
+
+import re
+import sys
+
+if len(sys.argv) != 3:
+ print("syntax: %s HEADER IMPL" % (sys.argv[0]), file=sys.stderr)
+ sys.exit(1)
+
+hdr = sys.argv[1]
+impl = sys.argv[2]
+
+perms = {}
+
+with open(hdr) as fh:
+ symprog = re.compile(r"^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$).*")
+ for line in fh:
+ symmatch = symprog.match(line)
+ if symmatch is not None:
+ perm = symmatch.group(1)
+
+ if not perm.endswith("_LAST"):
+ perms[perm] = 1
+
+warned = False
+
+with open(impl) as fh:
+ group = None
+ symlastprog = re.compile(r".*VIR_ACCESS_PERM_([_A-Z]+)_LAST.*")
+ alnumprog = re.compile(r'''.*"([_a-z]+)".*''')
+
+ for line in fh:
+ symlastmatch = symlastprog.match(line)
+ if symlastmatch is not None:
+ group = symlastmatch.group(1)
+ elif alnumprog.match(line) is not None:
+ bits = line.split(",")
+ for bit in bits:
+ m = alnumprog.match(bit)
+ if m is not None:
+ perm = (group + "_" + m.group(1)).upper()
+ if perm not in perms:
+ print("Unknown perm string %s for group %s" %
(m.group(1), group), file=sys.stderr)
+ warned = True
+
+ del perms[perm]
+
+for perm in perms.keys():
+ print("Perm %s had not string form" % perm, file=sys.stderr)
+ warned = True
+
+if warned:
+ sys.exit(1)
+sys.exit(0)
--
2.21.0
5:21 a.m.
New subject: [libvirt] [PATCH 04/11] docs: rewrite symfile sorting checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-symsorting.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
src/Makefile.am | 6 +--
src/check-symsorting.pl | 106 -------------------------------------
src/check-symsorting.py | 112 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 115 insertions(+), 109 deletions(-)
delete mode 100755 src/check-symsorting.pl
create mode 100755 src/check-symsorting.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 2eb4387231..f30b085d07 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -285,12 +285,12 @@ check-symfile:
check-admin-symfile:
endif ! WITH_LINUX
check-symsorting:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(SYM_FILES)
check-admin-symsorting:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(ADMIN_SYM_FILES)
-EXTRA_DIST += check-symfile.pl check-symsorting.pl
+EXTRA_DIST += check-symfile.pl check-symsorting.py
# Keep this list synced with RPC_PROBE_FILES
PROTOCOL_STRUCTS = \
diff --git a/src/check-symsorting.pl b/src/check-symsorting.pl
deleted file mode 100755
index 51e38bdedb..0000000000
--- a/src/check-symsorting.pl
+++ /dev/null
@@ -1,106 +0,0 @@
-#!/usr/bin/env perl
-
-# Copyright (C) 2012-2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# <http://www.gnu.org/licenses/>;.
-
-use strict;
-use warnings;
-
-die "syntax: $0 SRCDIR SYMFILE..." unless int(@ARGV) >= 2;
-
-my $ret = 0;
-my $srcdir = shift;
-my $lastgroup = undef;
-foreach my $symfile (@ARGV) {
- open SYMFILE, $symfile or die "cannot read $symfile: $!";
-
- my $line = 0;
- my $groupfile = "";
- my @group;
-
- while (<SYMFILE>) {
- chomp;
-
- if (/^#\s*((\w+\/)*(\w+\.h))\s*$/) {
- $groupfile = $1;
- } elsif (/^#/) {
- # Ignore comments
- } elsif (/^\s*$/) {
- if (@group) {
- &check_sorting(\@group, $symfile, $line, $groupfile);
- }
- @group = ();
- $line = $.;
- } else {
- $_ =~ s/;//;
- push @group, $_;
- }
- }
-
- close SYMFILE;
- if (@group) {
- &check_sorting(\@group, $symfile, $line, $groupfile);
- }
- $lastgroup = undef;
-}
-
-sub check_sorting {
- my $group = shift;
- my $symfile = shift;
- my $line = shift;
- my $groupfile = shift;
-
- my @group = @{$group};
- my @sorted = sort { lc $a cmp lc $b } @group;
- my $sorted = 1;
- my $first;
- my $last;
-
- # Check that groups are in order and groupfile exists
- if (defined $lastgroup && lc $lastgroup ge lc $groupfile) {
- print "Symbol block at $symfile:$line: block not sorted\n";
- print "Move $groupfile block before $lastgroup block\n";
- print "\n";
- $ret = 1;
- }
- if (! -e "$srcdir/$groupfile") {
- print "Symbol block at $symfile:$line: $groupfile not found\n";
- print "\n";
- $ret = 1;
- }
- $lastgroup = $groupfile;
-
- # Check that symbols within a group are in order
- for (my $i = 0 ; $i <= $#sorted ; $i++) {
- if ($sorted[$i] ne $group[$i]) {
- $first = $i unless defined $first;
- $last = $i;
- $sorted = 0;
- }
- }
- if (!$sorted) {
- @group = splice @group, $first, ($last-$first+1);
- @sorted = splice @sorted, $first, ($last-$first+1);
- print "Symbol block at $symfile:$line: symbols not sorted\n";
- print map { " " . $_ . "\n" } @group;
- print "Correct ordering\n";
- print map { " " . $_ . "\n" } @sorted;
- print "\n";
- $ret = 1;
- }
-}
-
-exit $ret;
diff --git a/src/check-symsorting.py b/src/check-symsorting.py
new file mode 100755
index 0000000000..a1d16a4c43
--- /dev/null
+++ b/src/check-symsorting.py
@@ -0,0 +1,112 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import os.path
+import re
+import sys
+
+if len(sys.argv) < 3:
+ print("syntax: %s SRCDIR SYMFILE..." % sys.argv[0], file=sys.stderr)
+ sys.exit(1)
+
+def check_sorting(group, symfile, line, groupfile, lastgroup):
+ sortedgroup = sorted(group, key=str.lower)
+ issorted = True
+ first = None
+ last = None
+
+ err = False
+ # Check that groups are in order and groupfile exists
+ if lastgroup is not None and lastgroup.lower() > groupfile.lower():
+ print("Symbol block at %s:%s: block not sorted" % (symfile, line),
file=sys.stderr)
+ print("Move %s block before %s block" % (groupfile, lastgroup),
file=sys.stderr)
+ print("", file=sys.stderr)
+ err = True
+
+ if not os.path.exists(os.path.join(srcdir, groupfile)):
+ print("Symbol block at %s:%s: %s not found" % (symfile, line,
groupfile), file=sys.stderr)
+ print("", file=sys.stderr)
+ err = True
+
+ # Check that symbols within a group are in order
+ for i in range(len(group)):
+ if sortedgroup[i] != group[i]:
+ if first is None:
+ first = i
+
+ last = i
+ issorted = False
+
+ if not issorted:
+ actual = group[first:(last-first+1)]
+ expect = sortedgroup[first:(last-first+1)]
+ print("Symbol block at %s:%s: symbols not sorted" % (symfile, line),
file=sys.stderr)
+ for g in actual:
+ print(" %s" % g, file=sys.stderr)
+ print("Correct ordering", file=sys.stderr)
+ for g in expect:
+ print(" %s" % g, file=sys.stderr)
+ print("", file=sys.stderr)
+ err = True
+
+ return err
+
+
+ret = 0
+srcdir = sys.argv[1]
+lastgroup = None
+for symfile in sys.argv[2:]:
+ with open(symfile, "r") as fh:
+ lineno = 0
+ groupfile = ""
+ group = []
+ thisline = 0
+
+ filenameprog =
re.compile(r'''^#\s*((\w+\/)*(\w+\.h))\s*$''')
+
+ for line in fh:
+ thisline = thisline + 1
+ line = line.strip()
+
+ filenamematch = filenameprog.match(line)
+ if filenamematch is not None:
+ groupfile = filenamematch.group(1)
+ elif line == "":
+ if len(group) > 0:
+ if check_sorting(group, symfile, lineno, groupfile, lastgroup):
+ ret = 1
+
+ group = []
+ lineno = thisline
+ lastgroup = groupfile
+ elif line[0] == '#':
+ # Ignore comments
+ pass
+ else:
+ line = line.strip(";")
+ group.append(line)
+
+ if len(group) > 0:
+ if check_sorting(group, symfile, lineno, groupfile, lastgroup):
+ ret = 1
+
+ lastgroup = None
+
+sys.exit(ret)
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 05/11] docs: rewrite symfile library checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-symfile.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
src/Makefile.am | 6 ++--
src/check-symfile.pl | 70 --------------------------------------
src/check-symfile.py | 80 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 83 insertions(+), 73 deletions(-)
delete mode 100755 src/check-symfile.pl
create mode 100755 src/check-symfile.py
diff --git a/src/Makefile.am b/src/Makefile.am
index f30b085d07..1724b3b809 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -275,10 +275,10 @@ PDWTAGS = \
# rule for libvirt.la. However, checking symbols relies on Linux ELF layout
if WITH_LINUX
check-symfile: libvirt.syms libvirt.la
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt.syms \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symfile.py libvirt.syms \
.libs/libvirt.so
check-admin-symfile: libvirt_admin.syms libvirt-admin.la
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt_admin.syms \
+ $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symfile.py libvirt_admin.syms \
.libs/libvirt-admin.so
else ! WITH_LINUX
check-symfile:
@@ -290,7 +290,7 @@ check-symsorting:
check-admin-symsorting:
$(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(ADMIN_SYM_FILES)
-EXTRA_DIST += check-symfile.pl check-symsorting.py
+EXTRA_DIST += check-symfile.py check-symsorting.py
# Keep this list synced with RPC_PROBE_FILES
PROTOCOL_STRUCTS = \
diff --git a/src/check-symfile.pl b/src/check-symfile.pl
deleted file mode 100755
index 4f88300864..0000000000
--- a/src/check-symfile.pl
+++ /dev/null
@@ -1,70 +0,0 @@
-#!/usr/bin/env perl
-
-# Copyright (C) 2012-2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# <http://www.gnu.org/licenses/>;.
-
-die "syntax: $0 SYMFILE ELFLIB(S)" unless int(@ARGV) >= 2;
-
-my $symfile = shift @ARGV;
-my @elflibs = @ARGV;
-
-my %wantsyms;
-my %gotsyms;
-
-my $ret = 0;
-
-open SYMFILE, $symfile or die "cannot read $symfile: $!";
-
-while (<SYMFILE>) {
- next if /{/;
- next if /}/;
- next if /global:/;
- next if /local:/;
- next if /^\s*$/;
- next if /^\s*#/;
- next if /\*/;
-
- die "malformed line $_" unless /^\s*(\S+);$/;
-
- if (exists $wantsyms{$1}) {
- print STDERR "Symbol $1 is listed twice\n";
- $ret = 1;
- } else {
- $wantsyms{$1} = 1;
- }
-}
-close SYMFILE;
-
-foreach my $elflib (@elflibs) {
- open NM, "-|", "nm", $elflib or die "cannot run 'nm
$elflib': $!";
-
- while (<NM>) {
- next unless /^\S+\s(?:[TBD])\s(\S+)\s*$/;
-
- $gotsyms{$1} = 1;
- }
-
- close NM;
-}
-
-foreach my $sym (keys(%wantsyms)) {
- next if exists $gotsyms{$sym};
-
- print STDERR "Expected symbol $sym is not in ELF library\n";
- $ret = 1;
-}
-
-exit($ret);
diff --git a/src/check-symfile.py b/src/check-symfile.py
new file mode 100755
index 0000000000..3154619bfe
--- /dev/null
+++ b/src/check-symfile.py
@@ -0,0 +1,80 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import subprocess
+import sys
+
+if len(sys.argv) < 3:
+ print("syntax: %s SYMFILE ELFLIB(S)" % sys.argv[0], file=sys.stderr)
+
+symfile = sys.argv[1]
+elflibs = sys.argv[2:]
+
+wantsyms = {}
+gotsyms = {}
+
+ret = 0
+
+with open(symfile, "r") as fh:
+ for line in fh:
+ line = line.strip()
+ if line.find("{") != -1:
+ continue
+ if line.find("}") != -1:
+ continue
+ if line in ["global:", "local:"]:
+ continue
+ if line == "":
+ continue
+ if line[0] == '#':
+ continue
+ if line.find("*") != -1:
+ continue
+
+ line = line.strip(";")
+
+ if line in wantsyms:
+ print("Symbol $1 is listed twice", file=sys.stderr)
+ ret = 1
+ else:
+ wantsyms[line] = True
+
+for elflib in elflibs:
+ nm = subprocess.Popen(["nm", elflib], shell=False,
stdout=subprocess.PIPE).stdout
+
+ symprog = re.compile(r'''^\S+\s(?:[TBD])\s(\S+)\s*$''')
+ for line in nm:
+ line = line.decode("utf-8")
+ symmatch = symprog.match(line)
+ if symmatch is None:
+ continue
+
+ gotsyms[symmatch.group(1)] = True
+
+
+for sym in wantsyms.keys():
+ if sym in gotsyms:
+ continue
+
+ print("Expected symbol '%s' is not in ELF library" % sym,
file=sys.stderr)
+ ret = 1
+
+sys.exit(ret)
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 06/11] docs: rewrite augest test generator in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the augeas-gentest.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/augeas-gentest.pl | 60 -------------------------------
build-aux/augeas-gentest.py | 72 +++++++++++++++++++++++++++++++++++++
src/Makefile.am | 2 +-
4 files changed, 74 insertions(+), 62 deletions(-)
delete mode 100755 build-aux/augeas-gentest.pl
create mode 100755 build-aux/augeas-gentest.py
diff --git a/Makefile.am b/Makefile.am
index cf9ff94f4f..e909d29d10 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -41,7 +41,7 @@ EXTRA_DIST = \
run.in \
README.md \
AUTHORS.in \
- build-aux/augeas-gentest.pl \
+ build-aux/augeas-gentest.py \
build-aux/check-spacing.pl \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
diff --git a/build-aux/augeas-gentest.pl b/build-aux/augeas-gentest.pl
deleted file mode 100755
index 65834b533b..0000000000
--- a/build-aux/augeas-gentest.pl
+++ /dev/null
@@ -1,60 +0,0 @@
-#!/usr/bin/env perl
-#
-# augeas-gentest.pl: Generate an augeas test file, from an
-# example config file + test file template
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# <http://www.gnu.org/licenses/>;.
-
-use strict;
-use warnings;
-
-die "syntax: $0 CONFIG TEMPLATE\n" unless @ARGV == 2;
-
-my $config = shift @ARGV;
-my $template = shift @ARGV;
-
-open CONFIG, "<", $config or die "cannot read $config: $!";
-open TEMPLATE, "<", $template or die "cannot read $template: $!";
-
-my $group = 0;
-while (<TEMPLATE>) {
- if (/\@CONFIG\@/) {
- my $group = 0;
- print " let conf = \"";
- while (<CONFIG>) {
- if (/^#\w/) {
- s/^#//;
- s/\"/\\\"/g;
- print $_;
- $group = /\[\s$/;
- } elsif ($group) {
- s/\"/\\\"/g;
- if (/#\s*\]/) {
- $group = 0;
- }
- if (/^#/) {
- s/^#//;
- print $_;
- }
- }
- }
- print "\"\n";
- } else {
- print $_;
- }
-}
-
-close TEMPLATE;
-close CONFIG;
diff --git a/build-aux/augeas-gentest.py b/build-aux/augeas-gentest.py
new file mode 100755
index 0000000000..b7f343ef42
--- /dev/null
+++ b/build-aux/augeas-gentest.py
@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# augeas-gentest.py: Generate an augeas test file, from an
+# example config file + test file template
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import sys
+
+if len(sys.argv) != 3:
+ print("syntax: %s CONFIG TEMPLATE" % sys.argv[0], file=sys.stderr)
+ sys.exit(1)
+
+config = sys.argv[1]
+template = sys.argv[2]
+
+def expand_config(config):
+ with open(config) as fh:
+ optprog = re.compile(r'''^#\w.*''')
+ groupstartprog = re.compile(r'''.*\[\s$''')
+ groupendprog = re.compile(r'''#\s*\].*$''')
+
+ group = False
+ for line in fh:
+ if optprog.match(line) is not None:
+ line = line[1:]
+ line = line.replace('"', '\\"')
+ print(line, end='')
+ if groupstartprog.match(line):
+ group = True
+ elif group:
+ line = line.replace('"', '\\"')
+
+ if groupendprog.match(line):
+ group = False
+
+ if line[0] == '#':
+ line = line[1:]
+ print(line, end='')
+
+
+def expand_template(template, config):
+ with open(template) as fh:
+ group = 0
+
+ markerprog = re.compile(r'''\s*@CONFIG@\s*''')
+ for line in fh:
+ if markerprog.match(line) is not None:
+ print(' let conf = "', end='')
+ expand_config(config)
+ print('"')
+ else:
+ print(line, end='')
+
+expand_template(template, config)
diff --git a/src/Makefile.am b/src/Makefile.am
index 1724b3b809..716c2000e2 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -422,7 +422,7 @@ check-augeas: $(augeas_DATA) $(augeastest_DATA)
fi
.PHONY: check-augeas
-AUG_GENTEST = $(PERL) $(top_srcdir)/build-aux/augeas-gentest.pl
+AUG_GENTEST = $(PYTHON) $(top_srcdir)/build-aux/augeas-gentest.py
#
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 07/11] docs: rewrite po file minimizer in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the minimize-po.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/minimize-po.pl | 37 -------------------------
build-aux/minimize-po.py | 60 ++++++++++++++++++++++++++++++++++++++++
po/Makefile.am | 2 +-
4 files changed, 62 insertions(+), 39 deletions(-)
delete mode 100755 build-aux/minimize-po.pl
create mode 100755 build-aux/minimize-po.py
diff --git a/Makefile.am b/Makefile.am
index e909d29d10..0a9d867610 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -45,7 +45,7 @@ EXTRA_DIST = \
build-aux/check-spacing.pl \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
- build-aux/minimize-po.pl \
+ build-aux/minimize-po.py \
build-aux/mock-noinline.pl \
build-aux/prohibit-duplicate-header.pl \
build-aux/useless-if-before-free \
diff --git a/build-aux/minimize-po.pl b/build-aux/minimize-po.pl
deleted file mode 100755
index 497533a836..0000000000
--- a/build-aux/minimize-po.pl
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/perl
-
-my @block;
-my $msgstr = 0;
-my $empty = 0;
-my $unused = 0;
-my $fuzzy = 0;
-while (<>) {
- if (/^$/) {
- if (!$empty && !$unused && !$fuzzy) {
- print @block;
- }
- @block = ();
- $msgstr = 0;
- $fuzzy = 0;
- push @block, $_;
- } else {
- if (/^msgstr/) {
- $msgstr = 1;
- $empty = 1;
- }
- if (/^#.*fuzzy/) {
- $fuzzy = 1;
- }
- if (/^#~ msgstr/) {
- $unused = 1;
- }
- if ($msgstr && /".+"/) {
- $empty = 0;
- }
- push @block, $_;
- }
-}
-
-if (@block && !$empty && !$unused) {
- print @block;
-}
diff --git a/build-aux/minimize-po.py b/build-aux/minimize-po.py
new file mode 100755
index 0000000000..5046bacede
--- /dev/null
+++ b/build-aux/minimize-po.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2018-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import sys
+
+block = []
+msgstr = False
+empty = False
+unused = False
+fuzzy = False
+
+strprog = re.compile(r'''.*".+".*''')
+
+for line in sys.stdin:
+ line = line.rstrip("\n")
+
+ if line == "":
+ if not empty and not unused and not fuzzy:
+ for b in block:
+ print(b)
+
+ block = []
+ msgstr = False
+ fuzzy = False
+ block.append(line)
+ else:
+ if line.startswith("msgstr"):
+ msgstr = True
+ empty = True
+
+ if line[0] == '#' and line.find("fuzzy") != -1:
+ fuzzy = True
+ if line.startswith("#~ msgstr"):
+ unused = True
+ if msgstr and strprog.match(line):
+ empty = False
+
+ block.append(line)
+
+if not empty and not unused and not fuzzy:
+ for b in block:
+ print(b)
diff --git a/po/Makefile.am b/po/Makefile.am
index da117edbd5..b4d2f956ab 100644
--- a/po/Makefile.am
+++ b/po/Makefile.am
@@ -58,7 +58,7 @@ update-mini-po: $(POTFILE)
$(MSGMERGE) --no-location --no-fuzzy-matching --sort-output \
$$lang.po $(POTFILE) | \
$(SED) $(SED_PO_FIXUP_ARGS) | \
- $(PERL) $(top_srcdir)/build-aux/minimize-po.pl > \
+ $(PYTHON) $(top_srcdir)/build-aux/minimize-po.py > \
$(srcdir)/$$lang.mini.po ; \
done
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 08/11] docs: rewrite duplicate header checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the prohibit-duplicate-header.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/prohibit-duplicate-header.pl | 26 -------------
build-aux/prohibit-duplicate-header.py | 54 ++++++++++++++++++++++++++
cfg.mk | 2 +-
4 files changed, 56 insertions(+), 28 deletions(-)
delete mode 100644 build-aux/prohibit-duplicate-header.pl
create mode 100644 build-aux/prohibit-duplicate-header.py
diff --git a/Makefile.am b/Makefile.am
index 0a9d867610..d0077a4f2a 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -47,7 +47,7 @@ EXTRA_DIST = \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
build-aux/mock-noinline.pl \
- build-aux/prohibit-duplicate-header.pl \
+ build-aux/prohibit-duplicate-header.py \
build-aux/useless-if-before-free \
build-aux/vc-list-files \
ci/Makefile \
diff --git a/build-aux/prohibit-duplicate-header.pl
b/build-aux/prohibit-duplicate-header.pl
deleted file mode 100644
index 4a2ea65665..0000000000
--- a/build-aux/prohibit-duplicate-header.pl
+++ /dev/null
@@ -1,26 +0,0 @@
-#!/usr/bin/env perl
-
-use strict;
-
-my $file = " ";
-my $ret = 0;
-my %includes = ( );
-my $lineno = 0;
-
-while (<>) {
- if (not $file eq $ARGV) {
- %includes = ( );
- $file = $ARGV;
- $lineno = 0;
- }
- $lineno++;
- if (/^# *include *[<"]([^>"]*\.h)[">]/) {
- $includes{$1}++;
- if ($includes{$1} == 2) {
- $ret = 1;
- print STDERR "$ARGV:$lineno: $_";
- print STDERR "Do not include a header more than once per file\n";
- }
- }
-}
-exit $ret;
diff --git a/build-aux/prohibit-duplicate-header.py
b/build-aux/prohibit-duplicate-header.py
new file mode 100644
index 0000000000..0ef86e8453
--- /dev/null
+++ b/build-aux/prohibit-duplicate-header.py
@@ -0,0 +1,54 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2016-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import sys
+
+def check_file(filename):
+ includes = {}
+ lineno = 0
+ errs = False
+ with open(filename, "r") as fh:
+ headerprog = re.compile(r'''^# *include
*[<"]([^>"]*\.h)[">]\s*$''')
+ for line in fh:
+ lineno = lineno + 1
+
+ headermatch = headerprog.match(line)
+ if headermatch is not None:
+ inc = headermatch.group(1)
+
+ if inc in includes:
+ print("%s:%d: %s" % (filename, lineno, inc),
file=sys.stderr)
+ errs = True
+ else:
+ includes[inc] = True
+
+ return errs
+
+ret = 0
+
+for filename in sys.argv[1:]:
+ if check_file(filename):
+ ret = 1
+
+if ret == 1:
+ print("Do not include a header more than once per file", file=sys.stderr)
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index d10dfa16a1..da1a1cb3cf 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1148,7 +1148,7 @@ endif
# Don't include duplicate header in the source (either *.c or *.h)
prohibit-duplicate-header:
$(AM_V_GEN)$(VC_LIST_EXCEPT) | $(GREP) '\.[chx]$$' | xargs \
- $(PERL) -W $(top_srcdir)/build-aux/prohibit-duplicate-header.pl
+ $(PYTHON) $(top_srcdir)/build-aux/prohibit-duplicate-header.py
spacing-check:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 09/11] docs: rewrite whitespace checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-spacing.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/check-spacing.pl | 198 -----------------------------------
build-aux/check-spacing.py | 204 +++++++++++++++++++++++++++++++++++++
cfg.mk | 2 +-
4 files changed, 206 insertions(+), 200 deletions(-)
delete mode 100755 build-aux/check-spacing.pl
create mode 100755 build-aux/check-spacing.py
diff --git a/Makefile.am b/Makefile.am
index d0077a4f2a..d28c3ebcea 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -42,7 +42,7 @@ EXTRA_DIST = \
README.md \
AUTHORS.in \
build-aux/augeas-gentest.py \
- build-aux/check-spacing.pl \
+ build-aux/check-spacing.py \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
diff --git a/build-aux/check-spacing.pl b/build-aux/check-spacing.pl
deleted file mode 100755
index 33377f3dd3..0000000000
--- a/build-aux/check-spacing.pl
+++ /dev/null
@@ -1,198 +0,0 @@
-#!/usr/bin/env perl
-#
-# check-spacing.pl: Report any usage of 'function (..args..)'
-# Also check for other syntax issues, such as correct use of ';'
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# <http://www.gnu.org/licenses/>;.
-
-use strict;
-use warnings;
-
-my $ret = 0;
-my $incomment = 0;
-
-foreach my $file (@ARGV) {
- # Per-file variables for multiline Curly Bracket (cb_) check
- my $cb_linenum = 0;
- my $cb_code = "";
- my $cb_scolon = 0;
-
- open FILE, $file;
-
- while (defined (my $line = <FILE>)) {
- my $data = $line;
- # For temporary modifications
- my $tmpdata;
-
- # Kill any quoted , ; = or "
- $data =~ s/'[";,=]'/'X'/g;
-
- # Kill any quoted strings
- $data =~ s,"(?:[^\\\"]|\\.)*","XXX",g;
-
- next if $data =~ /^#/;
-
- # Kill contents of multi-line comments
- # and detect end of multi-line comments
- if ($incomment) {
- if ($data =~ m,\*/,) {
- $incomment = 0;
- $data =~ s,^.*\*/,*/,;
- } else {
- $data = "";
- }
- }
-
- # Kill single line comments, and detect
- # start of multi-line comments
- if ($data =~ m,/\*.*\*/,) {
- $data =~ s,/\*.*\*/,/* */,;
- } elsif ($data =~ m,/\*,) {
- $incomment = 1;
- $data =~ s,/\*.*,/*,;
- }
-
- # We need to match things like
- #
- # int foo (int bar, bool wizz);
- # foo (bar, wizz);
- #
- # but not match things like:
- #
- # typedef int (*foo)(bar wizz)
- #
- # we can't do this (efficiently) without
- # missing things like
- #
- # foo (*bar, wizz);
- #
- # We also don't want to spoil the $data so it can be used
- # later on.
- $tmpdata = $data;
- while ($tmpdata =~ /(\w+)\s\((?!\*)/) {
- my $kw = $1;
-
- # Allow space after keywords only
- if ($kw =~ /^(?:if|for|while|switch|return)$/) {
- $tmpdata =~ s/(?:$kw\s\()/XXX(/;
- } else {
- print "Whitespace after non-keyword:\n";
- print "$file:$.: $line";
- $ret = 1;
- last;
- }
- }
-
- # Require whitespace immediately after keywords
- if ($data =~ /\b(?:if|for|while|switch|return)\(/) {
- print "No whitespace after keyword:\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # Forbid whitespace between )( of a function typedef
- if ($data =~ /\(\*\w+\)\s+\(/) {
- print "Whitespace between ')' and '(':\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # Forbid whitespace following ( or prior to )
- # but allow whitespace before ) on a single line
- # (optionally followed by a semicolon)
- if (($data =~ /\s\)/ && not $data =~ /^\s+\);?$/) ||
- $data =~ /\((?!$)\s/) {
- print "Whitespace after '(' or before ')':\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # Forbid whitespace before ";" or ",". Things like below are
allowed:
- #
- # 1) The expression is empty for "for" loop. E.g.
- # for (i = 0; ; i++)
- #
- # 2) An empty statement. E.g.
- # while (write(statuswrite, &status, 1) == -1 &&
- # errno == EINTR)
- # ;
- #
- if ($data =~ /\s[;,]/) {
- unless ($data =~ /\S; ; / ||
- $data =~ /^\s+;/) {
- print "Whitespace before semicolon or comma:\n";
- print "$file:$.: $line";
- $ret = 1;
- }
- }
-
- # Require EOL, macro line continuation, or whitespace after ";".
- # Allow "for (;;)" as an exception.
- if ($data =~ /;[^ \\\n;)]/) {
- print "Invalid character after semicolon:\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # Require EOL, space, or enum/struct end after comma.
- if ($data =~ /,[^ \\\n)}]/) {
- print "Invalid character after comma:\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # Require spaces around assignment '=', compounds and '=='
- if ($data =~ /[^ ]\b[!<>&|\-+*\/%\^=]?=/ ||
- $data =~ /=[^= \\\n]/) {
- print "Spacing around '=' or '==':\n";
- print "$file:$.: $line";
- $ret = 1;
- }
-
- # One line conditional statements with one line bodies should
- # not use curly brackets.
- if ($data =~ /^\s*(if|while|for)\b.*\{$/) {
- $cb_linenum = $.;
- $cb_code = $line;
- $cb_scolon = 0;
- }
-
- # We need to check for exactly one semicolon inside the body,
- # because empty statements (e.g. with comment only) are
- # allowed
- if ($cb_linenum == $. - 1 && $data =~ /^[^;]*;[^;]*$/) {
- $cb_code .= $line;
- $cb_scolon = 1;
- }
-
- if ($data =~ /^\s*}\s*$/ &&
- $cb_linenum == $. - 2 &&
- $cb_scolon) {
-
- print "Curly brackets around single-line body:\n";
- print "$file:$cb_linenum-$.:\n$cb_code$line";
- $ret = 1;
-
- # There _should_ be no need to reset the values; but to
- # keep my inner peace...
- $cb_linenum = 0;
- $cb_scolon = 0;
- $cb_code = "";
- }
- }
- close FILE;
-}
-
-exit $ret;
diff --git a/build-aux/check-spacing.py b/build-aux/check-spacing.py
new file mode 100755
index 0000000000..ec565585ad
--- /dev/null
+++ b/build-aux/check-spacing.py
@@ -0,0 +1,204 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# check-spacing.pl: Report any usage of 'function (..args..)'
+# Also check for other syntax issues, such as correct use of ';'
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import sys
+
+def check_whitespace(filename):
+ errs = False
+ with open(filename, 'r') as fh:
+ quotedmetaprog =
re.compile(r"""'[";,=]'""")
+ quotedstringprog =
re.compile(r'''"(?:[^\\\"]|\\.)*"''')
+ commentstartprog = re.compile(r'''^(.*)/\*.*$''')
+ commentendprog = re.compile(r'''^.*\*/(.*)$''')
+ commentprog = re.compile(r'''^(.*)/\*.*\*/(.*)''')
+ funcprog = re.compile(r'''(\w+)\s\((?!\*)''')
+ keywordprog =
re.compile(r'''^.*\b(?:if|for|while|switch|return)\(.*$''')
+ functypedefprog =
re.compile(r'''^.*\(\*\w+\)\s+\(.*$''')
+ whitespaceprog1 = re.compile(r'''^.*\s\).*$''')
+ whitespaceprog2 = re.compile(r'''^\s+\);?$''')
+ whitespaceprog3 = re.compile(r'''^.*\((?!$)\s.*''')
+ commasemiprog1 = re.compile(r'''.*\s[;,].*''')
+ commasemiprog2 = re.compile(r'''.*\S; ; .*''')
+ commasemiprog3 = re.compile(r'''^\s+;''')
+ semicolonprog = re.compile(r'''.*;[^ \\\n;)].*''')
+ commaprog = re.compile(r'''.*,[^ \\\n)}].*''')
+ assignprog1 = re.compile(r'''[^
]\b[!<>&|\-+*\/%\^=]?=''')
+ assignprog2 = re.compile(r'''=[^= \\\n]''')
+ condstartprog =
re.compile(r'''^\s*(if|while|for)\b.*\{$''')
+ statementprog = re.compile(r'''^[^;]*;[^;]*$''')
+ condendprog = re.compile(r'''^\s*}\s*$''')
+
+ incomment = False
+ # Per-file variables for multiline Curly Bracket (cb_) check
+ cb_lineno = 0
+ cb_code = ""
+ cb_scolon = False
+
+ lineno = 0
+ for line in fh:
+ lineno = lineno + 1
+ data = line
+ # For temporary modifications
+
+ # Kill any quoted , ; = or "
+ data = quotedmetaprog.sub("'X'", data)
+
+ # Kill any quoted strings
+ data = quotedstringprog.sub('"XXX"', data)
+
+ if data[0] == '#':
+ continue
+
+ # Kill contents of multi-line comments
+ # and detect end of multi-line comments
+ if incomment:
+ if commentendprog.match(data):
+ data = commentendprog.sub('*/\2', data)
+ incomment = False
+ else:
+ data = ""
+
+ # Kill single line comments, and detect
+ # start of multi-line comments
+ if commentprog.match(data):
+ data = commentprog.sub(r'''\1/* */\2''', data)
+ elif commentstartprog.match(data):
+ data = commentstartprog.sub(r'''\1/*''', data)
+ incomment = True
+
+ # We need to match things like
+ #
+ # int foo (int bar, bool wizz);
+ # foo (bar, wizz);
+ #
+ # but not match things like:
+ #
+ # typedef int (*foo)(bar wizz)
+ #
+ # we can't do this (efficiently) without
+ # missing things like
+ #
+ # foo (*bar, wizz);
+ #
+ for match in funcprog.finditer(data):
+ kw = match.group(1)
+
+ # Allow space after keywords only
+ if kw not in ["if", "for", "while",
"switch", "return"]:
+ print("Whitespace after non-keyword:", file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
file=sys.stderr)
+ errs = True
+ break
+
+ # Require whitespace immediately after keywords
+ if keywordprog.match(data):
+ print("No whitespace after keyword:", file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Forbid whitespace between )( of a function typedef
+ if functypedefprog.match(data):
+ print("Whitespace between ')' and '(':",
file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Forbid whitespace following ( or prior to )
+ # but allow whitespace before ) on a single line
+ # (optionally followed by a semicolon)
+ if ((whitespaceprog1.match(data) and
+ not whitespaceprog2.match(data))
+ or whitespaceprog3.match(data)):
+ print("Whitespace after '(' or before ')':",
file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Forbid whitespace before ";" or ",". Things like below
are allowed:
+ #
+ # 1) The expression is empty for "for" loop. E.g.
+ # for (i = 0; ; i++)
+ #
+ # 2) An empty statement. E.g.
+ # while (write(statuswrite, &status, 1) == -1 &&
+ # errno == EINTR)
+ # ;
+ #
+ if commasemiprog1.match(data) and not (
+ commasemiprog2.match(data) or
+ commasemiprog3.match(data)):
+ print("Whitespace before semicolon or comma:",
file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Require EOL, macro line continuation, or whitespace after ";".
+ # Allow "for (;;)" as an exception.
+ if semicolonprog.match(data):
+ print("Invalid character after semicolon:", file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Require EOL, space, or enum/struct end after comma.
+ if commaprog.match(data):
+ print("Invalid character after comma:", file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # Require spaces around assignment '=', compounds and '=='
+ if assignprog1.match(data) or assignprog2.match(data):
+ print("Spacing around '=' or '==':",
file=sys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line), file=sys.stderr)
+ errs = True
+
+ # One line conditional statements with one line bodies should
+ # not use curly brackets.
+ if condstartprog.match(data):
+ cb_lineno = lineno
+ cb_code = line
+ cb_scolon = False
+
+ # We need to check for exactly one semicolon inside the body,
+ # because empty statements (e.g. with comment only) are
+ # allowed
+ if (cb_lineno == lineno - 1) and statementprog.match(data):
+ cb_code = cb_code + line
+ cb_scolon = True
+
+ if condendprog.match(data) and (cb_lineno == lineno - 2) and cb_scolon:
+ print("Curly brackets around single-line body:",
file=sys.stderr)
+ print("%s:%d:\n%s%s"% (filename, cb_linenum - lineno, cb_code,
line), file=sys.stderr)
+ errs = True
+
+ # There _should_ be no need to reset the values; but to
+ # keep my inner peace...
+ cb_linenum = 0
+ cb_scolon = False
+ cb_code = ""
+
+ return errs
+
+ret = 0
+for filename in sys.argv[1:]:
+ if check_whitespace(filename):
+ ret = 1
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index da1a1cb3cf..02bade7f8a 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1152,7 +1152,7 @@ prohibit-duplicate-header:
spacing-check:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/check-spacing.pl || \
+ $(PYTHON) $(top_srcdir)/build-aux/check-spacing.py || \
{ echo '$(ME): incorrect formatting' 1>&2; exit 1; }
mock-noinline:
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 10/11] docs: rewrite mock inline checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the mock-noinline.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/mock-noinline.pl | 75 --------------------------------
build-aux/mock-noinline.py | 88 ++++++++++++++++++++++++++++++++++++++
cfg.mk | 2 +-
4 files changed, 90 insertions(+), 77 deletions(-)
delete mode 100644 build-aux/mock-noinline.pl
create mode 100644 build-aux/mock-noinline.py
diff --git a/Makefile.am b/Makefile.am
index d28c3ebcea..8bbc6e1747 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -46,7 +46,7 @@ EXTRA_DIST = \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
- build-aux/mock-noinline.pl \
+ build-aux/mock-noinline.py \
build-aux/prohibit-duplicate-header.py \
build-aux/useless-if-before-free \
build-aux/vc-list-files \
diff --git a/build-aux/mock-noinline.pl b/build-aux/mock-noinline.pl
deleted file mode 100644
index 958e133885..0000000000
--- a/build-aux/mock-noinline.pl
+++ /dev/null
@@ -1,75 +0,0 @@
-#!/usr/bin/env perl
-
-my %noninlined;
-my %mocked;
-
-# Functions in public header don't get the noinline annotation
-# so whitelist them here
-$noninlined{"virEventAddTimeout"} = 1;
-# This one confuses the script as its defined in the mock file
-# but is actually just a local helper
-$noninlined{"virMockStatRedirect"} = 1;
-
-foreach my $arg (@ARGV) {
- if ($arg =~ /\.h$/) {
- #print "Scan header $arg\n";
- &scan_annotations($arg);
- } elsif ($arg =~ /mock\.c$/) {
- #print "Scan mock $arg\n";
- &scan_overrides($arg);
- }
-}
-
-my $warned = 0;
-foreach my $func (keys %mocked) {
- next if exists $noninlined{$func};
-
- $warned++;
- print STDERR "$func is mocked at $mocked{$func} but missing noinline
annotation\n";
-}
-
-exit $warned ? 1 : 0;
-
-
-sub scan_annotations {
- my $file = shift;
-
- open FH, $file or die "cannot read $file: $!";
-
- my $func;
- while (<FH>) {
- if (/^\s*(\w+)\(/ || /^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(/) {
- my $name = $1;
- if ($name !~ /ATTRIBUTE/) {
- $func = $name;
- }
- } elsif (/^\s*$/) {
- $func = undef;
- }
- if (/ATTRIBUTE_NOINLINE/) {
- if (defined $func) {
- $noninlined{$func} = 1;
- }
- }
- }
-
- close FH
-}
-
-sub scan_overrides {
- my $file = shift;
-
- open FH, $file or die "cannot read $file: $!";
-
- my $func;
- while (<FH>) {
- if (/^(\w+)\(/ || /^\w+\s*(?:\*\s*)?(\w+)\(/) {
- my $name = $1;
- if ($name =~ /^vir/) {
- $mocked{$name} = "$file:$.";
- }
- }
- }
-
- close FH
-}
diff --git a/build-aux/mock-noinline.py b/build-aux/mock-noinline.py
new file mode 100644
index 0000000000..fdc60c5de9
--- /dev/null
+++ b/build-aux/mock-noinline.py
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2017-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+
+from __future__ import print_function
+
+import re
+import sys
+
+noninlined = {}
+mocked = {}
+
+# Functions in public header don't get the noinline annotation
+# so whitelist them here
+noninlined["virEventAddTimeout"] = True
+# This one confuses the script as its defined in the mock file
+# but is actually just a local helper
+noninlined["virMockStatRedirect"] = True
+
+def scan_annotations(filename):
+ funcprog1 = re.compile(r'''^\s*(\w+)\(.*''')
+ funcprog2 =
re.compile(r'''^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(.*''')
+ with open(filename, "r") as fh:
+ func = None
+ for line in fh:
+ line = line.strip()
+ m = funcprog1.match(line)
+ if m is None:
+ m = funcprog2.match(line)
+ if m is not None:
+ name = m.group(1)
+ if name.find("ATTRIBUTE") == -1:
+ func = name
+ elif line == "":
+ func = None
+
+ if line.find("ATTRIBUTE_NOINLINE") != -1:
+ if func is not None:
+ noninlined[func] = True
+
+def scan_overrides(filename):
+ funcprog1 = re.compile(r'''^(\w+)\(.*''')
+ funcprog2 = re.compile(r'''^\w+\s*(?:\*\s*)?(\w+)\(.*''')
+ with open(filename, "r") as fh:
+ func = None
+ lineno = 0
+ for line in fh:
+ lineno = lineno + 1
+ #line = line.strip()
+
+ m = funcprog1.match(line)
+ if m is None:
+ m = funcprog2.match(line)
+ if m is not None:
+ name = m.group(1)
+ if name.startswith("vir"):
+ mocked[name] = "%s:%d" % (filename, lineno)
+
+
+for filename in sys.argv[1:]:
+ if filename.endswith(".h"):
+ scan_annotations(filename)
+ elif filename.endswith("mock.c"):
+ scan_overrides(filename)
+
+warned = False
+for func in mocked.keys():
+ if func not in noninlined:
+ warned = True
+ print("%s is mocked at %s but missing noinline annotation" % (func,
mocked[func]), file=sys.stderr)
+
+if warned:
+ sys.exit(1)
+sys.exit(0)
diff --git a/cfg.mk b/cfg.mk
index 02bade7f8a..732ee8084c 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1157,7 +1157,7 @@ spacing-check:
mock-noinline:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[ch]$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/mock-noinline.pl
+ $(PYTHON) $(top_srcdir)/build-aux/mock-noinline.py
header-ifdef:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \
--
2.21.0
5:22 a.m.
New subject: [libvirt] [PATCH 11/11] docs: rewrite header ifdef checker in Python
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the header-ifdef.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com>
---
Makefile.am | 2 +-
build-aux/header-ifdef.pl | 182 ---------------------------------
build-aux/header-ifdef.py | 206 ++++++++++++++++++++++++++++++++++++++
cfg.mk | 2 +-
4 files changed, 208 insertions(+), 184 deletions(-)
delete mode 100644 build-aux/header-ifdef.pl
create mode 100644 build-aux/header-ifdef.py
diff --git a/Makefile.am b/Makefile.am
index 8bbc6e1747..22480882fc 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -44,7 +44,7 @@ EXTRA_DIST = \
build-aux/augeas-gentest.py \
build-aux/check-spacing.py \
build-aux/gitlog-to-changelog \
- build-aux/header-ifdef.pl \
+ build-aux/header-ifdef.py \
build-aux/minimize-po.py \
build-aux/mock-noinline.py \
build-aux/prohibit-duplicate-header.py \
diff --git a/build-aux/header-ifdef.pl b/build-aux/header-ifdef.pl
deleted file mode 100644
index dba3dbcbdc..0000000000
--- a/build-aux/header-ifdef.pl
+++ /dev/null
@@ -1,182 +0,0 @@
-#!/usr/bin/perl
-#
-# Validate that header files follow a standard layout:
-#
-# /*
-# ...copyright header...
-# */
-# <one blank line>
-# #pragma once
-# ....content....
-#
-#---
-#
-# For any file ending priv.h, before the #pragma once
-# We will have a further section
-#
-# #ifndef SYMBOL_ALLOW
-# # error ....
-# #endif /* SYMBOL_ALLOW */
-# <one blank line>
-#
-#---
-#
-# For public headers (files in include/), use the standard header guard instead of
#pragma once:
-# #ifndef SYMBOL
-# # define SYMBOL
-# ....content....
-# #endif /* SYMBOL */
-
-use strict;
-use warnings;
-
-my $STATE_COPYRIGHT_COMMENT = 0;
-my $STATE_COPYRIGHT_BLANK = 1;
-my $STATE_PRIV_START = 2;
-my $STATE_PRIV_ERROR = 3;
-my $STATE_PRIV_END = 4;
-my $STATE_PRIV_BLANK = 5;
-my $STATE_GUARD_START = 6;
-my $STATE_GUARD_DEFINE = 7;
-my $STATE_GUARD_END = 8;
-my $STATE_EOF = 9;
-my $STATE_PRAGMA = 10;
-
-my $file = " ";
-my $ret = 0;
-my $ifdef = "";
-my $ifdefpriv = "";
-my $publicheader = 0;
-
-my $state = $STATE_EOF;
-my $mistake = 0;
-
-sub mistake {
- my $msg = shift;
- warn $msg;
- $mistake = 1;
- $ret = 1;
-}
-
-while (<>) {
- if (not $file eq $ARGV) {
- if ($state == $STATE_COPYRIGHT_COMMENT) {
- &mistake("$file: missing copyright comment");
- } elsif ($state == $STATE_COPYRIGHT_BLANK) {
- &mistake("$file: missing blank line after copyright header");
- } elsif ($state == $STATE_PRIV_START) {
- &mistake("$file: missing '#ifndef $ifdefpriv'");
- } elsif ($state == $STATE_PRIV_ERROR) {
- &mistake("$file: missing '# error ...priv allow...'");
- } elsif ($state == $STATE_PRIV_END) {
- &mistake("$file: missing '#endif /* $ifdefpriv */'");
- } elsif ($state == $STATE_PRIV_BLANK) {
- &mistake("$file: missing blank line after priv header check");
- } elsif ($state == $STATE_GUARD_START) {
- if ($publicheader) {
- &mistake("$file: missing '#ifndef $ifdef'");
- } else {
- &mistake("$file: missing '#pragma once' header
guard");
- }
- } elsif ($state == $STATE_GUARD_DEFINE) {
- &mistake("$file: missing '# define $ifdef'");
- } elsif ($state == $STATE_GUARD_END) {
- &mistake("$file: missing '#endif /* $ifdef */'");
- }
-
- $ifdef = uc $ARGV;
- $ifdef =~ s,.*/,,;
- $ifdef =~ s,[^A-Z0-9],_,g;
- $ifdef =~ s,__+,_,g;
- unless ($ifdef =~ /^LIBVIRT_/ && $ARGV !~ /libvirt_internal.h/) {
- $ifdef = "LIBVIRT_" . $ifdef;
- }
- $ifdefpriv = $ifdef . "_ALLOW";
-
- $file = $ARGV;
- $state = $STATE_COPYRIGHT_COMMENT;
- $mistake = 0;
- $publicheader = ($ARGV =~ /include\//);
- }
-
- if ($mistake ||
- $ARGV =~ /config-post\.h$/ ||
- $ARGV =~ /vbox_(CAPI|XPCOM)/) {
- $state = $STATE_EOF;
- next;
- }
-
- if ($state == $STATE_COPYRIGHT_COMMENT) {
- if (m,\*/,) {
- $state = $STATE_COPYRIGHT_BLANK;
- }
- } elsif ($state == $STATE_COPYRIGHT_BLANK) {
- if (! /^$/) {
- &mistake("$file: missing blank line after copyright header");
- }
- if ($ARGV =~ /priv\.h$/) {
- $state = $STATE_PRIV_START;
- } else {
- $state = $STATE_GUARD_START;
- }
- } elsif ($state == $STATE_PRIV_START) {
- if (/^$/) {
- &mistake("$file: too many blank lines after copyright
header");
- } elsif (/#ifndef $ifdefpriv$/) {
- $state = $STATE_PRIV_ERROR;
- } else {
- &mistake("$file: missing '#ifndef $ifdefpriv'");
- }
- } elsif ($state == $STATE_PRIV_ERROR) {
- if (/# error ".*"$/) {
- $state = $STATE_PRIV_END;
- } else {
- &mistake("$file: missing '# error ...priv allow...'");
- }
- } elsif ($state == $STATE_PRIV_END) {
- if (m,#endif /\* $ifdefpriv \*/,) {
- $state = $STATE_PRIV_BLANK;
- } else {
- &mistake("$file: missing '#endif /* $ifdefpriv */'");
- }
- } elsif ($state == $STATE_PRIV_BLANK) {
- if (! /^$/) {
- &mistake("$file: missing blank line after priv guard");
- }
- $state = $STATE_GUARD_START;
- } elsif ($state == $STATE_GUARD_START) {
- if (/^$/) {
- &mistake("$file: too many blank lines after copyright
header");
- }
- if ($publicheader) {
- if (/#ifndef $ifdef$/) {
- $state = $STATE_GUARD_DEFINE;
- } else {
- &mistake("$file: missing '#ifndef $ifdef'");
- }
- } else {
- if (/#pragma once/) {
- $state = $STATE_PRAGMA;
- } else {
- &mistake("$file: missing '#pragma once' header
guard");
- }
- }
- } elsif ($state == $STATE_GUARD_DEFINE) {
- if (/# define $ifdef$/) {
- $state = $STATE_GUARD_END;
- } else {
- &mistake("$file: missing '# define $ifdef'");
- }
- } elsif ($state == $STATE_GUARD_END) {
- if (m,#endif /\* $ifdef \*/$,) {
- $state = $STATE_EOF;
- }
- } elsif ($state == $STATE_PRAGMA) {
- next;
- } elsif ($state == $STATE_EOF) {
- die "$file: unexpected content after '#endif /* $ifdef */'";
- } else {
- die "$file: unexpected state $state";
- }
-}
-exit $ret;
diff --git a/build-aux/header-ifdef.py b/build-aux/header-ifdef.py
new file mode 100644
index 0000000000..86c4781a7e
--- /dev/null
+++ b/build-aux/header-ifdef.py
@@ -0,0 +1,206 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2018-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# <http://www.gnu.org/licenses/>;.
+#
+# Validate that header files follow a standard layout:
+#
+# /*
+# ...copyright header...
+# */
+# <one blank line>
+# #pragma once
+# ....content....
+#
+#---
+#
+# For any file ending priv.h, before the #pragma once
+# We will have a further section
+#
+# #ifndef SYMBOL_ALLOW
+# # error ....
+# #endif /* SYMBOL_ALLOW */
+# <one blank line>
+#
+#---
+#
+# For public headers (files in include/), use the standard header guard instead of
#pragma once:
+# #ifndef SYMBOL
+# # define SYMBOL
+# ....content....
+# #endif /* SYMBOL */
+
+from __future__ import print_function
+
+import os.path
+import re
+import sys
+
+STATE_COPYRIGHT_COMMENT = 0
+STATE_COPYRIGHT_BLANK = 1
+STATE_PRIV_START = 2
+STATE_PRIV_ERROR = 3
+STATE_PRIV_END = 4
+STATE_PRIV_BLANK = 5
+STATE_GUARD_START = 6
+STATE_GUARD_DEFINE = 7
+STATE_GUARD_END = 8
+STATE_EOF = 9
+STATE_PRAGMA = 10
+
+def check_header(filename):
+ ifdef = ""
+ ifdefpriv = ""
+
+ state = STATE_EOF
+
+ ifdef = os.path.basename(filename).upper()
+ ifdef = re.sub(r"""[^A-Z0-9]""", "_", ifdef)
+ ifdef = re.sub(r"""__+""", "_", ifdef)
+
+ if not ifdef.startswith("LIBVIRT_") or
filename.find("libvirt_internal.h") != -1:
+ ifdef = "LIBVIRT_" + ifdef
+
+ ifdefpriv = ifdef + "_ALLOW"
+
+ state = STATE_COPYRIGHT_COMMENT
+ publicheader = False
+ if filename.find("include/") != -1:
+ publicheader = True
+
+ with open(filename, "r") as fh:
+ for line in fh:
+ line = line.rstrip("\n")
+ if state == STATE_COPYRIGHT_COMMENT:
+ if line.find("*/") != -1:
+ state = STATE_COPYRIGHT_BLANK
+ elif state == STATE_COPYRIGHT_BLANK:
+ if line != "":
+ print("%s: missing blank line after copyright header" %
filename, file=sys.stderr)
+ return True
+
+ if filename.endswith("priv.h"):
+ state = STATE_PRIV_START
+ else:
+ state = STATE_GUARD_START
+ elif state == STATE_PRIV_START:
+ if line == "":
+ print("%s: too many blank lines after copyright header" %
filename, file=sys.stderr)
+ return True
+ elif re.match(r"""#ifndef %s$""" %
ifdefpriv, line):
+ state = STATE_PRIV_ERROR
+ else:
+ print("%s: missing '#ifndef %s'" % (filename,
ifdefpriv), file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_ERROR:
+ if re.match(r"""# error ".*"$""",
line):
+ state = STATE_PRIV_END
+ else:
+ print("%s: missing '# error ...priv allow...'" %
filename, file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_END:
+ if re.match(r"""#endif /\* %s \*/""" %
ifdefpriv, line):
+ state = STATE_PRIV_BLANK
+ else:
+ print("%s: missing '#endif /* %s */'" % (filename,
ifdefpriv), file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_BLANK:
+ if line != "":
+ print("%s: missing blank line after priv guard" % filename,
file=sys.stderr)
+ return True
+ state = STATE_GUARD_START
+ elif state == STATE_GUARD_START:
+ if line == "":
+ print("%s: too many blank lines after copyright header" %
filename, file=sys.stderr)
+ return True
+ if publicheader:
+ if re.match(r"""#ifndef %s$""" % ifdef,
line):
+ state = STATE_GUARD_DEFINE
+ else:
+ print("%s: missing '#ifndef %s'" % (filename,
ifdef), file=sys.stderr)
+ return True
+ else:
+ if re.match(r"""#pragma once""",
line):
+ state = STATE_PRAGMA
+ else:
+ print("%s: missing '#pragma once' header guard"
% filename, file=sys.stderr)
+ return True
+ elif state == STATE_GUARD_DEFINE:
+ if re.match(r"""# define %s$""" % ifdef,
line):
+ state = STATE_GUARD_END
+ else:
+ print("%s: missing '# define %s'" % (filename,
ifdef), file=sys.stderr)
+ return True
+ elif state == STATE_GUARD_END:
+ if re.match(r"""#endif /\* %s \*/$""" %
ifdef, line):
+ state = STATE_EOF
+ elif state == STATE_PRAGMA:
+ next
+ elif state == STATE_EOF:
+ print("%s: unexpected content after '#endif /* %s */'"
% (filename, ifdef), file=sys.stderr)
+ return True
+ else:
+ print("%s: unexpected state $state" % filename,
file=sys.stderr)
+ return True
+
+
+ if state == STATE_COPYRIGHT_COMMENT:
+ print("%s: missing copyright comment" % filename, file=sys.stderr)
+ return True
+ elif state == STATE_COPYRIGHT_BLANK:
+ print("%s: missing blank line after copyright header" % filename,
file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_START:
+ print("%s: missing '#ifndef %s'" % (filename, ifdefpriv),
file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_ERROR:
+ print("%s: missing '# error ...priv allow...'" % filename,
file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_END:
+ print("%s: missing '#endif /* %s */'" % (filename, ifdefpriv),
file=sys.stderr)
+ return True
+ elif state == STATE_PRIV_BLANK:
+ print("%s: missing blank line after priv header check" % filename,
file=sys.stderr)
+ return True
+ elif state == STATE_GUARD_START:
+ if publicheader:
+ print("%s: missing '#ifndef %s'" % (filename, ifdef),
file=sys.stderr)
+ return True
+ else:
+ print("%s: missing '#pragma once' header guard" % filename,
file=sys.stderr)
+ return True
+ elif state == STATE_GUARD_DEFINE:
+ print("%s: missing '# define %s'" % (filename, ifdef),
file=sys.stderr)
+ return True
+ elif state == STATE_GUARD_END:
+ print("%s: missing '#endif /* %s */'" % (filename, ifdef),
file=sys.stderr)
+ return True
+
+ return False
+
+ret = 0
+
+for filename in sys.argv[1:]:
+ if filename.find("config-post.h") != -1:
+ continue
+ if filename.find("vbox_CAPI") != -1:
+ continue
+ if filename.find("vbox_XPCOM") != -1:
+ continue
+ if check_header(filename):
+ ret = 1
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index 732ee8084c..c398cf9d9f 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1161,7 +1161,7 @@ mock-noinline:
header-ifdef:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/header-ifdef.pl
+ $(PYTHON) $(top_srcdir)/build-aux/header-ifdef.py
test-wrap-argv:
$(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | xargs \
--
2.21.0
6:43 a.m.
On Thu, Sep 05, 2019 at 11:21:55AM +0100, Daniel P. Berrangé wrote:
[snipping the zoo]
Daniel P. Berrangé (11):
cfg.mk: fix comment detection for python semicolon check
docs: rewrite hvsupport.html page generator in python
docs: rewrite ACL permissions checker in Python
docs: rewrite symfile sorting checker in Python
docs: rewrite symfile library checker in Python
docs: rewrite augest test generator in Python
docs: rewrite po file minimizer in Python
docs: rewrite duplicate header checker in Python
docs: rewrite whitespace checker in Python
docs: rewrite mock inline checker in Python
docs: rewrite header ifdef checker in Python
As I said in RE: Language Consolidation,
this could use a preceding HACKING change with justification.
Also, rewrite to a new language does mean throwing away debugged and
optimized code - but if the end goal is to drop automake and configure,
overall the build will be much quicker anyway.
Also, this does not work with Python 3.6 on my Gentoo:
Traceback (most recent call last):
File "./build-aux/prohibit-duplicate-header.py", line 48, in <module>
if check_file(filename):
File "./build-aux/prohibit-duplicate-header.py", line 30, in check_file
for line in fh:
File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 548:
ordinal not in range(128)
makefile_conditionals
po_check
preprocessor_indentation
prohibit_HAVE_MBRTOWC
make: *** [cfg.mk:1150: prohibit-duplicate-header] Error 123
make: *** Waiting for unfinished jobs....
prohibit_PATH_MAX
Traceback (most recent call last):
File "./build-aux/header-ifdef.py", line 203, in <module>
if check_header(filename):
File "./build-aux/header-ifdef.py", line 85, in check_header
for line in fh:
File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 100:
ordinal not in range(128)
make: *** [cfg.mk:1163: header-ifdef] Error 123
Traceback (most recent call last):
File "./build-aux/mock-noinline.py", line 76, in <module>
scan_annotations(filename)
File "./build-aux/mock-noinline.py", line 39, in scan_annotations
for line in fh:
File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 100:
ordinal not in range(128)
make: *** [cfg.mk:1159: mock-noinline] Error 123
Traceback (most recent call last):
File "./build-aux/check-spacing.py", line 201, in <module>
if check_whitespace(filename):
File "./build-aux/check-spacing.py", line 59, in check_whitespace
for line in fh:
File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 548:
ordinal not in range(128)
maint.mk: incorrect formatting
make: *** [cfg.mk:1154: spacing-check] Error 1
Jano
6:48 a.m.
On Thu, Sep 05, 2019 at 01:43:32PM +0200, Ján Tomko wrote:
On Thu, Sep 05, 2019 at 11:21:55AM +0100, Daniel P. Berrangé wrote:
[snipping the zoo]
>
> Daniel P. Berrangé (11):
> cfg.mk: fix comment detection for python semicolon check
> docs: rewrite hvsupport.html page generator in python
> docs: rewrite ACL permissions checker in Python
> docs: rewrite symfile sorting checker in Python
> docs: rewrite symfile library checker in Python
> docs: rewrite augest test generator in Python
> docs: rewrite po file minimizer in Python
> docs: rewrite duplicate header checker in Python
> docs: rewrite whitespace checker in Python
> docs: rewrite mock inline checker in Python
> docs: rewrite header ifdef checker in Python
>
As I said in RE: Language Consolidation,
this could use a preceding HACKING change with justification.
Also, rewrite to a new language does mean throwing away debugged and
optimized code - but if the end goal is to drop automake and configure,
overall the build will be much quicker anyway.
Debugged yes, but I don't think I'd call this optimized code - it
is pretty much all a quick hack.
Also, this does not work with Python 3.6 on my Gentoo:
Traceback (most recent call last):
File "./build-aux/prohibit-duplicate-header.py", line 48, in <module>
if check_file(filename):
File "./build-aux/prohibit-duplicate-header.py", line 30, in check_file
for line in fh:
File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 548:
ordinal not in range(128)
Is your locale ($LANG / $LC_*) not using a UTF-8 character set perhaps ?
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
7:20 a.m.
On Thu, Sep 05, 2019 at 12:48:34PM +0100, Daniel P. Berrangé wrote:
On Thu, Sep 05, 2019 at 01:43:32PM +0200, Ján Tomko wrote:
> On Thu, Sep 05, 2019 at 11:21:55AM +0100, Daniel P. Berrangé wrote:
> [snipping the zoo]
>
> >
> > Daniel P. Berrangé (11):
> > cfg.mk: fix comment detection for python semicolon check
> > docs: rewrite hvsupport.html page generator in python
> > docs: rewrite ACL permissions checker in Python
> > docs: rewrite symfile sorting checker in Python
> > docs: rewrite symfile library checker in Python
> > docs: rewrite augest test generator in Python
> > docs: rewrite po file minimizer in Python
> > docs: rewrite duplicate header checker in Python
> > docs: rewrite whitespace checker in Python
> > docs: rewrite mock inline checker in Python
> > docs: rewrite header ifdef checker in Python
> >
>
> As I said in RE: Language Consolidation,
> this could use a preceding HACKING change with justification.
>
> Also, rewrite to a new language does mean throwing away debugged and
> optimized code - but if the end goal is to drop automake and configure,
> overall the build will be much quicker anyway.
Debugged yes, but I don't think I'd call this optimized code - it
is pretty much all a quick hack.
I stand by that. I actually ran a profiler on some of these back in 2016 :)
> Also, this does not work with Python 3.6 on my Gentoo:
> Traceback (most recent call last):
> File "./build-aux/prohibit-duplicate-header.py", line 48, in
<module>
> if check_file(filename):
> File "./build-aux/prohibit-duplicate-header.py", line 30, in check_file
> for line in fh:
> File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
> return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 548:
ordinal not in range(128)
Is your locale ($LANG / $LC_*) not using a UTF-8 character set perhaps ?
Running 'print (locale.getlocale())' in a manually invoked shell has it:
('en_US', 'UTF-8')
when invoked via 'make prohibit-duplicate-header' the locale is not set:
(None, None)
That seems to be gnulib's fault:
top/maint.mk:156:export LC_ALL = C
Jano
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
--
libvir-list mailing list
libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
7:22 a.m.
On Thu, Sep 05, 2019 at 02:20:55PM +0200, Ján Tomko wrote:
On Thu, Sep 05, 2019 at 12:48:34PM +0100, Daniel P. Berrangé wrote:
> On Thu, Sep 05, 2019 at 01:43:32PM +0200, Ján Tomko wrote:
> > On Thu, Sep 05, 2019 at 11:21:55AM +0100, Daniel P. Berrangé wrote:
> > [snipping the zoo]
> >
> > >
> > > Daniel P. Berrangé (11):
> > > cfg.mk: fix comment detection for python semicolon check
> > > docs: rewrite hvsupport.html page generator in python
> > > docs: rewrite ACL permissions checker in Python
> > > docs: rewrite symfile sorting checker in Python
> > > docs: rewrite symfile library checker in Python
> > > docs: rewrite augest test generator in Python
> > > docs: rewrite po file minimizer in Python
> > > docs: rewrite duplicate header checker in Python
> > > docs: rewrite whitespace checker in Python
> > > docs: rewrite mock inline checker in Python
> > > docs: rewrite header ifdef checker in Python
> > >
> >
> > As I said in RE: Language Consolidation,
> > this could use a preceding HACKING change with justification.
> >
> > Also, rewrite to a new language does mean throwing away debugged and
> > optimized code - but if the end goal is to drop automake and configure,
> > overall the build will be much quicker anyway.
>
> Debugged yes, but I don't think I'd call this optimized code - it
> is pretty much all a quick hack.
>
I stand by that. I actually ran a profiler on some of these back in 2016 :)
> > Also, this does not work with Python 3.6 on my Gentoo:
> > Traceback (most recent call last):
> > File "./build-aux/prohibit-duplicate-header.py", line 48, in
<module>
> > if check_file(filename):
> > File "./build-aux/prohibit-duplicate-header.py", line 30, in
check_file
> > for line in fh:
> > File
"/usr/lib/python-exec/python3.6/../../../lib64/python3.6/encodings/ascii.py",
line 26, in decode
> > return codecs.ascii_decode(input, self.errors)[0]
> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in
position 548: ordinal not in range(128)
>
> Is your locale ($LANG / $LC_*) not using a UTF-8 character set perhaps ?
>
Running 'print (locale.getlocale())' in a manually invoked shell has it:
('en_US', 'UTF-8')
when invoked via 'make prohibit-duplicate-header' the locale is not set:
(None, None)
That seems to be gnulib's fault:
top/maint.mk:156:export LC_ALL = C
Interesting, I wonder why that gnulib thing didn't affect me too.
Anyway, it should be easy enough to fix this to guarantee UTF8
for python
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
1905
days inactive
1905
days old
17 comments
3 participants
participants (3)
-
Daniel P. Berrangé -
Eric Blake -
Ján Tomko