[libvirt] [PATCH] schematestutils.sh: improve shell portability: avoid "echo -e"

echo -e is not portable. This is the sole remaining use in all of libvirt.
From a58cf340b5ebbca2157f43c6f23d4d6f56b848c7 Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering@redhat.com> Date: Wed, 14 Apr 2010 13:25:46 +0200 Subject: [PATCH] schematestutils.sh: improve shell portability: avoid "echo -e"
* tests/schematestutils.sh: Use printf rather than echo -e. --- tests/schematestutils.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/tests/schematestutils.sh b/tests/schematestutils.sh index f172857..f2b3b50 100644 --- a/tests/schematestutils.sh +++ b/tests/schematestutils.sh @@ -22,7 +22,7 @@ do test_result $n $(basename $(dirname $xml))"/"$(basename $xml) $ret if test "$verbose" = "1" && test $ret != 0 ; then - echo -e "$cmd\n$result" + printf '%s\n' "$cmd" "$result" fi if test "$ret" != 0 ; then f=`expr $f + 1` -- 1.7.1.rc1.248.gcefbb

On Wed, Apr 14, 2010 at 01:28:54PM +0200, Jim Meyering wrote:
echo -e is not portable. This is the sole remaining use in all of libvirt.
From a58cf340b5ebbca2157f43c6f23d4d6f56b848c7 Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering@redhat.com> Date: Wed, 14 Apr 2010 13:25:46 +0200 Subject: [PATCH] schematestutils.sh: improve shell portability: avoid "echo -e"
* tests/schematestutils.sh: Use printf rather than echo -e. --- tests/schematestutils.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tests/schematestutils.sh b/tests/schematestutils.sh index f172857..f2b3b50 100644 --- a/tests/schematestutils.sh +++ b/tests/schematestutils.sh @@ -22,7 +22,7 @@ do
test_result $n $(basename $(dirname $xml))"/"$(basename $xml) $ret if test "$verbose" = "1" && test $ret != 0 ; then - echo -e "$cmd\n$result" + printf '%s\n' "$cmd" "$result" fi if test "$ret" != 0 ; then f=`expr $f + 1`
I fail to see how printf(1) is any more portable than echo which is usually from the shell itself. Sounds to me that echo "$cmd" ; echo "$result" does what we want directly or am I mistaken ? Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/

Daniel Veillard wrote:
On Wed, Apr 14, 2010 at 01:28:54PM +0200, Jim Meyering wrote:
echo -e is not portable. This is the sole remaining use in all of libvirt.
From a58cf340b5ebbca2157f43c6f23d4d6f56b848c7 Mon Sep 17 00:00:00 2001 From: Jim Meyering <meyering@redhat.com> Date: Wed, 14 Apr 2010 13:25:46 +0200 Subject: [PATCH] schematestutils.sh: improve shell portability: avoid "echo -e"
* tests/schematestutils.sh: Use printf rather than echo -e. --- tests/schematestutils.sh | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/tests/schematestutils.sh b/tests/schematestutils.sh index f172857..f2b3b50 100644 --- a/tests/schematestutils.sh +++ b/tests/schematestutils.sh @@ -22,7 +22,7 @@ do
test_result $n $(basename $(dirname $xml))"/"$(basename $xml) $ret if test "$verbose" = "1" && test $ret != 0 ; then - echo -e "$cmd\n$result" + printf '%s\n' "$cmd" "$result" fi if test "$ret" != 0 ; then f=`expr $f + 1`
I fail to see how printf(1) is any more portable than echo which is usually from the shell itself. Sounds to me that echo "$cmd" ; echo "$result" does what we want directly or am I mistaken ?
printf is most definitely more portable than "echo -e". Note that printf has been a shell built-in in just about every shell in common use for at least 10 years. Documentation everywhere has been telling developers to prefer "printf" over "echo-with-any-options" for nearly 20 years. Here's an excerpt from "info coreutils echo": POSIX does not require support for any options, and says that the behavior of `echo' is implementation-defined if any STRING contains a backslash or if the first argument is `-n'. Portable programs can use the `printf' command if they need to omit trailing newlines or output control characters or backslashes. *Note printf invocation::. While my objection was to "echo -e", plain 'echo "$var"' (where $var may be arbitrary) is not terribly reliable. There are plenty of caveats when using plain "echo". I.e., the first argument must not start with a "-". Using printf the way I do above, even if $cmd or $value starts with "-" or contains backslashes, it will be faithfully reproduced. However, if you use your pair of "echo" commands and some $result starts with "-", you'll get misleading output. Here's the section on "echo" in autoconf's "Limitations of Builtins" section: `echo' The simple `echo' is probably the most surprising source of portability troubles. It is not possible to use `echo' portably unless both options and escape sequences are omitted. Don't expect any option. Do not use backslashes in the arguments, as there is no consensus on their handling. For `echo '\n' | wc -l', the `sh' of Solaris outputs 2, but Bash and Zsh (in `sh' emulation mode) output 1. The problem is truly `echo': all the shells understand `'\n'' as the string composed of a backslash and an `n'. Within a command substitution, `echo 'string\c'' will mess up the internal state of ksh88 on AIX 6.1 so that it will print the first character `s' only, followed by a newline, and then entirely drop the output of the next echo in a command substitution. Because of these problems, do not pass a string containing arbitrary characters to `echo'. For example, `echo "$foo"' is safe only if you know that FOO's value cannot contain backslashes and cannot start with `-'. If this may not be true, `printf' is in general safer and easier to use than `echo' and `echo -n'. Thus, scripts where portability is not a major concern should use `printf '%s\n'' whenever `echo' could fail, and similarly use `printf %s' instead of `echo -n'. For portable shell scripts, instead, it is suggested to use a here-document like this: cat <<EOF $foo EOF Alternatively, M4sh provides `AS_ECHO' and `AS_ECHO_N' macros which choose between various portable implementations: `echo' or `print' where they work, `printf' if it is available, or else other creative tricks in order to work around the above problems.

On 04/14/2010 05:28 AM, Jim Meyering wrote:
echo -e is not portable. This is the sole remaining use in all of libvirt.
ACK. How'd you find it? I know we've talked about adding a maint.mk check to help automate some of this (was it over on the bug-grep list?), but I still haven't thought of a good regexp that doesn't suffer from too many false positives (in particular, distinguishing between \ as arguments to echo vs. \ as line continuation can be hard) while still catching the worst offenders. Right now, about the best regex I can think of is: 'echo .?[$-]' which will filter out: echo -n oops echo -e oops echo "$oops" where the first two are blatantly wrong, and the third might be a false positive if you can audit $oops for safety. It would also have a FP on: echo a-b although that is less likely, and it totally misses echo 'a\\b', but you can only do so much. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org

Eric Blake wrote:
On 04/14/2010 05:28 AM, Jim Meyering wrote:
echo -e is not portable. This is the sole remaining use in all of libvirt.
ACK.
Thanks. Pushed.
How'd you find it? I know we've talked about adding a maint.mk check to
I simply used git grep 'echo -e' Maybe that's good enough for now.
help automate some of this (was it over on the bug-grep list?), but I still haven't thought of a good regexp that doesn't suffer from too many false positives (in particular, distinguishing between \ as arguments to echo vs. \ as line continuation can be hard) while still catching the worst offenders. Right now, about the best regex I can think of is:
'echo .?[$-]'
which will filter out:
echo -n oops echo -e oops echo "$oops"
where the first two are blatantly wrong, and the third might be a false positive if you can audit $oops for safety. It would also have a FP on:
echo a-b
although that is less likely, and it totally misses echo 'a\\b', but you can only do so much.

This patch kills translation of the daemon messages, but being a daemon, the messages probably didn't need translation in the first place. Other alternatives would be to make the script require * daemon/libvirtd.init.in (start, stop, reload): Drop bash-ism of $"". Use printf instead of echo -n. --- Just as 'echo -e' is non-portable, so is 'echo -n'. Plus, $"" and #!/bin/sh don't mix. Any opinions on the issue of translating the output of the libvirt daemon, or is this patch okay as-is? daemon/libvirtd.init.in | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/daemon/libvirtd.init.in b/daemon/libvirtd.init.in index 4c8821b..22f3305 100644 --- a/daemon/libvirtd.init.in +++ b/daemon/libvirtd.init.in @@ -54,7 +54,7 @@ fi RETVAL=0 start() { - echo -n $"Starting $SERVICE daemon: " + printf "Starting $SERVICE daemon: " mkdir -p @localstatedir@/cache/libvirt rm -rf @localstatedir@/cache/libvirt/* KRB5_KTNAME=$KRB5_KTNAME daemon --pidfile $PIDFILE --check $SERVICE $PROCESS --daemon $LIBVIRTD_CONFIG_ARGS $LIBVIRTD_ARGS @@ -64,7 +64,7 @@ start() { } stop() { - echo -n $"Stopping $SERVICE daemon: " + printf "Stopping $SERVICE daemon: " killproc -p $PIDFILE $PROCESS RETVAL=$? @@ -82,7 +82,7 @@ restart() { } reload() { - echo -n $"Reloading $SERVICE configuration: " + printf "Reloading $SERVICE configuration: " killproc -p $PIDFILE $PROCESS -HUP RETVAL=$? @@ -106,7 +106,7 @@ case "$1" in [ -f @localstatedir@/lock/subsys/$SERVICE ] && restart || : ;; *) - echo $"Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload|try-restart}" + echo "Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload|try-restart}" exit 2 ;; esac -- 1.6.6.1

On Wed, Apr 14, 2010 at 09:53:30AM -0600, Eric Blake wrote:
This patch kills translation of the daemon messages, but being a daemon, the messages probably didn't need translation in the first place. Other alternatives would be to make the script require
NACK, this current style, with translations is the stadard in Fedora. There isn't any portability issue with echo -n in the initscript, since this initscript is already Fedora/RHEL specific. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Eric Blake wrote:
This patch kills translation of the daemon messages, but being a daemon, the messages probably didn't need translation in the first place. Other alternatives would be to make the script require
* daemon/libvirtd.init.in (start, stop, reload): Drop bash-ism of $"". Use printf instead of echo -n.
IMHO, converting "echo -n" to printf is the way to go, if for no other reason than to set a proper example. However, you might want to keep the $"" for now, if only to remain consistent with the majority of other Fedora /etc/init.d/* scripts. On an F13 system I just checked, those that use that idiom outnumber the others more than 2-to-1: $ grep -l '\$"Startin' /etc/init.d/* |wc -l 59 $ grep -L '\$"Startin' /etc/init.d/* |wc -l 24
Just as 'echo -e' is non-portable, so is 'echo -n'. Plus, $"" and #!/bin/sh don't mix. Any opinions on the issue of translating the output of the libvirt daemon, or is this patch okay as-is?
daemon/libvirtd.init.in | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/daemon/libvirtd.init.in b/daemon/libvirtd.init.in index 4c8821b..22f3305 100644 --- a/daemon/libvirtd.init.in +++ b/daemon/libvirtd.init.in @@ -54,7 +54,7 @@ fi RETVAL=0
start() { - echo -n $"Starting $SERVICE daemon: " + printf "Starting $SERVICE daemon: " mkdir -p @localstatedir@/cache/libvirt rm -rf @localstatedir@/cache/libvirt/* KRB5_KTNAME=$KRB5_KTNAME daemon --pidfile $PIDFILE --check $SERVICE $PROCESS --daemon $LIBVIRTD_CONFIG_ARGS $LIBVIRTD_ARGS @@ -64,7 +64,7 @@ start() { }
stop() { - echo -n $"Stopping $SERVICE daemon: " + printf "Stopping $SERVICE daemon: "
killproc -p $PIDFILE $PROCESS RETVAL=$? @@ -82,7 +82,7 @@ restart() { }
reload() { - echo -n $"Reloading $SERVICE configuration: " + printf "Reloading $SERVICE configuration: "
killproc -p $PIDFILE $PROCESS -HUP RETVAL=$? @@ -106,7 +106,7 @@ case "$1" in [ -f @localstatedir@/lock/subsys/$SERVICE ] && restart || : ;; *) - echo $"Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload|try-restart}" + echo "Usage: $0 {start|stop|status|restart|condrestart|reload|force-reload|try-restart}" exit 2 ;; esac

On 04/14/2010 10:24 AM, Jim Meyering wrote:
Eric Blake wrote:
This patch kills translation of the daemon messages, but being a daemon, the messages probably didn't need translation in the first place. Other alternatives would be to make the script require
* daemon/libvirtd.init.in (start, stop, reload): Drop bash-ism of $"". Use printf instead of echo -n.
IMHO, converting "echo -n" to printf is the way to go, if for no other reason than to set a proper example.
But we're so heavily outnumbered by other scripts at the moment (that is, I repeated your grep /etc/init.d/* test, and the only 2 hits for printf were in awk subscripts), that the better thing to do here would be to file a bug on the upstream template that the init scripts borrow from, and get the template changed, rather than trying to brave it as the odd-man-out example. Daniel's argument that init scripts are only for Fedora, where we can guarantee that #!/bin/sh _is_ bash and therefore supports both echo -n and $"", is pretty convincing, so I'm discarding 1/2 from my queue. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org

* tests/virt-aa-helper-test (testme): Use printf instead. --- tests/virt-aa-helper-test | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/virt-aa-helper-test b/tests/virt-aa-helper-test index 3a2e74b..910c438 100755 --- a/tests/virt-aa-helper-test +++ b/tests/virt-aa-helper-test @@ -112,10 +112,10 @@ testme() { fi fi - echo -n " $outstr: " >$output - echo -n " '$extra_args $args" >$output + printf " $outstr: " >$output + printf " '$extra_args $args" >$output if [ -n "$input" ]; then - echo -n " < $input" >$output + printf " < $input" >$output fi echo "': " >$output set +e @@ -131,7 +131,7 @@ testme() { else echo "FAIL: exited with '$rc'" >$output echo "FAIL: exited with '$rc'" - echo -n " $outstr: " + printf " $outstr: " echo " '$extra_args $args': " errors=$(($errors + 1)) #exit $rc -- 1.6.6.1

Eric Blake wrote:
* tests/virt-aa-helper-test (testme): Use printf instead. --- tests/virt-aa-helper-test | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/tests/virt-aa-helper-test b/tests/virt-aa-helper-test index 3a2e74b..910c438 100755 --- a/tests/virt-aa-helper-test +++ b/tests/virt-aa-helper-test @@ -112,10 +112,10 @@ testme() { fi fi
- echo -n " $outstr: " >$output - echo -n " '$extra_args $args" >$output + printf " $outstr: " >$output + printf " '$extra_args $args" >$output
These are fine if you know that the printf arguments never (and will never) contain the likes of "%" and "\". If you add a "%s" argument, then we don't even have to think about it: printf %s " $outstr: " >$output printf %s " '$extra_args $args" >$output
if [ -n "$input" ]; then - echo -n " < $input" >$output + printf " < $input" >$output fi echo "': " >$output set +e @@ -131,7 +131,7 @@ testme() { else echo "FAIL: exited with '$rc'" >$output echo "FAIL: exited with '$rc'" - echo -n " $outstr: " + printf " $outstr: " echo " '$extra_args $args': " errors=$(($errors + 1)) #exit $rc

On 04/14/2010 10:14 AM, Jim Meyering wrote:
Eric Blake wrote:
* tests/virt-aa-helper-test (testme): Use printf instead. ---
- echo -n " $outstr: " >$output - echo -n " '$extra_args $args" >$output + printf " $outstr: " >$output + printf " '$extra_args $args" >$output
These are fine if you know that the printf arguments never (and will never) contain the likes of "%" and "\". If you add a "%s" argument, then we don't even have to think about it:
printf %s " $outstr: " >$output printf %s " '$extra_args $args" >$output
Okay, I modified 2/2 to use %s as you suggested, and pushed (I've already ditched 1/2, as discussed elsewhere in the thread). -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
participants (4)
-
Daniel P. Berrange
-
Daniel Veillard
-
Eric Blake
-
Jim Meyering