[libvirt] [PATCH 0/2] tests: Add nodeinfo test data utility scripts

Both scripts can be useful when adding new test cases to the nodeinfo test. Andrea Bolognani (2): tests: Add script to display nodeinfo test data tests: Add script to copy nodeinfo test data from host tests/nodeinfodata/copy-from-host.sh | 170 +++++++++++++++++++++++++++++++++++ tests/nodeinfodata/display.sh | 101 +++++++++++++++++++++ 2 files changed, 271 insertions(+) create mode 100755 tests/nodeinfodata/copy-from-host.sh create mode 100755 tests/nodeinfodata/display.sh -- 2.4.3

--- tests/nodeinfodata/display.sh | 101 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100755 tests/nodeinfodata/display.sh diff --git a/tests/nodeinfodata/display.sh b/tests/nodeinfodata/display.sh new file mode 100755 index 0000000..bc260e2 --- /dev/null +++ b/tests/nodeinfodata/display.sh @@ -0,0 +1,101 @@ +#!/bin/sh + +# display.sh - Display nodeinfo test data in a nice way + +# Copyright (C) 2015 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +die() { + typeset message=$1 + + echo "$message" >&2 + + exit 1 +} + +list_cpus() { + typeset datadir=$1 + + ls -d "$datadir/cpu/cpu"* | while read o; do + o=${o#*cpu/cpu} + case "$o" in + [0-9]*) + echo "$o" + ;; + esac + done | sort -un +} + +is_online() { + typeset datadir=$1 + typeset cpu=$2 + + typeset ret=1 + + if test -e "$datadir/cpu/cpu$cpu/online"; then + ret=$(cat "$datadir/cpu/cpu$cpu/online") + fi + + # Reverse boolean to use it as return code + case "$ret" in + 0) ret=1 ;; + 1) ret=0 ;; + esac + + return $ret; +} + +main() { + typeset datadir=$1 + typeset threads_per_core=$2 + + if ! test "$#" -eq "2"; then + die "Usage: $SELF DATADIR THREADS_PER_CORE" + fi + if ! test -d "$datadir"; then + die "$datadir: No such directory" + fi + + echo "Threads per core: $threads_per_core" + + echo -n 'Present CPUs: ' + if test -e "$datadir/cpu/present"; then + cat "$datadir/cpu/present" + else + echo 'information not available' + fi + + for cpu in $(list_cpus "$datadir"); do + + mod=$(expr "$cpu" % "$threads_per_core") + if test "$mod" = "0"; then + echo + fi + + if is_online "$datadir" "$cpu"; then + printf " %3d*" "$cpu" + else + printf " %3d " "$cpu" + fi + done + echo + + return 0 +} + +SELF=$0 +main "$@" +exit $? -- 2.4.3

Files we don't need, including symbolic links that might result problematic to make dist, are removed from the copied data. --- tests/nodeinfodata/copy-from-host.sh | 170 +++++++++++++++++++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100755 tests/nodeinfodata/copy-from-host.sh diff --git a/tests/nodeinfodata/copy-from-host.sh b/tests/nodeinfodata/copy-from-host.sh new file mode 100755 index 0000000..b5c5e8e --- /dev/null +++ b/tests/nodeinfodata/copy-from-host.sh @@ -0,0 +1,170 @@ +#!/bin/sh + +# copy-from-host.sh - Copy nodeinfo test data from a running host + +# Copyright (C) 2015 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +SYSFS_PATH="/sys/devices/system" +KEEP_LINKS="cpu[0-9]*" +KEEP_DIRECTORIES="cpu + cpu[0-9]* + node + node[0-9]* + topology" +KEEP_FILES="core_id + core_siblings* + kernel_max + meminfo + offline + online + physical_package_id + possible + present + thread_siblings*" + +die() { + typeset message=$1 + + echo "$message" >&2 + + exit 1 +} + +keep() { + typeset tag=$1 + typeset path=$2 + + echo " $tag $path" + + return 0 +} + +delete() { + typeset tag=$1 + typeset path=$2 + + rm -rf "$path" + + return $? +} + +is_valid_name() { + typeset name=$1 + typeset ret=0 + + case "$name" in + */*) + # We don't want to have to deal with subdirectories, so + # names containing slashes are rejected + ret=1 + ;; + esac + + return $ret +} + +matches_any_pattern() { + typeset name=$1 + typeset patterns=$2 + typeset ret=1 + + for pattern in $patterns; do + case "$name" in + $pattern) + ret=0 + ;; + esac + done + + return $ret +} + +main() { + typeset name=$1 + + if ! test "$name"; then + die "Usage: $SELF NAME" + fi + + if ! is_valid_name "$name"; then + die "Name '$name' is not valid" + fi + + # Create directory + if test -e "$name"; then + die "$name: File or directory already exists" + fi + + if ! mkdir "$name" >/dev/null 2>&1; then + die "$name: Unable to create directory" + fi + + echo "Copying host data..." + + # Copy data from host. Errors are ignored because some files we don't + # care about always give input/output error when read + cp -r "$SYSFS_PATH/cpu" "$name/" >/dev/null 2>&1 + cp -r "$SYSFS_PATH/node" "$name/" >/dev/null 2>&1 + + if ! test -d "$name/cpu" || ! test -d "$name/node"; then + die "Error while copying host data" + fi + + echo "Cleaning up data..." + + # Delete symbolic links + find "$name" -type l | while read l; do + b=${l##*/} + if matches_any_pattern "$b" "$KEEP_LINKS"; then + keep "l" "$l" + else + delete "l" "$l" + fi + done + + # Delete directories + find "$name" -type d | while read d; do + b=${d##*/} + # We don't want to delete the directory we just created :) + if test "$b" = "$name"; then + continue + fi + if matches_any_pattern "$b" "$KEEP_DIRECTORIES"; then + keep "d" "$d" + else + delete "d" "$d" + fi + done + + # Delete files + find "$name" -type f | while read f; do + b=${f##*/} + if matches_any_pattern "$b" "$KEEP_FILES"; then + keep "f" "$f" + else + delete "f" "$f" + fi + done + + echo "All done" + + return 0 +} + +SELF=$0 +main "$@" +exit $? -- 2.4.3

On Mon, Jul 20, 2015 at 11:24:51AM +0200, Andrea Bolognani wrote:
Files we don't need, including symbolic links that might result problematic to make dist, are removed from the copied data. --- tests/nodeinfodata/copy-from-host.sh | 170 +++++++++++++++++++++++++++++++++++
Since this script is not part of the _data_ for nodeinfo test, I'd rather put it in the tests/ directory. But I don't have a strict preference.
1 file changed, 170 insertions(+) create mode 100755 tests/nodeinfodata/copy-from-host.sh
diff --git a/tests/nodeinfodata/copy-from-host.sh b/tests/nodeinfodata/copy-from-host.sh new file mode 100755 index 0000000..b5c5e8e --- /dev/null +++ b/tests/nodeinfodata/copy-from-host.sh @@ -0,0 +1,170 @@ +#!/bin/sh + +# copy-from-host.sh - Copy nodeinfo test data from a running host + +# Copyright (C) 2015 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# <http://www.gnu.org/licenses/>. + +SYSFS_PATH="/sys/devices/system" +KEEP_LINKS="cpu[0-9]*" +KEEP_DIRECTORIES="cpu + cpu[0-9]* + node + node[0-9]* + topology" +KEEP_FILES="core_id + core_siblings* + kernel_max + meminfo + offline + online + physical_package_id + possible + present + thread_siblings*" + +die() { + typeset message=$1 + + echo "$message" >&2 + + exit 1 +} + +keep() { + typeset tag=$1 + typeset path=$2 + + echo " $tag $path" + + return 0 +} + +delete() { + typeset tag=$1 + typeset path=$2 + + rm -rf "$path" + + return $? +} + +is_valid_name() { + typeset name=$1 + typeset ret=0 + + case "$name" in + */*) + # We don't want to have to deal with subdirectories, so + # names containing slashes are rejected + ret=1 + ;; + esac + + return $ret +} + +matches_any_pattern() { + typeset name=$1 + typeset patterns=$2 + typeset ret=1 +
I'm not sure why typeset is better then (or compatible as): name="$1" and somewhere the parameters naming almost doubles the function length, but this is just a support script and not something we call on all machines, so I'm OK with keeping these as they are.
+ for pattern in $patterns; do + case "$name" in + $pattern) + ret=0 + ;; + esac + done + + return $ret +} + +main() { + typeset name=$1 + + if ! test "$name"; then + die "Usage: $SELF NAME" + fi + + if ! is_valid_name "$name"; then + die "Name '$name' is not valid" + fi + + # Create directory + if test -e "$name"; then + die "$name: File or directory already exists" + fi + + if ! mkdir "$name" >/dev/null 2>&1; then + die "$name: Unable to create directory" + fi + + echo "Copying host data..." + + # Copy data from host. Errors are ignored because some files we don't + # care about always give input/output error when read + cp -r "$SYSFS_PATH/cpu" "$name/" >/dev/null 2>&1 + cp -r "$SYSFS_PATH/node" "$name/" >/dev/null 2>&1 +
If there are errors we don't care about, why don't you just copy the files that match wither KEEP_LINKS or KEEP_DIRECTORIES? You can check that all of them copied correctly and you don't need to redirect the output, so any possible error gives a clue on what's wrong.
+ if ! test -d "$name/cpu" || ! test -d "$name/node"; then + die "Error while copying host data" + fi + + echo "Cleaning up data..." + + # Delete symbolic links + find "$name" -type l | while read l; do + b=${l##*/} + if matches_any_pattern "$b" "$KEEP_LINKS"; then + keep "l" "$l" + else + delete "l" "$l" + fi + done + + # Delete directories + find "$name" -type d | while read d; do + b=${d##*/} + # We don't want to delete the directory we just created :) + if test "$b" = "$name"; then + continue + fi + if matches_any_pattern "$b" "$KEEP_DIRECTORIES"; then + keep "d" "$d" + else + delete "d" "$d" + fi + done + + # Delete files + find "$name" -type f | while read f; do + b=${f##*/} + if matches_any_pattern "$b" "$KEEP_FILES"; then + keep "f" "$f" + else + delete "f" "$f" + fi + done + + echo "All done" + + return 0 +} + +SELF=$0 +main "$@" +exit $? -- 2.4.3
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

Wow, I had forgotten about this one :) On Thu, 2015-08-06 at 14:38 +0200, Martin Kletzander wrote:
Since this script is not part of the _data_ for nodeinfo test, I'd rather put it in the tests/ directory. But I don't have a strict preference.
The scripts are not themselves test data, but they're not test cases either, and they work on test data. So I'd rather have them inside nodeinfodata/, and since you don't feel too strongly about that I havent' moved them :)
+matches_any_pattern() { + typeset name=$1 + typeset patterns=$2 + typeset ret=1
I'm not sure why typeset is better then (or compatible as):
name="$1"
and somewhere the parameters naming almost doubles the function length, but this is just a support script and not something we call on all machines, so I'm OK with keeping these as they are.
Naming the parameters, even though it increases the script's size, improves its readability. Using typeset makes parameters and other variables local to the function, instead of messing with the caller's environment - which is a terrible default, by the way. But it's not very portable indeed (works on bash and ksh) so I've just dropped any pretense that the scripts are pure POSIX sh by changing the sha-bang line to explicitly use bash and switched from typeset to local.
+ # Copy data from host. Errors are ignored because some files we don't + # care about always give input/output error when read + cp -r "$SYSFS_PATH/cpu" "$name/" >/dev/null 2>&1 + cp -r "$SYSFS_PATH/node" "$name/" >/dev/null 2>&1 +
If there are errors we don't care about, why don't you just copy the files that match wither KEEP_LINKS or KEEP_DIRECTORIES? You can check that all of them copied correctly and you don't need to redirect the output, so any possible error gives a clue on what's wrong.
Very good point. I've changed the script to just copy the data we're interested in instead of copying everything and cleaning it up afterwards, and would you believe it? It's not only easier to understand and more robust, but also quite a bit shorter :) I'll send a v2 right away. Cheers. -- Andrea Bolognani Software Engineer - Virtualization Team

On Mon, Jul 20, 2015 at 11:24:49AM +0200, Andrea Bolognani wrote:
Both scripts can be useful when adding new test cases to the nodeinfo test.
Andrea Bolognani (2): tests: Add script to display nodeinfo test data tests: Add script to copy nodeinfo test data from host
Unless someone disagrees in a while or unless you're not wiling to do the optional amendments mentioned, weak-ACK series.
tests/nodeinfodata/copy-from-host.sh | 170 +++++++++++++++++++++++++++++++++++ tests/nodeinfodata/display.sh | 101 +++++++++++++++++++++ 2 files changed, 271 insertions(+) create mode 100755 tests/nodeinfodata/copy-from-host.sh create mode 100755 tests/nodeinfodata/display.sh
-- 2.4.3
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
participants (2)
-
Andrea Bolognani
-
Martin Kletzander