[libvirt] [PATCH] Init script for suspending/resuming guests on shutdown/boot
Example output during shutdown: Running guests on default URI: console, rhel6-1, rhel5-64 Running guests on lxc:/// URI: lxc-shell Running guests on xen:/// URI: error: no hypervisor driver available for xen:/// error: failed to connect to the hypervisor Running guests on vbox+tcp://orkuz/system URI: no running guests. Suspending guests on default URI... Suspending console: done Suspending rhel6-1: done Suspending rhel5-64: done Suspending guests on lxc:/// URI... Suspending lxc-shell: error: Failed to save domain 9cba8bfb-56f4-6589-2d12-8a58c886dd3b state error: this function is not supported by the hypervisor: virDomainManagedSave Note, the "Suspending $guest: " shows progress during the suspend phase if domjobinfo gives meaningful output. Example output during boot: Resuming guests on default URI... Resuming guest rhel6-1: done Resuming guest rhel5-64: done Resuming guest console: done Resuming guests on lxc:/// URI... Resuming guest lxc-shell: already active Configuration used for generating the examples above: URIS='default lxc:/// xen:/// vbox+tcp://orkuz/system' The script uses /var/lib/libvirt/libvirt-guests files to note all active guest it should try to resume on next boot. It's content looks like: default 7f8b9d93-30e1-f0b9-47a7-cb408482654b 085b4c95-5da2-e8e1-712f-6ea6a4156af2 fb4d8360-5305-df3a-2da1-07d682891b8c lxc:/// 9cba8bfb-56f4-6589-2d12-8a58c886dd3b --- daemon/Makefile.am | 16 +++- daemon/libvirt-guests.init.in | 194 +++++++++++++++++++++++++++++++++++++++++ daemon/libvirt-guests.sysconf | 3 + 3 files changed, 209 insertions(+), 4 deletions(-) create mode 100644 daemon/libvirt-guests.init.in create mode 100644 daemon/libvirt-guests.sysconf diff --git a/daemon/Makefile.am b/daemon/Makefile.am index a82e9a9..ed469bf 100644 --- a/daemon/Makefile.am +++ b/daemon/Makefile.am @@ -29,6 +29,8 @@ EXTRA_DIST = \ libvirtd.lxc.logrotate.in \ libvirtd.uml.logrotate.in \ test_libvirtd.aug \ + libvirt-guests.init.in \ + libvirt-guests.sysconf \ $(AVAHI_SOURCES) \ $(DAEMON_SOURCES) @@ -216,21 +218,27 @@ install-logrotate: $(LOGROTATE_CONFS) $(INSTALL_DATA) libvirtd.uml.logrotate $(DESTDIR)$(sysconfdir)/logrotate.d/libvirtd.uml if LIBVIRT_INIT_SCRIPT_RED_HAT -install-init: libvirtd.init +install-init: libvirtd.init libvirt-guests.init mkdir -p $(DESTDIR)$(sysconfdir)/rc.d/init.d $(INSTALL_SCRIPT) libvirtd.init \ $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd + $(INSTALL_SCRIPT) libvirt-guests.init \ + $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests mkdir -p $(DESTDIR)$(sysconfdir)/sysconfig $(INSTALL_SCRIPT) $(srcdir)/libvirtd.sysconf \ $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd + $(INSTALL_SCRIPT) $(srcdir)/libvirt-guests.sysconf \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests uninstall-init: rm -f $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirtd \ - $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd + $(DESTDIR)$(sysconfdir)/sysconfig/libvirtd \ + $(DESTDIR)$(sysconfdir)/rc.d/init.d/libvirt-guests \ + $(DESTDIR)$(sysconfdir)/sysconfig/libvirt-guests -BUILT_SOURCES += libvirtd.init +BUILT_SOURCES += libvirtd.init libvirt-guests.init -libvirtd.init: libvirtd.init.in +%.init: %.init.in $(AM_V_GEN)sed \ -e s!\@localstatedir\@!@localstatedir@!g \ -e s!\@sbindir\@!@sbindir@!g \ diff --git a/daemon/libvirt-guests.init.in b/daemon/libvirt-guests.init.in new file mode 100644 index 0000000..e930115 --- /dev/null +++ b/daemon/libvirt-guests.init.in @@ -0,0 +1,194 @@ +#!/bin/sh + +# the following is the LSB init header see +# http://www.linux-foundation.org/spec//booksets/LSB-Core-generic/LSB-Core-gen... +# +### BEGIN INIT INFO +# Provides: libvirt-guests +# Required-Start: libvirtd +# Required-Stop: libvirtd +# Default-Start: 3 4 5 +# Short-Description: suspend/resume libvirt guests on shutdown/boot +# Description: This is a script for suspending active libvirt guests +# on shutdown and resuming them on next boot +# See http://libvirt.org +### END INIT INFO + +# the following is chkconfig init header +# +# libvirt-guests: suspend/resume libvirt guests on shutdown/boot +# +# chkconfig: 345 98 02 +# description: This is a script for suspending active libvirt guests +# on shutdown and resuming them on next boot +# See http://libvirt.org +# + +sysconfdir=@sysconfdir@ +localstatedir=@localstatedir@ + +# Source function library. +. $sysconfdir/rc.d/init.d/functions + +URIS=default + +test -f $sysconfdir/sysconfig/libvirt-guests && . $sysconfdir/sysconfig/libvirt-guests + +LISTFILE=$localstatedir/lib/libvirt/libvirt-guests + +RETVAL=0 + +retval() { + "$@" + if [ $? -ne 0 ]; then + RETVAL=1 + return 1 + else + return 0 + fi +} + +run_virsh() { + uri=$1 + shift + + if [ "x$uri" = xdefault ]; then + conn= + else + conn="-c $uri" + fi + + virsh $conn "$@" +} + +run_virsh_c() { + ( export LC_ALL=C; run_virsh "$@" ) +} + +list_guests() { + uri=$1 + + list=`run_virsh_c $uri list` + if [ $? -ne 0 ]; then + RETVAL=1 + return 1 + fi + + for id in `echo "$list" | awk 'NR > 2 {print $1}'`; do + run_virsh_c $uri dominfo $id | awk '/^UUID:/{print $2}' + done +} + +guest_name() { + uri=$1 + uuid=$2 + + name=`run_virsh_c $uri dominfo $uuid 2>/dev/null | \ + awk '/^Name:/{print $2}'` + [ -n "$name" ] || name=$uuid + + echo "$name" +} + +guest_is_on() { + uri=$1 + uuid=$2 + + id=`run_virsh_c $uri dominfo $uuid 2>/dev/null | \ + awk '/^Id:/{print $2}'` + + [ -n "$id" ] && ! [ "x$id" = x- ] +} + +start() { + while read uri list; do + configured=false + for confuri in $URIS; do + if [ $confuri = $uri ]; then + configured=true + break + fi + done + if ! $configured; then + echo $"Ignoring guests on $uri URI" + continue + fi + + echo $"Resuming guests on $uri URI..." + for guest in $list; do + name=`guest_name $uri $guest` + echo -n $"Resuming guest $name: " + if retval guest_is_on $uri $guest; then + echo $"already active" + else + retval run_virsh $uri start "$name" >/dev/null && echo $"done" + fi + done + done <$LISTFILE + + rm -f $LISTFILE +} + +stop() { + >$LISTFILE + for uri in $URIS; do + echo -n $"Running guests on $uri URI: " + list=`list_guests $uri` + if [ $? -eq 0 ]; then + empty=true + for uuid in $list; do + $empty || echo -n ", " + echo -n `guest_name $uri $uuid` + empty=false + done + if $empty; then + echo $"no running guests." + else + echo + echo $uri $list >>$LISTFILE + fi + fi + done + + while read uri list; do + echo $"Suspending guests on $uri URI..." + for guest in $list; do + name=`guest_name $uri $guest` + label=$"Suspending $name: " + echo -n "$label" + run_virsh $uri managedsave $guest >/dev/null & + virsh_pid=$! + while true; do + sleep .5 + kill -0 $virsh_pid >&/dev/null || break + progress=`run_virsh_c $uri domjobinfo $guest 2>/dev/null | \ + awk '/^Data processed:/{print $3, $4}'` + if [ -n "$progress" ]; then + printf '\r%s%12s ' "$label" "$progress" + else + printf '\r%s%-12s ' "$label" "..." + fi + done + retval wait $virsh_pid && printf '\r%s%-12s\n' "$label" $"done" + done + done <$LISTFILE +} + +gueststatus() { + for uri in $URIS; do + echo "* $uri URI:" + retval run_virsh $uri list || echo + done +} + +# See how we were called. +case "$1" in + start|stop|gueststatus) + $1 + ;; + *) + echo $"Usage: $0 {start|stop|gueststatus}" + exit 3 + ;; +esac +exit $RETVAL diff --git a/daemon/libvirt-guests.sysconf b/daemon/libvirt-guests.sysconf new file mode 100644 index 0000000..eb70ebf --- /dev/null +++ b/daemon/libvirt-guests.sysconf @@ -0,0 +1,3 @@ +# specify URIs to check for running guests +# example: URIS='default xen:/// vbox+tcp://host/system lxc:///' +#URIS=default -- 1.7.1
On 05/14/2010 07:50 AM, Jiri Denemark wrote:
Example output during shutdown:
Running guests on default URI: console, rhel6-1, rhel5-64 Running guests on lxc:/// URI: lxc-shell Running guests on xen:/// URI: error: no hypervisor driver available for xen:/// error: failed to connect to the hypervisor Running guests on vbox+tcp://orkuz/system URI: no running guests. Suspending guests on default URI... Suspending console: done Suspending rhel6-1: done Suspending rhel5-64: done Suspending guests on lxc:/// URI... Suspending lxc-shell: error: Failed to save domain 9cba8bfb-56f4-6589-2d12-8a58c886dd3b state error: this function is not supported by the hypervisor: virDomainManagedSave
Nice recipes for running multiple guests - I'll have to branch out on my machine and run some non-qemu guests.
+++ b/daemon/libvirt-guests.init.in @@ -0,0 +1,194 @@ +#!/bin/sh
This script has bash-isms, but this situation is no different than daemon/libvirtd.init.in - since we know that init scripts only run on Fedora/RHEL systems where we are guaranteed that /bin/sh==bash, there's no issue.
+ +# the following is the LSB init header see +# http://www.linux-foundation.org/spec//booksets/LSB-Core-generic/LSB-Core-gen...
Is the second // necessary, or can it just be spec/booksets? For that matter, that URL didn't work for me; I found: http://refspecs.freestandards.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generi...
+ +sysconfdir=@sysconfdir@ +localstatedir=@localstatedir@ + +# Source function library. +. $sysconfdir/rc.d/init.d/functions
Technically, it would be safer to quote $sysconfdir, even if in practice, it never contains whitespace.
+ +URIS=default + +test -f $sysconfdir/sysconfig/libvirt-guests && . $sysconfdir/sysconfig/libvirt-guests
Likewise. Also, should we fail if sourcing the config file failed?
+ +run_virsh() { + uri=$1 + shift + + if [ "x$uri" = xdefault ]; then + conn= + else + conn="-c $uri" + fi + + virsh $conn "$@"
Fails if $uri contains spaces, but that's not a valid URI in the first place, so no big deal.
+} + +run_virsh_c() { + ( export LC_ALL=C; run_virsh "$@" ) +} + +list_guests() { + uri=$1 + + list=`run_virsh_c $uri list`
While bashisms in init scripts is questionable, use of POSIX is not; you can safely use $() instead of `` for readability.
+ if [ $? -ne 0 ]; then + RETVAL=1 + return 1 + fi + + for id in `echo "$list" | awk 'NR > 2 {print $1}'`; do + run_virsh_c $uri dominfo $id | awk '/^UUID:/{print $2}' + done
Failure to run virsh doesn't affect our exit status? Maybe that's okay, but even so, we should probably log if a virsh invocation unexpectedly fails.
+} + +guest_name() { + uri=$1 + uuid=$2 + + name=`run_virsh_c $uri dominfo $uuid 2>/dev/null | \ + awk '/^Name:/{print $2}'` + [ -n "$name" ] || name=$uuid + + echo "$name" +} + +guest_is_on() { + uri=$1 + uuid=$2 + + id=`run_virsh_c $uri dominfo $uuid 2>/dev/null | \ + awk '/^Id:/{print $2}'` + + [ -n "$id" ] && ! [ "x$id" = x- ]
If we were worried about portability to non-POSIX, I would have written this as '[ "x$id" != x- ]' rather than '! [ "x$id" = x- ]', but either way works here since we assume POSIX.
+start() { + while read uri list; do + configured=false + for confuri in $URIS; do + if [ $confuri = $uri ]; then + configured=true + break + fi + done + if ! $configured; then + echo $"Ignoring guests on $uri URI" + continue + fi + + echo $"Resuming guests on $uri URI..." + for guest in $list; do + name=`guest_name $uri $guest` + echo -n $"Resuming guest $name: "
Bash-isms. Neither 'echo -n' nor $"" are portable, but as this style is used in LOTS of other places for i18n of init scripts, you are no worse than existing style (that is, fixing all init scripts to use /bin/bash instead of /bin/sh, or to use alternative constructs that are portable to POSIX, is outside the scope of this patch).
+ if retval guest_is_on $uri $guest; then + echo $"already active" + else + retval run_virsh $uri start "$name" >/dev/null && echo $"done" + fi + done + done <$LISTFILE
This fails if $LISTFILE does not exist, which will be the case if you run start() twice in a row. But LSB requires that a start() on an already started service be a successful no-op. Maybe all you need is a line at the start of the function: test -f $LISTFILE || return 0
+ + rm -f $LISTFILE +} + +stop() { + >$LISTFILE
Bash-ism; trivial to use ': >$LISTFILE' instead to be portable to POSIX.
+ for uri in $URIS; do + echo -n $"Running guests on $uri URI: " + list=`list_guests $uri` + if [ $? -eq 0 ]; then + empty=true + for uuid in $list; do + $empty || echo -n ", "
Another echo -n. Unlike the $"" case (where the style is pervasive), here, I'd like to see the use of 'printf ", "'.
+ echo -n `guest_name $uri $uuid` + empty=false + done + if $empty; then + echo $"no running guests." + else + echo + echo $uri $list >>$LISTFILE + fi + fi + done + + while read uri list; do + echo $"Suspending guests on $uri URI..." + for guest in $list; do + name=`guest_name $uri $guest` + label=$"Suspending $name: " + echo -n "$label" + run_virsh $uri managedsave $guest >/dev/null & + virsh_pid=$! + while true; do + sleep .5
sleep .5 is a GNU-ism, but both coreutils and busybox support it, so I think you're okay.
+ kill -0 $virsh_pid >&/dev/null || break + progress=`run_virsh_c $uri domjobinfo $guest 2>/dev/null | \ + awk '/^Data processed:/{print $3, $4}'` + if [ -n "$progress" ]; then + printf '\r%s%12s ' "$label" "$progress" + else + printf '\r%s%-12s ' "$label" "..." + fi
Don't you also need to print the ANSI sequence for clear-to-eol, so that a shorter line overwriting a previous longer line doesn't leave garbage from the longer line?
+ done + retval wait $virsh_pid && printf '\r%s%-12s\n' "$label" $"done" + done + done <$LISTFILE +} + +gueststatus() { + for uri in $URIS; do + echo "* $uri URI:" + retval run_virsh $uri list || echo + done +} + +# See how we were called. +case "$1" in + start|stop|gueststatus) + $1 + ;; + *) + echo $"Usage: $0 {start|stop|gueststatus}" + exit 3 + ;; +esac
Missing restart, force-reload, and status actions, per the LSB. Also, status should be logged prior to exit, with log_*_msg. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
+ +# the following is the LSB init header see +# http://www.linux-foundation.org/spec//booksets/LSB-Core-generic/LSB-Core-gen...
Is the second // necessary, or can it just be spec/booksets? For that matter, that URL didn't work for me; I found:
http://refspecs.freestandards.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generi...
Hmm, just copied from libvirtd.init.in, better remove it completely I guess.
+sysconfdir=@sysconfdir@ +localstatedir=@localstatedir@ + +# Source function library. +. $sysconfdir/rc.d/init.d/functions
Technically, it would be safer to quote $sysconfdir, even if in practice, it never contains whitespace.
Done.
+ +URIS=default + +test -f $sysconfdir/sysconfig/libvirt-guests && . $sysconfdir/sysconfig/libvirt-guests
Likewise. Also, should we fail if sourcing the config file failed?
Hmm, not sure what's the policy for this. libvirtd.init.in does exactly this.
+run_virsh() { + uri=$1 + shift + + if [ "x$uri" = xdefault ]; then + conn= + else + conn="-c $uri" + fi + + virsh $conn "$@"
Fails if $uri contains spaces, but that's not a valid URI in the first place, so no big deal.
Yes, exactly. Also we have a space-separated list of URIs in URIS...
+} + +run_virsh_c() { + ( export LC_ALL=C; run_virsh "$@" ) +} + +list_guests() { + uri=$1 + + list=`run_virsh_c $uri list`
While bashisms in init scripts is questionable, use of POSIX is not; you can safely use $() instead of `` for readability.
Cool, I wasn't sure about this one... I also prefer $().
+ if [ $? -ne 0 ]; then + RETVAL=1 + return 1 + fi + + for id in `echo "$list" | awk 'NR > 2 {print $1}'`; do + run_virsh_c $uri dominfo $id | awk '/^UUID:/{print $2}' + done
Failure to run virsh doesn't affect our exit status?
In v2 it does.
+guest_is_on() { + uri=$1 + uuid=$2 + + id=`run_virsh_c $uri dominfo $uuid 2>/dev/null | \ + awk '/^Id:/{print $2}'` + + [ -n "$id" ] && ! [ "x$id" = x- ]
If we were worried about portability to non-POSIX, I would have written this as '[ "x$id" != x- ]' rather than '! [ "x$id" = x- ]', but either way works here since we assume POSIX.
OK, fixed. I actually prefer the != way.
This fails if $LISTFILE does not exist, which will be the case if you run start() twice in a row. But LSB requires that a start() on an already started service be a successful no-op. Maybe all you need is a line at the start of the function: test -f $LISTFILE || return 0
Fixed.
+stop() { + >$LISTFILE
Bash-ism; trivial to use ': >$LISTFILE' instead to be portable to POSIX.
Fixed.
+ for uri in $URIS; do + echo -n $"Running guests on $uri URI: " + list=`list_guests $uri` + if [ $? -eq 0 ]; then + empty=true + for uuid in $list; do + $empty || echo -n ", "
Another echo -n. Unlike the $"" case (where the style is pervasive), here, I'd like to see the use of 'printf ", "'.
OK, if it makes you feel better ;-)
+ kill -0 $virsh_pid >&/dev/null || break + progress=`run_virsh_c $uri domjobinfo $guest 2>/dev/null | \ + awk '/^Data processed:/{print $3, $4}'` + if [ -n "$progress" ]; then + printf '\r%s%12s ' "$label" "$progress" + else + printf '\r%s%-12s ' "$label" "..." + fi
Don't you also need to print the ANSI sequence for clear-to-eol, so that a shorter line overwriting a previous longer line doesn't leave garbage from the longer line?
Hmm, not sure how well it would work with various terminals and serial consoles. I chose enough space so that it shouldn't happen.
Missing restart, force-reload, and status actions, per the LSB. Also, status should be logged prior to exit, with log_*_msg.
Fixed. Thanks for reviewing, I'll send v2 with the fixes and enhancements later once I finish some testing. Jirka
On 05/18/2010 04:54 AM, Jiri Denemark wrote:
+test -f $sysconfdir/sysconfig/libvirt-guests && . $sysconfdir/sysconfig/libvirt-guests
Likewise. Also, should we fail if sourcing the config file failed?
Hmm, not sure what's the policy for this. libvirtd.init.in does exactly this.
Well, sounds like I should do an audit of libvirtd.init.in for potential cleanups, then :)
Thanks for reviewing, I'll send v2 with the fixes and enhancements later once I finish some testing.
Looking forward to it. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Hi, just an idea: wouldn't it make sense to add a global configure switch allowing to shut down the vms instead of suspending them? E.g. in /etc/sysconfig/libvirtd: ON_SHUTDOWN=suspend (this is the default) or ON_SHUTDOWN=shutdown This could be made available through the init script too: /etc/init.d/libvirt-guests shutdown Usecase: I want to be able to make sure the diskspace is unmounted and the filesystems of my vms are clean state. This is needed e.g. before copying the raw blockdevices. Another thing is timing: a shutdown is sometimes a very time-constrained thing (think of an ups running out of power). Depending on the programs in the vms, memory size and disk speed, suspending can take longer than a clean shutdown. Kind regards, Gerd
just an idea: wouldn't it make sense to add a global configure switch allowing to shut down the vms instead of suspending them?
E.g. in /etc/sysconfig/libvirtd:
ON_SHUTDOWN=suspend (this is the default) or ON_SHUTDOWN=shutdown
This could be made available through the init script too: /etc/init.d/libvirt-guests shutdown
That sounds like a good idea. Not as simple as it sounds but I implemented it in v2, which I will send later today. Jirka
participants (3)
-
Eric Blake -
Gerd v. Egidy -
Jiri Denemark