[libvirt] [PATCH] qemu: Add ability to disable autostarting of guests on daemon start
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily. This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel. Unfortunately, there's no simple way we could do this in libvirt. The problem is how to detect that the guest has finished booting as it's mostly a black box for libvirt. There area few options: 1) Wait until I/O load of the guest drops. - This is insufficient on guests that do heavy I/O all the time. 2) Let the guest tell us. - Not every OS has the ability to have a guest agent. 3) Wait for certain amount of time. - Very limited amount of control on diverse guests. With this functionality, the user may write his own script that load balances the autostart in a sane way with accordance to his infrastructure (Query agents, check I/O, use configurable timeouts) while stil being able to mark the domains as autostartable with all the benefits (virsh list --all --autostart). --- We could also add an example script based on parts of libvirt-guests to show how to use this. --- src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_driver.c | 5 ++++- 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index dd853c8..91115bf 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -2,6 +2,13 @@ # All settings described here are optional - if omitted, sensible # defaults are used. +# When the qemu driver starts all guests that are marked to be +# started automaticaly are started (at once). This option disables +# autostarting of the guests. The user can use a script to do so in +# a more sensible way. Eg. wait for guest to start up, before continuing. +# +#disable_autostart = 1 + # VNC is configured to listen on 127.0.0.1 by default. # To make it listen on all public interfaces, uncomment # this next option. diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c index dc4d680..598b34e 100644 --- a/src/qemu/qemu_conf.c +++ b/src/qemu/qemu_conf.c @@ -153,6 +153,10 @@ int qemudLoadDriverConfig(struct qemud_driver *driver, return -1; \ } + p = virConfGetValue(conf, "disable_autostart"); + CHECK_TYPE("disable_autostart", VIR_CONF_LONG); + if (p && p->l) driver->disableAutoStart = true; + p = virConfGetValue(conf, "vnc_auto_unix_socket"); CHECK_TYPE("vnc_auto_unix_socket", VIR_CONF_LONG); if (p) driver->vncAutoUnixSocket = p->l; diff --git a/src/qemu/qemu_conf.h b/src/qemu/qemu_conf.h index 2c7f70c..fb6e317 100644 --- a/src/qemu/qemu_conf.h +++ b/src/qemu/qemu_conf.h @@ -156,6 +156,8 @@ struct qemud_driver { int keepAliveInterval; unsigned int keepAliveCount; int seccompSandbox; + + bool disableAutoStart; }; typedef struct _qemuDomainCmdlineDef qemuDomainCmdlineDef; diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c index 978af57..3b4928f 100644 --- a/src/qemu/qemu_driver.c +++ b/src/qemu/qemu_driver.c @@ -888,7 +888,10 @@ qemudStartup(int privileged) { qemuDriverUnlock(qemu_driver); - qemuAutostartDomains(qemu_driver); + if (!qemu_driver->disableAutoStart) + qemuAutostartDomains(qemu_driver); + else + VIR_WARN("QEMU domain autostarting is disabled"); if (conn) virConnectClose(conn); -- 1.8.0
On 05.11.2012 16:02, Peter Krempa wrote:
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily.
This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel.
Unfortunately, there's no simple way we could do this in libvirt. The problem is how to detect that the guest has finished booting as it's mostly a black box for libvirt. There area few options:
1) Wait until I/O load of the guest drops. - This is insufficient on guests that do heavy I/O all the time.
2) Let the guest tell us. - Not every OS has the ability to have a guest agent.
3) Wait for certain amount of time. - Very limited amount of control on diverse guests.
With this functionality, the user may write his own script that load balances the autostart in a sane way with accordance to his infrastructure (Query agents, check I/O, use configurable timeouts) while stil being able to mark the domains as autostartable with all the benefits (virsh list --all --autostart). --- We could also add an example script based on parts of libvirt-guests to show how to use this. --- src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_driver.c | 5 ++++- 4 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/src/qemu/qemu.conf b/src/qemu/qemu.conf index dd853c8..91115bf 100644 --- a/src/qemu/qemu.conf +++ b/src/qemu/qemu.conf @@ -2,6 +2,13 @@ # All settings described here are optional - if omitted, sensible # defaults are used.
+# When the qemu driver starts all guests that are marked to be +# started automaticaly are started (at once). This option disables +# autostarting of the guests. The user can use a script to do so in +# a more sensible way. Eg. wait for guest to start up, before continuing. +# +#disable_autostart = 1 +
While I agree that disabling autostart may be desired, I wonder if we can introduce command line argument instead/as well, e.g.: libvirtd --disable-autostart I think it's a bit more handy. Michal
On 11/05/2012 08:02 AM, Peter Krempa wrote:
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily.
This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel.
Hmm, this config option effectively renders autostart useless. If that's the case, I wonder if we should then also forbid people from using virDomainSetAutostart to turn autostart on for a given domain. [Reading further - oh, I found your example of 'virsh list --all --autostart] I really wonder if a better solution would be to add something to domain XML that says whether autostart is enabled, and more importantly, gives a delay for how long after libvirtd starts before that particular domain's autostart is acted on. That is, right now, autostart is an invisible property (not present in virDomainGetXMLDesc; you have to use virDomainGetAutostart to learn about it), with just a bool value of on or off. But what if we modified domain XML to display: <autostart enabled='yes' delay='5'/> meaning that this domain is set to autostart, but only after 5 seconds have elapsed since the start of libvirtd?
Unfortunately, there's no simple way we could do this in libvirt. The problem is how to detect that the guest has finished booting as it's mostly a black box for libvirt. There area few options:
1) Wait until I/O load of the guest drops. - This is insufficient on guests that do heavy I/O all the time.
2) Let the guest tell us. - Not every OS has the ability to have a guest agent.
Agree that these two won't work.
3) Wait for certain amount of time. - Very limited amount of control on diverse guests.
Limited, but at least expressible if you take my idea of exposing an autostart delay in the domain XML.
With this functionality, the user may write his own script that load balances the autostart in a sane way with accordance to his infrastructure (Query agents, check I/O, use configurable timeouts) while stil being able to mark the domains as autostartable with all the
s/stil/still/
benefits (virsh list --all --autostart).
If you use the config option to disable autostart, then there is no autostart. But I guess I see where you are coming from for at least allowing the designation, even if libvirtd won't honor it, so that you can at least filter domains based on those that were marked. I also wonder if Michal's suggestion of making it a command-line option rather than a config option is better - how easy or hard is it to inject additional command line options into a systemd startup script? -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 11/05/12 16:48, Eric Blake wrote:
On 11/05/2012 08:02 AM, Peter Krempa wrote:
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily.
This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel.
Hmm, this config option effectively renders autostart useless. If that's the case, I wonder if we should then also forbid people from using virDomainSetAutostart to turn autostart on for a given domain. [Reading further - oh, I found your example of 'virsh list --all --autostart]
I meant this to be exactly this way. The user can use this field as a metadata field for the script or whatever he is using instead of the autostart behavior.
I really wonder if a better solution would be to add something to domain XML that says whether autostart is enabled, and more importantly, gives a delay for how long after libvirtd starts before that particular domain's autostart is acted on.
That is, right now, autostart is an invisible property (not present in virDomainGetXMLDesc; you have to use virDomainGetAutostart to learn about it), with just a bool value of on or off. But what if we modified domain XML to display:
<autostart enabled='yes' delay='5'/>
meaning that this domain is set to autostart, but only after 5 seconds have elapsed since the start of libvirtd?
Hmm, that's strange that we're not exposing the autostart property. I agree that we should add that. For the delay property, the absolute meaning is OK if we assume a management application is sitting on top of libvirt and tweaking those values. But if we have a regular user that doesn't have an automated magic on top of this, then removing machines from one of selected time create holes in this scheduling. Also if the user wishes to allocate more time for a machine to boot, he has to manually shift other guests away.
Unfortunately, there's no simple way we could do this in libvirt. The problem is how to detect that the guest has finished booting as it's mostly a black box for libvirt. There area few options:
1) Wait until I/O load of the guest drops. - This is insufficient on guests that do heavy I/O all the time.
2) Let the guest tell us. - Not every OS has the ability to have a guest agent.
Agree that these two won't work.
Well, unless you know what you're doing. If you know that you have guest agent installed, you might take that into account and actually start the machines after they stop booting (or some timeout expires - 2+3)
3) Wait for certain amount of time. - Very limited amount of control on diverse guests.
Limited, but at least expressible if you take my idea of exposing an autostart delay in the domain XML.
With this functionality, the user may write his own script that load balances the autostart in a sane way with accordance to his infrastructure (Query agents, check I/O, use configurable timeouts) while stil being able to mark the domains as autostartable with all the
s/stil/still/
benefits (virsh list --all --autostart).
If you use the config option to disable autostart, then there is no autostart. But I guess I see where you are coming from for at least allowing the designation, even if libvirtd won't honor it, so that you can at least filter domains based on those that were marked.
And you can use the result of the list query to start them in the script or manualy.
I also wonder if Michal's suggestion of making it a command-line option rather than a config option is better - how easy or hard is it to inject additional command line options into a systemd startup script?
Hm the harder part here is to propagate the values from commandline to the drivers. We'd need to modify the driver registration function to allow us passing arguments. That's the reason I chose qemu.conf.
On Mon, Nov 05, 2012 at 04:02:28PM +0100, Peter Krempa wrote:
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily.
This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel.
Unfortunately, there's no simple way we could do this in libvirt. The problem is how to detect that the guest has finished booting as it's mostly a black box for libvirt. There area few options:
1) Wait until I/O load of the guest drops. - This is insufficient on guests that do heavy I/O all the time.
2) Let the guest tell us. - Not every OS has the ability to have a guest agent.
3) Wait for certain amount of time. - Very limited amount of control on diverse guests.
With this functionality, the user may write his own script that load balances the autostart in a sane way with accordance to his infrastructure (Query agents, check I/O, use configurable timeouts) while stil being able to mark the domains as autostartable with all the benefits (virsh list --all --autostart). --- We could also add an example script based on parts of libvirt-guests to show how to use this. --- src/qemu/qemu.conf | 7 +++++++ src/qemu/qemu_conf.c | 4 ++++ src/qemu/qemu_conf.h | 2 ++ src/qemu/qemu_driver.c | 5 ++++- 4 files changed, 17 insertions(+), 1 deletion(-)
I sort wonder whether it is better to expose this as a command line argument rather than a config option. It would make life much easier for people who want to have autostart running most of the time, but occassionally want to start libvirtd without it. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Nov 06, 2012 at 14:57:30 +0100, Daniel P. Berrange wrote:
On Mon, Nov 05, 2012 at 04:02:28PM +0100, Peter Krempa wrote:
When the daemon is starting it autostarts all guests marked as autostartable. This is not an ideal solution for autostarting if there's a lot of domains configured so. After autostarting all right away the guests start competing for disk I/O and the boot is prolonged unnecessarily.
This patch adds a config option to disable autostarting to leave it on user scripts that may use mechanisms to reduce load of the host by limiting the number of guests started in parallel.
I sort wonder whether it is better to expose this as a command line argument rather than a config option. It would make life much easier for people who want to have autostart running most of the time, but occassionally want to start libvirtd without it.
I also think command line argument is better than a config option. And if someone wants to permanently disable autostarting, we have service configuration files for this (such as /etc/sysconfig/libvirtd, /etc/conf.d/libvirtd, ...). Jirka
participants (5)
-
Daniel P. Berrange -
Eric Blake -
Jiri Denemark -
Michal Privoznik -
Peter Krempa