10:51 p.m.
Is there a way that you can set the guests startup order and stagger
their startup at boot?
Regards,
Gerry
4:50 a.m.
On Mon, May 25, 2009 at 11:51:07PM -0400, Gerry Reno wrote:
Is there a way that you can set the guests startup order and stagger
their startup at boot?
Hum, no. We only have the autostart flag on the domain but nothing
about ordering of starts on a wider basis.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
8:49 a.m.
Quick intro since this is my first post to the list: My name is
Michael Ansel, and I am third year Electrical and Computer Engineering
/ Computer Science double major at Duke University in Durham, North
Carolina. I've done a little bit of work off and on over the past
several years with virtualization on Linux, and my
project/self-inflicted torture for the summer is to put together a
stable, object-oriented wrapper for the libvirt ruby bindings so that
VM management is extremely easy from within ruby (though... I haven't
quite gotten that idea off the ground).
Anyways, enough about me and onto the issue at hand:
How hard would something like this (staggered boot order) be to
add/integrate with an optional, external config file (or as a
[true|false|1|2|3|..]. style directive in the main XML)? I'd be
interested in putting something together if it is a feasible addition
and someone can point me to the general area of the code that needs to
be changed. Also, is there an indicator in libvirt for when a domain
has finished booting, or would this boot staggering need to be time
based? Thanks!
Michael
On Tue, May 26, 2009 at 5:50 AM, Daniel Veillard <veillard(a)redhat.com> wrote:
On Mon, May 25, 2009 at 11:51:07PM -0400, Gerry Reno wrote:
> Is there a way that you can set the guests startup order and stagger
> their startup at boot?
Hum, no. We only have the autostart flag on the domain but nothing
about ordering of starts on a wider basis.
Daniel
--
Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/
daniel(a)veillard.com | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library http://libvirt.org/
--
Libvir-list mailing list
Libvir-list(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
9:11 a.m.
Quoting Michael Ansel <libvir(a)anselcomputers.com>:
[...]
How hard would something like this (staggered boot order) be to
add/integrate with an optional, external config file (or as a
Disclaimer: I have no clue as to how difficult it would be to add
or integrate.
However, considering that the boot order is most likely dependent
on a higher level application (the app hosted in the guest) and
it's dependencies, does it really make sense to add that integration
to libvirt/libvirtd? To truly get a staggered order, it should really
know something about the application preferences etc. I'd
argue that this level of capability is something that should be
handled by, for instance a cluster resource manager?
[true|false|1|2|3|..]. style directive in the main XML)? I'd be
interested in putting something together if it is a feasible addition
and someone can point me to the general area of the code that needs to
be changed. Also, is there an indicator in libvirt for when a domain
has finished booting, or would this boot staggering need to be time
Time based seems way too simplistic since there, in a production
environment could be a number of issues determining the time it takes
for a specific guest to boot, including the load on the host system.
Also, the fact that the guest is done booting in by no means an
indication that the application its hosting is ready to be interacted
with by one of the dependent guests (example: database recovery for
some of the commercial database vendors occurs asynchronously to
the completion of the boot process, yet the DB isn't ready for
transactions until the recovery is complete, which could be long
after the boot process is "complete" from an OS perspective.)
// Thomas
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
9:17 a.m.
On Tue, May 26, 2009 at 10:11:52AM -0400, Thomas Sjolshagen wrote:
Quoting Michael Ansel <libvir(a)anselcomputers.com>:
>[true|false|1|2|3|..]. style directive in the main XML)? I'd be
>interested in putting something together if it is a feasible addition
>and someone can point me to the general area of the code that needs to
>be changed. Also, is there an indicator in libvirt for when a domain
>has finished booting, or would this boot staggering need to be time
Time based seems way too simplistic since there, in a production
environment could be a number of issues determining the time it takes
for a specific guest to boot, including the load on the host system.
Also, the fact that the guest is done booting in by no means an
indication that the application its hosting is ready to be interacted
with by one of the dependent guests (example: database recovery for
some of the commercial database vendors occurs asynchronously to
the completion of the boot process, yet the DB isn't ready for
transactions until the recovery is complete, which could be long
after the boot process is "complete" from an OS perspective.)
Yeah, this is what makes ordered startup of guests a 'feature' that
is doomed to fail. It is essentially impossible to determine 'finished
booing' from the host OS, and time delays are inevitably going to fail
on some percentage of boot attempts. The only reliable way is to have
the applications which depend on resources from other guests automatically
retry connections, or have something in the guest which probes to see if
the external resource is ready before attempting to start the app in
question.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
10:48 a.m.
Daniel P. Berrange wrote:
On Tue, May 26, 2009 at 10:11:52AM -0400, Thomas Sjolshagen wrote:
> Quoting Michael Ansel <libvir(a)anselcomputers.com>:
>
>
>> [true|false|1|2|3|..]. style directive in the main XML)? I'd be
>> interested in putting something together if it is a feasible addition
>> and someone can point me to the general area of the code that needs to
>> be changed. Also, is there an indicator in libvirt for when a domain
>> has finished booting, or would this boot staggering need to be time
>>
> Time based seems way too simplistic since there, in a production
> environment could be a number of issues determining the time it takes
> for a specific guest to boot, including the load on the host system.
>
> Also, the fact that the guest is done booting in by no means an
> indication that the application its hosting is ready to be interacted
> with by one of the dependent guests (example: database recovery for
> some of the commercial database vendors occurs asynchronously to
> the completion of the boot process, yet the DB isn't ready for
> transactions until the recovery is complete, which could be long
> after the boot process is "complete" from an OS perspective.)
>
Yeah, this is what makes ordered startup of guests a 'feature' that
is doomed to fail. It is essentially impossible to determine 'finished
booing' from the host OS, and time delays are inevitably going to fail
on some percentage of boot attempts. The only reliable way is to have
the applications which depend on resources from other guests automatically
retry connections, or have something in the guest which probes to see if
the external resource is ready before attempting to start the app in
question.
Daniel
Just a simple time-based solution would be sufficient. Right now when
multiple guests all try to boot at the same time it takes about 4 times
as long for all of them to boot up as when you boot them individually.
Apparently too much context switching. So if we could check off an
autoboot flag and then say +120 seconds, that guest would start booting
120 seconds after the autoboot process started. Even this simple
mechansim would help quite a bit.
Regards,
Gerry
11:39 a.m.
In addition to Dan's reply, operating systems ought to indicate what
stage they have reached in the boot process (eg. "BIOS up", "kernel
up", "userspace up", "applications started"), perhaps by writing
a
status code into a register.
Some PC BIOSes do this already, writing to IO port 0x80, see for
example:
http://lkml.indiana.edu/hypermail/linux/kernel/0203.1/0770.html
If you are using a watchdog driver, feasibly you might use this to
detect when userspace in the guest is active.
Anyway, this is worth discussing with the relevant upstream projects,
QEMU, IPMI, various watchdog hardware ...
Rich.
--
Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://et.redhat.com/~rjones/libguestfs/
See what it can do: http://et.redhat.com/~rjones/libguestfs/recipes.html
11:42 a.m.
Daniel P. Berrange wrote:
On Tue, May 26, 2009 at 10:11:52AM -0400, Thomas Sjolshagen wrote:
> Quoting Michael Ansel <libvir(a)anselcomputers.com>:
>
>
>> [true|false|1|2|3|..]. style directive in the main XML)? I'd be
>> interested in putting something together if it is a feasible addition
>> and someone can point me to the general area of the code that needs to
>> be changed. Also, is there an indicator in libvirt for when a domain
>> has finished booting, or would this boot staggering need to be time
>>
> Time based seems way too simplistic since there, in a production
> environment could be a number of issues determining the time it takes
> for a specific guest to boot, including the load on the host system.
>
> Also, the fact that the guest is done booting in by no means an
> indication that the application its hosting is ready to be interacted
> with by one of the dependent guests (example: database recovery for
> some of the commercial database vendors occurs asynchronously to
> the completion of the boot process, yet the DB isn't ready for
> transactions until the recovery is complete, which could be long
> after the boot process is "complete" from an OS perspective.)
>
Yeah, this is what makes ordered startup of guests a 'feature' that
is doomed to fail. It is essentially impossible to determine 'finished
booing' from the host OS, and time delays are inevitably going to fail
on some percentage of boot attempts. The only reliable way is to have
the applications which depend on resources from other guests automatically
retry connections, or have something in the guest which probes to see if
the external resource is ready before attempting to start the app in
question.
Daniel
Just a simple time-based solution would be sufficient. Right now when
multiple guests all try to boot at the same time it takes about 4 times
as long for all of them to boot up as when you boot them individually.
Apparently too much context switching. So if we could check off an
autoboot flag and then say +120 seconds, that guest would start booting
120 seconds after the autoboot process started. Even this simple
mechansim would help quite a bit.
Regards,
Gerry
2:50 p.m.
On Tue, May 26, 2009 at 12:42:47PM -0400, Gerry Reno wrote:
Just a simple time-based solution would be sufficient. Right now
when
multiple guests all try to boot at the same time it takes about 4 times
as long for all of them to boot up as when you boot them individually.
Apparently too much context switching. So if we could check off an
autoboot flag and then say +120 seconds, that guest would start booting
120 seconds after the autoboot process started. Even this simple
mechansim would help quite a bit.
Now this is a interesting problem :-) It doesn't so much suggest the
provision of guest startup ordering, but rather the serialization of
guest startup - only if there were no other viable solution elsewhere
though.
I'd be surprised if context switching were the problem & if it was,
then I'd suggest beating up on the kernel guys to fix their schedular.
More likely is competition for disk access, and the parallel guest
startup causing storm of disk seeks, destroying overall I/O throughput.
I would still wonder if this could be solved in the kernel I/O schedular
either automatically, or with libvirt setting some tunable knob to tell
it to try and batch up I/O for the VM to avoid so much seeking. Of course
this is all conjecture without profiling / benchmarks to identify just
where the problem is.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5693
days inactive
5693
days old
8 comments
6 participants
participants (6)
-
Daniel P. Berrange -
Daniel Veillard -
Gerry Reno -
Michael Ansel -
Richard W.M. Jones -
Thomas Sjolshagen