2:32 p.m.
On Solaris-based xen 3.1.2 I am seeing virDomainGetInfo() often returning
VIR_DOMAIN_NOSTATE in dominfo.state for domUs that are powered on and
running. What exactly does VIR_DOMAIN_NOSTATE mean? Why does it exist? I
suspect it is a catch-all in the API for hosts that don't entirely have
their act together, that do not always return a valid guest state. Is that
the case?
Is anyone else seeing this on Xen?
--
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
3:23 a.m.
Russ Blaine wrote:
On Solaris-based xen 3.1.2 I am seeing virDomainGetInfo() often
returning VIR_DOMAIN_NOSTATE in dominfo.state for domUs that are powered
on and running. What exactly does VIR_DOMAIN_NOSTATE mean? Why does it
exist? I suspect it is a catch-all in the API for hosts that don't
entirely have their act together, that do not always return a valid
guest state. Is that the case?
Is anyone else seeing this on Xen?
It means that we query xcnd and there is no domain/state part of the
sexpr for the domain.
What does 'xm list --long' say?
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
3:28 a.m.
Richard W.M. Jones wrote:
It means that we query xcnd
^^^
Not sure what happened there ... xend
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
8:39 p.m.
Richard W.M. Jones wrote:
Russ Blaine wrote:
> On Solaris-based xen 3.1.2 I am seeing virDomainGetInfo() often
> returning VIR_DOMAIN_NOSTATE in dominfo.state for domUs that are
> powered on and running. What exactly does VIR_DOMAIN_NOSTATE mean? Why
> does it exist? I suspect it is a catch-all in the API for hosts that
> don't entirely have their act together, that do not always return a
> valid guest state. Is that the case?
>
> Is anyone else seeing this on Xen?
It means that we query xcnd and there is no domain/state part of the
sexpr for the domain.
What does 'xm list --long' say?
[ libvirt 0.4.0; I just subscribed to the list so it might take a bit for
traffic to flow to me. ]
When the problem occurs, the state line in xm list --long is:
(state ------)
And in the 'xm list' it looks like this:
Name ID Mem VCPUs State Time(s)
abc2 27 512 1 ------ 9.9
It's an XP domain, and this happens while it's booting up.
- Russ
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
9:16 p.m.
On Fri, Jan 25, 2008 at 06:39:00PM -0800, Russ Blaine wrote:
Richard W.M. Jones wrote:
>Russ Blaine wrote:
>>On Solaris-based xen 3.1.2 I am seeing virDomainGetInfo() often
>>returning VIR_DOMAIN_NOSTATE in dominfo.state for domUs that are
>>powered on and running. What exactly does VIR_DOMAIN_NOSTATE mean? Why
>>does it exist? I suspect it is a catch-all in the API for hosts that
>>don't entirely have their act together, that do not always return a
>>valid guest state. Is that the case?
>>
>>Is anyone else seeing this on Xen?
>
>It means that we query xcnd and there is no domain/state part of the
>sexpr for the domain.
Actually we will most likely be getting this data direct from the hypervisor
rather than XenD.
[ libvirt 0.4.0; I just subscribed to the list so it might take a bit
for
traffic to flow to me. ]
When the problem occurs, the state line in xm list --long is:
(state ------)
And in the 'xm list' it looks like this:
Name ID Mem VCPUs State Time(s)
abc2 27 512 1 ------ 9.9
Can you edit the src/xen_internal.c file and in the xenHypervisorGetDomInfo()
method, add a printf() for the 'domain_flags' flags variable. IMHO the
way we deal with this isn't quite correct. We merely mask out the high
bits and then switch on the resulting value. This isn't the way the Xen
hypervisor uses this field though. Xen more or less uses the whole thing
as as bitmask, allowing near arbitrary combinations of bits to be set.
So I think what is happening is there's a combo of bits set which cause
the switch() statement to fail all cases, resulting in NOSTATE.
We'll probably need to replace the switch(domain_flags) with something
which explicitly tests for the bits we're interested in, rather than
looking at the value as a whole.
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
12:01 p.m.
Daniel P. Berrange wrote:
Can you edit the src/xen_internal.c file and in the
xenHypervisorGetDomInfo()
method, add a printf() for the 'domain_flags' flags variable. IMHO the
way we deal with this isn't quite correct. We merely mask out the high
bits and then switch on the resulting value. This isn't the way the Xen
hypervisor uses this field though. Xen more or less uses the whole thing
as as bitmask, allowing near arbitrary combinations of bits to be set.
So I think what is happening is there's a combo of bits set which cause
the switch() statement to fail all cases, resulting in NOSTATE.
We'll probably need to replace the switch(domain_flags) with something
which explicitly tests for the bits we're interested in, rather than
looking at the value as a whole.
when xenHypervisorGetDomInfo() returns VIR_DOMAIN_NONE, domain_flags is 0
(sampled after the HVM bit has been masked out). So is there some other
state implied by domain_flags being 0? In all cases I have seen, the domain
is actually running -- accumulating CPU time, etc.
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
12:14 p.m.
Russ Blaine wrote:
Daniel P. Berrange wrote:
> Can you edit the src/xen_internal.c file and in the
> xenHypervisorGetDomInfo()
> method, add a printf() for the 'domain_flags' flags variable. IMHO the
> way we deal with this isn't quite correct. We merely mask out the high
> bits and then switch on the resulting value. This isn't the way the Xen
> hypervisor uses this field though. Xen more or less uses the whole thing
> as as bitmask, allowing near arbitrary combinations of bits to be set.
> So I think what is happening is there's a combo of bits set which cause
> the switch() statement to fail all cases, resulting in NOSTATE.
>
> We'll probably need to replace the switch(domain_flags) with something
> which explicitly tests for the bits we're interested in, rather than
> looking at the value as a whole.
when xenHypervisorGetDomInfo() returns VIR_DOMAIN_NONE, domain_flags is
0 (sampled after the HVM bit has been masked out). So is there some
other state implied by domain_flags being 0? In all cases I have seen,
the domain is actually running -- accumulating CPU time, etc.
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
This begs a question actually ... Is the host (or guest) Solaris?
Rich.
--
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in
England and Wales under Company Registration No. 03798903
12:18 p.m.
Host is Solaris, guest is WinXP.
Richard W.M. Jones wrote:
Russ Blaine wrote:
> Daniel P. Berrange wrote:
>> Can you edit the src/xen_internal.c file and in the
>> xenHypervisorGetDomInfo()
>> method, add a printf() for the 'domain_flags' flags variable. IMHO the
>> way we deal with this isn't quite correct. We merely mask out the
>> high bits and then switch on the resulting value. This isn't the way
>> the Xen
>> hypervisor uses this field though. Xen more or less uses the whole thing
>> as as bitmask, allowing near arbitrary combinations of bits to be set.
>> So I think what is happening is there's a combo of bits set which cause
>> the switch() statement to fail all cases, resulting in NOSTATE.
>>
>> We'll probably need to replace the switch(domain_flags) with
>> something which explicitly tests for the bits we're interested in,
>> rather than
>> looking at the value as a whole.
>
> when xenHypervisorGetDomInfo() returns VIR_DOMAIN_NONE, domain_flags
> is 0 (sampled after the HVM bit has been masked out). So is there some
> other state implied by domain_flags being 0? In all cases I have seen,
> the domain is actually running -- accumulating CPU time, etc.
>
> -----------------------------------------------------
> Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
This begs a question actually ... Is the host (or guest) Solaris?
Rich.
--
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
12:15 p.m.
On Tue, Jan 29, 2008 at 10:01:35AM -0800, Russ Blaine wrote:
Daniel P. Berrange wrote:
>Can you edit the src/xen_internal.c file and in the
>xenHypervisorGetDomInfo()
>method, add a printf() for the 'domain_flags' flags variable. IMHO the
>way we deal with this isn't quite correct. We merely mask out the high
>bits and then switch on the resulting value. This isn't the way the Xen
>hypervisor uses this field though. Xen more or less uses the whole thing
>as as bitmask, allowing near arbitrary combinations of bits to be set.
>So I think what is happening is there's a combo of bits set which cause
>the switch() statement to fail all cases, resulting in NOSTATE.
>
>We'll probably need to replace the switch(domain_flags) with something
>which explicitly tests for the bits we're interested in, rather than
>looking at the value as a whole.
when xenHypervisorGetDomInfo() returns VIR_DOMAIN_NONE, domain_flags is 0
(sampled after the HVM bit has been masked out). So is there some other
state implied by domain_flags being 0? In all cases I have seen, the domain
is actually running -- accumulating CPU time, etc.
What hypervisor version are you running. I'm struggling to see the codepath
in the hypervisor 'getdomaininfo' call which could lead to domain_flags
being zero. AFAICT, as well as the HVM flags, there must always be at
least one other bit set.
int flags = XEN_DOMINF_blocked;
for_each_vcpu ( d, v )
{
....
if ( !test_bit(_VPF_down, &v->pause_flags) )
{
if ( !(v->pause_flags & VPF_blocked) )
flags &= ~XEN_DOMINF_blocked;
if ( v->is_running )
flags |= XEN_DOMINF_running;
info->nr_online_vcpus++;
}
}
Guarenteed either XEN_DOMINF_blocked or XEN_DOMINF_running is
set now.
And this next block, simply sets a few more bits
info->flags = flags |
((d->is_dying == DOMDYING_dead) ? XEN_DOMINF_dying : 0) |
(d->is_shut_down ? XEN_DOMINF_shutdown : 0) |
(d->is_paused_by_controller ? XEN_DOMINF_paused : 0) |
(d->debugger_attached ? XEN_DOMINF_debugged : 0) |
d->shutdown_code << XEN_DOMINF_shutdownshift;
if ( is_hvm_domain(d) )
info->flags |= XEN_DOMINF_hvm_guest;
This is all in getdomaininfo, from xen/common/domctl.c
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
12:21 p.m.
Daniel P. Berrange wrote:
What hypervisor version are you running. I'm struggling to see
the codepath
in the hypervisor 'getdomaininfo' call which could lead to domain_flags
being zero. AFAICT, as well as the HVM flags, there must always be at
least one other bit set.
This is opensolaris 81 with xen 3.1.2. Not having studied this code
extensively, I do see a path that could cause this:
flags starts out life as XEN_DOMINF_blocked
That flag then gets cleared in this marked line:
int flags = XEN_DOMINF_blocked;
for_each_vcpu ( d, v )
{
....
if ( !test_bit(_VPF_down, &v->pause_flags) )
{
if ( !(v->pause_flags & VPF_blocked) )
flags &= ~XEN_DOMINF_blocked; <-----here
if ( v->is_running )
flags |= XEN_DOMINF_running;
info->nr_online_vcpus++;
}
}
and then no bits are set in the next block.
Guarenteed either XEN_DOMINF_blocked or XEN_DOMINF_running is
set now.
And this next block, simply sets a few more bits
info->flags = flags |
((d->is_dying == DOMDYING_dead) ? XEN_DOMINF_dying : 0) |
(d->is_shut_down ? XEN_DOMINF_shutdown : 0) |
(d->is_paused_by_controller ? XEN_DOMINF_paused : 0) |
(d->debugger_attached ? XEN_DOMINF_debugged : 0) |
d->shutdown_code << XEN_DOMINF_shutdownshift;
if ( is_hvm_domain(d) )
info->flags |= XEN_DOMINF_hvm_guest;
This is all in getdomaininfo, from xen/common/domctl.c
-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine(a)sun.com
3:39 a.m.
[please subscribe to the mailing-list, I can't guarantee proper delivery
of mail bounced by the list tool if you are not subscibed, thanks !]
On Thu, Jan 24, 2008 at 12:32:32PM -0800, Russ Blaine wrote:
On Solaris-based xen 3.1.2 I am seeing virDomainGetInfo() often
returning
VIR_DOMAIN_NOSTATE in dominfo.state for domUs that are powered on and
running. What exactly does VIR_DOMAIN_NOSTATE mean? Why does it exist? I
that we could not extract the state status from the hypervisor.
suspect it is a catch-all in the API for hosts that don't
entirely have
their act together, that do not always return a valid guest state. Is that
the case?
Well, info->state is set to VIR_DOMAIN_NOSTATE only if in the SEXPR returned
by xend when asking about the domain, the (domain ... (state X) ...)
is either missing or empty, assuming the call ended up using a Xend RPC.
If the call used an hypercall, then the call should go though
xenHypervisorGetDomInfo and the switch on the domain state extracted
seems to be failing setting info->state to VIR_DOMAIN_NOSTATE (actually it
sets it to VIR_DOMAIN_NONE which is the wrong enum value, but 0 so equivalent
in practice, I will fix this in CVS).
Is anyone else seeing this on Xen?
What version of libvirt are you using ? Its behaviour might be different
from the upstream version I looked at, but in both case it really should
imply that we failed to extract the state informations from the hypervisor.
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard(a)redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
6143
days inactive
6148
days old
10 comments
4 participants
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Richard W.M. Jones -
Russ Blaine