5:44 a.m.
Hello,
I regularly observe the problem, that depending on the libvirt-URL I get
different information:
root@xen4# virsh -c xen://xen4.domain.name/ list
Id Name State
----------------------------------
root@xen4# virsh -c xen:/// list
Id Name State
----------------------------------
0 Domain-0 running
1 dos4 idle
root@xen4# virsh -c xen+unix:/// list
Id Name State
----------------------------------
root@xen4# xm li
Name ID Mem VCPUs State
Time(s)
Domain-0 0 2044 2 r----- 419.7
dos4 1 4 1 -b---- 33.4
Has somebody observed the same problem and/or can give me any advise on where
to look for the problem?
I'm running a Debian-Lenny based OS with libvirt-0.8.2 on Xen-3.4.3 with a
2.6.32 kernel.
Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn(a)univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 28359 Bremen fax: +49 421 22 232-99
http://www.univention.de
7:51 a.m.
On Thu, Jul 22, 2010 at 12:44:53PM +0200, Philipp Hahn wrote:
Hello,
I regularly observe the problem, that depending on the libvirt-URL I get
different information:
root@xen4# virsh -c xen://xen4.domain.name/ list
Id Name State
----------------------------------
This will go to the libvirtd daemon, which will in turn open
a 'xen:///' connection
root@xen4# virsh -c xen:/// list
Id Name State
----------------------------------
0 Domain-0 running
1 dos4 idle
This is uses a combination of talking to the hypervisor, xenstore
and xend.
root@xen4# virsh -c xen+unix:/// list
Id Name State
----------------------------------
This goes to the libvirtd daemon, using UNIX sockets, which
will in turn open a 'xen:///' connection.
Assuming the libvirtd daemon you've connected to is on the
same host as virsh process, then since they all ultimately
end up using 'xen:///', the data should be identical in
all cases.
To verify that libvirtd is actually openening the same drivers as virsh
you can edit /etc/libvirt/libvirtd.conf and set
log_filters="1:libvirt 1:xen"
log_outputs="1:file:/var/log/libvirt/libvirtd.log"
and restart the libvirtd daemon. Then try the 'xen+unix:///' URI again
After that use
export LIBVIRT_LOG_FILTERS="1:libvirt 1:xen"
export LIBVIRT_LOG_OUTPUTS="1:file:/var/log/libvirt/libvirtd.log"
and run 'virsh -c xen:///'. It should be possible to then compare that
both sets of logs show same Xen drivers
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
11:02 a.m.
Hello,
thank you for your answer.
Am Donnerstag 22 Juli 2010 14:51:22 schrieb Daniel P. Berrange:
> root@xen4# virsh -c xen://xen4.domain.name/ list
> root@xen4# virsh -c xen:/// list
> root@xen4# virsh -c xen+unix:/// list
...
Assuming the libvirtd daemon you've connected to is on the
same host as virsh process, then since they all ultimately
end up using 'xen:///', the data should be identical in
all cases.
Yes, I run all those commands locally on the same host (xen4).
The problem only occurs after some time of normal operation: just after
starting xend, libvirtd, ... everything is fine. Then sometime later after
starting and stoping some domains, the systems gets unresponsive and the
above described problem occurs: Only than do the different connection types
provide different data.
To me it looks like some connection between libvirtd / virsh and xen gets
broken, so no data on running domains can be retrieved any more, until I
restart libvirtd. After that everything is back to normal until the next
hickup.
To verify that libvirtd is actually openening the same drivers as
virsh
you can edit /etc/libvirt/libvirtd.conf and set
log_filters="1:libvirt 1:xen"
log_outputs="1:file:/var/log/libvirt/libvirtd.log"
I'v now enabled that and see if I can gather more data.
Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn(a)univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 28359 Bremen fax: +49 421 22 232-99
http://www.univention.de
9:17 a.m.
New subject: [libvirt] [RFC, PATCH] Re: xen:/// vs. xen://FQDN/ vs xen+unix:/// discrepancy
Hello,
Am Freitag 23 Juli 2010 18:02:27 schrieb Philipp Hahn:
Am Donnerstag 22 Juli 2010 14:51:22 schrieb Daniel P. Berrange:
> > root@xen4# virsh -c xen://xen4.domain.name/ list
> > root@xen4# virsh -c xen:/// list
> > root@xen4# virsh -c xen+unix:/// list
...
To me it looks like some connection between libvirtd / virsh and xen
gets
broken, so no data on running domains can be retrieved any more, until I
restart libvirtd. After that everything is back to normal until the next
hickup.
Today I encountered the bug again: "virsh list" was not showing any domain,
while "virsh -c xen+unix:///" did show "Domain 0" and my other
domain.
I attached gdb, "thread apply all bt" showed no thread still executing
xenHypervisorInit(), but I found the following inconsistent state:
in_init == 1
hypervisor_version == -1
sys_interface_version == -1
dom_interface_version == -1
Notice in_init=1: Currently there are two error path, where that can happen:
1. ret = open(XEN_HYPERVISOR_SOCKET, O_RDWR);
2. if (VIR_ALLOC(ipt) < 0) {
In both cases in_init remains 1, which disables further useful logging:
#define virXenError(code, ...)
\
if (in_init == 0) \
virReportErrorHelper(NULL, VIR_FROM_XEN, code, __FILE__, \
__FUNCTION__, __LINE__, __VA_ARGS__)
This leads me to the following question:
1. Why is error reporting deactivated during init?
Since in_init==1 is valid only during the call xenHypervisorInit(), explicitly
using that macro there for logging seems very wrong.
2. xenHypervisorInit() also allocates three regexp_t structures, which are
only freed when one of them can not be compiled, but they remain in memory
when determin the Xen hypervisor version fails.
On the other hand these 3 structures are used in
xenHypervisorMakeCapabilitiesInternal(). This isn't checked directly, but
only indirectly, because in the error case no one will ever call it with a
valid virConnectPtr.
Should the structures be freed in all error cases?
The attached patch is a first try at improving that situation.
> To verify that libvirtd is actually openening the same drivers
as virsh
> you can edit /etc/libvirt/libvirtd.conf and set
>
> log_filters="1:libvirt 1:xen"
> log_outputs="1:file:/var/log/libvirt/libvirtd.log"
I've now enabled that and see if I can gather more data.
This didn't work because of the bug described above.
Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn(a)univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99
http://www.univention.de/
3:11 p.m.
New subject: [libvirt] [RFC, PATCH] Re: xen:/// vs. xen://FQDN/ vs xen+unix:/// discrepancy
Philipp Hahn wrote:
Hello,
Am Freitag 23 Juli 2010 18:02:27 schrieb Philipp Hahn:
> Am Donnerstag 22 Juli 2010 14:51:22 schrieb Daniel P. Berrange:
>
>>> root@xen4# virsh -c xen://xen4.domain.name/ list
>>> root@xen4# virsh -c xen:/// list
>>> root@xen4# virsh -c xen+unix:/// list
>>>
...
> To me it looks like some connection between libvirtd / virsh and xen gets
> broken, so no data on running domains can be retrieved any more, until I
> restart libvirtd. After that everything is back to normal until the next
> hickup.
>
Today I encountered the bug again: "virsh list" was not showing any domain,
while "virsh -c xen+unix:///" did show "Domain 0" and my other
domain.
Are you using xen unstable or Xen 4.1 (i.e. something greater than Xen
4.0)? If so, the sysctl and domctl versions have changed, which
essentially breaks the xen_hypervisor sub-driver. I ran into this
problem while trying to get the libxl and xen-unified drivers to play
nicely together - wasted a bunch of my testing time :-(. I'll send a
patch to handle the new sysctl and domctl versions.
Regards,
Jim
12:13 a.m.
New subject: [libvirt] [RFC, PATCH] Re: xen:/// vs. xen://FQDN/ vs xen+unix:/// discrepancy
Hello Jim,
Am Donnerstag 17 März 2011 21:11:34 schrieb Jim Fehlig:
Philipp Hahn wrote:
> Today I encountered the bug again: "virsh list" was not showing any
> domain, while "virsh -c xen+unix:///" did show "Domain 0" and my
other
> domain.
Are you using xen unstable or Xen 4.1 (i.e. something greater than Xen
4.0)?
No, this was with Xen 3.4.3.
If so, the sysctl and domctl versions have changed, which
essentially breaks the xen_hypervisor sub-driver. I ran into this
problem while trying to get the libxl and xen-unified drivers to play
nicely together - wasted a bunch of my testing time :-(. I'll send a
patch to handle the new sysctl and domctl versions.
Good to head your XenLight work is making progress.
Sincerely
Philipp Hahn
--
Philipp Hahn Open Source Software Engineer hahn(a)univention.de
Univention GmbH Linux for Your Business fon: +49 421 22 232- 0
Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99
http://www.univention.de/
4811
days inactive
5050
days old
5 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Jim Fehlig -
Philipp Hahn