[Libvir] [PATCH] Fix the value of XEN_V2_OP_GETDOMAININFOLIST
Hi, The value of "Max memory" shown by using "virsh dominfo" is different from xm command. # virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 300000 kB Used memory: 264064 kB # xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5) So, I changed the definition as follows, (before) #define XEN_V2_OP_GETDOMAININFOLIST 6 (changed) #define XEN_V2_OP_GETDOMAININFOLIST 5 and I could get the value that I want. # ./virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 460800 kB Used memory: 299008 kB # xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5) I think that is not mistake because the XEN_V2_OP_GETDOMAININFOLIST is a system operation, so the definition is not exactly wrong. /usr/include/xen/sysctl.h:#define XEN_SYSCTL_getdomaininfolist 6 Do you know any reason for this? Signed-off-by: Saori Fukuta <fukuta.saori@jp.fujitsu.com> Thanks, Saori Fukuta. Index: xen_internal.c =================================================================== RCS file: /data/cvs/libvirt/src/xen_internal.c,v retrieving revision 1.71 diff -u -p -r1.71 xen_internal.c --- xen_internal.c 13 Apr 2007 00:43:57 -0000 1.71 +++ xen_internal.c 26 Apr 2007 11:41:03 -0000 @@ -117,7 +117,7 @@ static regex_t xen_cap_rec; #define XEN_V0_OP_GETDOMAININFOLIST 38 #define XEN_V1_OP_GETDOMAININFOLIST 38 -#define XEN_V2_OP_GETDOMAININFOLIST 6 +#define XEN_V2_OP_GETDOMAININFOLIST 5 struct xen_v0_getdomaininfo { domid_t domain; /* the domain number */
On Thu, Apr 26, 2007 at 09:58:53PM +0900, Saori Fukuta wrote:
Hi,
The value of "Max memory" shown by using "virsh dominfo" is different from xm command.
# virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 300000 kB Used memory: 264064 kB
# xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5)
So, I changed the definition as follows, (before) #define XEN_V2_OP_GETDOMAININFOLIST 6 (changed) #define XEN_V2_OP_GETDOMAININFOLIST 5 and I could get the value that I want.
# ./virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 460800 kB Used memory: 299008 kB
# xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5)
I think that is not mistake because the XEN_V2_OP_GETDOMAININFOLIST is a system operation, so the definition is not exactly wrong. /usr/include/xen/sysctl.h:#define XEN_SYSCTL_getdomaininfolist 6
Do you know any reason for this?
No. I really can't apply such a patch as-is, the operation #5 is /* Interface for controlling Xen software performance counters. */ #define XEN_SYSCTL_perfc_op 5 this really isn't what we are looking at. What version of xen and libvirt are you using and on which architecture. that makes no sense to me, if we were off by one, on this it would have broken a long time ago. What version of the xen kernel are you using too ? Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hi, I am using virsion of libvirt: revision 1.521(ChangeLog) and xen: Xen version 3.0-unstable (root@) () Tue Apr 17 16:32:35 JST 2007 Latest ChangeSet: Fri Apr 13 16:07:48 2007 +0100 14854:039daabebad5 on Fedora Core 7 test3(i686). It may depend on architecture, so I will test other architectures. Thanks, Saori Fukuta On Thu, 26 Apr 2007 09:10:25 -0400 Daniel Veillard wrote:
I think that is not mistake because the XEN_V2_OP_GETDOMAININFOLIST is a system operation, so the definition is not exactly wrong. /usr/include/xen/sysctl.h:#define XEN_SYSCTL_getdomaininfolist 6
Do you know any reason for this?
No. I really can't apply such a patch as-is, the operation #5 is /* Interface for controlling Xen software performance counters. */ #define XEN_SYSCTL_perfc_op 5
this really isn't what we are looking at.
What version of xen and libvirt are you using and on which architecture. that makes no sense to me, if we were off by one, on this it would have broken a long time ago. What version of the xen kernel are you using too ?
Daniel
-- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Fri, Apr 27, 2007 at 10:01:52AM +0900, Saori Fukuta wrote:
Hi,
I am using virsion of libvirt: revision 1.521(ChangeLog) and xen: Xen version 3.0-unstable (root@) () Tue Apr 17 16:32:35 JST 2007 Latest ChangeSet: Fri Apr 13 16:07:48 2007 +0100 14854:039daabebad5 on Fedora Core 7 test3(i686).
I've just updated one of my i686 boxes to use the kernel-xen-2.6-2.6.20-2925.5.fc7 kernel xen-3.0.4-9.fc7 userspace and libvirt-0.2.2-1.fc7. With that the 'virsh dominfo demo' does show correct memory usage. If I do a 'virsh vcpuinfo demo' though the whole box hangs & I get a kernel trace ! At this think I think this is most likely to be something wrong with the Fedora kernel rather than libvirt, since we know there's anumber of other wierd bugs with this 2.6.20 kernel. x86_64 worked fine AFAICT Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Hi Dan, On Fri, 27 Apr 2007 19:07:43 +0100 "Daniel P. Berrange" wrote:
With that the 'virsh dominfo demo' does show correct memory usage. If I do a 'virsh vcpuinfo demo' though the whole box hangs & I get a kernel trace ! At this think I think this is most likely to be something wrong with the Fedora kernel rather than libvirt, since we know there's anumber of other wierd bugs with this 2.6.20 kernel. x86_64 worked fine AFAICT
Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
I'm sorry for my delay in replying. I made a mistake when I confirmed this problem before.
# ./virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 460800 kB Used memory: 299008 kB
I said I could get the value after I changed the definition of XEN_V2_OP_GETDOMAININFOLIST. But actually the virsh command showed the value using "xend_internal.c", not using "xen_internal.c". Here is a result of a comparing between "xend_internal.c" and "xen_internal.c". # xm list Name ID Mem VCPUs State Time(s) Domain-0 0 729 2 r----- 483.5 HVM_RH5_sda6 5 195 2 ------ 10294.2 # virsh setmaxmem HVM_RH5_sda6 500000 # xm list -l HVM_RH5_sda6 | grep mem (maxmem 400) (memory 195) (shadow_memory 6) # ./virsh dominfo HVM_RH5_sda6 [DEBUG] using: xenDaemonDomainGetInfo State: no state CPU(s): 5 CPU time: 10202.0s Max memory: 409600 kB <--- Used memory: 199680 kB <--- # ./virsh dominfo HVM_RH5_sda6 [DEBUG] using: xenHypervisorGetDomainInfo State: no state CPU(s): 5 CPU time: 10214.1s Max memory: 500000 kB <--- Used memory: 207744 kB <--- I think the value is correct when using "xenDaemonDomainGetInfo". So I want to use only "xenDaemonDomainGetInfo" for Xen when executing "virsh dominfo" command. How does that sound ? Thanks, Saori Fukuta
On Tue, May 08, 2007 at 11:01:04AM +0900, Saori Fukuta wrote:
Here is a result of a comparing between "xend_internal.c" and "xen_internal.c".
# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 729 2 r----- 483.5 HVM_RH5_sda6 5 195 2 ------ 10294.2
# virsh setmaxmem HVM_RH5_sda6 500000
# xm list -l HVM_RH5_sda6 | grep mem (maxmem 400) (memory 195) (shadow_memory 6)
# ./virsh dominfo HVM_RH5_sda6 [DEBUG] using: xenDaemonDomainGetInfo State: no state CPU(s): 5 CPU time: 10202.0s Max memory: 409600 kB <--- Used memory: 199680 kB <---
# ./virsh dominfo HVM_RH5_sda6 [DEBUG] using: xenHypervisorGetDomainInfo State: no state CPU(s): 5 CPU time: 10214.1s Max memory: 500000 kB <--- Used memory: 207744 kB <---
I think the value is correct when using "xenDaemonDomainGetInfo". So I want to use only "xenDaemonDomainGetInfo" for Xen when executing "virsh dominfo" command. How does that sound ?
Hmm, this is interesting. So what your example is showing is that the current & max-memory settings disagree between the hypervisor and XenD. Whether the HV is more correct, or whether XenD is more correct is a difficult question to decide upon ! What has happened in this scenario I think is that the guest was booted with maxmem == 500, hence the Hypervisor shows '500000 kB'. Later the guest's max memory was reduced to 400 using XenD, however, since you can't truely change the max-memory maps of the guest, the HV still shows 500, while XenD now shows the reduced 400 MV limit. IMHO, XenD is wrong here because it is showing an artificially low limit which the HV has no intention of enforcing. xm list is merely showing the info it gets back from XenD, so if XenD disagrees with the HV, then so will xm list. Making libvirt always use xenDaemonDomainGetInfo is also not practical, since the virDomainGetInfo is the single most performance critical API in libvirt. If we used always called xenDaemonDomainGetInfo instead of xenHypervisorGetDomainInfo, then we reduce the performance of this API by at least x1000 which is not really acceptable. Since we have the libvirt_proxy, even when running as non-root apps will (indirectly) be calling xenHypervisorGetDomainInfo, so it will be very rare that libvirt ever calls xenDaemonDomainGetInfo. So I think we just need to accept that virsh dominfo, will disagree with xm list in some scenarios & document this difference to explain to people why libvirt is more correct ;-) Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Tue, 8 May 2007 03:14:26 +0100 "Daniel P. Berrange" wrote:
Hmm, this is interesting. So what your example is showing is that the current & max-memory settings disagree between the hypervisor and XenD. Whether the HV is more correct, or whether XenD is more correct is a difficult question to decide upon !
Yeah, it is difficult to decide which of HV or XenD is correct.
What has happened in this scenario I think is that the guest was booted with maxmem == 500, hence the Hypervisor shows '500000 kB'. Later the guest's max memory was reduced to 400 using XenD, however, since you can't truely change the max-memory maps of the guest, the HV still shows 500, while XenD now shows the reduced 400 MV limit. IMHO, XenD is wrong here because it is showing an artificially low limit which the HV has no intention of enforcing. xm list is merely showing the info it gets back from XenD, so if XenD disagrees with the HV, then so will xm list.
I see. Now it makes sense. Thank you for your explanations !
Since we have the libvirt_proxy, even when running as non-root apps will (indirectly) be calling xenHypervisorGetDomainInfo, so it will be very rare that libvirt ever calls xenDaemonDomainGetInfo.
So I think we just need to accept that virsh dominfo, will disagree with xm list in some scenarios & document this difference to explain to people why libvirt is more correct ;-)
Okay, I quite agree. thanks a lot ! Saori
On Thu, Apr 26, 2007 at 09:58:53PM +0900, Saori Fukuta wrote:
Hi,
The value of "Max memory" shown by using "virsh dominfo" is different from xm command.
# virsh dominfo HVM_RH5_sda6 | grep mem Max memory: 300000 kB Used memory: 264064 kB
# xm li -l HVM_RH5_sda6 | grep mem (maxmem 450) (memory 292) (shadow_memory 5)
So, I changed the definition as follows, (before) #define XEN_V2_OP_GETDOMAININFOLIST 6 (changed) #define XEN_V2_OP_GETDOMAININFOLIST 5 and I could get the value that I want.
This doesn't make any sense to me. There are two hypercalls in the Xen headers, XEN_SYSCTL_getdomaininfolist and XEN_DOMCTL_getdomaininfo, which use the same struct. The former gets info on multiple domains, the latter on a single domain. In the libvirt code we always use the sysctl even when getting info for a single domain. So XEN_V2_OP_GETDOMAININFOLIST should match XEN_SYSCTL_getdomaininfolist, which AFAICT we already have correct. Does this problem occurr for you on all architectures ? Also what version number of xen & libvirt have you got installed ? Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
Saori Fukuta