RE: [Libvirt-cim] What does NumberOfBlocks and ConsumableBlocks in the Xen_Memory class represent?

Hi Dayne, Let me sort of work backwards here.. > Currently, on this system ConsumableBlocks represent MemTotal or the > current memory allocated to the guest, even though it is not > completely accurately representing the free memory on Dom0 that is > available to new DomUs. The issue is that between version 0.4.1 and > 0.5.2 NumberOfBlocks for a Dom0 changed to be MaxInt. It used to > match ConsumableBlocks.

Correct - that's what the bug is here. A regression was introduced.

In 0.4.1: -NumberOfBlocks the memory currently assigned to the guest -ConsumableBlocks the maximum memory allocated to the guest\

In 0.5.2: -NumberOfBlocks the maximum memory allocated to the guest -ConsumableBlocks the memory currently assigned to the guest

So this should definitely be fixed since we aren't adhering to the definitions in the mof... > I think there is something else going on here. As far as I can tell > the numbers are not swapped. Here is what I think is going on. The > NumberOfBlock x BlockSize equates to about 16TB (not what my system Ah, my mistake. When I read your previous message, I thought you were talking about the issue above.

The values do seem strange. We use the following calculation: NumberOfBlocks = (max_mem * 1024) / BlockSize ConsumableBlocks = (used_mem * 1024) / BlockSize

Agreed - NumberOfBlocks is puzzling: (4294967040 / 1024) * 4096 = 17179865088 KB

ConsumableBlocks is correct though: (6595584 * 1024) / 4096 = 1648896 What does "xm list -l Domain-0" return for memory and maxmem? I don't have a Xen system with that much mem to test on.

> has - my system only has 8GB). If you look at virsh dominfo Domain- 0 > on my box I get: > > Max memory: no limit > Used memory: 6595584 kB > > If NumberOfBlocks is MaxInt (of some sort) than this would make some sort of sense and ConsumableBlocks contains the correct value. It is a little bit misleading since processes in Dom0 take memory away from what can be allocated to a DomU. This is evident from this /proc/meminfo snippet: > > MemTotal: 6595584 kB > MemFree: 5940684 kB > Buffers: 16112 kB > Cached: 264716 kB > SwapCached: 0 kB > Active: 138332 kB > Inactive: 216032 kB > SwapTotal: 2626544 kB > SwapFree: 2626544 kB > Dirty: 436 kB > ... > VmallocTotal: 34359738367 kB > VmallocUsed: 267184 kB > VmallocChunk: 34359470859 kB > DirectMap4k: 8066284 kB > DirectMap2M: 0 kB > > > The current WBEM values: > > -ConsumableBlocks=1648896 > -NumberOfBlocks=4294967040 > -BlockSize=4096 Medlyn, Dayne (VSL - Ft Collins) wrote: >> Medlyn, Dayne (VSL - Ft Collins) wrote: >>> Kaitlin, >>> >>> Thanks for the correction. It seems were trying to use these >> properties correctly and there is something just not right. Using the >> same wbemcli command I get: >>> -SystemCreationClassName="Xen_ComputerSystem" >>> -SystemName="Domain-0" >>> -CreationClassName="Xen_Memory" >>> -DeviceID="Domain-0/mem" >>> >>> -ConsumableBlocks=1717760 >>> -NumberOfBlocks=4294967040 >>> -BlockSize=4096 >> Yes, that's definitely a bug. The values for ConsumableBlocks and >> NumberOfBlocks should be swapped. I'd hoped to have a bugfix out >> today, >> but it looks like it'll be tomorrow. > > > I think there is something else going on here. As far as I can tell the numbers are not swapped. Here is what I think is going on. The NumberOfBlock x BlockSize equates to about 16TB (not what my system has - my system only has 8GB). If you look at virsh dominfo Domain-0 on my box I get: > > Max memory: no limit > Used memory: 6595584 kB > > If NumberOfBlocks is MaxInt (of some sort) than this would make some sort of sense and ConsumableBlocks contains the correct value. It is a little bit misleading since processes in Dom0 take memory away from what can be allocated to a DomU. This is evident from this /proc/meminfo snippet: > > MemTotal: 6595584 kB > MemFree: 5940684 kB > Buffers: 16112 kB > Cached: 264716 kB > SwapCached: 0 kB > Active: 138332 kB > Inactive: 216032 kB > SwapTotal: 2626544 kB > SwapFree: 2626544 kB > Dirty: 436 kB > ... > VmallocTotal: 34359738367 kB > VmallocUsed: 267184 kB > VmallocChunk: 34359470859 kB > DirectMap4k: 8066284 kB > DirectMap2M: 0 kB > > > The current WBEM values: > > -ConsumableBlocks=1648896 > -NumberOfBlocks=4294967040 > -BlockSize=4096 > > Currently, on this system ConsumableBlocks represent MemTotal or the current memory allocated to the guest, even though it is not completely accurately representing the free memory on Dom0 that is available to new DomUs. The issue is that between version 0.4.1 and 0.5.2 NumberOfBlocks for a Dom0 changed to be MaxInt. It used to match ConsumableBlocks. I don't think there is an issue and I can work with this now that I understand it. > > > As you point out below, I am probably not going to be able to get the information I want out of Xen_MemoryPool. > > Thanks for your help and insight. > > > Dayne > > > > > >>> One difference I did notice is that we are trying to use these values >> from Dom0 to determine the amount of available memory for guests to >> use. Perhaps for Dom0 these values just map differently. >> >> >>> My objective is to identify how much memory is available on the >> hypervisor that can be allocated to new guests. Looking more closely, >> I wonder if we should be using Xen_MemoryPool somehow to do this >> instead. What is the relationship between the Capacity and Reserved >> properties? I have not quite been able to make sense out of what these >> values mean. What I have noticed is that a host with no defined guests >> starts with Reserved smaller than Capacity: >>> -PoolID="MemoryPool/0" >>> -Primordial=FALSE >>> -Capacity=8385536 >>> -Reserved=8064748 >>> -ResourceType=4 >>> -OtherResourceType= >>> -ResourceSubType= >>> -AllocationUnits="KiloBytes" >>> >>> As guests are create and start the Reserved count increases and grows >> beyond the capacity. I am not quite sure how to make use of this >> information. Do you have any insights? >> >> The Capacity value is the memory value libvirt reports for the host >> (you'd also get this value if you use: virsh nodeinfo). >> >> The Reserved value is the some of all the memory that is currently >> allocated to the guests on the system (as reported by libvirt). This >> includes guests that aren't running, which is why you are seeing the >> value grow beyond capacity. >> >> We don't represent the host capabilities, but in the case of Xen, you >> can get around that by pulling some things from Dom0. >> >> However, using Dom0's attribute may not give you the full picture >> you're >> looking for. I would suggest taking a look at a provider set that >> represents the host information. Something like the sblim-base >> providers should this info. >> >>> >>> Dayne >>> >>> >>> >>>> -----Original Message----- >>>> From: libvirt-cim-bounces(a)redhat.com [mailto:libvirt-cim- >>>> bounces(a)redhat.com] On Behalf Of Kaitlin Rupert >>>> Sent: Wednesday, May 13, 2009 7:10 PM >>>> To: List for discussion and development of libvirt CIM >>>> Subject: Re: [Libvirt-cim] What does NumberOfBlocks and >>>> ConsumableBlocks in theXen_Memory class represent? >>>> >>>> Medlyn, Dayne (VSL - Ft Collins) wrote: >>>>> All, >>>>> >>>>> I am trying to understand the use of NumberOfBlocks and >>>> ConsumableBlocks in the Xen_Memory class, specifically for the Xen >>>> host. >>>>> What I have noticed is that between libvirt-cim-0.4.1 and libvirt- >>>> cim-0.5.2 the values for NumberOfBlock is now different than >>>> ConsumableBlocks and >>>> > much larger than the physical memory installed on the system. >>>>> Is it the case that NumberOfBlocks represents the maximum possible >>>> blocks for the hardware, >>>> > or some such number ConsumableBlocks is the memory that is >>>>> actually installed in the system? On my system, however, >>>> NumberOfBlocks reports 16TB where /proc/meminfo >>>>> reports 32Tb for VmallocTotal. In short, should I be using >>>> ConsumableBlocks to determine the total physical memory on the >> system? >>>> >>>> Hi Dayne, >>>> >>>> It looks like there is a bug here. Currently, the providers use the >>>> following representation: >>>> >>>> NumberOfBlocks: max amount of memory that can be allocated to a >> guest >>>> ConsumableBlocks: current memory allocated to the guest >>>> >>>> However, these values should be reversed based on the attribute >>>> definitions. >>>> >>>> Here's an example using one of the guests on my system: >>>> >>>> # virsh dominfo rstest_domainId: - >>>> Name: rstest_domain >>>> UUID: 746de06d-cb45-4efd-bc18-bf91d10bec84 >>>> State: shut off >>>> CPU(s): 1 >>>> Max memory: 131072 kB >>>> Used memory: 130048 kB >>>> Autostart: disable >>>> >>>> We take the max and used memory values libvirt reports and then >> convert >>>> them based on the block size. >>>> >>>> # wbemcli gi >>>> >> 'http://localhost:5988/root/virt:Xen_Memory.CreationClassName="Xen_Memo >> ry",DeviceID="rstest_domain/mem",SystemCreationClassName="Xen_ComputerS >>>> ystem",SystemName="rstest_domain"' >>>> -nl >>>> >> localhost:5988/root/virt:Xen_Memory.CreationClassName="Xen_Memory",Devi >> ceID="rstest_domain/mem",SystemCreationClassName="Xen_ComputerSystem",S >>>> ystemName="rstest_domain" >>>> <snip> >>>> >>>> -TransitioningToState=12 >>>> -SystemCreationClassName="Xen_ComputerSystem" >>>> -SystemName="rstest_domain" >>>> -CreationClassName="Xen_Memory" >>>> -DeviceID="rstest_domain/mem" >>>> >>>> <snip> >>>> >>>> -BlockSize=4096 >>>> -NumberOfBlocks=32768 >>>> -ConsumableBlocks=32512 >>>> >>>> <snip> >>>> -- >>>> Kaitlin Rupert >>>> IBM Linux Technology Center >>>> kaitlin(a)linux.vnet.ibm.com >>>> >>>> _______________________________________________ >>>> Libvirt-cim mailing list >>>> Libvirt-cim(a)redhat.com >>>> https://www.redhat.com/mailman/listinfo/libvirt-cim >>> _______________________________________________ >>> Libvirt-cim mailing list >>> Libvirt-cim(a)redhat.com >>> https://www.redhat.com/mailman/listinfo/libvirt-cim >> >> -- >> Kaitlin Rupert >> IBM Linux Technology Center >> kaitlin(a)linux.vnet.ibm.com >> >> _______________________________________________ >> Libvirt-cim mailing list >> Libvirt-cim(a)redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-cim > > _______________________________________________ > Libvirt-cim mailing list > Libvirt-cim(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-cim -- Kaitlin Rupert IBM Linux Technology Center kaitlin(a)linux.vnet.ibm.com _______________________________________________ Libvirt-cim mailing list Libvirt-cim(a)redhat.com https://www.redhat.com/mailman/listinfo/libvirt-cim