On Fri, May 16, 2014 at 03:01:14AM +0000, Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) wrote:

Hi all,

I have a question about NUMA. User configured vNode(guest virtual numa node), but he didn't configure cputune and numatune. Now we want to get the information that each vNode run in which pNode(host numa node). It's run-time information that may be modified with high frequency. In current Libvirt's API, we can get the information that each vCpu running on which pCpu through virsh vcpuinfo(there should be a corresponding Libvirt API function). But we didn't find any APIs to get the information that each vNode uses which pNode's memory, or just each vCpu consumes which pNode's memory.

Yes, you are correct - the libvirt APIs only provide a way to figure out the vCPU<->pCPU placement, nothing about memory.

We find a command "numastat -mcn -p qemu" that can get the memory consume data of each VM, but it still loses the information that we want(vNode memory consume data), as following: # numastat -mcn -p qemu

Per-node process memory usage (in MBs) PID Node 0 Node 1 Total --------------- ------ ------ ----- 8900 (qemu-kvm) 2032 50 2083 17716 (qemu-kvm) 1546 663 2209 22484 (qemu-kvm) 621 1524 2146 29694 (qemu-kvm) 892 1350 2242 --------------- ------ ------ ----- Total 5092 3588 8680 .........

My question is:

1. In Libvirt, are there any ways that we can get our needed data?

Not at this point in time.

2. If no ways in Libvirt, do you have any other suggestions to collect the information?

I don't believe there is any easy way. The 'numastat' command can only see things at process-level granularity - it has no way of knowing about the fact that the KVM process has two virtual guest NUMA nodes. With current QEMU there's not even any way for libvirt to know where guest NUMA node memory is allocated from. There is working taking place in QEMU to make it possible to associated guest NUMA nodes with host NUMA nodes. This only helps if the guest / host nodes are specified though. If you are letting the guest NUMA nodes float across any host NUMA node, I'm not sure that KVM will provide us enough info to determine what was allocated from where. You might want to send this query to the qemu-devel mailing list instead to see if they have better suggestions Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|