[libvirt-users] Inconsistent behavior between x86_64 and ppc64 when creating guests with NUMA node placement
Hello all, I was recently trying out NUMA placement for my guests on both x86_64 and ppc64 machines. When booting a guest on the x86_64 machine, the following specs were valid (obviously, just notable excepts from the xml): <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <vcpu placement='static'>4</vcpu> ... <cpu> <topology sockets='4' cores='1' threads='1'/> <numa> <cell cpus='0-2' memory='6144'/> <cell cpus='3' memory='2048'/> </numa> </cpu> However, on ppc64 this causes the following error: error: Failed to create domain from sample_guest.xml error: internal error: early end of file from monitor: possible problem: 2014-09-11T18:44:25.502140Z qemu-system-ppc64: total memory for NUMA nodes (8388608) should equal RAM size (200000000) The 200000000 is actually 8192 MB in bytes and hexidecimal. This is apparently just an issue with the error message. The following specs work on ppc64: <cpu> <topology sockets='4' cores='1' threads='1'/> <numa> <cell cpus='0-2' memory='6291456'/> <cell cpus='3' memory='2097152'/> </numa> </cpu> Note that the memory for each cell is 6144*1024 and 2048*1024 respectively. The issue is that the memory size for each NUMA cell should be specified in KiB, not MB (http://libvirt.org/formatdomain.html#resPartition "|memory| specifies the node memory in kibibyte"). In short, it seems that specifying NUMA cell memory in MB works on x86_64 but not on ppc64. Does anyone have any insight to what's causing this, or if I'm misunderstanding something? Any help is appreciated, thank you! Regards, Michael Turek
On 12.09.2014 21:10, Michael Turek wrote:
Hello all,
I was recently trying out NUMA placement for my guests on both x86_64 and ppc64 machines. When booting a guest on the x86_64 machine, the following specs were valid (obviously, just notable excepts from the xml):
<memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <vcpu placement='static'>4</vcpu> ... <cpu> <topology sockets='4' cores='1' threads='1'/> <numa> <cell cpus='0-2' memory='6144'/> <cell cpus='3' memory='2048'/> </numa> </cpu>
6144 + 2048 != 8388608. Either you need to multiply those values by factor of 1024 (6144*1024 + 2048*1024 == 8388608) or add @unit='MiB' attribute, so libvirt knows what units you mean.
However, on ppc64 this causes the following error:
error: Failed to create domain from sample_guest.xml error: internal error: early end of file from monitor: possible problem: 2014-09-11T18:44:25.502140Z qemu-system-ppc64: total memory for NUMA nodes (8388608) should equal RAM size (200000000)
The 200000000 is actually 8192 MB in bytes and hexidecimal. This is apparently just an issue with the error message.
The following specs work on ppc64: <cpu> <topology sockets='4' cores='1' threads='1'/> <numa> <cell cpus='0-2' memory='6291456'/> <cell cpus='3' memory='2097152'/> </numa> </cpu>
Note that the memory for each cell is 6144*1024 and 2048*1024 respectively. The issue is that the memory size for each NUMA cell should be specified in KiB, not MB (http://libvirt.org/formatdomain.html#resPartition "|memory| specifies the node memory in kibibyte").
In short, it seems that specifying NUMA cell memory in MB works on x86_64 but not on ppc64. Does anyone have any insight to what's causing this, or if I'm misunderstanding something? Any help is appreciated, thank you!
What does the corresponding part of qemu command line look like? In both cases. I don't see anything architecture dependent there. What may happen here is that qemu on x86_64 does the checking while on ppc64 it does not. Michal
participants (2)
-
Michael Turek -
Michal Privoznik