Am Mittwoch, 11. September 2013, 11:27:30 schrieb Daniel P. Berrange:
On Wed, Sep 11, 2013 at 10:47:08AM +0200, David Weber wrote:
Am Freitag, 6. September 2013, 12:10:04 schrieb Daniel P. Berrange:
On Tue, Aug 27, 2013 at 09:09:25AM +0200, David Weber wrote:
Hi,
we try to use vcpu pinning on a 2 socket server with Intel Xeon E5620 cpus, HT enabled and 2*6*16GiB Ram but experience problems if we try to start a guest on the second socket: error: Failed to start domain test error: internal error: process exited while connecting to monitor: kvm_init_vcpu failed: Cannot allocate memory
# virsh freecell 0 0: 86071624 KiB
# virsh freecell 1 1: 75258628 KiB
# virsh edit test <domain type='kvm'>
<name>test</name> <uuid>08cdc389-78bf-450c-89f4-b4728edabdbf</uuid> <memory unit='KiB'>1048576</memory> <currentMemory unit='KiB'>1048576</currentMemory> <vcpu placement='static' cpuset='4-7'>1</vcpu> <numatune>
<memory mode='strict' nodeset='1'/>
</numatune> <os>
<type arch='x86_64' machine='pc-i440fx-1.5'>hvm</type> <boot dev='hd'/>
</os> <features>
<acpi/> <apic/> <pae/>
</features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices>
<emulator>/usr/bin/qemu-kvm</emulator> <controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
</controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
</controller> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes'/> <video>
<model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
</video> <memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</memballoon>
</devices>
</domain>
# virsh start test
error: Failed to start domain test error: internal error: process exited while connecting to monitor: kvm_init_vcpu failed: Cannot allocate memory
Allocating memory on this node with numactl works fine # numactl --cpubind=1 --membind=1 -- dd if=/dev/zero of=/dev/null bs=2G count=1 0+1 records in 0+1 records out 2147479552 bytes (2.1 GB) copied, 0.60816 s, 3.5 GB/s
Hmm, this makes no sense at all to me. Your configuration looks totally valid and you have plenty of memory in both nodes.
It also fails if try to assign it to another node through the cgroup interface. So it's probably a kernel and not a libvirt issue: # /bin/echo 1 > /sys/fs/cgroup/cpuset/machine/Ubuntu.libvirt-qemu/cpuset.mems /bin/echo: write error: Device or resource busy It has also been reported here: https://bugzilla.redhat.com/show_bug.cgi?id=920406 David
Daniel