Laine, Thank you for your response. I did use the numa element instead of using qemu commandline and that worked: 16 <cpu mode='custom' match='exact'> ... 34 <numa> 35 <cell id='0' cpus='0-1' memory='1048576' unit='KiB'/> 36 <cell id='1' cpus='2-3' memory='1048576' unit='KiB'/> 37 </numa> 38 </cpu> However, whenever I try to use the pci-expander-bus I get: error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng Extra element devices in interleave Element domain failed to validate content I tried the following 93 <controller type='pci' index='1' model='pci-expander-bus'> 94 <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> 95 </controller> and 93 <controller type='pci' index='1' model='pci-expander-bus'> 94 <target busNr='254'> 95 <node>1</node> 96 </target> 97 <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> 98 </controller> I got the syntax from: https://www.redhat.com/archives/libvir-list/2016-March/msg01224.html Just to give you a bit more info. This is running on: Linux ashehata-DellPE 3.19.0-51-generic #58-Ubuntu SMP Fri Feb 26 21:22:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux below is a dump of the working XML file I'm using thanks amir <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>MR_test01</name> <uuid>59eafc3c-9d9e-4dab-bac9-48003048c6dc</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>4</vcpu> <os> <type arch='x86_64' machine='pc-i440fx-utopic'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <pae/> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>core2duo</model> <vendor>Intel</vendor> <feature policy='require' name='pbe'/> <feature policy='require' name='tm2'/> <feature policy='require' name='ds'/> <feature policy='require' name='ss'/> <feature policy='require' name='dtes64'/> <feature policy='require' name='ht'/> <feature policy='require' name='dca'/> <feature policy='require' name='lahf_lm'/> <feature policy='require' name='tm'/> <feature policy='require' name='cx16'/> <feature policy='require' name='vmx'/> <feature policy='require' name='ds_cpl'/> <feature policy='require' name='pdcm'/> <feature policy='require' name='xtpr'/> <feature policy='require' name='acpi'/> <numa> <cell id='0' cpus='0-1' memory='1048576' unit='KiB'/> <cell id='1' cpus='2-3' memory='1048576' unit='KiB'/> </numa> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/media/ashehata/Lustre03/kvm-linux2/disk1.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/media/ashehata/Lustre03/kvm-linux2/disk4.qcow2'/> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/> </disk> <disk type='block' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='hda' bus='ide'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <controller type='ide' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </controller> <interface type='network'> <mac address='52:54:00:da:1b:af'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <interface type='network'> <mac address='52:54:00:84:0a:66'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </interface> <interface type='network'> <mac address='52:54:00:17:9d:cd'/> <source network='default'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='spice' autoport='yes'/> <sound model='ich6'> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </sound> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/> </memballoon> </devices> <qemu:commandline> <qemu:arg value='-gdb'/> <qemu:arg value='tcp::1201'/> </qemu:commandline> </domain> On 21 November 2016 at 11:40, Laine Stump <laine@laine.org> wrote:
On 11/21/2016 12:34 PM, Amir Shehata wrote:
Hello,
Hope all is well.
I've been looking at how I can create a virtual machine which is NUMA capable. I was able to do that by:
140 <qemu:commandline> 143 <qemu:arg value='-numa'/> 144 <qemu:arg value='node'/> 145 <qemu:arg value='-numa'/> 146 <qemu:arg value='node'/> 147 </qemu:commandline>
qemu commandline passthrough should only be used for qemu features that libvirt doesn't support directly (and even then it's only intended as a temporary stopgap until the feature gains official libvirt support). NUMA has been supported by libvirt for quite a long time.
If you open http://www.libvirt.org/formatdomain.html and search for "numa" you'll find descriptions of the configuration for defining NUMA memory regions, and assigning particular CPUs to particular nodes. The is also a small bit about defining a pci-expander-bus (a type of pci controller) and tying it to a particular NUMA node - this is how you can place a guest device on a particular NUMA node; just assign the device a PCI address that has the given pci-expander-bus' "index" as its "bus" attribute.
This creates two NUMA nodes.
Now I would like to assign my virtual network interfaces, of which I have three, to the different NUMA nodes on the VM.
Anyone know how I can do that? Any help/pointers would be much appreciated.
I'm trying to create a VM that will enable me to test a NUMA based feature.
The one thing I'm unclear about is whether the guest can be setup with NUMA nodes when the host itself doesn't have multiple NUMA nodes...