[libvirt] Problem setting CPU topology
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this: <domain type='kvm'> .. <vcpu placement='static'>4</vcpu> <os> <type arch='i686' machine='pc-0.15'>hvm</type> <boot dev='hd'/> </os> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='2'/> </cpu> .. The qemu commandline launched for this domain looks like this: /usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/home/zeenix/.local/share/gnome-boxes/images/fedora17-2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/extra-data/ISOs/Fedora-17-i686-Live-Desktop.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev user,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:21:6f:6e,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing -vga qxl -global qxl-vga.vram_size=67108864 -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 Despite the '-smp 4,sockets=1,cores=2,threads=2', I see only 1 CPU reported by '/proc/cpuinfo' in the guest F17 live. However, when I manually launch a very similar commandline,': /usr/bin/qemu-kvm -name fedora17-2 -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -m 1G -smp 4,cores=2,threads=2,sockets=1 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -enable-kvm -cdrom /home/zeenix/ISOs/Fedora-17-i686-Live-Desktop.iso I see 4 CPUs reported by '/proc/cpuinfo in the guest. Any clues? -- Regards, Zeeshan Ali (Khattak) FSF member#5124
On Sat, Jul 7, 2012 at 11:10 AM, Zeeshan Ali (Khattak) <zeeshanak@gnome.org> wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<snip>
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid
<snip>
Despite the '-smp 4,sockets=1,cores=2,threads=2', I see only 1 CPU reported by '/proc/cpuinfo' in the guest F17 live.
However, when I manually launch a very similar commandline,':
/usr/bin/qemu-kvm -name fedora17-2 -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -m 1G -smp 4,cores=2,threads=2,sockets=1 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -enable-kvm -cdrom /home/zeenix/ISOs/Fedora-17-i686-Live-Desktop.iso
I see 4 CPUs reported by '/proc/cpuinfo in the guest. Any clues?
What version of qemu-kvm are you using? Sounds like a bug in qemu-kvm's command line arg parsing code. -- Doug Goldstein
On Sun, Jul 8, 2012 at 1:36 AM, Doug Goldstein <cardoe@gentoo.org> wrote:
On Sat, Jul 7, 2012 at 11:10 AM, Zeeshan Ali (Khattak) <zeeshanak@gnome.org> wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<snip>
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid
<snip>
Despite the '-smp 4,sockets=1,cores=2,threads=2', I see only 1 CPU reported by '/proc/cpuinfo' in the guest F17 live.
However, when I manually launch a very similar commandline,':
/usr/bin/qemu-kvm -name fedora17-2 -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -m 1G -smp 4,cores=2,threads=2,sockets=1 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -enable-kvm -cdrom /home/zeenix/ISOs/Fedora-17-i686-Live-Desktop.iso
I see 4 CPUs reported by '/proc/cpuinfo in the guest. Any clues?
What version of qemu-kvm are you using? Sounds like a bug in qemu-kvm's command line arg parsing code.
I was able to reproduce this with both qemu from git master and qemu-kvm from stock F17 packages (1.0). -- Regards, Zeeshan Ali (Khattak) FSF member#5124
On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<domain type='kvm'> .. <vcpu placement='static'>4</vcpu> <os> <type arch='i686' machine='pc-0.15'>hvm</type> <boot dev='hd'/> </os> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='2'/> </cpu> ..
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag which libvirt added because the domain XML was missing <features><acpi/></features> So in the end that was a bug in Boxes, not really a libvirt/qemu issue. Unless libvirt/qemu should be taught that CPU topology won't work without ACPI and complain about it? Christophe
On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<domain type='kvm'> .. <vcpu placement='static'>4</vcpu> <os> <type arch='i686' machine='pc-0.15'>hvm</type> <boot dev='hd'/> </os> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='2'/> </cpu> ..
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag which libvirt added because the domain XML was missing <features><acpi/></features> So in the end that was a bug in Boxes, not really a libvirt/qemu issue. Unless libvirt/qemu should be taught that CPU topology won't work without ACPI and complain about it?
I don't think it's really impossible for any guest OS to recognize the CPU topology or boot the other CPUs without ACPI. It looks like it's just a limitation of (most?) guest OSes. If that's the case, libvirt or QEMU can't prevent the user from running a (possibly valid) configuration just because it's not very common. -- Eduardo
On Tue, Jul 10, 2012 at 2:02 PM, Eduardo Habkost <ehabkost@redhat.com> wrote:
On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<domain type='kvm'> .. <vcpu placement='static'>4</vcpu> <os> <type arch='i686' machine='pc-0.15'>hvm</type> <boot dev='hd'/> </os> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='2'/> </cpu> ..
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag which libvirt added because the domain XML was missing <features><acpi/></features> So in the end that was a bug in Boxes, not really a libvirt/qemu issue. Unless libvirt/qemu should be taught that CPU topology won't work without ACPI and complain about it?
I don't think it's really impossible for any guest OS to recognize the CPU topology or boot the other CPUs without ACPI. It looks like it's just a limitation of (most?) guest OSes. If that's the case, libvirt or QEMU can't prevent the user from running a (possibly valid) configuration just because it's not very common.
-- Eduardo
Isn't MPS and APIC support required for a guest OS to really handle the topology correctly? Now days ACPI provides most of that information so its likely that QEMU doesn't support plain old MPS + APIC. Its also possible that QEMU does support this but the kernel got confused because I see the CPUID +acpi specified in the command line while -no-acpi is on the command line as you noted. -- Doug Goldstein
On Tue, Jul 10, 2012 at 09:19:37PM -0500, Doug Goldstein wrote:
On Tue, Jul 10, 2012 at 2:02 PM, Eduardo Habkost <ehabkost@redhat.com> wrote:
On Tue, Jul 10, 2012 at 11:54:05AM +0200, Christophe Fergeau wrote:
On Sat, Jul 07, 2012 at 07:10:53PM +0300, Zeeshan Ali (Khattak) wrote:
Hi, I'm trying to set exact CPU topology to qemu-kvm domains to match host's topology. In my case, host topology is: 1 socket, 2 cores and 2 threads. If I set the XML like this:
<domain type='kvm'> .. <vcpu placement='static'>4</vcpu> <os> <type arch='i686' machine='pc-0.15'>hvm</type> <boot dev='hd'/> </os> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='2'/> </cpu> ..
The qemu commandline launched for this domain looks like this:
/usr/bin/qemu-kvm -name fedora17-2 -S -M pc-0.15 -cpu core2duo,+lahf_lm,+rdtscp,+aes,+popcnt,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds -enable-kvm -m 1152 -smp 4,sockets=1,cores=2,threads=2 -uuid c573342b-2876-05b8-098e-6d5314cab062 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/home/zeenix/.config/libvirt/qemu/lib/fedora17-2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -no-acpi
I debugged this together with Zeeshan, and the issue comes from this -no-acpi flag which libvirt added because the domain XML was missing <features><acpi/></features> So in the end that was a bug in Boxes, not really a libvirt/qemu issue. Unless libvirt/qemu should be taught that CPU topology won't work without ACPI and complain about it?
I don't think it's really impossible for any guest OS to recognize the CPU topology or boot the other CPUs without ACPI. It looks like it's just a limitation of (most?) guest OSes. If that's the case, libvirt or QEMU can't prevent the user from running a (possibly valid) configuration just because it's not very common.
-- Eduardo
Isn't MPS and APIC support required for a guest OS to really handle the topology correctly? Now days ACPI provides most of that information so its likely that QEMU doesn't support plain old MPS + APIC. Its also possible that QEMU does support this but the kernel got confused because I see the CPUID +acpi specified in the command line while -no-acpi is on the command line as you noted.
QEMU + Seabios have plain old MPS working, and from my tests it looks like: - Seabios is able to init all VCPUs correctly. - The MP-table contains only 1 entry for each CPU package (not for each core+thread). I don't understand why exactly, but I am assuming that this is a feature, not a bug (there's some discussion about it at [1]). - The CPUID information exposed by QEMU reflect the topology correctly. With that information, nothing really prevents the guest from trying to init the other cores/threads in each package after parsing the MP-table. On the other hand, I don't think there's any spec that recommends an algorithm for that (as the MPS spec is too old to have any mention of multi-core or hyper-threading). [1] http://www.coreboot.org/pipermail/coreboot/2009-November/054119.html -- Eduardo
participants (4)
-
Christophe Fergeau -
Doug Goldstein -
Eduardo Habkost -
Zeeshan Ali (Khattak)