Libvirt qemu-system-x86_64 on ppc64le no multi threading

hi, i’m running Debian 10 on POWER9 and would like to spawn x86_64 emulated VMs. The virtual machine is configured to run with 8 vpus but it’s very slow. On the host you can see that the qemu-system-x86_64 process is using just one core! Ppc64le guests are using multi cores, so its looks like an config problem or software bug with the x86 emulator. <domain type='qemu'> <name>ubuntu</name> <uuid>a38148e3-6a85-4793-8d58-8725e8bef25a</uuid> <metadata> <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://debian.org/debian/10"/> </libosinfo:libosinfo> </metadata> <memory unit='KiB'>33554432</memory> <currentMemory unit='KiB'>33554432</currentMemory> <vcpu placement='static'>8</vcpu> <os> <type arch='x86_64' machine='pc-q35-5.0'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='custom' match='exact' check='none'> <model fallback='forbid'>qemu64</model> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/ubuntu-x86_64.qcow2'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sda' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0x16'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/> </controller> <interface type='bridge'> <mac address='52:54:00:cc:08:18'/> <source bridge='br0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes'> <listen type='address'/> </graphics> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <backend model='random'>/dev/urandom</backend> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </rng> </devices> </domain> # ps ax|grep qemu 31546 ? Sl 2:38 /usr/bin/qemu-system-x86_64 -name guest=ubuntu,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-59-ubuntu/master-key.aes -machine pc-q35-5.0,accel=tcg,usb=off,dump-guest-core=off -cpu qemu64 -m 32768 -overcommit mem-lock=off -smp 8,sockets=8,cores=1,threads=1 -uuid a38148e3-6a85-4793-8d58-8725e8bef25a -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=32,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-hpet -no-reboot -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/ubuntu-x86_64.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-2-format","read-only":false,"driver":"qcow2","file":"libvirt-2-storage","backing":null} -device virtio-blk-pci,bus=pci.4,addr=0x0,drive=libvirt-2-format,id=virtio-disk0,bootindex=2 -blockdev {"driver":"file","filename":"/var/lib/libvirt/images/ubuntu-20.04-live-server-amd64.iso","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} -blockdev {"node-name":"libvirt-1-format","read-only":true,"driver":"raw","file":"libvirt-1-storage"} -device ide-cd,bus=ide.0,drive=libvirt-1-format,id=sata0-0-0,bootindex=1 -netdev tap,fd=34,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:cc:08:18,bus=pci.1,addr=0x0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=35,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:1 -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on Does anybody knows this problem? Best regards Kim

On Mon, Aug 03, 2020 at 01:45:45PM +0000, Kim-Norman Sahm wrote:
hi,
i’m running Debian 10 on POWER9 and would like to spawn x86_64 emulated VMs. The virtual machine is configured to run with 8 vpus but it’s very slow. On the host you can see that the qemu-system-x86_64 process is using just one core!
Ppc64le guests are using multi cores, so its looks like an config problem or software bug with the x86 emulator.
snip
Does anybody knows this problem?
You've not mentioned what QMEU version or what libvirt version you have. Historically though emulation was single threaded. Only with MTTCG has it become multi-threaded and thus able to use all cores for vCPUs. THat is a few years old now though, so most distros should have MTTCG in some form or another if reasonably up2date. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

Hi Daniel, attached the versions: # dpkg -l|grep qemu ii ipxe-qemu 1.0.0+git-20190125.36a4c85-5 all PXE boot firmware - ROM images for qemu ii libvirt-daemon-driver-qemu 6.4.0-2 ppc64el Virtualization daemon QEMU connection driver ii qemu 1:5.0-13 ppc64el fast processor emulator, dummy package ii qemu-block-extra 1:5.0-13 ppc64el extra block backend modules for qemu-system and qemu-utils ii qemu-system-common 1:5.0-13 ppc64el QEMU full system emulation binaries (common files) ii qemu-system-data 1:5.0-13 all QEMU full system emulation (data files) ii qemu-system-gui:ppc64el 1:5.0-13 ppc64el QEMU full system emulation binaries (user interface and audio support) ii qemu-system-ppc 1:5.0-13 ppc64el QEMU full system emulation binaries (ppc) ii qemu-system-x86 1:5.0-13 ppc64el QEMU full system emulation binaries (x86) ii qemu-utils 1:5.0-13 ppc64el QEMU utilities # dpkg -l|grep libvirt ii libvirt-clients 6.4.0-2 ppc64el Programs for the libvirt library ii libvirt-daemon 6.4.0-2 ppc64el Virtualization daemon ii libvirt-daemon-driver-qemu 6.4.0-2 ppc64el Virtualization daemon QEMU connection driver ii libvirt-daemon-system 6.4.0-2 ppc64el Libvirt daemon configuration files ii libvirt-daemon-system-systemd 6.4.0-2 ppc64el Libvirt daemon configuration files (systemd) ii libvirt-dbus 1.4.0-1 ppc64el libvirt D-Bus API bindings ii libvirt-glib-1.0-0:ppc64el 3.0.0-1 ppc64el libvirt GLib and GObject mapping library ii libvirt0:ppc64el 6.4.0-2 ppc64el library for interfacing with different virtualization systems ii python3-libvirt 6.1.0-1+b1 ppc64el libvirt Python 3 bindings # qemu-system-x86_64 --version QEMU emulator version 5.0.0 (Debian 1:5.0-13) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers # libvirtd --version libvirtd (libvirt) 6.4.0 Regards, Kim Am 03.08.20, 16:36 schrieb "Daniel P. Berrangé" <berrange@redhat.com>: On Mon, Aug 03, 2020 at 01:45:45PM +0000, Kim-Norman Sahm wrote: > hi, > > i’m running Debian 10 on POWER9 and would like to spawn x86_64 emulated VMs. > The virtual machine is configured to run with 8 vpus but it’s very slow. > On the host you can see that the qemu-system-x86_64 process is using just one core! > > Ppc64le guests are using multi cores, so its looks like an config problem or software bug with the x86 emulator. snip > Does anybody knows this problem? You've not mentioned what QMEU version or what libvirt version you have. Historically though emulation was single threaded. Only with MTTCG has it become multi-threaded and thus able to use all cores for vCPUs. THat is a few years old now though, so most distros should have MTTCG in some form or another if reasonably up2date. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Aug 03, 2020 at 03:35:49PM +0100, Daniel P. Berrangé wrote:
On Mon, Aug 03, 2020 at 01:45:45PM +0000, Kim-Norman Sahm wrote:
hi,
i’m running Debian 10 on POWER9 and would like to spawn x86_64 emulated VMs. The virtual machine is configured to run with 8 vpus but it’s very slow. On the host you can see that the qemu-system-x86_64 process is using just one core!
Ppc64le guests are using multi cores, so its looks like an config problem or software bug with the x86 emulator.
snip
Does anybody knows this problem?
You've not mentioned what QMEU version or what libvirt version you have.
Actually this doens't matter.
Historically though emulation was single threaded. Only with MTTCG has it become multi-threaded and thus able to use all cores for vCPUs. THat is a few years old now though, so most distros should have MTTCG in some form or another if reasonably up2date.
x86_64 guest on ppc64 host simply cannot use MTTCG at all, so will always be restricted to 1 thread. If you want concurrency you'll need to run on a real x86_64 host. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (3)
-
Daniel P. Berrange
-
Daniel P. Berrangé
-
Kim-Norman Sahm