- Users - libvirt List Archives

[libvirt-users] New Application to add to web site
by Matthew 11 Aug '18

11 Aug '18

Hello, Could you please add the OpenVM Dashboard to the list of applications that use libvirt. Below is the URL and description. Also, to all the developers who put libvirt-php together...THANK YOU! Thanks, Matthew Penning URL: https://openvm.tech <https://openvm.tech/> Description: The OpenVM Dashboard is an open source HTML5 and PHP based web interface for the KVM and QEMU hypervisor. It is designed to be a easy-to-use management platform allowing users to create and manage domains (virtual machines) using the libvirt-php module.

1 0

Re: [libvirt-users] Windows Guest I/O performance issues (already using virtio) (Matt Schumacher)
by Allence 09 Aug '18

09 Aug '18

I think performance is not just about your xml, the host system will have a bigger impact. Maybe you can see this link: Https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/ht… >Date: Wed, 8 Aug 2018 17:35:11 +0000 >From: Matt Schumacher <matt.s(a)aptalaska.com> >To: "libvirt-users(a)redhat.com" <libvirt-users(a)redhat.com> >Subject: [libvirt-users] Windows Guest I/O performance issues (already > using virtio) >Message-ID: <30EE4896-CF82-4E23-82D5-0C56B38F5E09(a)aptalaska.com> >Content-Type: text/plain; charset="utf-8" > >List, > >I have a number of Windows 2016 servers I am deploying, but I?m having some I/O performance issues. I have done all of the obvious things like virtio drivers, but am finding there is more performance to be found with hyper-v extensions, how we virtualize the hardware clock, and iothreads. I?m using ZVOLs to back the VM, and I?m using 4k block sizes, which seems to offer the best 4k random read/write performance (mail and database workloads), but maybe I?m missing something at this layer too. > >Questions: > > > 1. Does my VM config look reasonable for the latest releases of windows? Are there features I should be using that will help performance? > 2. Why does the hypervclock timer make so much performance difference in windows VMs? > 3. Does my virtualized CPU model make sense? I defined Haswell-noTSX-IBRS and libvirt added the features. > 4. Which kernel branch offers the best stability and performance? > 5. Are there performance gains in using UEFI booting the windows guest and defining ?<blockio logical_block_size='4096' physical_block_size='4096'/>?? Perhaps better block size consistency through to the zvol? > > >Here is my setup: > >48 core Haswell CPU >192G Ram >Linux 4.14.61 or 4.9.114 (testing both) >ZFS file system on optane SSD drive or ZFS file system on dumb HBA with 8 spindles of 15k disks (testing both) >4k block size zvol for virtual machines >32G arc cache > >Here is my VM: > ><domain type='kvm' id='12'> > <name>testvm</name> > <memory unit='KiB'>33554432</memory> > <currentMemory unit='KiB'>33554432</currentMemory> > <vcpu placement='static'>12</vcpu> > <iothreads>1</iothreads> > <os> > <type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type> > <boot dev='cdrom'/> > <boot dev='hd'/> > </os> > <features> > <acpi/> > <hyperv> > <relaxed state='on'/> > <vapic state='on'/> > <spinlocks state='on' retries='8191'/> > <vpindex state='on'/> > <runtime state='on'/> > <synic state='on'/> > <reset state='on'/> > <vendor_id state='on' value='KVM Hv'/> > </hyperv> > </features> > <cpu mode='custom' match='exact' check='full'> > <model fallback='forbid'>Haswell-noTSX-IBRS</model> > <topology sockets='1' cores='6' threads='2'/> > <feature policy='require' name='vme'/> > <feature policy='require' name='f16c'/> > <feature policy='require' name='rdrand'/> > <feature policy='require' name='hypervisor'/> > <feature policy='require' name='arat'/> > <feature policy='disable' name='spec-ctrl'/> > <feature policy='require' name='xsaveopt'/> > <feature policy='require' name='abm'/> > </cpu> > <clock offset='localtime'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='yes'/> > <timer name='hypervclock' present='yes'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <devices> > <emulator>/usr/bin/qemu-system-x86_64</emulator> > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native' ioeventfd='on' iothread='1'/> > <source dev='/dev/zvol/datastore/vm/testvm-vda'/> > <backingStore/> > <target dev='vda' bus='virtio'/> > <alias name='virtio-disk0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> > </disk> > <disk type='file' device='cdrom'> > <driver name='qemu'/> > <target dev='hdc' bus='ide'/> > <readonly/> > <alias name='ide0-1-0'/> > <address type='drive' controller='0' bus='1' target='0' unit='0'/> > </disk> > <controller type='ide' index='0'> > <alias name='ide'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> > </controller> > <controller type='usb' index='0' model='piix3-uhci'> > <alias name='usb'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> > </controller> > <controller type='pci' index='0' model='pci-root'> > <alias name='pci.0'/> > </controller> > <controller type='virtio-serial' index='0'> > <alias name='virtio-serial0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> > </controller> > <interface type='bridge'> > <source bridge='lan'/> > <target dev='vnet0'/> > <model type='virtio'/> > <alias name='net0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > </interface> > <channel type='unix'> > <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-12-testvm/org.qemu.guest_agent.0'/> > <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/> > <alias name='channel0'/> > <address type='virtio-serial' controller='0' bus='0' port='1'/> > </channel> > <input type='tablet' bus='usb'> > <alias name='input0'/> > <address type='usb' bus='0' port='1'/> > </input> > <input type='mouse' bus='ps2'> > <alias name='input1'/> > </input> > <input type='keyboard' bus='ps2'> > <alias name='input2'/> > </input> > <graphics type='vnc' port='5901' autoport='no' listen='0.0.0.0'> > <listen type='address' address='0.0.0.0'/> > </graphics> > <video> > <model type='cirrus' vram='16384' heads='1' primary='yes'/> > <alias name='video0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> > </video> > <memballoon model='none'/> > </devices> > <seclabel type='dynamic' model='dac' relabel='yes'> > <label>+0:+100</label> > <imagelabel>+0:+100</imagelabel> > </seclabel> ></domain>

1 0

[libvirt-users] Mount URL as cdrom/iso KVM/QEMU
by Inception Hosting 09 Aug '18

09 Aug '18

Hi Folks, According to the examples in http://libvirt.org/formatdomain.html#elementsDisks it should be possible to mount an iso /url as a cdrom, the example given is: </disk> <disk type='network' device='cdrom'> <driver name='qemu' type='raw'/> <source protocol="http" name="url_path"> <host name="hostname" port="80"/> </source> <target dev='hde' bus='ide' tray='open'/> <readonly/> </disk> I am unable to get this to work at all and wondered if I am just missing something obvious: Boot failed: Could not read from CDROM (code 0003) Actual XML in use snippet: <disk type='network' device='cdrom'> <driver name='qemu' type='raw'/> <source protocol="http" name="/debian-cd/current/amd64/iso-cd/debian-9.5.0-amd64-netinst.iso"> <host name="mirror.bytemark.co.uk" port="80"/> </source> <target dev='hdb' bus='ide' tray='open'/> <readonly/> </disk> libvirtd (libvirt) 3.9.0 QEMU emulator version 2.10.0 Is anyone able to offer any assistance or tips? I have tried putting the complete path including fqdn in the url_path as well, with and without http:// Many Thanks. Anthony.

3 8

[libvirt-users] Windows Guest I/O performance issues (already using virtio)
by Matt Schumacher 08 Aug '18

08 Aug '18

List, I have a number of Windows 2016 servers I am deploying, but I’m having some I/O performance issues. I have done all of the obvious things like virtio drivers, but am finding there is more performance to be found with hyper-v extensions, how we virtualize the hardware clock, and iothreads. I’m using ZVOLs to back the VM, and I’m using 4k block sizes, which seems to offer the best 4k random read/write performance (mail and database workloads), but maybe I’m missing something at this layer too. Questions: 1. Does my VM config look reasonable for the latest releases of windows? Are there features I should be using that will help performance? 2. Why does the hypervclock timer make so much performance difference in windows VMs? 3. Does my virtualized CPU model make sense? I defined Haswell-noTSX-IBRS and libvirt added the features. 4. Which kernel branch offers the best stability and performance? 5. Are there performance gains in using UEFI booting the windows guest and defining “<blockio logical_block_size='4096' physical_block_size='4096'/>”? Perhaps better block size consistency through to the zvol? Here is my setup: 48 core Haswell CPU 192G Ram Linux 4.14.61 or 4.9.114 (testing both) ZFS file system on optane SSD drive or ZFS file system on dumb HBA with 8 spindles of 15k disks (testing both) 4k block size zvol for virtual machines 32G arc cache Here is my VM: <domain type='kvm' id='12'> <name>testvm</name> <memory unit='KiB'>33554432</memory> <currentMemory unit='KiB'>33554432</currentMemory> <vcpu placement='static'>12</vcpu> <iothreads>1</iothreads> <os> <type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> </os> <features> <acpi/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <vpindex state='on'/> <runtime state='on'/> <synic state='on'/> <reset state='on'/> <vendor_id state='on' value='KVM Hv'/> </hyperv> </features> <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Haswell-noTSX-IBRS</model> <topology sockets='1' cores='6' threads='2'/> <feature policy='require' name='vme'/> <feature policy='require' name='f16c'/> <feature policy='require' name='rdrand'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='arat'/> <feature policy='disable' name='spec-ctrl'/> <feature policy='require' name='xsaveopt'/> <feature policy='require' name='abm'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='yes'/> <timer name='hypervclock' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native' ioeventfd='on' iothread='1'/> <source dev='/dev/zvol/datastore/vm/testvm-vda'/> <backingStore/> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu'/> <target dev='hdc' bus='ide'/> <readonly/> <alias name='ide0-1-0'/> <address type='drive' controller='0' bus='1' target='0' unit='0'/> </disk> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='usb' index='0' model='piix3-uhci'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <interface type='bridge'> <source bridge='lan'/> <target dev='vnet0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-12-testvm/org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <alias name='input0'/> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <graphics type='vnc' port='5901' autoport='no' listen='0.0.0.0'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='cirrus' vram='16384' heads='1' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='none'/> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+100</label> <imagelabel>+0:+100</imagelabel> </seclabel> </domain>

2 1

[libvirt-users] Copy volume from one storage to another
by Shashwat shagun 08 Aug '18

08 Aug '18

Hi, I want to copy a volume from one to pool to another pool through libvirt golang api but I’m unaware of any such functions. Can anybody guide me a little bit here? Best Regards, Shashwat Shagun me(a)shashwat.tech

3 2

[libvirt-users] LIBVIRT-4.6.0 can't work with QEMU 3.0.0
by Holger Schranz 08 Aug '18

08 Aug '18

Hello, if I try to use Llibvirt-4.6.0 together with qemu-3.0.0-rc4. I run into an issue. Please see the following Memo. Best regards Holger ================================================================================ Procduce QEMU-3.0.0-RC4 and LIBVIRT-4.6.0: QEMU-3-0-0-RC4: make clean ./configure --enable-libiscsi --enable-libusb --enable-bzip2 --enable-libnfs --enable-spice --enable-user --enable-virtfs --enable-opengl --enable-sdl --enable-gtk --enable-virglrenderer --disable-seccomp --prefix=/usr;make -j$(grep -c ^processor /proc/cpuinfo) su make install LIBVIRT-4.6.0 make clean ./configure -q --with-lxc --with-storage-iscsi --with-storage-scsi --with-interface --with-storage-lvm --with-storage-fs --with-udev --with-vmware --with-storage-mpath --prefix=/usr;make -j$(grep -c ^processor /proc/cpuinfo) su systemctl stop libvirtd.service cd libvirt-4.5.0 make uninstall cd .. cd libvirt-4.6.0 make install systemctl status libvirtd.service -l systemctl daemon-reload systemctl start libvirtd.service systemctl status libvirtd.service -l ? libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2018-08-08 09:05:26 CEST; 3s ago Docs: man:libvirtd(8) https://libvirt.org Main PID: 137948 (libvirtd) Tasks: 19 (limit: 32768) CGroup: /system.slice/libvirtd.service ?? 4725 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt_leaseshelper ?? 4726 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt_leaseshelper ??137948 /usr/sbin/libvirtd Aug 08 09:05:26 etcsvms5 systemd[1]: Starting Virtualization daemon... Aug 08 09:05:26 etcsvms5 systemd[1]: Started Virtualization daemon. Aug 08 09:05:26 etcsvms5 libvirtd[137948]: 2018-08-08 07:05:26.876+0000: 137964: info : libvirt version: 4.6.0 Aug 08 09:05:26 etcsvms5 libvirtd[137948]: 2018-08-08 07:05:26.876+0000: 137964: info : hostname: etcsvms5 Aug 08 09:05:26 etcsvms5 libvirtd[137948]: 2018-08-08 07:05:26.876+0000: 137964: error : virNetworkObjAssignDefLocked:589 : operation failed: network 'default' already exists with uuid 52abe250-4f23-40fe-875e-0808d57ce72d Aug 08 09:05:27 etcsvms5 dnsmasq[4725]: read /etc/hosts - 8 addresses Aug 08 09:05:27 etcsvms5 dnsmasq[4725]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses Aug 08 09:05:27 etcsvms5 dnsmasq-dhcp[4725]: read /var/lib/libvirt/dnsmasq/default.hostsfile Aug 08 09:05:27 etcsvms5 libvirtd[137948]: 2018-08-08 07:05:27.711+0000: 137964: warning : virLXCDriverCapsInit:82 : Failed to get host CPU cache info Aug 08 09:05:27 etcsvms5 libvirtd[137948]: 2018-08-08 07:05:27.719+0000: 137964: warning : umlCapsInit:73 : Failed to get host CPU cache info etcsvms5:/home/shl/Install/libvirt-4.6.0 # ---> test1: Start virt-manager etcsvms5:/home/shl/Install/libvirt-4.6.0 # virsh version Compiled against library: libvirt 4.6.0 Using library: libvirt 4.6.0 Using API: LXC 4.6.0 Running hypervisor: LXC 4.4.140 etcsvms5:/home/shl/Install/libvirt-4.6.0 # Information on virt-manager window: Unable to connect to libvirt qemu:///system. no connection driver available for qemu:///system Libvirt URI is: qemu:///system Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/connection.py", line 1036, in _do_open self._backend.open(self._do_creds_password) File "/usr/share/virt-manager/virtinst/connection.py", line 144, in open open_flags) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 104, in openAuth if ret is None:raise libvirtError('virConnectOpenAuth() failed') libvirtError: no connection driver available for qemu:///system ---> TEst2: Call VM via virsh create etcsvms5:/home/shl/Install/libvirt-4.6.0 # cd /kvm/CS8400/M4 etcsvms5:/kvm/CS8400/M4 # cd VLP0 etcsvms5:/kvm/CS8400/M4/VLP0 # virsh create M4-VLP0.xml error: Failed to create domain from M4-VLP0.xml error: invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm etcsvms5:/kvm/CS8400/M4/VLP0 # ----- switch back to LIBVIRT.4-5-0 and all runs o.k. --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus

3 6

[libvirt-users] Qemu guest display black after reboot
by Allence 06 Aug '18

06 Aug '18

Hey, every friend. I have a question. My guests are not working, unbelievable. I can use it the first time I use it. But after I close my guest and start over, my guests will not appear. My screen is black. My environment: Guest: windows 10 vm-bios: ovmf host: ubuntu 1604 qemu: 2.12 libvirt: 4.2 kernel: 4.17.2 xml config: gpu: <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='vfio-pci,host=0000:01:00.0,romfile=/root/rom.file,multifunction=on,x-vga=on'/> </qemu:commandline> features: <kvm> <hidden state='on'/> </kvm>

1 0

[libvirt-users] Trouble passing PCI device in isolated IOMMU group
by Quincy Wofford 06 Aug '18

06 Aug '18

I'm attempting PCI-passthrough from host to guest on an HP ProLiant 380P, which has an outdated BIOS (2014), but it does support VT-d. I'm running CentOS 7, kernel 3.10.0-862.9.1.el7.x86_64. I have an Intel 82580 NIC installed with 4 ports. Each of these ports is in its own IOMMU group (I enabled SR-IOV at the BIOS, which might be the reason they show up separately) After detaching and adding a 'hostdev' device with the appropriate pci address, I attempt to start my VM. I get " failed to set iommu for container: Operation not permitted". As recommended here ( http://vfio.blogspot.com/2014/08/vfiovga-faq.html) I parsed dmesg in an attempt to find: ------------------- No interrupt remapping support. Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform ------------------- ...but nothing similar exists in my logs. Since this device is showing up in its own IOMMU group, I assume ACS override won't get me any further. In any case, it is not an option for me to leave ACS override on. I can turn it on for testing, the server is not currently in production. Any idea why this could be failing?

2 2

[libvirt-users] How to set the MTU size of VM interface
by netsurfed 06 Aug '18

06 Aug '18

Dear all: I start a VM with MTU size 1450, and login VM to check the mtu size, but this setting didn't work. The VM xml is like this: <interface type='bridge'> <mac address='52:54:00:54:14:f8'/> <source bridge='ovsbr1'/> <virtualport type='openvswitch'> <parameters interfaceid='a42e5b42-09db-4cfa-b198-d2ce62843378'/> </virtualport> <target dev='vnet0'/> <model type='virtio'/> <mtu size='1450'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> However, the actual mtu size of VM is still 1500, as following: root@ubuntu-zhf:~# ip addr 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:54:14:f8 brd ff:ff:ff:ff:ff:ff The MTU size of both ovs bridge and vnet are 1450 in my hypervisor: root@ubuntu-191:~# ip addr show ovsbr1 16: ovsbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1 link/ether 12:01:7b:5b:85:49 brd ff:ff:ff:ff:ff:ff inet6 fe80::1001:7bff:fe5b:8549/64 scope link valid_lft forever preferred_lft forever root@ubuntu-191:~# ip addr show vnet0 22: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master ovs-system state UNKNOWN group default qlen 1000 link/ether fe:54:00:54:14:f8 brd ff:ff:ff:ff:ff:ff inet6 fe80::fc54:ff:fe54:14f8/64 scope link valid_lft forever preferred_lft forever I use ovs to create vxlan networks and add VMs to the ovs bridge. Because of the limitations of vxlan, I need to set the mtu of vm to 1450. How to set the MTU of VM? Thanks. Below some information about my hypervisor: root@ubuntu-192:~# virsh -V Virsh command line tool of libvirt 3.4.0 See web site at http://libvirt.org/ Compiled with support for: Hypervisors: QEMU/KVM LXC UML OpenVZ VMware VirtualBox Test Networking: Remote Network Bridging Interface udev Nwfilter VirtualPort Storage: Dir Filesystem SCSI Multipath iSCSI LVM Miscellaneous: Daemon Nodedev SELinux Secrets Debug Modular root@ubuntu-192:~# qemu-x86_64 --version qemu-x86_64 version 2.9.0 Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers root@ubuntu-192:~# uname -a Linux ubuntu-192 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

3 2

[libvirt-users] multiple devices in the same iommu group in L1 guest
by Yalan Zhang 06 Aug '18

06 Aug '18

Hi, I have a guest enabled vIOMMU, but on the guest there are several devices in the same iommu group. Could someone help to check if I missed something? Thank you very much! 1. guest xml: # virsh edit q ... <os> <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type> <loader readonly='yes' secure='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader> <nvram>/var/lib/libvirt/qemu/nvram/q_VARS.fd</nvram> </os> ... <features> ... <ioapic driver='qemu'/> </features> <cpu mode='host-passthrough' check='none'> <feature policy='require' name='vmx'/> </cpu> ... <devices> ... <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <interface type='network'> <mac address='52:54:00:b9:ff:90'/> <source network='default'/> <model type='e1000'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </interface> <iommu model='intel'> <driver intremap='on' caching_mode='on' iotlb='on'/> </iommu> </devices> ... 2. guest has 'intel_iommu=on' enabled in kernel cmdline, then reboot guest 3. log in guest to check: # dmesg | grep -i DMAR [ 0.000000] ACPI: DMAR 000000007d83f000 00050 (v01 BOCHS BXPCDMAR 00000001 BXPC 00000001) [ 0.000000] DMAR: IOMMU enabled [ 0.155178] DMAR: Host address width 39 [ 0.155180] DMAR: DRHD base: 0x000000fed90000 flags: 0x1 [ 0.155221] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260286 ecap f00f5e [ 0.155228] DMAR: ATSR flags: 0x1 [ 0.155231] DMAR-IR: IOAPIC id 0 under DRHD base 0xfed90000 IOMMU 0 [ 0.155232] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 0.156843] DMAR-IR: Enabled IRQ remapping in x2apic mode [ 2.112369] DMAR: No RMRR found [ 2.112505] DMAR: dmar0: Using Queued invalidation [ 2.112669] DMAR: Setting RMRR: [ 2.112671] DMAR: Prepare 0-16MiB unity mapping for LPC [ 2.112820] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff] [ 2.211577] DMAR: Intel(R) Virtualization Technology for Directed I/O ===> This is expected # dmesg | grep -i iommu |grep device [ 2.212267] iommu: Adding device 0000:00:00.0 to group 0 [ 2.212287] iommu: Adding device 0000:00:01.0 to group 1 [ 2.212372] iommu: Adding device 0000:00:02.0 to group 2 [ 2.212392] iommu: Adding device 0000:00:02.1 to group 2 [ 2.212411] iommu: Adding device 0000:00:02.2 to group 2 [ 2.212444] iommu: Adding device 0000:00:02.3 to group 2 [ 2.212464] iommu: Adding device 0000:00:02.4 to group 2 [ 2.212482] iommu: Adding device 0000:00:02.5 to group 2 [ 2.212520] iommu: Adding device 0000:00:1d.0 to group 3 [ 2.212533] iommu: Adding device 0000:00:1d.1 to group 3 [ 2.212541] iommu: Adding device 0000:00:1d.2 to group 3 [ 2.212550] iommu: Adding device 0000:00:1d.7 to group 3 [ 2.212567] iommu: Adding device 0000:00:1f.0 to group 4 [ 2.212576] iommu: Adding device 0000:00:1f.2 to group 4 [ 2.212585] iommu: Adding device 0000:00:1f.3 to group 4 [ 2.212599] iommu: Adding device 0000:01:00.0 to group 2 [ 2.212605] iommu: Adding device 0000:02:01.0 to group 2 [ 2.212621] iommu: Adding device 0000:04:00.0 to group 2 [ 2.212634] iommu: Adding device 0000:05:00.0 to group 2 [ 2.212646] iommu: Adding device 0000:06:00.0 to group 2 [ 2.212657] iommu: Adding device 0000:07:00.0 to group 2 ====> several devices in the same iommu group # virsh nodedev-dumpxml pci_0000_07_00_0 <device> <name>pci_0000_07_00_0</name> <path>/sys/devices/pci0000:00/0000:00:02.5/0000:07:00.0</path> <parent>pci_0000_00_02_5</parent> <driver> <name>e1000</name> </driver> <capability type='pci'> <domain>0</domain> <bus>7</bus> <slot>0</slot> <function>0</function> <product id='0x100e'>82540EM Gigabit Ethernet Controller</product> <vendor id='0x8086'>Intel Corporation</vendor> <iommuGroup number='2'> <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> <address domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> <address domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> <address domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> <address domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> <address domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> <address domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> <address domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> <address domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </iommuGroup> </capability> </device> Thus, can not attach the device to L2 guest: # cat hostdev.xml <hostdev mode='subsystem' type='pci' managed='yes'> <source> <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </source> </hostdev> # virsh attach-device rhel hostdev.xml error: Failed to attach device from hostdev.xml error: internal error: unable to execute QEMU command 'device_add': vfio error: 0000:07:00.0: group 2 is not viable ------- Best Regards, Yalan Zhang IRC: yalzhang Internal phone: 8389413

2 1