- Users - libvirt List Archives

Re: [libvirt-users] Sr-iov passthrough - no packet arrive to guest
by Yaron Illouz 29 Jan '15

29 Jan '15

I can see from different post that if working with sr-iov, i should work with vlan Is this an obligation to work with vlan if working with sr-iov? If not according to which parameter will the different vf get the traffic. Let's say i declare max_vfs=7, how will the traffic be seperated between the vm? However till i get an answer i tried to work with vlan And i still don't get traffic in vm, but i see packet arriving in the physical interface of the host I tried to do the following to define the vlan In virsh edit <interface type='hostdev' managed='yes'> <mac address='52:54:00:60:7f:57'/> <source> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x0'/> </source> <vlan> <tag id='44'/> </vlan> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </interface> In host ip link 23: eth6: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000 link/ether 38:ea:a7:8f:fa:98 brd ff:ff:ff:ff:ff:ff vf 0 MAC 52:54:00:60:7f:57, vlan 44 In guest modprob 8021q eth1 Link encap:Ethernet HWaddr 52:54:00:60:7F:57 inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:14 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:1156 (1.1 KiB) eth1.44 Link encap:Ethernet HWaddr 52:54:00:60:7F:57 inet6 addr: fe80::5054:ff:fe60:7f57/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:7 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 ________________________________ From: Yaron Illouz Sent: Tuesday, January 27, 2015 9:56 PM To: 'libvirt-users(a)redhat.com' Subject: Sr-iov passthrough - no packet arrive to guest Hi I am trying to do passthrough to guest using sriov and libvirt I tried to use both techniques from here http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E <http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E> Assignment from a pool of SRIOV VFs in a libvirt <network> definition Assignment with <interface type='hostdev'> (SRIOV devices only) <network> <name>passthrough-eth6</name> <forward mode='hostdev' managed='yes'> <pf dev='eth6'/> </forward> </network> I launch the vm, there is a vm with virt-install .... --network network=passthrough-eth6 I inject traffic with a generator so I defined it in promisc mode without an ip address.(ifconfig eth6 promisc up) In my physical device eth6 i can see RX increasing. But in the guest side nothing arrive in the RX? Why? I don't know in which log to look at to understand the problem I defined the network-script/eth1 file in the guest is defined as static (not dhcp) as the interface it is connected to in host have no ip. I have a another interface on the host connected via ovs and libvirt and it is working fine - you can see it in log as vnet0(thanks to this mailing-list help). In the guest part I attached the log but I think interesting part is Jan 27 17:03:21 localhost kernel: ixgbevf 0000:00:04.0: NIC Link is Up, 10 Gbps I also attached dmesg output from guest Here is the log from the host side, I can't see an error. /var/log/message Jan 27 12:02:29 1235 avahi-daemon[2598]: Withdrawing address record for fe80::fc54:ff:fe92:7352 on vnet0. Jan 27 12:02:29 1235 kernel: device vnet0 left promiscuous mode Jan 27 12:02:30 1235 kernel: ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 Jan 27 12:02:30 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --if-exists del-port vnet0 Jan 27 12:02:30 1235 ovs-vswitchd: 00180|bridge|ERR|bridge virbr0: mirror mirror-1-to-3 does not specify output; ignoring Jan 27 12:02:31 1235 ntpd[2905]: Deleting interface #28 vnet0, fe80::fc54:ff:fe92:7352#123, interface stats: received=0, sent=0, dropped=0, active_time=981 secs Jan 27 12:02:31 1235 ntpd[2905]: peers refreshed Jan 27 12:02:43 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled rdmsr: 0x345 Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0 ... Jan 27 12:02:51 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port br3 vnet0 -- set Interface vnet0 "external-ids:attached-mac=\"52:54:00:43:BF:8C\"" -- set Interface vnet0 "external-ids:iface-id=\"309ed431-417f-2843-89b8-179a066aa595\"" -- set Interface vnet0 "external-ids:vm-id=\"870e3b65-953b-01ee-84a3-c2d05300c2d9\"" -- set Interface vnet0 external-ids:iface-status=active Jan 27 12:02:51 1235 ovs-vswitchd: 00183|bridge|ERR|bridge virbr0: mirror mirror-1-to-3 does not specify output; ignoring Jan 27 12:02:51 1235 kernel: device vnet0 entered promiscuous mode Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0 Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. Jan 27 12:02:51 1235 kernel: pci-stub 0000:15:10.0: enabling device (0000 -> 0002) Jan 27 12:02:52 1235 avahi-daemon[2598]: Registering new address record for fe80::fc54:ff:fe43:bf8c on vnet0.*. Jan 27 12:02:54 1235 ntpd[2905]: Listen normally on 29 vnet0 fe80::fc54:ff:fe43:bf8c UDP 123 Jan 27 12:02:54 1235 ntpd[2905]: peers refreshed Jan 27 12:03:17 1235 kernel: __ratelimit: 23 callbacks suppressed Jan 27 12:03:17 1235 kernel: kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd Jan 27 12:03:20 1235 kernel: ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 And this is the /var/log/libvirt/qemu/probe8.log from the host 2015-01-27 17:02:51.762+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name probe8 -S -M rhel6.6.0 -enable-kvm -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 870e3b65-953b-01ee-84a3-c2d05300c2d9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/probe8.monitor,server,n owait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,ad dr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -drive file=/data/KVM/comp_VNF_MaveriQ_Probe2-cloud8.qcow2,if=none,id=drive-ide 0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/var/lib/libvirt/images/configuration8.iso,if=none,id=drive-virtio- disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=v irtio-disk0 -netdev tap,fd=25,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:43:bf:8c,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -vga cirrus -device pci-assign,host=15:10.0,id=hostdev0,configfd=26,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on char device redirected to /dev/pts/1 I can see that virtual function succeeded because I got the following output virsh net-dumpxml passthrough-eth6 <network connections='1'> <name>passthrough-eth6</name> <uuid>f8fd37cd-3215-e4b0-17b1-0b5515140db9</uuid> <forward mode='hostdev' managed='yes'> <pf dev='eth6'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x0'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x2'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x4'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x6'/> ... </forward> </network> lspci | grep Intel 15:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 15:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 15:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 15:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) .... And Intel-IOMMU: enabled in the dmesg >From host dmesg ixgbe 0000:15:00.0: setting MAC 52:54:00:97:c6:86 on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present __ratelimit: 23 callbacks suppressed kvm: 6896: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3 pci-stub 0000:15:10.6: claimed by stub device vnet0 left promiscuous mode pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. kvm: 7207: cpu0 unhandled rdmsr: 0x345 kvm: 7207: cpu0 unhandled wrmsr: 0x680 data 0 ... ixgbe 0000:15:00.0: setting MAC 52:54:00:16:b6:0c on VF 3 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode pci-stub 0000:15:10.6: enabling device (0000 -> 0002) assign device 0:15:10.6 vnet0: no IPv6 routers present __ratelimit: 24 callbacks suppressed kvm: 7254: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3 pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X device vnet0 left promiscuous mode pci-stub 0000:15:10.6: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC ba:c1:bd:19:c1:d5 on VF 3 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode ixgbe 0000:15:00.0: setting MAC 52:54:00:e2:39:62 on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present kvm: 7639: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X device vnet0 left promiscuous mode pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. kvm: 7957: cpu0 unhandled rdmsr: 0x345 kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0 ... device vnet0 entered promiscuous mode ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present __ratelimit: 23 callbacks suppressed kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X ...

1 1

[libvirt-users] Sr-iov passthrough - no packet arrive to guest
by Yaron Illouz 27 Jan '15

27 Jan '15

Hi I am trying to do passthrough to guest using sriov and libvirt I tried to use both techniques from here http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E <http://wiki.libvirt.org/page/Networking#Assignment_with_.3Chostdev.3E> Assignment from a pool of SRIOV VFs in a libvirt <network> definition Assignment with <interface type='hostdev'> (SRIOV devices only) <network> <name>passthrough-eth6</name> <forward mode='hostdev' managed='yes'> <pf dev='eth6'/> </forward> </network> I launch the vm, there is a vm with virt-install .... --network network=passthrough-eth6 I inject traffic with a generator so I defined it in promisc mode without an ip address.(ifconfig eth6 promisc up) In my physical device eth6 i can see RX increasing. But in the guest side nothing arrive in the RX? Why? I don't know in which log to look at to understand the problem I defined the network-script/eth1 file in the guest is defined as static (not dhcp) as the interface it is connected to in host have no ip. I have a another interface on the host connected via ovs and libvirt and it is working fine - you can see it in log as vnet0(thanks to this mailing-list help). In the guest part I attached the log but I think interesting part is Jan 27 17:03:21 localhost kernel: ixgbevf 0000:00:04.0: NIC Link is Up, 10 Gbps I also attached dmesg output from guest Here is the log from the host side, I can't see an error. /var/log/message Jan 27 12:02:29 1235 avahi-daemon[2598]: Withdrawing address record for fe80::fc54:ff:fe92:7352 on vnet0. Jan 27 12:02:29 1235 kernel: device vnet0 left promiscuous mode Jan 27 12:02:30 1235 kernel: ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 Jan 27 12:02:30 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --if-exists del-port vnet0 Jan 27 12:02:30 1235 ovs-vswitchd: 00180|bridge|ERR|bridge virbr0: mirror mirror-1-to-3 does not specify output; ignoring Jan 27 12:02:31 1235 ntpd[2905]: Deleting interface #28 vnet0, fe80::fc54:ff:fe92:7352#123, interface stats: received=0, sent=0, dropped=0, active_time=981 secs Jan 27 12:02:31 1235 ntpd[2905]: peers refreshed Jan 27 12:02:43 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled rdmsr: 0x345 Jan 27 12:02:43 1235 kernel: kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0 ... Jan 27 12:02:51 1235 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port br3 vnet0 -- set Interface vnet0 "external-ids:attached-mac=\"52:54:00:43:BF:8C\"" -- set Interface vnet0 "external-ids:iface-id=\"309ed431-417f-2843-89b8-179a066aa595\"" -- set Interface vnet0 "external-ids:vm-id=\"870e3b65-953b-01ee-84a3-c2d05300c2d9\"" -- set Interface vnet0 external-ids:iface-status=active Jan 27 12:02:51 1235 ovs-vswitchd: 00183|bridge|ERR|bridge virbr0: mirror mirror-1-to-3 does not specify output; ignoring Jan 27 12:02:51 1235 kernel: device vnet0 entered promiscuous mode Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0 Jan 27 12:02:51 1235 kernel: ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. Jan 27 12:02:51 1235 kernel: pci-stub 0000:15:10.0: enabling device (0000 -> 0002) Jan 27 12:02:52 1235 avahi-daemon[2598]: Registering new address record for fe80::fc54:ff:fe43:bf8c on vnet0.*. Jan 27 12:02:54 1235 ntpd[2905]: Listen normally on 29 vnet0 fe80::fc54:ff:fe43:bf8c UDP 123 Jan 27 12:02:54 1235 ntpd[2905]: peers refreshed Jan 27 12:03:17 1235 kernel: __ratelimit: 23 callbacks suppressed Jan 27 12:03:17 1235 kernel: kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd Jan 27 12:03:20 1235 kernel: ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 And this is the /var/log/libvirt/qemu/probe8.log from the host 2015-01-27 17:02:51.762+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name probe8 -S -M rhel6.6.0 -enable-kvm -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 870e3b65-953b-01ee-84a3-c2d05300c2d9 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/probe8.monitor,server,n owait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,ad dr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -drive file=/data/KVM/comp_VNF_MaveriQ_Probe2-cloud8.qcow2,if=none,id=drive-ide 0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/var/lib/libvirt/images/configuration8.iso,if=none,id=drive-virtio- disk0,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=v irtio-disk0 -netdev tap,fd=25,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:43:bf:8c,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -vga cirrus -device pci-assign,host=15:10.0,id=hostdev0,configfd=26,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on char device redirected to /dev/pts/1 I can see that virtual function succeeded because I got the following output virsh net-dumpxml passthrough-eth6 <network connections='1'> <name>passthrough-eth6</name> <uuid>f8fd37cd-3215-e4b0-17b1-0b5515140db9</uuid> <forward mode='hostdev' managed='yes'> <pf dev='eth6'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x0'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x2'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x4'/> <address type='pci' domain='0x0000' bus='0x15' slot='0x10' function='0x6'/> ... </forward> </network> lspci | grep Intel 15:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 15:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 15:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) 15:10.1 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) .... And Intel-IOMMU: enabled in the dmesg >From host dmesg ixgbe 0000:15:00.0: setting MAC 52:54:00:97:c6:86 on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present __ratelimit: 23 callbacks suppressed kvm: 6896: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3 pci-stub 0000:15:10.6: claimed by stub device vnet0 left promiscuous mode pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. kvm: 7207: cpu0 unhandled rdmsr: 0x345 kvm: 7207: cpu0 unhandled wrmsr: 0x680 data 0 ... ixgbe 0000:15:00.0: setting MAC 52:54:00:16:b6:0c on VF 3 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode pci-stub 0000:15:10.6: enabling device (0000 -> 0002) assign device 0:15:10.6 vnet0: no IPv6 routers present __ratelimit: 24 callbacks suppressed kvm: 7254: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 3 pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X pci-stub 0000:15:10.6: irq 62 for MSI/MSI-X pci-stub 0000:15:10.6: irq 63 for MSI/MSI-X device vnet0 left promiscuous mode pci-stub 0000:15:10.6: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC ba:c1:bd:19:c1:d5 on VF 3 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. device vnet0 entered promiscuous mode ixgbe 0000:15:00.0: setting MAC 52:54:00:e2:39:62 on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present kvm: 7639: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X pci-stub 0000:15:10.0: irq 63 for MSI/MSI-X device vnet0 left promiscuous mode pci-stub 0000:15:10.0: restoring config space at offset 0x1 (was 0x100000, writing 0x100004) ixgbe 0000:15:00.0: setting MAC a2:fc:a8:89:2a:8f on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. kvm: 7957: cpu0 unhandled rdmsr: 0x345 kvm: 7957: cpu0 unhandled wrmsr: 0x680 data 0 ... device vnet0 entered promiscuous mode ixgbe 0000:15:00.0: setting MAC 52:54:00:d9:f1:bd on VF 0 ixgbe 0000:15:00.0: Reload the VF driver to make this change effective. pci-stub 0000:15:10.0: enabling device (0000 -> 0002) assign device 0:15:10.0 vnet0: no IPv6 routers present __ratelimit: 23 callbacks suppressed kvm: 8006: cpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd ixgbe 0000:15:00.0: eth6: VF Reset msg received from vf 0 pci-stub 0000:15:10.0: irq 62 for MSI/MSI-X ...

1 0

[libvirt-users] live migration taking forever
by Xu (Simon) Chen 27 Jan '15

27 Jan '15

Hey folks, I am running libvirt 1.2.4 and qemu 2.1 on a 3.14.27 kernel. I've found that live migrating a relatively large VM (16 cores and 64G ram) is taking forever - close to 15 hours now, and still not done... With "lsof -i", I can see a connection is established from my source hypervisor to a target hypervisor, likely for the purpose of copying data. nettop shows that this connection is constantly sending 50-60MBps traffic. The VM being migrated has a disk on ceph by using librbd. I wonder if anyone has seen similar issues, and how I could troubleshoot further. (I tried but failed to get qemu monitor to work on the VM...) Thanks. -Simon

4 3

[libvirt-users] Probem with Windows 2012 R2 server
by Dominique Ramaekers 26 Jan '15

26 Jan '15

Hi, I'm having a lot of trouble with a Server 2012R2 evaluation version. I'm testing it to see how good it works in libvirt-kvm. It seems the server won't run longer than a few days. After a few days, when I shut down the server, she won't start again. Has anybody have experienced the same??? My Windows 8.1, and Linux VM's works perfectly... Greetings

1 0

[libvirt-users] questions around using numatune/numa/schedinfo
by Chris Friesen 26 Jan '15

26 Jan '15

Hi, I'm running into some problems with libvirt and hoping someone can point me at some instructions or maybe even help me out. First, are there any requirements on qemu version in order to use the "numatune" and/or "cpu/numa/cell" elements? Or do they use cgroups and not the native qemu numa support? Second, are there any instructions on how to set up cgroups? I initially hadn't had cgroups mounted and running "virsh schedinfo <domain>" gave an error. So I mounted "/sys/fs/cgroup/memory" and "/sys/fs/cgroup/cpu" and restarted libvirtd but running "virsh schedinfo <domain>" still gives an error: root@compute-1:~# virsh schedinfo 8 Scheduler : Unknown error: Requested operation is not valid: cgroup CPU controller is not mounted What am I missing? Thanks, Chris

3 3

[libvirt-users] libvirt + xen assertion
by CloudPatch Staff 22 Jan '15

22 Jan '15

We're hitting an assert whenever we try to create an HVM instance under Xen via libvirtd. System is running on Gentoo, package information as follows: app-emulation/xen-4.5.0 USE="api debug flask hvm pam pygrub python qemu screen" app-emulation/xen-tools-4.5.0 USE="api debug flask hvm pam pygrub python qemu screen" app-emulation/libvirt-1.2.11-r2:0/1.2.11 USE="caps libvirtd lvm macvtap nls qemu udev vepa virtualbox xen" The following commands are run in parallel: vmmachine ~ # libvirtd --listen 2015-01-22 16:33:13.596+0000: 2620: info : libvirt version: 1.2.11 2015-01-22 16:33:13.596+0000: 2620: error : udevGetDMIData:1607 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' libvirtd: libxl_fork.c:350: sigchld_installhandler_core: Assertion `((void)"application must negotiate with libxl about SIGCHLD", !(sigchld_saved_action.sa_flags & 4) && (sigchld_saved_action.__sigaction_handler.sa_handler == ((__sighandler_t) 0) || sigchld_saved_action.__sigaction_handler.sa_handler == ((__sighandler_t) 1)))' failed. Aborted vmmachine ~ # VIRSH_DEBUG=0 virsh create xml create: file(optdata): xml libvirt: XML-RPC error : End of file while reading data: Input/output error error: Failed to create domain from xml error: End of file while reading data: Input/output error libvirt: Domain Config error : Requested operation is not valid: A different callback was requested

1 0

[libvirt-users] openvswitch port with no ip
by Yaron Illouz 21 Jan '15

21 Jan '15

I used openvswitch with virt-intall I defined a network <network> <name>private</name> <bridge name='virbr0'/> <forward mode='bridge'/> <virtualport type='openvswitch'/> </network> But get no ip for the ovs port when I launch the instance. Is this correct? ci-info: +++++++++++++++++++++++++Net device info++++++++++++++++++++++++++ ci-info: +--------+------+--------------+-------------+-------------------+ ci-info: | Device | Up | Address | Mask | Hw-Address | ci-info: +--------+------+--------------+-------------+-------------------+ ci-info: | lo | True | 127.0.0.1 | 255.0.0.0 | . | ci-info: | eth1 | True | . | . | 52:54:00:24:16:29 | ci-info: | eth0 | True | 172.16.5.136 | 255.255.0.0 | 52:54:00:44:01:28 | In the bridge port vnet1 get assigned to virbr0 Bridge "virbr0" Port "virbr0" Interface "virbr0" type: internal Port "vnet1" Interface "vnet1" And i launch the instance the following way virt-install --name probe9 --ram 4096 --vcpus=4 --check-cpu --os-type=linux --import --disk /data/KVM/Probe2-cloud3.qcow2,format=qcow2 --network network=ext --network network=private --graphics vnc,listen=0.0.0.0 --noautoconsole --disk vol=default/configuration2.iso,bus=virtio Regards

2 1

Re: [libvirt-users] [libvirt] libvirt 1.2.10 and latest EL6 qemu-kvm
by Kashyap Chamarthy 20 Jan '15

20 Jan '15

[Dropping libvir list and adding libvir-users list.] On Mon, Jan 19, 2015 at 04:12:46PM +0100, liedekef(a)telenet.be wrote: > Hi all, > > I had libvirt 1.2.10 running without issues together with the qemu package > qemu-kvm-0.12.1.2-2.415.el6_5.14.x86_64.rpm That's a really old QEMU and the libvirt you're using is relatively newer. > Now, after updating my server, qemu-kvm became version > qemu-kvm-0.12.1.2-2.448.el6_6.x86_64 > > And as a result, when doing 'virsh start' (virsh define worked ok), the following error occured: > > 2015-01-19 15:05:26.539+0000: 4347: error : qemuMonitorJSONCheckError:379 : internal error: unable to execute QEMU command 'qom-list': The command qom-list has not been You seem to be hitting this bug (although the bug is for RHEL7, looks like the same behavior): https://bugzilla.redhat.com/show_bug.cgi?id=1082124 Where Michal Privoznik says, it should already be fixed with these commits in libvirt: 730af8f2cd7bc0e4c98b97200857909f42ea817f -- qemuMonitorJSONGetCPUx86Data: Don't fail on ancient qemus 84f69602143551433e3e0a5226dc572ecb33c059 -- qemu: Check for presence of device and properities when getting CPUID a6a6f84af92a506f83fdecf56f292bcb89905492 -- qemu: Change return type of qemuMonitorGetGuestCPU() > > The only thing I could do (even downgrading libvirt to 1.2.9 didn't > work), was to downgrade qemu-kvm to the older version again ... Any > hints on this? For the EL distrubtione you're using, I think using whatever the newest supported combination of libvirt and QEMU should help. -- /kashyap

2 2

[libvirt-users] Guests using more ram than specified
by Dennis Jacobfeuerborn 19 Jan '15

19 Jan '15

Hi, today I noticed that one of my HVs started swapping aggressively and noticed that the two guests running on it use quite a bit more ram than I assigned to them. They respectively were assigned 124G and 60G with the idea that the 192G system then has 8G for other purposes. In top I see the VMs using about 128G and 64G which means there is nothing left for the system. This is on a CentOS 7 system. Any ideas what causes this or how I can calculate the actual maximum amount of RAM I can assign to the guests on a HV without overcommitting RAM? Regards, Dennis

3 3

[libvirt-users] volume can't be attached after libvirt upgrade
by Chengyuan Li 19 Jan '15

19 Jan '15

Hi, My nova-compute nodes was running libvirt 0.9.13 and QEMU 1.0, after libvirt was upgraded to 1.2.2, the VMs launched by libvirt 0.9.13 can't attach volume any more, i.e. "nova volume-attach <vm-uuid> <volume-uuid>" doesn't work. But before the libvirt upgrade, the same VM can attach volume disk. I did some debuging and find that, it failed in QEMU when try to open the volume device, and the return errno is EPERM. QEMU instances is running as user libvirt-qemu always, so is that possible some problem in Linux capabilites or libvirt apparmor? ------------------------------------- int qemu_open(const char *name, int flags, ...) { int ret; int mode = 0; long euid, egid; if (flags & O_CREAT) { va_list ap; va_start(ap, flags); mode = va_arg(ap, int); va_end(ap); } euid = (long)geteuid(); egid = (long)getegid(); #ifdef O_CLOEXEC ret = open(name, flags | O_CLOEXEC, mode); #else ret = open(name, flags, mode); if (ret >= 0) { qemu_set_cloexec(ret); } #endif return ret; } Regards, CY.

1 1