-----Original Message----- From: Erik Skultety [mailto:eskultet@redhat.com] Sent: Thursday, March 16, 2017 10:41 PM To: Chen, Xiaoguang <xiaoguang.chen@intel.com> Cc: libvir-list@redhat.com; He, Yongli <yongli.he@intel.com> Subject: Re: [libvirt] [RFC PATCH v2 REBASE 00/18] Introduce vGPU mdev framework to libvirt
[2] 2005 ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ *************************************************** start libvirt-d 2017-03-09 19:04:57.211+0000: 2059: info : libvirt version: 3.1.0 2017-03-09 19:04:57.211+0000: 2059: info : hostname: z-nuc-11.maas 2017-03-09 19:04:57.211+0000: 2059: error : qemuMonitorOpenUnix:367 : failed to connect to monitor socket: No such process 2017-03-09 19:04:57.213+0000: 2059: error : virMediatedDeviceGetIOMMUGroupDev:153 : internal error: IOMMU group file /sys/bus/mdev/devices/894f3983-1a36-42b3-b52c- 1024aca216be/iommu_group is not a symlink
When I saw this error message for the first time in the original thread, I got confused, since this just checks whether the symlink exists, if it doesn't, the vfio device probably also doesn't exist (but take this with a grain of salt, I haven't investigated that deep) and libvirt needs it to pass it onto qemu command line. I hit this issue once by accident in the past and at that time I didn't understand what caused it, but after a reboot it was gone. So seeing it here it caught my eye and I investigated it last week. What I found out was that it's caused by the vfio-mdev module not being loaded automatically as a dependency. I solved it by autoloading the module on system boot. So this is not a libvirt issue, but just for a reference, there is a BZ on this [1].
2017-03-09 19:04:57.213+0000: 2003: info : libvirt version: 3.1.0 2017-03-09 19:04:57.213+0000: 2003: info : hostname: z-nuc-11.maas 2017-03-09 19:04:57.213+0000: 2003: error : virNetSocketReadWire:1800 : End of file while reading data: Input/output error
I suppose this corresponds to the problem above, do you hit this error if you work around the vfio-mdev module problem described above? Based on the KVMGT setup guide we should add kvmgt, vfio-iommu-type1 and vfio-mdev modules into initfamfs. So I don't think it is the same problem. But we will double confirm it later. Yong li is on vacation this week when he come back we will do the test again.
the screen call trace while start the VM (same for Ubuntu, Win10 etc) ======================================================
ubuntu@z-nuc-11:~/vgpu-meta/libvirt-stage$ myvirsh start vgpu-ubuntu 2017-03-09 19:06:50.483+0000: 2232: info : libvirt version: 3.1.0 2017-03-09 19:06:50.483+0000: 2232: info : hostname: z-nuc-11.maas 2017-03-09 19:06:50.483+0000: 2232: warning : qemuDomainObjTaint:4056 : Domain id=1 name='vgpu-ubuntu' uuid=972b5e38-0437-11e7-8f97-d36dba74552d is tainted: high-privileges 2017-03-09 19:06:50.819+0000: 2204: info : libvirt version: 3.1.0 2017-03-09 19:06:50.819+0000: 2232: warning : virDomainAuditHostdev:456 : Unexpected hostdev type while encoding audit message: 4
This one's interesting, again, are you able to hit the error when you work around the missing vfio-mdev module? I'll have a look at this if you actually can hit the error, even if the XML is correct.
I posted v3 of the series and also created a new branch 'mdev-next' on my github [2]. I dropped the attribute 'type' from the source address element, so follow the example in the updated docs.
Will test the v3 when yong li back from vacation.
Thanks for giving it a try. Erik
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1420572 [2] https://github.com/eskultety/libvirt/commits/mdev-next