failed to start vm after add vsock device
Hi, all: there is no error when launch qemu-kvm from cli directly, but vm fails to start via libvirtd. i have tried to chmod 0666 /dev/vhost-vsock. error: internal error: qemu unexpectedly closed the monitor: 2021-01-26T13:06:06.403097Z qemu-kvm: -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4: vhost-vsock: failed to open vhost device: Unknown error -13 <qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4'/> </qemu:commandline> thanks
On 1/26/21 2:13 PM, longguang.yue wrote:
Hi, all:
there is no error when launch qemu-kvm from cli directly, but vm fails to start via libvirtd. i have tried to chmod 0666 /dev/vhost-vsock.
error: internal error: qemu unexpectedly closed the monitor: 2021-01-26T13:06:06.403097Z qemu-kvm: -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4: vhost-vsock: failed to open vhost device: Unknown error -13
Errno 13 is EACCES (Permission denied) which means that libvirt didn't set seclabel on something ...
<qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4'/> </qemu:commandline>
.. and this is explains why. Anything that's added via qemu commandline passthru is opaque to libvirt. Libvirt does not examine it, nor it sets any labels, nothing. If you use it, you're on your own. However, vsock was added to libvirt (almost 3 years ago) and instead of passing through a command line you can define vsock device: https://libvirt.org/formatdomain.html#vsock For instance like this: <vsock model='virtio'> <cid auto='no' address='3'/> </vsock> Michal
Michal, thanks. i have another question which is related to kata-container. when there is only one virtiofs-device , how does it do that in guest there are 4 times of virtiofs-mounts that have same src and different targets. # in guest [root@kvm kata-containers]# docker exec efda32ca6a93 mount | grep kataShared kataShared on / type virtiofs (rw,relatime) kataShared on /etc/resolv.conf type virtiofs (rw,relatime) kataShared on /etc/hostname type virtiofs (rw,relatime) kataShared on /etc/hosts type virtiofs (rw,relatime) # qemu-kvm -chardev socket,id=char-c91f3c6a619cec75,path=/run/vc/vm/efda32ca6a93491ac173dc2ad8a38ac095abab3bd8147a1101851f2a0a8d9012/vhost-fs.sock -device vhost-user-fs-pci,chardev=char-c91f3c6a619cec75,tag=kataShared,romfile= At 2021-01-27 21:31:49, "Michal Privoznik" <mprivozn@redhat.com> wrote:
On 1/26/21 2:13 PM, longguang.yue wrote:
Hi, all:
there is no error when launch qemu-kvm from cli directly, but vm fails to start via libvirtd. i have tried to chmod 0666 /dev/vhost-vsock.
error: internal error: qemu unexpectedly closed the monitor: 2021-01-26T13:06:06.403097Z qemu-kvm: -device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4: vhost-vsock: failed to open vhost device: Unknown error -13
Errno 13 is EACCES (Permission denied) which means that libvirt didn't set seclabel on something ...
<qemu:commandline> <qemu:arg value='-device'/> <qemu:arg value='vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=4'/> </qemu:commandline>
.. and this is explains why. Anything that's added via qemu commandline passthru is opaque to libvirt. Libvirt does not examine it, nor it sets any labels, nothing. If you use it, you're on your own. However, vsock was added to libvirt (almost 3 years ago) and instead of passing through a command line you can define vsock device:
https://libvirt.org/formatdomain.html#vsock
For instance like this:
<vsock model='virtio'> <cid auto='no' address='3'/> </vsock>
Michal
On 1/28/21 12:08 PM, longguang.yue wrote:
Michal, thanks.
i have another question which is related to kata-container.
when there is only one virtiofs-device , how does it do that in guest there are 4 times of virtiofs-mounts that have same src and different targets.
# in guest [root@kvm kata-containers]# docker exec efda32ca6a93 mount | grep kataShared kataShared on / type virtiofs (rw,relatime) kataShared on /etc/resolv.conf type virtiofs (rw,relatime) kataShared on /etc/hostname type virtiofs (rw,relatime) kataShared on /etc/hosts type virtiofs (rw,relatime)
I'm not sure how this is related to libvirt, but I'll try to answer anyway. I believe these mount points are set up by the initrd in the guest. And this confusion you are seeing is not related to virtiofs at all. It's non-intuitive way of how 'mount' shows bind mounts. One can bind mount a file. For instance: # touch /tmp/a /tmp/b # mount --bind /tmp/a /tmp/b # mount | grep /tmp/b tmpfs on /tmp/b type tmpfs (rw,nosuid,nodev,seclabel,nr_inodes=409600,inode64) As you can see, mount doesn't show /tmp/a as the source of the mount point but the FS associated. And I believe this is what's happening. kataShared is mount as root, but then those three files from /etc are bind mounted and thus mount shows kataShared as their source. I agree it's misleading (and I remember running into this same problem when developing private namespaces for QEMU VMs - but that's another story). Michal
participants (2)
-
longguang.yue -
Michal Privoznik