Hello, On KubeVirt, we are trying to pre-create a tap device, then instruct libvirt to consume it (via the type=ethernet , managed='no' attributes). It works as expected, **unless** when we create a multi-queue tap device. The difference when creating the tap device is that we set the multi-queue flag; libvirt throws the following error when consuming it: ``` LibvirtError(Code=38, Domain=0, Message='Unable to create tap device tap0: Invalid argument') ``` After digging a bit on the libvirt code (we're using libvirt 6.0.0), I see this on the logs (immediately before the error): {"component":"virt-launcher","level":"info","msg":"Enabling IFF_VNET_HDR","pos":"virNetDevProbeVnetHdr:190","subcomponent":"libvirt","thread":"33" ,"timestamp":"2020-09-22T10:34:29.335000Z"} I do not understand how it can try to set the VNET_HDR flag, since I have not set it when I created it, which, as per [0] should only happen when requested. Here's the tap device I'm creating: (output of `ip tuntap show`) - tap0: tap persist0x100 user 107 group 107

I'm confused by this, since on [1] it says that only one flag can be used (persist *or* vnet_hdr). Also, from my limited understanding, in order to open a pre-created tap device, it must be set to persistent.

Am I right (in the sense that these flags cannot co-exist) ? If so, why would libvirt be setting the VNET_HDR flag on the tap device ? [0] - https://github.com/libvirt/libvirt/blob/v6.0.0/src/util/virnetdevtap.c#L261 [1] - https://github.com/libvirt/libvirt/blob/v6.0.0/src/util/virnetdevtap.c#L209

On Wed, Sep 23, 2020 at 05:44:28PM +0100, Daniel P. Berrangé wrote: > On Tue, Sep 22, 2020 at 01:48:08PM +0200, Miguel Duarte de Mora Barroso wrote: > > Hello, > > > > On KubeVirt, we are trying to pre-create a tap device, then instruct > > libvirt to consume it (via the type=ethernet , managed='no' > > attributes). > > > > It works as expected, **unless** when we create a multi-queue tap device. > > > > The difference when creating the tap device is that we set the > > multi-queue flag; libvirt throws the following error when consuming > > it: > > > > ``` > > LibvirtError(Code=38, Domain=0, Message='Unable to create tap device > > tap0: Invalid argument') > > ``` > > > > After digging a bit on the libvirt code (we're using libvirt 6.0.0), I > > see this on the logs (immediately before the error): > > {"component":"virt-launcher","level":"info","msg":"Enabling > > IFF_VNET_HDR","pos":"virNetDevProbeVnetHdr:190","subcomponent":"libvirt","thread":"33" > > ,"timestamp":"2020-09-22T10:34:29.335000Z"} > > > > I do not understand how it can try to set the VNET_HDR flag, since I > > have not set it when I created it, which, as per [0] should only > > happen when requested. Here's the tap device I'm creating: (output of > > `ip tuntap show`) > > - tap0: tap persist0x100 user 107 group 107 > > IIUC the kernel code correctly, the VNET_HDR flag is not required > to be set when you first create the tap device - it appears to be > permitted any time you open a file descriptor for it. > > AFAIK, there is no problem with VNET_HDR, as it is a standard flag > we've set on all tap devices on Linux for 10 years. Looking at the kernel code, you need to set the MULTI_QUEUE flag at time you create the device and also set it when opening the device. In tun_set_iff(): if (!!(ifr->ifr_flags & IFF_MULTI_QUEUE) != !!(tun->flags & IFF_MULTI_QUEUE)) return -EINVAL; so if you've configured QEMU to use multiqueue, the you need to use: $ ip tuntap add dev mytap mode tap vnet_hdr multi_queue actually vnet_hdr doesn't matter as that can be set on the fly but multi_queue is mandatory. Without it, I get the same EINVAL error as you mention.

Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|