Hi all, I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0]. My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs. I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces. Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ? [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367

On 4/6/20 9:54 AM, Daniel P. Berrangé wrote: > > Would you be able to shed some light into this ? Is it possible on > > libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ? > > > > [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 > This links to the following message, which illustrates how to use pre-create > tap and macvtap devices: > > https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html > > Laine: it would be useful to add something like this short guide to the > knowledge base docs You mean the wiki? Sure, I can do that.

(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))

On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump(a)redhat.com&gt; wrote: > > On 4/6/20 9:54 AM, Daniel P. Berrangé wrote: > > On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote: > >> Hi all, > >> > >> I'm aware that it is possible to plug pre-created macvtap devices to > >> libvirt guests - tracked in RFE [0]. > >> > >> My interpretation of the wording in [1] and [2] is that it is also > >> possible to plug pre-created tap devices into libvirt guests - that > >> would be a requirement to allow kubevirt to run with less capabilities > >> in the pods that encapsulate the VMs. > >> > >> I took a look at the libvirt code ([3] & [4]), and, from my limited > >> understanding, I got the impression that plugging existing interfaces > >> via `managed='no' ` is only possible for macvtap interfaces. > > > No, it works for standard tap devices as well. > > > The reason the BZs and commit logs talk mostly about macvtap rather than > tap is because 1) that's what kubevirt people had asked for and 2) it > already *mostly* worked for tap devices, so most of the work was related > to macvtap (my memory is already fuzzy, but I think there were a couple > privileged operations we still tried to do for standard tap devices even > if they were precreated (standard disclaimer: I often misremember, so > this memory could be wrong! But definitely precreated tap devices do work). > It's been a while since I've started this thread, but lately I've understood better how tap devices work, and that new insight makes me wonder about a couple of things. Our ultimate goal In kubevirt is to consume a pre-created tap device by a kubernetes pod that doesn't have the NET_ADMIN capability. After looking at the current libvirt code, I don't think that is currently supported, since we'll *always* enter the `virNetDevTapCreate` function in [1] (I'm interested in the *tap* scenario). The tap device is effectively created in that function - [2] - by opening the clone device (/dev/net/tun), and calling `ioctl(fd, TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the NET_ADMIN capability. If I'm correct, this means that the current libvirt implementation makes our goals impossible to achieve. I'd first like to know if I read the code correctly, and if I'm ultimately right - i.e. since libvirt's implementation for consuming a pre-existent tap device opens the clone device, where ever libvirt is run on will *always* require the NET_ADMIN capability.

> > I think though that when someone from kubevirt actually tried using a > precreated macvtap device, they found that their precreated device > wasn't visible at all to the unprivileged libvirtd in the pod, because > it was in a different network namespace, or something like that. So > there may still be more work to do (or, again, my info might be out of > date and they figured out a proper solution). > > > >> > >> Would you be able to shed some light into this ? Is it possible on > >> libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ? > >> > >> [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 > > This links to the following message, which illustrates how to use pre-create > > tap and macvtap devices: > > > > https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html > > > > Laine: it would be useful to add something like this short guide to the > > knowledge base docs > > > You mean the wiki? Sure, I can do that. > > > (BTW - that was admirable reading / searching / responding - 7 minutes > and it wasn't even your patch! How do you do that? :-)) > > On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump(a)redhat.com&gt; wrote: > > On 4/6/20 9:54 AM, Daniel P. Berrangé wrote: > > On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote: > >> Hi all, > >> > >> I'm aware that it is possible to plug pre-created macvtap devices to > >> libvirt guests - tracked in RFE [0]. > >> > >> My interpretation of the wording in [1] and [2] is that it is also > >> possible to plug pre-created tap devices into libvirt guests - that > >> would be a requirement to allow kubevirt to run with less capabilities > >> in the pods that encapsulate the VMs. > >> > >> I took a look at the libvirt code ([3] & [4]), and, from my limited > >> understanding, I got the impression that plugging existing interfaces > >> via `managed='no' ` is only possible for macvtap interfaces. > > > No, it works for standard tap devices as well. > > > The reason the BZs and commit logs talk mostly about macvtap rather than > tap is because 1) that's what kubevirt people had asked for and 2) it > already *mostly* worked for tap devices, so most of the work was related > to macvtap (my memory is already fuzzy, but I think there were a couple > privileged operations we still tried to do for standard tap devices even > if they were precreated (standard disclaimer: I often misremember, so > this memory could be wrong! But definitely precreated tap devices do work). > > > I think though that when someone from kubevirt actually tried using a > precreated macvtap device, they found that their precreated device > wasn't visible at all to the unprivileged libvirtd in the pod, because > it was in a different network namespace, or something like that. So > there may still be more work to do (or, again, my info might be out of > date and they figured out a proper solution). > > > >> > >> Would you be able to shed some light into this ? Is it possible on > >> libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ? > >> > >> [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 > > This links to the following message, which illustrates how to use pre-create > > tap and macvtap devices: > > > > https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html > > > > Laine: it would be useful to add something like this short guide to the > > knowledge base docs > > > You mean the wiki? Sure, I can do that. > > > (BTW - that was admirable reading / searching / responding - 7 minutes > and it wasn't even your patch! How do you do that? :-)) > >

On Tue, Jun 30, 2020 at 12:59:03PM +0200, Miguel Duarte de Mora Barroso wrote: > On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump(a)redhat.com&gt; wrote: > > > > On 4/6/20 9:54 AM, Daniel P. Berrangé wrote: > > > On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote: > > >> Hi all, > > >> > > >> I'm aware that it is possible to plug pre-created macvtap devices to > > >> libvirt guests - tracked in RFE [0]. > > >> > > >> My interpretation of the wording in [1] and [2] is that it is also > > >> possible to plug pre-created tap devices into libvirt guests - that > > >> would be a requirement to allow kubevirt to run with less capabilities > > >> in the pods that encapsulate the VMs. > > >> > > >> I took a look at the libvirt code ([3] & [4]), and, from my limited > > >> understanding, I got the impression that plugging existing interfaces > > >> via `managed='no' ` is only possible for macvtap interfaces. > > > > > > No, it works for standard tap devices as well. > > > > > > The reason the BZs and commit logs talk mostly about macvtap rather than > > tap is because 1) that's what kubevirt people had asked for and 2) it > > already *mostly* worked for tap devices, so most of the work was related > > to macvtap (my memory is already fuzzy, but I think there were a couple > > privileged operations we still tried to do for standard tap devices even > > if they were precreated (standard disclaimer: I often misremember, so > > this memory could be wrong! But definitely precreated tap devices do work). > > > > It's been a while since I've started this thread, but lately I've > understood better how tap devices work, and that new insight makes me > wonder about a couple of things. > > Our ultimate goal In kubevirt is to consume a pre-created tap device > by a kubernetes pod that doesn't have the NET_ADMIN capability. > > After looking at the current libvirt code, I don't think that is > currently supported, since we'll *always* enter the > `virNetDevTapCreate` function in [1] (I'm interested in the *tap* > scenario). > > The tap device is effectively created in that function - [2] - by > opening the clone device (/dev/net/tun), and calling `ioctl(fd, > TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the > NET_ADMIN capability. If I'm correct, this means that the current > libvirt implementation makes our goals impossible to achieve. AFAIK, that is not correct - CAP_NET_ADMIN isn't required to open or create a tap device - only to add the tap device to a bridge. So if you create the tap device & attach it to a bridge ahead of time, libvirt should then be able to open it and give it to QEMU