plug pre-created tap devices to libvirt guests
Hi all, I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0]. My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs. I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces. Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ? [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367#c2 [2] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367#c3 [3] - https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_interface.c#L43... [4] - https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_interface.c#L44...
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices: https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html Laine: it would be useful to add something like this short guide to the knowledge base docs Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well. The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work). I think though that when someone from kubevirt actually tried using a precreated macvtap device, they found that their precreated device wasn't visible at all to the unprivileged libvirtd in the pod, because it was in a different network namespace, or something like that. So there may still be more work to do (or, again, my info might be out of date and they figured out a proper solution).
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that. (BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
On Mon, Apr 06, 2020 at 10:03:41AM -0400, Laine Stump wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
[0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that.
No, i mean docs/kbase/
(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
I just read the BZ & followed the link :-) Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
It's been a while since I've started this thread, but lately I've understood better how tap devices work, and that new insight makes me wonder about a couple of things. Our ultimate goal In kubevirt is to consume a pre-created tap device by a kubernetes pod that doesn't have the NET_ADMIN capability. After looking at the current libvirt code, I don't think that is currently supported, since we'll *always* enter the `virNetDevTapCreate` function in [1] (I'm interested in the *tap* scenario). The tap device is effectively created in that function - [2] - by opening the clone device (/dev/net/tun), and calling `ioctl(fd, TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the NET_ADMIN capability. If I'm correct, this means that the current libvirt implementation makes our goals impossible to achieve. I'd first like to know if I read the code correctly, and if I'm ultimately right - i.e. since libvirt's implementation for consuming a pre-existent tap device opens the clone device, where ever libvirt is run on will *always* require the NET_ADMIN capability.
I think though that when someone from kubevirt actually tried using a precreated macvtap device, they found that their precreated device wasn't visible at all to the unprivileged libvirtd in the pod, because it was in a different network namespace, or something like that. So there may still be more work to do (or, again, my info might be out of date and they figured out a proper solution).
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that.
(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
I think though that when someone from kubevirt actually tried using a precreated macvtap device, they found that their precreated device wasn't visible at all to the unprivileged libvirtd in the pod, because it was in a different network namespace, or something like that. So there may still be more work to do (or, again, my info might be out of date and they figured out a proper solution).
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that.
(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
On Tue, Jun 30, 2020 at 12:59 PM Miguel Duarte de Mora Barroso <mdbarroso@redhat.com> wrote:
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
It's been a while since I've started this thread, but lately I've understood better how tap devices work, and that new insight makes me wonder about a couple of things.
Our ultimate goal In kubevirt is to consume a pre-created tap device by a kubernetes pod that doesn't have the NET_ADMIN capability.
After looking at the current libvirt code, I don't think that is currently supported, since we'll *always* enter the `virNetDevTapCreate` function in [1] (I'm interested in the *tap* scenario).
The tap device is effectively created in that function - [2] - by opening the clone device (/dev/net/tun), and calling `ioctl(fd, TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the NET_ADMIN capability. If I'm correct, this means that the current libvirt implementation makes our goals impossible to achieve.
I'd first like to know if I read the code correctly, and if I'm ultimately right - i.e. since libvirt's implementation for consuming a pre-existent tap device opens the clone device, where ever libvirt is run on will *always* require the NET_ADMIN capability.
Adding the links that I've forgotten to add ... [0] - https://bugzilla.redhat.com/show_bug.cgi?id=1723367 [1] - https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_interface.c#L44... [2] - https://github.com/libvirt/libvirt/blob/master/src/util/virnetdevtap.c#L243
I think though that when someone from kubevirt actually tried using a precreated macvtap device, they found that their precreated device wasn't visible at all to the unprivileged libvirtd in the pod, because it was in a different network namespace, or something like that. So there may still be more work to do (or, again, my info might be out of date and they figured out a proper solution).
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that.
(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
I think though that when someone from kubevirt actually tried using a precreated macvtap device, they found that their precreated device wasn't visible at all to the unprivileged libvirtd in the pod, because it was in a different network namespace, or something like that. So there may still be more work to do (or, again, my info might be out of date and they figured out a proper solution).
Would you be able to shed some light into this ? Is it possible on libvirt-5.6.0 to plug pre-created tap devices to libvirt guests ?
This links to the following message, which illustrates how to use pre-create tap and macvtap devices:
https://www.redhat.com/archives/libvir-list/2019-August/msg01256.html
Laine: it would be useful to add something like this short guide to the knowledge base docs
You mean the wiki? Sure, I can do that.
(BTW - that was admirable reading / searching / responding - 7 minutes and it wasn't even your patch! How do you do that? :-))
On Tue, Jun 30, 2020 at 12:59:03PM +0200, Miguel Duarte de Mora Barroso wrote:
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
It's been a while since I've started this thread, but lately I've understood better how tap devices work, and that new insight makes me wonder about a couple of things.
Our ultimate goal In kubevirt is to consume a pre-created tap device by a kubernetes pod that doesn't have the NET_ADMIN capability.
After looking at the current libvirt code, I don't think that is currently supported, since we'll *always* enter the `virNetDevTapCreate` function in [1] (I'm interested in the *tap* scenario).
The tap device is effectively created in that function - [2] - by opening the clone device (/dev/net/tun), and calling `ioctl(fd, TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the NET_ADMIN capability. If I'm correct, this means that the current libvirt implementation makes our goals impossible to achieve.
AFAIK, that is not correct - CAP_NET_ADMIN isn't required to open or create a tap device - only to add the tap device to a bridge. So if you create the tap device & attach it to a bridge ahead of time, libvirt should then be able to open it and give it to QEMU Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Jun 30, 2020 at 04:02:05PM +0100, Daniel P. Berrangé wrote:
On Tue, Jun 30, 2020 at 12:59:03PM +0200, Miguel Duarte de Mora Barroso wrote:
On Mon, Apr 6, 2020 at 4:03 PM Laine Stump <lstump@redhat.com> wrote:
On 4/6/20 9:54 AM, Daniel P. Berrangé wrote:
On Mon, Apr 06, 2020 at 03:47:01PM +0200, Miguel Duarte de Mora Barroso wrote:
Hi all,
I'm aware that it is possible to plug pre-created macvtap devices to libvirt guests - tracked in RFE [0].
My interpretation of the wording in [1] and [2] is that it is also possible to plug pre-created tap devices into libvirt guests - that would be a requirement to allow kubevirt to run with less capabilities in the pods that encapsulate the VMs.
I took a look at the libvirt code ([3] & [4]), and, from my limited understanding, I got the impression that plugging existing interfaces via `managed='no' ` is only possible for macvtap interfaces.
No, it works for standard tap devices as well.
The reason the BZs and commit logs talk mostly about macvtap rather than tap is because 1) that's what kubevirt people had asked for and 2) it already *mostly* worked for tap devices, so most of the work was related to macvtap (my memory is already fuzzy, but I think there were a couple privileged operations we still tried to do for standard tap devices even if they were precreated (standard disclaimer: I often misremember, so this memory could be wrong! But definitely precreated tap devices do work).
It's been a while since I've started this thread, but lately I've understood better how tap devices work, and that new insight makes me wonder about a couple of things.
Our ultimate goal In kubevirt is to consume a pre-created tap device by a kubernetes pod that doesn't have the NET_ADMIN capability.
After looking at the current libvirt code, I don't think that is currently supported, since we'll *always* enter the `virNetDevTapCreate` function in [1] (I'm interested in the *tap* scenario).
The tap device is effectively created in that function - [2] - by opening the clone device (/dev/net/tun), and calling `ioctl(fd, TUNSETIFF,...)` in it. AFAIK, both of those operations *require* the NET_ADMIN capability. If I'm correct, this means that the current libvirt implementation makes our goals impossible to achieve.
AFAIK, that is not correct - CAP_NET_ADMIN isn't required to open or create a tap device - only to add the tap device to a bridge.
So if you create the tap device & attach it to a bridge ahead of time, libvirt should then be able to open it and give it to QEMU
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/driv... ((uid_valid(tun->owner) && !uid_eq(cred->euid, tun->owner)) || (gid_valid(tun->group) && !in_egroup_p(tun->group))) && !ns_capable(net->user_ns, CAP_NET_ADMIN); This is called by the TUNSETIFF code. AFAICT, that means if you fchown(tapfd, uid, gid), to the uid+gid of libvirtd, it should not require CAP_NET_ADMIN. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (3)
-
Daniel P. Berrangé -
Laine Stump -
Miguel Duarte de Mora Barroso