Sent: Thursday, August 08, 2024 at 6:16 AM From: "Laine Stump" <lstump(a)redhat.com&gt; To: users(a)lists.libvirt.org Cc: "daggs" <daggs(a)gmx.com&gt; Subject: Re: cap seach for session libvirt On 8/7/24 2:25 PM, daggs via Users wrote: > Greetings, > > I'm working on allowing a session vm to create a tap iface. > the vm has this defintion: > <interface type='ethernet'> If you were able to create a tap device with session-mode libvirt, what would you do with it? It's not connected to anything, and your session-mode libvirt wouldn't be able to set an IP address (which would permit traffic to be *routed* to it by the host ip networking). > <mac address='52:54:00:a7:79:6b'/> > <target dev='veth0'/> Is there a particular reason you're wanting to name the device "veth0"? That naming is usually use for veth devices (these are two-sided network devices that are created in pairs, both looking like an ethernet. They can be used to connect an LXC container running in a separate network name space to the host's network name space (they are, as a matter of fact, what libvirt's LXC driver uses to implement networking in LXC containers). They are of really no use to a QEMU VM though. I'm pointing this out because it's possible that the Linux kernel makes some assumptions based on a device's name (unlikely, but also humans might make incorrect assumptions based on the name). > <model type='virtio'/> > <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> > </interface> > > when I try to start the vm, I get this error: Unable to create tap device veth0: Operation not permitted > searching the code led me to this line: https://github.com/libvirt/libvirt/blob/0caacf47d7b423db9126660fb0382ed56... > I've looked on line and found out I need the net_admin_cap set. so I took the relevant code in to a dedicated test file and using pam_cap I've defined such cap The whole point of session mode libvirt is that it *can't* get all those extra caps which would permit it (or the guest it runs) to wreak havoc on the host by (for example) creating a tap device connected to some random place. The way to get tap devices created for guests running under session mode libvirt is to make sure the qemu-bridge-helper is installed/setup properly on the host (including adding a line for the bridge device you want to connect to into /etc/qemu/bridge.conf), then just use <interface type='bridge'>, including a <source bridge='yourbridgename'/>, but *not* including a <target dev='blah'/> (since qemu-bridge-helper will determine the name for the tap device it creates). At guest start time. libvirt will see that a tap device is needed, then exec qemu-bridge-helper (which is a suid-root binary with the necessary caps to create a tap and connect it to a bridge) to create the tap device and connect it to the specified bridge. It passes the name of the tap device back to libvirt, which in turn provides the tap device to QEMU. Of course your example suggests that you don't want the tap device connected to a bridge, but in that case again I would ask "Why?". > for the test file, all went well. > so I took it back to virsh and defined that cap to virsh but I'm still getting the same issue, see: https://ibb.co/zHggRQZ > the os is debian 12 > > any ideas why I'm still getting this error? > > Thanks, > > Dagg >

Sent: Thursday, August 08, 2024 at 7:17 PM From: "Laine Stump" <lstump(a)redhat.com&gt; To: &quot;users(a)lists.libvirt.org&quot; <users(a)lists.libvirt.org&gt; Cc: "daggs" <daggs(a)gmx.com&gt; Subject: Re: cap seach for session libvirt On 8/8/24 2:40 AM, daggs wrote: > Greetings Laine, > >> Sent: Thursday, August 08, 2024 at 6:16 AM >> From: "Laine Stump" <lstump(a)redhat.com&gt; >> To: users(a)lists.libvirt.org >> Cc: "daggs" <daggs(a)gmx.com&gt; >> Subject: Re: cap seach for session libvirt >> >> On 8/7/24 2:25 PM, daggs via Users wrote: >>> Greetings, >>> >>> I'm working on allowing a session vm to create a tap iface. >>> the vm has this defintion: >>> <interface type='ethernet'> >> >> If you were able to create a tap device with session-mode libvirt, what >> would you do with it? It's not connected to anything, and your >> session-mode libvirt wouldn't be able to set an IP address (which would >> permit traffic to be *routed* to it by the host ip networking). >> >> >>> <mac address='52:54:00:a7:79:6b'/> >>> <target dev='veth0'/> >> >> Is there a particular reason you're wanting to name the device "veth0"? >> That naming is usually use for veth devices (these are two-sided network >> devices that are created in pairs, both looking like an ethernet. They >> can be used to connect an LXC container running in a separate network >> name space to the host's network name space (they are, as a matter of >> fact, what libvirt's LXC driver uses to implement networking in LXC >> containers). They are of really no use to a QEMU VM though. I'm pointing >> this out because it's possible that the Linux kernel makes some >> assumptions based on a device's name (unlikely, but also humans might >> make incorrect assumptions based on the name). >> >> >>> <model type='virtio'/> >>> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> >>> </interface> >>> >>> when I try to start the vm, I get this error: Unable to create tap device veth0: Operation not permitted >>> searching the code led me to this line: https://github.com/libvirt/libvirt/blob/0caacf47d7b423db9126660fb0382ed56... >>> I've looked on line and found out I need the net_admin_cap set. so I took the relevant code in to a dedicated test file and using pam_cap I've defined such cap >> >> The whole point of session mode libvirt is that it *can't* get all those >> extra caps which would permit it (or the guest it runs) to wreak havoc >> on the host by (for example) creating a tap device connected to some >> random place. The way to get tap devices created for guests running >> under session mode libvirt is to make sure the qemu-bridge-helper is >> installed/setup properly on the host (including adding a line for the >> bridge device you want to connect to into /etc/qemu/bridge.conf), then >> just use <interface type='bridge'>, including a <source >> bridge='yourbridgename'/>, but *not* including a <target dev='blah'/> >> (since qemu-bridge-helper will determine the name for the tap device it >> creates). >> >> >> At guest start time. libvirt will see that a tap device is needed, then >> exec qemu-bridge-helper (which is a suid-root binary with the necessary >> caps to create a tap and connect it to a bridge) to create the tap >> device and connect it to the specified bridge. It passes the name of the >> tap device back to libvirt, which in turn provides the tap device to QEMU. >> >> >> Of course your example suggests that you don't want the tap device >> connected to a bridge, but in that case again I would ask "Why?". >> >>> for the test file, all went well. >>> so I took it back to virsh and defined that cap to virsh but I'm still getting the same issue, see: https://ibb.co/zHggRQZ >>> the os is debian 12 >>> >>> any ideas why I'm still getting this error? >>> >>> Thanks, >>> >>> Dagg >>> >> > my goal is rather simple (at least for me), I have a server that runs two vms, one (denote vm1) is running openwrt, it has 5 nic via pt, What is "pt"? Do you mean VFIO device assignment (i.e. "passthrough"?)

> the other vm (denote vm2) runs libreelec. > I need to allow both the host and vm2 access to the lan. When you say "access the lan", do you mean that they need to be on the same subnet as the physical lan? And are you implying here that the openwrt VM should *not* be directly on the lan?

> I think I was able to connect vm2 to vm1 using a virtual switch What do you mean by "virtual switch"? A Linux host bridge? What is the configuration of that bridge? Is it attached to a host physical ethernet? what is the <interface> config of the two VMs? (AIn short - "You haven't given me any details, and so I don't know what I'm working with, and that makes it nearly impossible to tell you what to do next." :-))

> but I still need to connect the host to vm1. If you are connecting vm2 to vm1 using a Linux host bridge, then all you need to do is give that host bridge an IP address - then both vm1 and vm2 can communicate with the host via that IP address.

> that is why I'm trying the tap way, I've tried to connect the host using the virtual switch but didn't got any ip from vm1. What did you do to "try connecting the host using the virtual switch"? Instead of a generalized summary, give me the exact changes you made to libvirt and host system config.

Also, this statement implies that you have a dhcp server running on vm1, and that you want the host to get its IP from vm1's dhcp server. Is that correct? If so, how were you planning to enable dhclient for the host side of the tap device after creating it at runtime? This just doesn't seem practically achievable.

> maybe my problem is that I'm not using qemu-bridge-helper, I tried to get it to work but wasn't able, ... which implies that the connection you made between vm1 & vm2 was not via a Linux host bridge, since qemu-bridge-helper is the only method of connection a session mode libvirt guest to a Linux host bridge, so now I'm just confused :-P.

What exactly was the <interface> config of the guest, and what what the symptom(s) of "not working"? What was in the <interface> section of "virsh

> so I assume I either didn't configured the device properly in vm1 or didn't configured libvirt to use it properly. > I used this tutorial: https://mike42.me/blog/2019-08-how-to-use-the-qemu-bridge-helper-on-debia.... I took a look there and that really is all there is to it - one difference I see from the situation on Fedora is that the qemu-bridge-helper is apparently in /usr/lib/qemu on debian, rather than in /usr/libexec as it is on Fedora. Is that correct?

If you follow those instructions exactly, then your VMs should have IP addresses on 192.168.112.0/24, as will the host (it will be visible to the VMs at 192.168.122.1) - exactly the same as if you were using qemu:///system and <interface type='network'><source network='default'/>.

> I'll try again, maybe now it will work > other ideas are welcomed It would really help to see a drawing (either ASCII drawing directly in email, or a link to an image of a drawing (either computer-generated or hand-drawn) on imgur or some similar site (please don't send graphic attachments to the mailing list :-)). This drawing should show 5 entities (host, vm1, vm2, physical lan, internet connection) with lines showing which interfaces of which machines should be on the same subnet. Having that would greatly help in being able to give you productive advice.