Re: Guests: Unable to get IP address
Please keep the list on CC so that others can benefit from the conversation too. On 5/21/24 14:34, Arun Mani J wrote:
Thanks for the reply!
I recreated the NAT network using the guide from https://wiki.libvirt.org/Networking.html.
The configuration now reads as: <network> <name>default</name> <uuid>33376669-6292-44c9-b26b-9d2ee69826a7</uuid> <forward mode="nat"> <nat> <port start="1024" end="65535"/> </nat> </forward> <bridge name="virbr0" stp="on" delay="0"/> <mac address="52:54:00:78:76:0f"/> <ip address="192.168.122.1" netmask="255.255.255.0"> <dhcp> <range start="192.168.122.2" end="192.168.122.254"/> </dhcp> </ip> </network>
There is a dhcp section but still I'm not able to get my guest working.
I have dnsmasq-base package installed in my host. However I'm not sure how to see if that is running. Like I opened Systems Monitor and can't find any process with that name.
So may be that's the problem?
Well, have you restarted the network? You need the 'virsh net-destroy default && virsh net-start default' combo to restart it. After that you should see dnsmasq process, e.g. via 'ps axf | grep dnsmasq'. Michal
Sorry I thought I clicked Reply instead of Reply All. So I restarted my laptop, ran virsh net-destroy default && virsh net-start default. Then created a new VM out of Debian 12 KDE Live ISO (to avoid any trailing configurations). Still the issue persists. nmcli in the guest says enp1s0: disconnected. But ps axf | grep dnsmasq gives this: 4341 pts/0 S+ 0:00 | \_ grep --color=auto dnsmasq 3995 ? S 0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper 3996 ? S 0:00 \_ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper What am I missing :( Thanks you On Tuesday, May 21st, 2024 at 6:51 PM, Michal Prívozník <mprivozn@redhat.com> wrote:
Please keep the list on CC so that others can benefit from the conversation too.
On 5/21/24 14:34, Arun Mani J wrote:
Thanks for the reply!
I recreated the NAT network using the guide from https://wiki.libvirt.org/Networking.html.
The configuration now reads as: <network> <name>default</name> <uuid>33376669-6292-44c9-b26b-9d2ee69826a7</uuid> <forward mode="nat"> <nat> <port start="1024" end="65535"/> </nat> </forward> <bridge name="virbr0" stp="on" delay="0"/> <mac address="52:54:00:78:76:0f"/> <ip address="192.168.122.1" netmask="255.255.255.0"> <dhcp> <range start="192.168.122.2" end="192.168.122.254"/> </dhcp> </ip> </network>
There is a dhcp section but still I'm not able to get my guest working.
I have dnsmasq-base package installed in my host. However I'm not sure how to see if that is running. Like I opened Systems Monitor and can't find any process with that name.
So may be that's the problem?
Well, have you restarted the network? You need the 'virsh net-destroy default && virsh net-start default' combo to restart it. After that you should see dnsmasq process, e.g. via 'ps axf | grep dnsmasq'.
Michal
On 5/21/24 18:02, Arun Mani J wrote:
Sorry I thought I clicked Reply instead of Reply All.
So I restarted my laptop, ran virsh net-destroy default && virsh net-start default. Then created a new VM out of Debian 12 KDE Live ISO (to avoid any trailing configurations).
Still the issue persists. nmcli in the guest says enp1s0: disconnected.
This looks weird. I'm not familiar with networkmanager, but I suspect this is not telling the state of the link, is it? Because the link should be up no matter the host side configuration.
But ps axf | grep dnsmasq gives this: 4341 pts/0 S+ 0:00 | \_ grep --color=auto dnsmasq 3995 ? S 0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper 3996 ? S 0:00 \_ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
What am I missing :(
One thing that comes to my mind is - ip forwarding. Libvirt sets up NAT and should set enable ip forwarding too, but maybe that failed? Inside the host - can you share the output of: cat /proc/sys/net/ipv4/ip_forward brctl show virbr0 for i in nat filter mangle; do iptables -t $i -L -v ; done Michal
I have attached the screenshot of `nmcli` inside guest (clipboard doesn't work, but that's for another day may be): https://imgur.com/NlDtDtc The guest is stuck in two states basically - connecting and after a few seconds it reaches disconnected state. $ cat /proc/sys/net/ipv4/ip_forward 1 (I'm using bridge instead of brctl because it is not available and seems to be deprecated in favor of bridge, please let me know if that's not the case) $ sudo bridge link show virbr0 The command gave no output, so I tried ip link (apologies if that doesn't help) $ ip link show virbr0 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:78:76:0f brd ff:ff:ff:ff:ff:ff (It says state is DOWN ??) $ for i in nat filter mangle; do sudo iptables -t $i -L -v ; done Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 1099 packets, 77803 bytes) pkts bytes target prot opt in out source destination 1016 71246 ts-postrouting all -- any any anywhere anywhere 1099 77803 LIBVIRT_PRT all -- any any anywhere anywhere Chain LIBVIRT_PRT (1 references) pkts bytes target prot opt in out source destination 0 0 RETURN all -- any any 192.168.122.0/24 base-address.mcast.net/24 0 0 RETURN all -- any any 192.168.122.0/24 255.255.255.255 0 0 MASQUERADE tcp -- any any 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535 0 0 MASQUERADE udp -- any any 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535 0 0 MASQUERADE all -- any any 192.168.122.0/24 !192.168.122.0/24 Chain ts-postrouting (1 references) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- any any anywhere anywhere mark match 0x40000/0xff0000 Chain INPUT (policy ACCEPT 4723 packets, 3908K bytes) pkts bytes target prot opt in out source destination 5260 3961K ts-input all -- any any anywhere anywhere 4723 3908K LIBVIRT_INP all -- any any anywhere anywhere Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 ts-forward all -- any any anywhere anywhere 0 0 LIBVIRT_FWX all -- any any anywhere anywhere 0 0 LIBVIRT_FWI all -- any any anywhere anywhere 0 0 LIBVIRT_FWO all -- any any anywhere anywhere Chain OUTPUT (policy ACCEPT 5305 packets, 604K bytes) pkts bytes target prot opt in out source destination 5305 604K LIBVIRT_OUT all -- any any anywhere anywhere Chain LIBVIRT_FWI (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- any virbr0 anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED 0 0 REJECT all -- any virbr0 anywhere anywhere reject-with icmp-port-unreachable Chain LIBVIRT_FWO (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- virbr0 any 192.168.122.0/24 anywhere 0 0 REJECT all -- virbr0 any anywhere anywhere reject-with icmp-port-unreachable Chain LIBVIRT_FWX (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- virbr0 virbr0 anywhere anywhere Chain LIBVIRT_INP (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- virbr0 any anywhere anywhere udp dpt:domain 0 0 ACCEPT tcp -- virbr0 any anywhere anywhere tcp dpt:domain 0 0 ACCEPT udp -- virbr0 any anywhere anywhere udp dpt:bootps 0 0 ACCEPT tcp -- virbr0 any anywhere anywhere tcp dpt:67 Chain LIBVIRT_OUT (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- any virbr0 anywhere anywhere udp dpt:domain 0 0 ACCEPT tcp -- any virbr0 anywhere anywhere tcp dpt:domain 0 0 ACCEPT udp -- any virbr0 anywhere anywhere udp dpt:bootpc 0 0 ACCEPT tcp -- any virbr0 anywhere anywhere tcp dpt:68 Chain ts-forward (1 references) pkts bytes target prot opt in out source destination 0 0 MARK all -- tailscale0 any anywhere anywhere MARK xset 0x40000/0xff0000 0 0 ACCEPT all -- any any anywhere anywhere mark match 0x40000/0xff0000 0 0 DROP all -- any tailscale0 100.64.0.0/10 anywhere 0 0 DROP all -- any tailscale0 anywhere anywhere ! ctstate RELATED,ESTABLISHED 0 0 ACCEPT all -- any tailscale0 anywhere anywhere Chain ts-input (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- lo any thinking.taila514c.ts.net anywhere 0 0 RETURN all -- !tailscale0 any 100.115.92.0/23 anywhere 0 0 DROP all -- !tailscale0 any 100.64.0.0/10 anywhere 469 65919 ACCEPT all -- tailscale0 any anywhere anywhere 223 16320 ACCEPT udp -- any any anywhere anywhere udp dpt:41641 Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination Chain POSTROUTING (policy ACCEPT 5331 packets, 607K bytes) pkts bytes target prot opt in out source destination 5331 607K LIBVIRT_PRT all -- any any anywhere anywhere Chain LIBVIRT_PRT (1 references) pkts bytes target prot opt in out source destination 0 0 CHECKSUM udp -- any virbr0 anywhere anywhere udp dpt:bootpc CHECKSUM fill Please let me know if some other information is required. Thanks for helping! Arun Mani J On Wednesday, May 22nd, 2024 at 3:48 PM, Michal Prívozník <mprivozn@redhat.com> wrote:
On 5/21/24 18:02, Arun Mani J wrote:
Sorry I thought I clicked Reply instead of Reply All.
So I restarted my laptop, ran virsh net-destroy default && virsh net-start default. Then created a new VM out of Debian 12 KDE Live ISO (to avoid any trailing configurations).
Still the issue persists. nmcli in the guest says enp1s0: disconnected.
This looks weird. I'm not familiar with networkmanager, but I suspect this is not telling the state of the link, is it? Because the link should be up no matter the host side configuration.
But ps axf | grep dnsmasq gives this: 4341 pts/0 S+ 0:00 | \_ grep --color=auto dnsmasq 3995 ? S 0:00 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper 3996 ? S 0:00 \_ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
What am I missing :(
One thing that comes to my mind is - ip forwarding. Libvirt sets up NAT and should set enable ip forwarding too, but maybe that failed?
Inside the host - can you share the output of:
cat /proc/sys/net/ipv4/ip_forward brctl show virbr0 for i in nat filter mangle; do iptables -t $i -L -v ; done
Michal
On 5/22/24 19:22, Arun Mani J wrote:
I have attached the screenshot of `nmcli` inside guest (clipboard doesn't work, but that's for another day may be): https://imgur.com/NlDtDtc
The guest is stuck in two states basically - connecting and after a few seconds it reaches disconnected state.
$ cat /proc/sys/net/ipv4/ip_forward 1
(I'm using bridge instead of brctl because it is not available and seems to be deprecated in favor of bridge, please let me know if that's not the case)
$ sudo bridge link show virbr0
The command gave no output, so I tried ip link (apologies if that doesn't help)
So, when I'm running a VM what I get is: # bridge link show virbr0 18: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 master virbr0 state forwarding priority 32 cost 2 Something looks broken and vnet0 is not getting plugged into the bridge.
$ ip link show virbr0 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:78:76:0f brd ff:ff:ff:ff:ff:ff
(It says state is DOWN ??)
Yeah, this is suspicious too.
$ for i in nat filter mangle; do sudo iptables -t $i -L -v ; done
Firewall rules look more or less the same. Now, I'm assuming your guest is under qemu:///system URI, right? Can you enable debug logs, start the guest and then share the logs somewhere (e.g. pastebin-like service) please? In the logs there should be a section where libvirt creates vnetX interface and plugs it into the bridge. https://libvirt.org/kbase/debuglogs.html If you're running a monolithic daemon (libvirtd) then all the work happens inside it and logs from it are sufficient. But if you're running split daemons then I'd need to see logs from virtqemud and virtnetworkd. Michal
Yea I'm using qemu:///system. This should have the logs https://gitlab.com/Arun-Mani-J/snippets/-/blob/main/virtlogs. Just to confirm that I got it right: 1. Found that my installation is a monolithic daemon. 2. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "3:journald 1:file:/home/arun-mani-j/virtd". 3. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:event 3:util.json 3:util.object 3:util.dbus 3:util.netlink 3:node_device 3:rpc 3:access 1:*" 4. Launched virt-manager. 5. Created and started a new guest that boots Debian 12 KDE Live. 6. Waited for the network-manager in guest to give up acquiring IP and reach deactivated state (few minutes of waiting). 7. Shutdown the guest. 8. Closed virt-manager. 9. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "" 10. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:util.json 4:rpc" Please let me know if I have missed something, so that I can redo. Thanks for being so helpful, really awesome :) Arun Mani J On Thursday, May 23rd, 2024 at 1:11 PM, Michal Prívozník <mprivozn@redhat.com> wrote:
On 5/22/24 19:22, Arun Mani J wrote:
I have attached the screenshot of `nmcli` inside guest (clipboard doesn't work, but that's for another day may be): https://imgur.com/NlDtDtc
The guest is stuck in two states basically - connecting and after a few seconds it reaches disconnected state.
$ cat /proc/sys/net/ipv4/ip_forward 1
(I'm using bridge instead of brctl because it is not available and seems to be deprecated in favor of bridge, please let me know if that's not the case)
$ sudo bridge link show virbr0
The command gave no output, so I tried ip link (apologies if that doesn't help)
So, when I'm running a VM what I get is:
# bridge link show virbr0 18: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 master virbr0
state forwarding priority 32 cost 2
Something looks broken and vnet0 is not getting plugged into the bridge.
$ ip link show virbr0 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:78:76:0f brd ff:ff:ff:ff:ff:ff
(It says state is DOWN ??)
Yeah, this is suspicious too.
$ for i in nat filter mangle; do sudo iptables -t $i -L -v ; done
Firewall rules look more or less the same.
Now, I'm assuming your guest is under qemu:///system URI, right? Can you enable debug logs, start the guest and then share the logs somewhere (e.g. pastebin-like service) please? In the logs there should be a section where libvirt creates vnetX interface and plugs it into the bridge.
https://libvirt.org/kbase/debuglogs.html
If you're running a monolithic daemon (libvirtd) then all the work happens inside it and logs from it are sufficient. But if you're running split daemons then I'd need to see logs from virtqemud and virtnetworkd.
Michal
Hii Any help on this or is it just that my installation is messed up? 0_0. Perhaps I can try reinstalling the packages after purging everything related to libvirt. Thanks! Arun Mani J On Thursday, May 23rd, 2024 at 10:53 PM, Arun Mani J <J.ArunMani@proton.me> wrote:
Yea I'm using qemu:///system.
This should have the logs https://gitlab.com/Arun-Mani-J/snippets/-/blob/main/virtlogs.
Just to confirm that I got it right: 1. Found that my installation is a monolithic daemon. 2. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "3:journald 1:file:/home/arun-mani-j/virtd". 3. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:event 3:util.json 3:util.object 3:util.dbus 3:util.netlink 3:node_device 3:rpc 3:access 1:*" 4. Launched virt-manager. 5. Created and started a new guest that boots Debian 12 KDE Live. 6. Waited for the network-manager in guest to give up acquiring IP and reach deactivated state (few minutes of waiting). 7. Shutdown the guest. 8. Closed virt-manager. 9. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "" 10. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:util.json 4:rpc"
Please let me know if I have missed something, so that I can redo.
Thanks for being so helpful, really awesome :) Arun Mani J
On Thursday, May 23rd, 2024 at 1:11 PM, Michal Prívozník mprivozn@redhat.com wrote:
On 5/22/24 19:22, Arun Mani J wrote:
I have attached the screenshot of `nmcli` inside guest (clipboard doesn't work, but that's for another day may be): https://imgur.com/NlDtDtc
The guest is stuck in two states basically - connecting and after a few seconds it reaches disconnected state.
$ cat /proc/sys/net/ipv4/ip_forward 1
(I'm using bridge instead of brctl because it is not available and seems to be deprecated in favor of bridge, please let me know if that's not the case)
$ sudo bridge link show virbr0
The command gave no output, so I tried ip link (apologies if that doesn't help)
So, when I'm running a VM what I get is:
# bridge link show virbr0 18: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 master virbr0
state forwarding priority 32 cost 2
Something looks broken and vnet0 is not getting plugged into the bridge.
$ ip link show virbr0 4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 52:54:00:78:76:0f brd ff:ff:ff:ff:ff:ff
(It says state is DOWN ??)
Yeah, this is suspicious too.
$ for i in nat filter mangle; do sudo iptables -t $i -L -v ; done
Firewall rules look more or less the same.
Now, I'm assuming your guest is under qemu:///system URI, right? Can you enable debug logs, start the guest and then share the logs somewhere (e.g. pastebin-like service) please? In the logs there should be a section where libvirt creates vnetX interface and plugs it into the bridge.
https://libvirt.org/kbase/debuglogs.html
If you're running a monolithic daemon (libvirtd) then all the work happens inside it and logs from it are sufficient. But if you're running split daemons then I'd need to see logs from virtqemud and virtnetworkd.
Michal
On 6/9/24 07:54, Arun Mani J wrote:
Hii
Any help on this or is it just that my installation is messed up? 0_0.
Perhaps I can try reinstalling the packages after purging everything related to libvirt.
Thanks!
Arun Mani J
On Thursday, May 23rd, 2024 at 10:53 PM, Arun Mani J <J.ArunMani@proton.me> wrote:
Yea I'm using qemu:///system.
This should have the logs https://gitlab.com/Arun-Mani-J/snippets/-/blob/main/virtlogs.
Just to confirm that I got it right: 1. Found that my installation is a monolithic daemon. 2. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "3:journald 1:file:/home/arun-mani-j/virtd". 3. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:event 3:util.json 3:util.object 3:util.dbus 3:util.netlink 3:node_device 3:rpc 3:access 1:*" 4. Launched virt-manager. 5. Created and started a new guest that boots Debian 12 KDE Live. 6. Waited for the network-manager in guest to give up acquiring IP and reach deactivated state (few minutes of waiting). 7. Shutdown the guest. 8. Closed virt-manager. 9. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "" 10. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:util.json 4:rpc"
Please let me know if I have missed something, so that I can redo.
I haven't found anything wrong in the logs. The only thing that comes to my mind is NetworkManager on the host clashing with what libvirt does (which is supported by the fact that virbr0 link state was down in one of previous e-mails). Michal
Hii Okay, then let me see if I can reproduce this bug in a fresh Debian installation. If yes, then could be a bug in the Debian side, else I will reinstall my system :) Thank you! -------- Original Message -------- On 6/10/24 12:19 PM, Michal Prívozník <mprivozn@redhat.com> wrote:
On 6/9/24 07:54, Arun Mani J wrote:
Hii
Any help on this or is it just that my installation is messed up? 0_0.
Perhaps I can try reinstalling the packages after purging everything related to libvirt.
Thanks!
Arun Mani J
On Thursday, May 23rd, 2024 at 10:53 PM, Arun Mani J <J.ArunMani@proton.me> wrote:
Yea I'm using qemu:///system.
This should have the logs https://gitlab.com/Arun-Mani-J/snippets/-/blob/main/virtlogs.
Just to confirm that I got it right: 1. Found that my installation is a monolithic daemon. 2. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "3:journald 1:file:/home/arun-mani-j/virtd". 3. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:event 3:util.json 3:util.object 3:util.dbus 3:util.netlink 3:node_device 3:rpc 3:access 1:*" 4. Launched virt-manager. 5. Created and started a new guest that boots Debian 12 KDE Live. 6. Waited for the network-manager in guest to give up acquiring IP and reach deactivated state (few minutes of waiting). 7. Shutdown the guest. 8. Closed virt-manager. 9. Did sudo virt-admin -c libvirtd:///system daemon-log-outputs "" 10. Did sudo virt-admin -c libvirtd:///system daemon-log-filters "3:remote 4:util.json 4:rpc"
Please let me know if I have missed something, so that I can redo.
I haven't found anything wrong in the logs. The only thing that comes to my mind is NetworkManager on the host clashing with what libvirt does (which is supported by the fact that virbr0 link state was down in one of previous e-mails).
Michal
participants (2)
-
Arun Mani J -
Michal Prívozník