[libvirt-users] assign static external IP to container
hello i have a server colocated in a datacenter with several external IP addresses available to use. the physical server is using one of these IPs, and i want to assign another, unused IP to the virtual machine. i thought i could just do this by editing the container's /etc/network/interfaces, setting a static IP address for eth0 much like i did for br0 on the host machine.... but doesn't seem to be working. ifconfig shows eth0 has the external address but i can't resolve any hostnames nor telnet to a direct IP address (no route to host). if i change back to dhcp and let eth0 get an internal address, i can at least access the internet but cannot access the virtual machine from the internet. what is the trick to giving a VM a routable, external IP address? thanks
On 12/09/2013 05:14 AM, scar wrote:
hello i have a server colocated in a datacenter with several external IP addresses available to use. the physical server is using one of these IPs, and i want to assign another, unused IP to the virtual machine. i thought i could just do this by editing the container's /etc/network/interfaces, setting a static IP address for eth0 much like i did for br0 on the host machine.... but doesn't seem to be working. ifconfig shows eth0 has the external address but i can't resolve any hostnames nor telnet to a direct IP address (no route to host). if i change back to dhcp and let eth0 get an internal address, i can at least access the internet but cannot access the virtual machine from the internet. what is the trick to giving a VM a routable, external IP address? thanks
I have a machine running libvirt lxc, on this machine(host), the network configure looks like this eth0 is the physical nic, virbr0 is the bridge libvirt created, and vnet0 is the veth device created for libvirt lxc container. you can see, the eth0 is a port of virbr0, and the ip is assigned on virbr0. ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::5054:ff:fe39:626f prefixlen 64 scopeid 0x20<link> ether 52:54:00:39:62:6f txqueuelen 1000 (Ethernet) RX packets 1502 bytes 93029 (90.8 KiB) RX errors 0 dropped 10 overruns 0 frame 0 TX packets 546 bytes 52825 (51.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.122.178 netmask 255.255.255.0 broadcast 192.168.122.255 ether 52:54:00:39:62:6f txqueuelen 0 (Ethernet) RX packets 419 bytes 26899 (26.2 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 328 bytes 42246 (41.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 vnet0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::b4a1:32ff:fe56:5706 prefixlen 64 scopeid 0x20<link> ether b6:a1:32:56:57:06 txqueuelen 1000 (Ethernet) RX packets 21 bytes 1642 (1.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 214 bytes 11884 (11.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 brctl show bridge name bridge id STP enabled interfaces virbr0 8000.52540039626f yes eth0 vnet0 And in container I change the ip of eth0 to 192.168.122.33 and add a route rule ip route add default via 192.168.122.1 dev eth0 then the container can access the outside. I think it should work even you change the 192.168.x.x to the global ip.
Gao feng @ 12/11/2013 01:49 AM:
I have a machine running libvirt lxc, on this machine(host), the network configure looks like this eth0 is the physical nic, virbr0 is the bridge libvirt created, and vnet0 is the veth device created for libvirt lxc container. you can see, the eth0 is a port of virbr0, and the ip is assigned on virbr0.
[snip]
And in container
I change the ip of eth0 to 192.168.122.33 and add a route rule ip route add default via 192.168.122.1 dev eth0
then the container can access the outside.
I think it should work even you change the 192.168.x.x to the global ip.
arrg thank you but this is all so confusing to me >< this is what my ifconfig and brctl show looks like on host machine: http://pastebin.com/9f1HzXi5 i want to use 67.180.246.123 for the container
On 12/13/2013 12:33 PM, scar wrote:
Gao feng @ 12/11/2013 01:49 AM:
I have a machine running libvirt lxc, on this machine(host), the network configure looks like this eth0 is the physical nic, virbr0 is the bridge libvirt created, and vnet0 is the veth device created for libvirt lxc container. you can see, the eth0 is a port of virbr0, and the ip is assigned on virbr0.
[snip]
And in container
I change the ip of eth0 to 192.168.122.33 and add a route rule ip route add default via 192.168.122.1 dev eth0
then the container can access the outside.
I think it should work even you change the 192.168.x.x to the global ip.
arrg thank you but this is all so confusing to me ><
this is what my ifconfig and brctl show looks like on host machine: http://pastebin.com/9f1HzXi5
i want to use 67.180.246.123 for the container
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ? I think you may setup the wrong bridge. try to set ip for virbr0 and make eth0 as a port of virbr0.
On 12/13/2013 01:18 PM, Gao feng wrote:
On 12/13/2013 12:33 PM, scar wrote:
Gao feng @ 12/11/2013 01:49 AM:
I have a machine running libvirt lxc, on this machine(host), the network configure looks like this eth0 is the physical nic, virbr0 is the bridge libvirt created, and vnet0 is the veth device created for libvirt lxc container. you can see, the eth0 is a port of virbr0, and the ip is assigned on virbr0.
[snip]
And in container
I change the ip of eth0 to 192.168.122.33 and add a route rule ip route add default via 192.168.122.1 dev eth0
then the container can access the outside.
I think it should work even you change the 192.168.x.x to the global ip.
arrg thank you but this is all so confusing to me ><
this is what my ifconfig and brctl show looks like on host machine: http://pastebin.com/9f1HzXi5
i want to use 67.180.246.123 for the container
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ?
and #virsh net-dumpxml default
I think you may setup the wrong bridge. try to set ip for virbr0 and make eth0 as a port of virbr0.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Gao feng @ 12/12/2013 10:18 PM:
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ?
well i don't quite understand bridges and their function, i'm kinda just following directions. br0 replaced eth0 though in my interfaces file according to this guide[1] i followed to setup the bridge. i'm pretty sure virbr0 is what libvirt uses.... host$ cat /etc/network/interfaces auto lo iface lo inet loopback auto br0 iface br0 inet static address 67.180.246.122 netmask 255.255.255.248 gateway 67.180.246.121 dns-nameservers 67.180.241.11 67.180.241.12 bridge_ports eth0 bridge_fd 9 bridge_maxage 12 bridge_stp off host$ virsh net-list Name State Autostart - ----------------------------------------- default active yes host$ virsh net-dumpxml default <network> <name>default</name> <uuid>e6d3f97a-5870-8ee7-1d82-64b3434f138b</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0' /> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254' /> </dhcp> </ip> </network> $ 1. https://help.ubuntu.com/12.04/serverguide/network-configuration.html#bridgin... -----BEGIN PGP SIGNATURE----- iEYEAREIAAYFAlKrx58ACgkQXhfCJNu98qDHhwCguJBus/7kZj+5R2EAYSzSENg8 SFwAoLVCxi6jJQBaUhMl4Ter+Jnvq/rI =4wm8 -----END PGP SIGNATURE-----
On 12/14/2013 10:51 AM, scar wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Gao feng @ 12/12/2013 10:18 PM:
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ?
well i don't quite understand bridges and their function, i'm kinda just following directions. br0 replaced eth0 though in my interfaces file according to this guide[1] i followed to setup the bridge. i'm pretty sure virbr0 is what libvirt uses....
host$ cat /etc/network/interfaces auto lo iface lo inet loopback
auto br0 iface br0 inet static address 67.180.246.122 netmask 255.255.255.248 gateway 67.180.246.121 dns-nameservers 67.180.241.11 67.180.241.12 bridge_ports eth0 bridge_fd 9 bridge_maxage 12 bridge_stp off
remove this br0 sector from the interfaces file. and restart the network. (remove the bridge br0) I'm not familiar with the configure files in ubuntu, let's use the command. 1, ifconfig eth0 0.0.0.0 2, brctl addif virbr0 eth0 3, ifconfig virbr0 67.180.246.122/29 4，ip route add default via 67.180.246.121 dev virbr0 5, ifconfig eth0 x.x.x.x/29 (in container) 6, ip route add default via 67.180.246.121 dev eth0 (in container) but this will not work when you restart libvirt.
host$ virsh net-list Name State Autostart - ----------------------------------------- default active yes
host$ virsh net-dumpxml default <network> <name>default</name> <uuid>e6d3f97a-5870-8ee7-1d82-64b3434f138b</uuid> <forward mode='nat'/> <bridge name='virbr0' stp='on' delay='0' /> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.2' end='192.168.122.254' /> </dhcp> </ip> </network>
$
1. https://help.ubuntu.com/12.04/serverguide/network-configuration.html#bridgin... -----BEGIN PGP SIGNATURE-----
iEYEAREIAAYFAlKrx58ACgkQXhfCJNu98qDHhwCguJBus/7kZj+5R2EAYSzSENg8 SFwAoLVCxi6jJQBaUhMl4Ter+Jnvq/rI =4wm8 -----END PGP SIGNATURE-----
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Gao feng @ 12/15/2013 07:47 PM:
remove this br0 sector from the interfaces file. and restart the network. (remove the bridge br0) I'm not familiar with the configure files in ubuntu, let's use the command.
1, ifconfig eth0 0.0.0.0 2, brctl addif virbr0 eth0 3, ifconfig virbr0 67.180.246.122/29 4，ip route add default via 67.180.246.121 dev virbr0 5, ifconfig eth0 x.x.x.x/29 (in container) 6, ip route add default via 67.180.246.121 dev eth0 (in container)
but this will not work when you restart libvirt.
hehe well i kinda need something that will work :P also if i take out br0 from interfaces file i'm pretty sure my server will become inaccessible :\
On 12/14/2013 10:51 AM, scar wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ? well i don't quite understand bridges and their function, i'm kinda just following directions. br0 replaced eth0 though in my interfaces file according to this guide[1] i followed to setup the bridge. i'm
Gao feng @ 12/12/2013 10:18 PM: pretty sure virbr0 is what libvirt uses....
host$ cat /etc/network/interfaces auto lo iface lo inet loopback
auto br0 iface br0 inet static address 67.180.246.122 netmask 255.255.255.248 gateway 67.180.246.121 dns-nameservers 67.180.241.11 67.180.241.12 bridge_ports eth0 bridge_fd 9 bridge_maxage 12 bridge_stp off
remove this br0 sector from the interfaces file. and restart the network. (remove the bridge br0) I'm not familiar with the configure files in ubuntu, let's use the command.
1, ifconfig eth0 0.0.0.0 2, brctl addif virbr0 eth0 3, ifconfig virbr0 67.180.246.122/29 4，ip route add default via 67.180.246.121 dev virbr0 5, ifconfig eth0 x.x.x.x/29 (in container) 6, ip route add default via 67.180.246.121 dev eth0 (in container)
but this will not work when you restart libvirt. This advice is incorrect and should not be followed! The br0 that he has created is *exactly* what he needs, so he shouldn't remove it from the config. And you should *never* manually mess with the bridges setup by
On 12/16/2013 04:47 AM, Gao feng wrote: libvirt's virtual networks. What you've done here is modify virbr0 to temporarily appear exactly as he already had br0 setup, and this temporary change will be undone the next time you stop/start libvirt's default network, or restart the host. Sorry for coming in so late on this discussion, but it sounds like the guest (aka "vm") is probably configured to use libvirt's "default" network, which means that it is connected to the bridge "virbr0", and that bridge is *not* directly connected to the physical network - it instead sets up a private network on 192.168.122.0/24 and uses IP masquerading (NAT) to make all the guests appear to the outside world as the host's IP address). You can verify this by looking for the <interface> section in the guest's to see if it looks like this: <interface type='network'> ... <source network='default'/> ... </interface> (also, when configured for DHCP, your guest will get an IP address in the 192.168.122.0/24 range). Since virbr0 has no L2 connection to the physical network - its only connection is via the host's IP routing stack, i.e. L3 - simply changing the IP configuration in the guest just gives you an interface with an IP from the physical net that is effectively disconnected from that network, exactly as you've described. If you want your guest to be connected directly to the physical network, you will need to change the guest's config so that the <interface> section instead looks like this: <interface type='bridge'> ... <source bridge='br0'/> ... </interface> You can make this change by running "virsh edit $guestname" at a root shell prompt (that brings up the guest config in vi, or whatever is set as $EDITOR in root's environment), making the modifications and saving the file. After that, you will need to shutdown the guest completely, then restart it for the changes to take effect. At that time you should be able to modify the guest's network config files (in the guest's own system config) with the secondary static IP address your COLO provider has allocated for you. (Note that, if the COLO has restrictions on MAC address (e.g. Hetzner), then you will need to request a new MAC address to go with the 2nd IP address, and edit the guest config's <interface> section to use that MAC address instead of the one automatically generated by libvirt). TL;DR - put back the config for br0 if you've removed it, then modify the guest's <interface> config to use br0 instead of the default network.
On 12/20/2013 04:04 PM, Laine Stump wrote:
On 12/14/2013 10:51 AM, scar wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
I saw there are two bridge br0 and virbr0 in your host. which bridge the libvirt uses? what's result of #virsh net-list ? well i don't quite understand bridges and their function, i'm kinda just following directions. br0 replaced eth0 though in my interfaces file according to this guide[1] i followed to setup the bridge. i'm
Gao feng @ 12/12/2013 10:18 PM: pretty sure virbr0 is what libvirt uses....
host$ cat /etc/network/interfaces auto lo iface lo inet loopback
auto br0 iface br0 inet static address 67.180.246.122 netmask 255.255.255.248 gateway 67.180.246.121 dns-nameservers 67.180.241.11 67.180.241.12 bridge_ports eth0 bridge_fd 9 bridge_maxage 12 bridge_stp off
remove this br0 sector from the interfaces file. and restart the network. (remove the bridge br0) I'm not familiar with the configure files in ubuntu, let's use the command.
1, ifconfig eth0 0.0.0.0 2, brctl addif virbr0 eth0 3, ifconfig virbr0 67.180.246.122/29 4，ip route add default via 67.180.246.121 dev virbr0 5, ifconfig eth0 x.x.x.x/29 (in container) 6, ip route add default via 67.180.246.121 dev eth0 (in container)
but this will not work when you restart libvirt. This advice is incorrect and should not be followed! The br0 that he has created is *exactly* what he needs, so he shouldn't remove it from the config. And you should *never* manually mess with the bridges setup by
On 12/16/2013 04:47 AM, Gao feng wrote: libvirt's virtual networks.
What you've done here is modify virbr0 to temporarily appear exactly as he already had br0 setup, and this temporary change will be undone the next time you stop/start libvirt's default network, or restart the host.
Sorry for coming in so late on this discussion, but it sounds like the guest (aka "vm") is probably configured to use libvirt's "default" network, which means that it is connected to the bridge "virbr0", and that bridge is *not* directly connected to the physical network - it instead sets up a private network on 192.168.122.0/24 and uses IP masquerading (NAT) to make all the guests appear to the outside world as the host's IP address).
You can verify this by looking for the <interface> section in the guest's to see if it looks like this:
<interface type='network'> ... <source network='default'/> ... </interface>
(also, when configured for DHCP, your guest will get an IP address in the 192.168.122.0/24 range). Since virbr0 has no L2 connection to the physical network - its only connection is via the host's IP routing stack, i.e. L3 - simply changing the IP configuration in the guest just gives you an interface with an IP from the physical net that is effectively disconnected from that network, exactly as you've described.
If you want your guest to be connected directly to the physical network, you will need to change the guest's config so that the <interface> section instead looks like this:
<interface type='bridge'> ... <source bridge='br0'/> ... </interface>
You can make this change by running "virsh edit $guestname" at a root shell prompt (that brings up the guest config in vi, or whatever is set as $EDITOR in root's environment), making the modifications and saving the file. After that, you will need to shutdown the guest completely, then restart it for the changes to take effect. At that time you should be able to modify the guest's network config files (in the guest's own system config) with the secondary static IP address your COLO provider has allocated for you.
(Note that, if the COLO has restrictions on MAC address (e.g. Hetzner), then you will need to request a new MAC address to go with the 2nd IP address, and edit the guest config's <interface> section to use that MAC address instead of the one automatically generated by libvirt).
TL;DR - put back the config for br0 if you've removed it, then modify the guest's <interface> config to use br0 instead of the default network.
You are right :)
Laine Stump @ 12/20/2013 01:04 AM:
TL;DR - put back the config for br0 if you've removed it, then modify the guest's <interface> config to use br0 instead of the default network.
thanks that's all i had to do :) then in the guest i just configured /etc/network/interfaces to use the static IP i wanted, and the vm is accessible from the internet like another server!
participants (3)
-
Gao feng -
Laine Stump -
scar