[libvirt] IPv6: getting it to work

Before I start creating patches (since it is not only source code but also documentation, schemas, tests, etc), I thought I would run this by you folks for comments/suggestions. With IPv4, it is relatively easy to set up working networks: just use nat/MASQUERADE and things pretty much just work. With IPv6, it is a bit more difficult because IPv6 is route-only with (theoretically) unique addresses known across the Internet. It took me a while but once I realized that I needed to define some static routes on my default router, it because much easier. The default route(s) on the default router connect the IPv6 guest network used by a qemu-kvm/libvirt with the virtual host's IPv6 address. My recommendation is to use a /48 or /56 IPv6 network and assign it to a specific virtual host. This virtual host needs to have a fixed IPv6 address either with manual configuration or using a client-id to pin a specific IPv6 address. Then make each of the virtual networks on that host be a /64 network. So far so good and really not too much of an administrative burden. NetworkManager can be used to set this default route. Now lets add some additional virtual network layers/segments to the mix. For example: guest10 <-> net-a <-> guest20 <-> net-b <-> guest30 <-> virbr<n> host40 <-> router50 <-> host60 guest30 can talk to other systems such as host40, router50, and host60 on the real network since it is covered by the static route on the default router. guest10, guest20, and guest30 can talk to each other with some additional static routes (or just default routes). The problem: guest10 cannot talk to other real hosts such as host40 or host60. The problem is that NetworkManager will not set a static route for any network on a libvirt bridge device (or any bridge device which NM does not "own"). At first I thought this was a NM problem but I now believe that this should be fixed by libvirt. I did some manual configuration to figure out what needed to be done so that guest10 could use IPv6 to talk to another host on the real network. 1. You have defined a static route on the default router for fd00:aa:bb::/48 to the virtual host. 2. You have an libvirt network defined with (for example) fd00:aa:bb:10::/64 and a guest on that network with the address of fd00:aa:bb:10::2/128. Lets say it is on virbr4. 3. Your secondary (isolated) virtual network is fd00:aa:bb:11::/64. 4. You need to issue firewalld commands so it will pass the additional network on virbr4. These are of the form: firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -d fd00:aa:bb:11::/64 -o virbr4 -j ACCEPT firewall-cmd --direct --passthrough ipv6 -I FORWARD -1 -s fd00:aa:bb:11::/64 -i virbr4 -j ACCEPT 5. Create the route: ip -6 route fd00:aa:bb:11::/64 via fd00:aa:bb:10::2 dev virbr4 proto static metric 1 OK, that is what has to be done but I want libvirt to do all of this for me after some simple configuration. I propose adding a new optional xml-element to the <ip> element: <via> <via> would be an exclusive alternate to <dhcp> and both <via> and <dhcp> could not be used under a single <ip> definition. As implied by the name, <via> would specify the gateway address which is to receive the packets on the designated network. Right now, if you specify an additional IPv6 address to a network definition, you get the correct ip6tables rules but you also get a ip-addr for that additional definition and an ip-route for the related network. With <via>, this last part would be replaced with NO additional ip-addr and a static route for the network to the gateway. Anticipated code changes (besides the tests, schemas, and documentation) are: network_conf.c to handle the new <via> element. virnetdev.c to create and issue the ip-6-route command. bridge_driver.c to detect when an IPv6 address is a "via" and do the ip-6-route instead of adding the address. Although this is being done for IPv6, there is no reason not to make sure it also works with IPv4. Comments, suggestions appreciated. Gene