Hello Libvirt Users,
I would like to setup a two node bare-metal cluster. I need to guidance on
the network configuration. I have attached a small diagram, the same
diagram can be seen here: https://i.imgur.com/SOk6a6G.png
I would like to configure the following details:
- Each node has a DHCP enabled guest network where VMs will run. (eg,
*192.168.1.0/24
<http://192.168.1.0/24>* for Host1, and *192.168.2.0/24
<http://192.168.2.0/24>* for Host2)
- Any guest in Host1 should be able to ping guests in Host2, and vice versa.
- All guests have routes to reach the open internet (so that '*yum update*'
will work "out-of-the-box")
- Each node will be able to operate fully if the other physical node fails.
(no central DHCP server, etc)
- I would like to *add more physical nodes later* when I need the resources.
This is what I have done so far:
- Installed latest Ubuntu 18.04, with latest version of libvirt and
supporting software from ubuntu's apt repo.
- Each node can reach the other via its own eth0.
- Each node has a working vxlan0, which can ping the other via its vxlan0,
so it looks like the vxlan config is working. (I used *ip link add vxlan0
type vxlan...*)
- Configured route on Host1 like so: *ip route add 192.168.2.0/24
<http://192.168.2.0/24> via 172.20.0.1*
- Configured route on Host2 also: *ip route add 192.168.1.0/24
<http://192.168.1.0/24> via 172.20.0.2*
- All guests on Host1 (and Host1) can ping eth0 and vxlan0 on Host2, and
vice versa, yay.
- Guests on Host1 *cannot* ping guests on Host2, I suspect because the the
default NAT config of the libvirt network.
So, at this point I started to search for tutorials or more
information/documentation, but I am a little overwhelmed by the sheer
amount of information, as well as a lot of "stale" information on blogs etc.
I have learned that I can *virsh net-edit default*, and then change it to
an "open" network:* <forward mode='open'/>*
After doing this, the guests cannot reach outside their own network, nor
reach the internet, so I assume that I would need to add some routes, or
something else to get the network functioning like I want it. There is
also *<forward
mode="route"/>*, but I dont fully understand the scenarios where one would
need an *open* or a *route* forward mode. I have also shied away from using
openvswitch, and have opted for ifupdown2.
(I have taken most of my inspiration from this blog post:
https://joejulian.name/post/how-to-configure-linux-vxlans-with-multiple-uni…
)
Some questions that I have for the mailing list, any help would be greatly
appreciated:
- Is my target configuration of a KVM cluster uncommon? Do you see
drawbacks of this setup, or does it go against "typical convention"?
- Would my scenario be better suited for an "*open*" network or a "*route*"
network?
- What would be the approach to complete this setup?
