Heya Laine, Here's some quick notes to associate libvirt guests to Open vSwitch. Configure Open vSwitch ---------------------- Now that a regular Linux bridge is configured, let's try to configure an OVS brdige and get IP addresses from that space: Create an Open vSwitch bridge device called 'ovsbr', and display the current state of OpenvSwitch database contents: $ ovs-vsctl add-br ovsbr $ ovs-vsctl show Add a virtual ethernet interface called 'veth0' with

$ ip link add name veth0 \ type veth peer name veth1 Add 'veth0' ethernet device to the Linux bridge 'br0', and enumerate all bridge devices: $ brctl addif br0 veth0 $ brctl show Now, associate virtual ethernet device 'veth1' to the OVS bridge, and display the current state of OpenvSwitch database contents $ ovs-vsctl add-port ovsbr veth1 $ ovs-vsctl show Bring up both the virtual ethernet interfaces 'veth0' and 'veth1' $ ip link set veth0 up && \ ip link set veth1 up Update libvirt guest's bridge source to OVS ------------------------------------------- Install a minimal Fedora guest with Oz (or any other mechanism): $ wget \ https://github.com/kashyapc/virt-scripts/blob/master/oz/oz-jeos.bash $ ./oz-jeos f19-min f19 Once install is finished, define the guest XML from the current dir: $ virsh define f19-minJul_12_2013-12 Now let's edit libvirt's guest XML file to reflect its bridge source is OVS bridge: $ virsh edit f19-min The contents of the guest XML should reflect something along the below lines: $ virsh dumpxml f19-min | grep bridge -A8 <interface type='bridge'> <mac address='52:54:00:a6:08:70'/> <source bridge='ovsbr'/> <virtualport type='openvswitch'> <parameters interfaceid='ecdff22d-ce80-4ae7-a008-42994415084e'/> </virtualport> <target dev='vnet2'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> Start the guest, and check the IP of it: $ virsh start f19-jeos --console $ ifconfig eth0 Please note, this is just a simple test, I haven't done any further experiments with VLAN tagging, etc. Slightly verbose notes: http://kashyapc.fedorapeople.org/virt/openvswitch-and-libvirt-kvm.txt

On Fri, Jul 12, 2013 at 05:51:14PM +0530, Kashyap Chamarthy wrote: > Heya Laine, > > Here's some quick notes to associate libvirt guests to Open vSwitch. > > Configure Open vSwitch > ---------------------- > > Now that a regular Linux bridge is configured, let's try to configure an > OVS brdige and get IP addresses from that space: > > Create an Open vSwitch bridge device called 'ovsbr', and display the > current state of OpenvSwitch database contents: > > $ ovs-vsctl add-br ovsbr > $ ovs-vsctl show > > > Add a virtual ethernet interface called 'veth0' with > > $ ip link add name veth0 \ > type veth peer name veth1 > > Add 'veth0' ethernet device to the Linux bridge 'br0', and enumerate all > bridge devices: > > $ brctl addif br0 veth0 > $ brctl show > > Now, associate virtual ethernet device 'veth1' to the OVS bridge, > and display the current state of OpenvSwitch database contents > > $ ovs-vsctl add-port ovsbr veth1 > $ ovs-vsctl show > > > Bring up both the virtual ethernet interfaces 'veth0' and 'veth1' > > $ ip link set veth0 up && \ > ip link set veth1 up > > > Update libvirt guest's bridge source to OVS > ------------------------------------------- > > Install a minimal Fedora guest with Oz (or any other mechanism): > > $ wget \ > https://github.com/kashyapc/virt-scripts/blob/master/oz/oz-jeos.bash > $ ./oz-jeos f19-min f19 > > Once install is finished, define the guest XML from the current dir: > > $ virsh define f19-minJul_12_2013-12 > > > Now let's edit libvirt's guest XML file to reflect its bridge source is > OVS bridge: > > $ virsh edit f19-min > > The contents of the guest XML should reflect something along the below > lines: > > $ virsh dumpxml f19-min | grep bridge -A8 > <interface type='bridge'> > <mac address='52:54:00:a6:08:70'/> > <source bridge='ovsbr'/> > <virtualport type='openvswitch'> > <parameters interfaceid='ecdff22d-ce80-4ae7-a008-42994415084e'/> > </virtualport> > <target dev='vnet2'/> > <model type='virtio'/> > <alias name='net0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> > </interface> > > Start the guest, and check the IP of it: > > $ virsh start f19-jeos --console > $ ifconfig eth0 > > > Please note, this is just a simple test, I haven't done any further experiments with VLAN > tagging, etc. > > > Slightly verbose notes: > > http://kashyapc.fedorapeople.org/virt/openvswitch-and-libvirt-kvm.txt > Cool ! It would be good if we could get that on some permanent web page on libvirt.org :-)

On Fri, Jul 12, 2013 at 05:51:14PM +0530, Kashyap Chamarthy wrote: > Heya Laine, > > Here's some quick notes to associate libvirt guests to Open vSwitch. > > Configure Open vSwitch > ---------------------- > > Now that a regular Linux bridge is configured, let's try to configure an > OVS brdige and get IP addresses from that space: > > Create an Open vSwitch bridge device called 'ovsbr', and display the > current state of OpenvSwitch database contents: > > $ ovs-vsctl add-br ovsbr > $ ovs-vsctl show > > > Add a virtual ethernet interface called 'veth0' with > > $ ip link add name veth0 \ > type veth peer name veth1 > > Add 'veth0' ethernet device to the Linux bridge 'br0', and enumerate all > bridge devices: > > $ brctl addif br0 veth0 > $ brctl show I don't really see why you are linking ovs to a traditional software bridge.

You now have the overheads of both bridging & ovs code in your data path. Surely it is better to connect ovs with your physical NIC, taking traditional bridges out of the loop completely.

> it appears that OpenStack uses Linux bridge in conjunction with an OVS bridge: > > There are four distinct type of virtual networking devices: TAP > devices, veth pairs, Linux bridges, and Open vSwitch bridgesFor an > ethernet frame to travel from eth0 of virtual machine vm01, to the > physical network, it must pass through nine devices inside of the > host: TAP vnet0, Linux bridge qbrXXX, veth pair (qcbXXX, qvoXXX), > Open vSwitch bridge br-int, veth pair (int-br-eth1, phy-br-eth1), > and, finally, the physical network interface card eth1. That depends on how you configure openstack to operate. The reason openstack links ovs to a bridge, is that you can't setup iptables rules with ovs.

So for each guest, openstack creates a separate bridge + veth pair, and then sets iptables rules on that. This is pretty undesirable from a performance POV due to the number of devices the traffic must traverse :-( So I wouldn't take openstack's usage as an example of good practice here.