On 07/12/2013 06:30 PM, Daniel Veillard wrote:

On Fri, Jul 12, 2013 at 05:51:14PM +0530, Kashyap Chamarthy wrote:

...

Heya Laine,

Here's some quick notes to associate libvirt guests to Open vSwitch.

Configure Open vSwitch ----------------------

Now that a regular Linux bridge is configured, let's try to configure an OVS brdige and get IP addresses from that space:

Create an Open vSwitch bridge device called 'ovsbr', and display the current state of OpenvSwitch database contents:

$ ovs-vsctl add-br ovsbr $ ovs-vsctl show

Add a virtual ethernet interface called 'veth0' with

$ ip link add name veth0 \ type veth peer name veth1

Add 'veth0' ethernet device to the Linux bridge 'br0', and enumerate all bridge devices:

$ brctl addif br0 veth0 $ brctl show

Now, associate virtual ethernet device 'veth1' to the OVS bridge, and display the current state of OpenvSwitch database contents

$ ovs-vsctl add-port ovsbr veth1 $ ovs-vsctl show

Bring up both the virtual ethernet interfaces 'veth0' and 'veth1'

$ ip link set veth0 up && \ ip link set veth1 up

Update libvirt guest's bridge source to OVS -------------------------------------------

Install a minimal Fedora guest with Oz (or any other mechanism):

$ wget \ https://github.com/kashyapc/virt-scripts/blob/master/oz/oz-jeos.bash $ ./oz-jeos f19-min f19

Once install is finished, define the guest XML from the current dir:

$ virsh define f19-minJul_12_2013-12

Now let's edit libvirt's guest XML file to reflect its bridge source is OVS bridge:

$ virsh edit f19-min

The contents of the guest XML should reflect something along the below lines:

$ virsh dumpxml f19-min | grep bridge -A8 <interface type='bridge'> <mac address='52:54:00:a6:08:70'/> <source bridge='ovsbr'/> <virtualport type='openvswitch'> <parameters interfaceid='ecdff22d-ce80-4ae7-a008-42994415084e'/> </virtualport> <target dev='vnet2'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>

Start the guest, and check the IP of it:

$ virsh start f19-jeos --console $ ifconfig eth0

Please note, this is just a simple test, I haven't done any further experiments with VLAN tagging, etc.

Slightly verbose notes:

http://kashyapc.fedorapeople.org/virt/openvswitch-and-libvirt-kvm.txt

Cool !

It would be good if we could get that on some permanent web page on libvirt.org :-)

Sure. Will submit a doc patch once I'm done with experimenting a bit with other use-cases like pure OVS bridging, etc. -- /kashyap

On 07/12/2013 06:32 PM, Daniel P. Berrange wrote:

On Fri, Jul 12, 2013 at 05:51:14PM +0530, Kashyap Chamarthy wrote:

...

Heya Laine,

Here's some quick notes to associate libvirt guests to Open vSwitch.

Configure Open vSwitch ----------------------

Now that a regular Linux bridge is configured, let's try to configure an OVS brdige and get IP addresses from that space:

Create an Open vSwitch bridge device called 'ovsbr', and display the current state of OpenvSwitch database contents:

$ ovs-vsctl add-br ovsbr $ ovs-vsctl show

Add a virtual ethernet interface called 'veth0' with

$ ip link add name veth0 \ type veth peer name veth1

Add 'veth0' ethernet device to the Linux bridge 'br0', and enumerate all bridge devices:

$ brctl addif br0 veth0 $ brctl show

I don't really see why you are linking ovs to a traditional software bridge.

I had no specific reason on mind. The only test machine I had free was already having a Linux bridge. I thought I'd try on it anyway. Meanwhile, from this networking notes page, http://docs.openstack.org/trunk/openstack-network/admin/content/under_the_ho... it appears that OpenStack uses Linux bridge in conjunction with an OVS bridge: There are four distinct type of virtual networking devices: TAP devices, veth pairs, Linux bridges, and Open vSwitch bridgesFor an ethernet frame to travel from eth0 of virtual machine vm01, to the physical network, it must pass through nine devices inside of the host: TAP vnet0, Linux bridge qbrXXX, veth pair (qcbXXX, qvoXXX), Open vSwitch bridge br-int, veth pair (int-br-eth1, phy-br-eth1), and, finally, the physical network interface card eth1. And further, it notes the distinction between how a Linux bridge and OVS in this context: A Linux bridge behaves like a hub: you can connect multiple (physical or virtual) network interfaces devices to a Linux bridge. Any ethernet frames that come in from one interface attached to the bridge is transmitted to all of the other devices. An Open vSwitch bridge behaves like a virtual switch: network interface devices connect to Open vSwitch bridge's ports, and the ports can be configured much like a physical switch's ports, including VLAN configurations.

You now have the overheads of both bridging & ovs code in your data path. Surely it is better to connect ovs with your physical NIC, taking traditional bridges out of the loop completely.

That's my next test :) -- /kashyap

[...]

...

it appears that OpenStack uses Linux bridge in conjunction with an OVS bridge:

There are four distinct type of virtual networking devices: TAP devices, veth pairs, Linux bridges, and Open vSwitch bridgesFor an ethernet frame to travel from eth0 of virtual machine vm01, to the physical network, it must pass through nine devices inside of the host: TAP vnet0, Linux bridge qbrXXX, veth pair (qcbXXX, qvoXXX), Open vSwitch bridge br-int, veth pair (int-br-eth1, phy-br-eth1), and, finally, the physical network interface card eth1.

That depends on how you configure openstack to operate. The reason openstack links ovs to a bridge, is that you can't setup iptables rules with ovs.

This is useful insight, didn't know about it.

So for each guest, openstack creates a separate bridge + veth pair, and then sets iptables rules on that. This is pretty undesirable from a performance POV due to the number of devices the traffic must traverse :-( So I wouldn't take openstack's usage as an example of good practice here.

Noted. I see your recommendation: from Libvirt guests POV, OVS bridge connected to physical eth0 is the least over-head . I was wondering does an OVS bridge perform any better than Linux bridge. I had a brief chat with Thomas Graf, he mentions, w.r.t performance, from some numbers they did, ovs /slightly/ did better. However, he hasn't seen any numbers that would indicate that ovs is better in performance but i haven't seen anything the other way around either. -- /kashyap