Update: I have upgraded from libvirt 0.9.8 on the virtualization server to libvirt 1.0.4. Now the net-define command works with the "virtualport type='openvswitch'" directive in it. So I did a "net-edit" on the networks, added that line in, and saved them. So now I am assuming libvirt knows that the underlying bridge type is Open vSwitch. However, when I try to use virt-manager (which is 0.9.4 running on Fedora 18) to spin up a new VM and try to attach it to one of the networks I defined on the virtualization host (runs Ubuntu 12.04 Server with libvirt 1.0.4 as I had mentioned above), I am now getting the following different traceback: Unable to complete install: 'Unable to add port vnet0 to OVS bridge vl20-ovsbr0: Operation not permitted' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/create.py", line 1943, in do_install guest.start_install(False, meter=meter) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1249, in start_install noboot) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1317, in _create_guest dom = self.conn.createLinux(start_xml or final_xml, 0) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: Unable to add port vnet0 to OVS bridge vl20-ovsbr0: Operation not permitted So the error went from "Operation not supported" to "Operation not permitted"... Is this some sort of a permissions error now?? If someone could please help me figure this out, that would be great, as I'm up against a bit of a deadline with this to get it working... Thanks, Will -----Original Message----- From: Will Dennis Sent: Friday, April 05, 2013 12:12 PM To: libvirt-users(a)redhat.com Subject: Problem with net-define using Open vSwitch bridge Hi all, I used "net-define" to create a network to use to attach VMs to an Open vSwitch bridge. The net-define worked fine, I can see the network in "net-list" (I set it to started and to autostart) but when I try to attach a new VM's NIC to this network (via virt-manager), I'm getting the following traceback: Unable to complete install: 'Unable to add bridge vl10-ovsbr0 port vnet0: Operation not supported' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/create.py", line 1943, in do_install guest.start_install(False, meter=meter) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1249, in start_install noboot) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1317, in _create_guest dom = self.conn.createLinux(start_xml or final_xml, 0) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: Unable to add bridge vl10-ovsbr0 port vnet0: Operation not supported I'm going to assume the problem is that the bridge is not a linux bridge, but an Open vSwitch bridge. But I did have a line in the XML file that I used for net-define as follows: <virtualport type='openvswitch'/> But when I ran net-define, it failed to create the network with the error: XML error: unexpected virtualport type -1 Removing that line and re-running the net-define worked, as I have stated above. How do I tell libvirt that this bridge is an Open vSwitch bridge and not a regular Linux bridge? Thanks, Will

Thanks, Laine, for your reply. The OVS bridge named 'vl20-ovsbr0' does exist, but it is an Open vSwitch "fake bridge" -- see http://blog.scottlowe.org/2012/10/19/vlans-with-open-vswitch-fake-bridge s/ for why I'm doing this. Here's some relevant output from my system: # ifconfig | grep encap eth0 Link encap:Ethernet HWaddr 00:25:90:aa:bb:cc lo Link encap:Local Loopback ovsbr0 Link encap:Ethernet HWaddr a2:c6:37:cd:90:4a vl10-ovsbr0 Link encap:Ethernet HWaddr b6:c4:8c:69:2b:06 vl20-ovsbr0 Link encap:Ethernet HWaddr d6:d0:be:6e:b6:fb # ovs-vsctl --real list-br ovsbr0 # ovs-vsctl --fake list-br vl10-ovsbr0 vl20-ovsbr0 # ovs-vsctl show 3e0d861b-efb7-46b1-af1b-4a76cd833558 Bridge "ovsbr0" Controller "tcp:127.0.0.1:6633" is_connected: true Port "ovsbr0" Interface "ovsbr0" type: internal Port "vl10-ovsbr0" tag: 10 Interface "vl10-ovsbr0" type: internal Port "vl20-ovsbr0" tag: 20 Interface "vl20-ovsbr0" type: internal # ovs-vsctl br-to-parent vl20-ovsbr0 ovsbr0 virsh # net-dumpxml 20-net <network> <name>20-net</name> <uuid>9e2c2ce7-a193-17ec-d6c0-3f115e759e29</uuid> <forward mode='bridge'/> <bridge name='vl20-ovsbr0' /> <virtualport type='openvswitch'/> </network> (I have no XML from the guest, as virt-manager will not let me create it with the desired network...) Is the problem that libvirt only can define networks with "real" OVS bridges? That would be a bummer, since I need the VM's port tagged with a specific VLAN... Or is there another workable way to do this? Thanks, Will On 04/08/2013 04:15 PM, Laine Stump wrote: -----Original Message----- From: Will Dennis Sent: Monday, April 08, 2013 2:45 PM To: libvirt-users(a)redhat.com Subject: RE: Problem with net-define using Open vSwitch bridge Update: I have upgraded from libvirt 0.9.8 on the virtualization server to libvirt 1.0.4. Now the net-define command works with the "virtualport type='openvswitch'" directive in it. So I did a "net-edit" on the networks, added that line in, and saved them. So now I am assuming libvirt knows that the underlying bridge type is Open vSwitch. However, when I try to use virt-manager (which is 0.9.4 running on Fedora 18) to spin up a new VM and try to attach it to one of the networks I defined on the virtualization host (runs Ubuntu 12.04 Server with libvirt 1.0.4 as I had mentioned above), I am now getting the following different traceback: Unable to complete install: 'Unable to add port vnet0 to OVS bridge vl20-ovsbr0: Operation not permitted' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/create.py", line 1943, in do_install guest.start_install(False, meter=meter) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1249, in start_install noboot) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1317, in _create_guest dom = self.conn.createLinux(start_xml or final_xml, 0) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: Unable to add port vnet0 to OVS bridge vl20-ovsbr0: Operation not permitted So the error went from "Operation not supported" to "Operation not permitted"... Is this some sort of a permissions error now?? If someone could please help me figure this out, that would be great, as I'm up against a bit of a deadline with this to get it working... Thanks, Will -----Original Message----- From: Will Dennis Sent: Friday, April 05, 2013 12:12 PM To: libvirt-users(a)redhat.com Subject: Problem with net-define using Open vSwitch bridge Hi all, I used "net-define" to create a network to use to attach VMs to an Open vSwitch bridge. The net-define worked fine, I can see the network in "net-list" (I set it to started and to autostart) but when I try to attach a new VM's NIC to this network (via virt-manager), I'm getting the following traceback: Unable to complete install: 'Unable to add bridge vl10-ovsbr0 port vnet0: Operation not supported' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/create.py", line 1943, in do_install guest.start_install(False, meter=meter) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1249, in start_install noboot) File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line 1317, in _create_guest dom = self.conn.createLinux(start_xml or final_xml, 0) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in createLinux if ret is None:raise libvirtError('virDomainCreateLinux() failed', conn=self) libvirtError: Unable to add bridge vl10-ovsbr0 port vnet0: Operation not supported I'm going to assume the problem is that the bridge is not a linux bridge, but an Open vSwitch bridge. But I did have a line in the XML file that I used for net-define as follows: <virtualport type='openvswitch'/> But when I ran net-define, it failed to create the network with the error: XML error: unexpected virtualport type -1 Removing that line and re-running the net-define worked, as I have stated above. How do I tell libvirt that this bridge is an Open vSwitch bridge and not a regular Linux bridge? Thanks, Will

So, looks like a permissions problem.. From /var/log/libvirt/libvirtd.log: 2013-04-08 16:58:53.897+0000: 6743: info : libvirt version: 1.0.4 2013-04-08 16:58:53.897+0000: 6743: error : virCommandWait:2315 : internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 -- --may-exist add-port vl20-ovsbr0 vnet0 -- set Interface vnet0 'external-ids:attached-mac="52:54:00:10:B0:EE"' -- set Interface vnet0 'external-ids:iface-id="6e4a326a-1595-e712-15ec-c4905725663e"' -- set Interface vnet0 'external-ids:vm-id="67e7842b-a47b-54df-bb7c-aca75368cd45"' -- set Interface vnet0 external-ids:iface-status=active) unexpected exit status 1: libvirt: error : cannot execute binary /usr/local/bin/ovs-vsctl: Permission denied 2013-04-08 16:58:53.897+0000: 6743: error : virNetDevOpenvswitchAddPort:136 : Unable to add port vnet0 to OVS bridge vl20-ovsbr0: Operation not permitted 2013-04-08 16:58:53.937+0000: 6743: error : virCommandWait:2315 : internal error Child process (/usr/local/bin/ovs-vsctl --timeout=5 -- --if-exists del-port) unexpected exit status 1: libvirt: error : cannot execute binary /usr/local/bin/ovs-vsctl: Permission denied But ovs-vsctl does have read+exec set for 'others': # ls -la /usr/local/bin/ovs-vsctl -rwxr-xr-x 1 root root 3930719 Feb 26 17:12 /usr/local/bin/ovs-vsctl What else is needed? The reason I want to use OVS "fake bridges" is to keep it simple for the team that will be using this virtualization server. If they just select the correct libvirt network (which is tied to the OVS "fake bridge") then the ports will be automatically tagged with the correct VLAN. (I think of OVS fake bridges as something akin to VMware "Port Groups" on an OVS level.) I have defined libvirt networks as detailed in Kyle's blogpost you referenced, but there does not seem to be a way in the virt-manager UI to select the desired portgroup as defined in the network XML. (Yes, you can edit the VM's XML config and hack the portgroup into there, but that's not what I want these users to have to do...) Thanks again for your assistance! Will -----Original Message----- From: sendmail [mailto:justsendmailnothingelse@gmail.com] On Behalf Of Laine Stump Sent: Tuesday, April 09, 2013 10:47 AM To: libvirt-users(a)redhat.com Cc: Will Dennis Subject: Re: [libvirt-users] Problem with net-define using Open vSwitch bridge On 04/08/2013 05:20 PM, Will Dennis wrote: This looks fine. Since this is the first time I've heard of an ovs "fake bridge", I can't really say for sure if it works correctly. However, I now see from your 2nd message that the reason for your initial failure was that you were using a version of libvirt (0.9.4) that had no OVS support at all, and that you've since upgraded to libvirt 1.0.4. Meanwhile, the blog post that you reference above as the reason you're using a "fake bridge" is saying that the reason to do this is to overcome some unnamed bug in libvirt's OVS vlan support. That blog entry was written in October 2012, and mentions that "Kyle" (probably Kyle Mestery) had submitted a patch for [whatever the problem was]. I'm unable to determine which bug or which patch they're talking about from that information, but I don't believe there are any open bugs in OVS vlan support, so it must have been pushed and so would definitely be fixed in libvirt 1.0.4. So, my advice is 1) stop using "fake bridges" and 2) use libvirt's ovs vlan support, as described here: http://www.siliconloons.com/?p=305 (It would be useful to make these "fake bridges" work properly though. You can probably find the error message output by ovs-vsctl by looking in /var/log/messages or /var/log/libvirt/libvirtd.log (it will be logged by the libvirtd process). If you can post that message to this thread, maybe someone will understand why it's failing, and we can push a fix for it.)

(Replies inlined as requested...) I assume you can run /usr/local/bin/ovs-vsctl from a root shell, correct? Yes. logs that could indicate the reason for the denial? Anybody else have an idea? Have AppArmor, no SELinux. I did a status on the AppArmor, and I see that libvirtd was in 'enforce' mode: # apparmor_status apparmor module is loaded. 6 profiles are loaded. 6 profiles are in enforce mode. /sbin/dhclient /usr/lib/NetworkManager/nm-dhcp-client.action /usr/lib/connman/scripts/dhclient-script /usr/lib/libvirt/virt-aa-helper /usr/sbin/libvirtd /usr/sbin/tcpdump 0 profiles are in complain mode. 1 processes have profiles defined. 1 processes are in enforce mode. /usr/sbin/libvirtd (6941) 0 processes are in complain mode. 0 processes are unconfined but have a profile defined. So I changed the mode across the board to 'complain': # aa-complain /etc/apparmor.d/* Setting /etc/apparmor.d/sbin.dhclient to complain mode. Setting /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper to complain mode. Setting /etc/apparmor.d/usr.sbin.libvirtd to complain mode. Setting /etc/apparmor.d/usr.sbin.rsyslogd to complain mode. Setting /etc/apparmor.d/usr.sbin.tcpdump to complain mode. # apparmor_status apparmor module is loaded. 7 profiles are loaded. 0 profiles are in enforce mode. 7 profiles are in complain mode. /sbin/dhclient /usr/lib/NetworkManager/nm-dhcp-client.action /usr/lib/connman/scripts/dhclient-script /usr/lib/libvirt/virt-aa-helper /usr/sbin/libvirtd /usr/sbin/rsyslogd /usr/sbin/tcpdump 2 processes have profiles defined. 0 processes are in enforce mode. 1 processes are in complain mode. /usr/sbin/libvirtd (6941) 1 processes are unconfined but have a profile defined. /usr/sbin/rsyslogd (815) And then virt-manager could indeed attach the VM nic to the defined network! (used "10-net" on this one, which corresponds to the "vl10-ovsbr0" fake bridge): # /usr/local/bin/ovs-vsctl show 3e0d861b-efb7-46b1-af1b-4a76cd833558 Bridge "ovsbr0" Controller "tcp:127.0.0.1:6633" is_connected: true Port "ovsbr0" Interface "ovsbr0" type: internal Port "vnet0" <<<<<<<<<<< tag: 10 <<<<<<<<<<< Interface "vnet0" Port "vl10-ovsbr0" tag: 10 Interface "vl10-ovsbr0" type: internal Port "vl20-ovsbr0" tag: 20 Interface "vl20-ovsbr0" type: internal Yay. detailed way then telling the users to select the proper portgroup would be the right way to do what you want, new libvirt features in virt-manager for "awhile" now, and the addition of portgroups to libvirt I'm sure virt-manager would *gladly* accept a patch. Otherwise, hopefully there will be some new take on. Sadly I have (very) little python-fu, so no doubt am unqualified to try to patch virt-manager at this point... (Hopefully someday when I have more time to dive into Python!) It would be a great thing to add to the virt-manager UI, IMHO... Is there a separate list for virt-manager (or is virt-manager, virt-viewer, etc. devel discussed here?) Thanks again for your guidance, Laine - community rocks! Best, Will