[libvirt] Libvirt, nwfilter, openvswitch
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels). But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand... Now.. Is there a way to define nwfilter rules _without_ ebtables ? What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements ) Thanks
On Wed, Nov 06, 2013 at 02:44:57PM +0100, Jan De Landtsheer wrote:
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels).
But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand...
Now.. Is there a way to define nwfilter rules _without_ ebtables ?
What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements )
There's no support for nwfilter at all when using openvswitch, due to the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
yeah, I know... that's not gonna happen any time soon in my setup... tested OpenStack, and all is 'working' until it doesn't. Then try to find out why... OTOH I like openvswitch very much, and not only for performance. I'll define a hooks/qemu then. Had one running with some mangle marks for TC Aight, Thanks Jan On Thu, Nov 7, 2013 at 5:04 AM, Daniel P. Berrange <berrange@redhat.com>wrote:
On Wed, Nov 06, 2013 at 02:44:57PM +0100, Jan De Landtsheer wrote:
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels).
But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand...
Now.. Is there a way to define nwfilter rules _without_ ebtables ?
What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements )
There's no support for nwfilter at all when using openvswitch, due to the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev
Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/:| |: http://libvirt.org -o- http://virt-manager.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/:| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc:|
On 11/07/2013 06:04 AM, Daniel P. Berrange wrote:
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels).
But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand...
Now.. Is there a way to define nwfilter rules _without_ ebtables ?
What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements ) There's no support for nwfilter at all when using openvswitch, due to
On Wed, Nov 06, 2013 at 02:44:57PM +0100, Jan De Landtsheer wrote: the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev
It would be an "interesting exercise" to see if it was possible to write a backend for nwfilter that used OVS flows or whatever it is they have. My suspicion is that the configuration model used by nwfilter may be too low level and (ip|eb)tables-specific to be mapped into a different backend implementation, so it could require some new higher level elements that would then need to be implemented for the (ip|eb)tables backend as well (and the original config elements deprecated).
Why not use ovs-ofctl add-flow for this? Translate iptables rules to openflow flows not very hard.... 2013/11/7 Laine Stump <laine@laine.org>:
On 11/07/2013 06:04 AM, Daniel P. Berrange wrote:
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels).
But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand...
Now.. Is there a way to define nwfilter rules _without_ ebtables ?
What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements ) There's no support for nwfilter at all when using openvswitch, due to
On Wed, Nov 06, 2013 at 02:44:57PM +0100, Jan De Landtsheer wrote: the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev
It would be an "interesting exercise" to see if it was possible to write a backend for nwfilter that used OVS flows or whatever it is they have. My suspicion is that the configuration model used by nwfilter may be too low level and (ip|eb)tables-specific to be mapped into a different backend implementation, so it could require some new higher level elements that would then need to be implemented for the (ip|eb)tables backend as well (and the original config elements deprecated).
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru
Not very hard ? please enlighten me ;-) I'm _really_ struggling, mainly because I'm a neophyte in OpenVSwitch and Flow the only thing I want is block outgoing dhcp{4,6} and routing advertisements coming out of VM's On Thu, Nov 7, 2013 at 9:05 PM, Vasiliy Tolstov <v.tolstov@selfip.ru> wrote:
Why not use ovs-ofctl add-flow for this? Translate iptables rules to openflow flows not very hard....
2013/11/7 Laine Stump <laine@laine.org>:
On 11/07/2013 06:04 AM, Daniel P. Berrange wrote:
Hello, I _was_ trying to set up an nwfilter for our networking set-up with VXLAN and openvswitch, where we use VXLAN as carrier for separate networks (unlike OpenStack gre-tunnels).
But with OVS, ebtables do not work, and the basic setup of nwfilter rules are based on this premise... or so I understand...
Now.. Is there a way to define nwfilter rules _without_ ebtables ?
What I would like to do is quite simple (block out dhcp{4,6} services from VM's, and ipv6 router advertisements ) There's no support for nwfilter at all when using openvswitch, due to
On Wed, Nov 06, 2013 at 02:44:57PM +0100, Jan De Landtsheer wrote: the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev
It would be an "interesting exercise" to see if it was possible to write a backend for nwfilter that used OVS flows or whatever it is they have. My suspicion is that the configuration model used by nwfilter may be too low level and (ip|eb)tables-specific to be mapped into a different backend implementation, so it could require some new higher level elements that would then need to be implemented for the (ip|eb)tables backend as well (and the original config elements deprecated).
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru
2013/11/8 Jan De Landtsheer <jan.delandtsheer@gmail.com>:
Not very hard ? please enlighten me ;-) I'm _really_ struggling, mainly because I'm a neophyte in OpenVSwitch and Flow
the only thing I want is block outgoing dhcp{4,6} and routing advertisements coming out of VM's
https://github.com/xapi-project/xen-api/blob/master/scripts/setup-vif-rules#... see this for examples. flows added by ovs-ofctl add-flow BRIDGE xxx,xxx,xx -- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru
The 07/11/13, Daniel P. Berrange wrote:
There's no support for nwfilter at all when using openvswitch, due to the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev
Why is it "disgusting"? -- Nicolas Sebrecht
On 11/12/2013 11:10 AM, Nicolas Sebrecht wrote:
The 07/11/13, Daniel P. Berrange wrote:
There's no support for nwfilter at all when using openvswitch, due to the kernel limitations you mention. The (disgusting) way openstack deals with this is to create a traditional bridge per vm so you have
phys nic <-> openvswitch \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev \---> vm bridge <-> vm tap dev Why is it "disgusting"?
Because it's terribly inefficient. You may, on the other hand, view it as "clever", because it is able to work around deficiencies in the individual components to make something that works at all. It certainly is true, though, that a lot of cycles are being wasted on each packet's trip through all that network linkage, and it would sure be nice if that waste could be avoided.
2013/11/12 Laine Stump <laine@laine.org>:
Because it's terribly inefficient.
You may, on the other hand, view it as "clever", because it is able to work around deficiencies in the individual components to make something that works at all. It certainly is true, though, that a lot of cycles are being wasted on each packet's trip through all that network linkage, and it would sure be nice if that waste could be avoided.
May be the best way to create minimal daemon that able to talk via rpc with libvirt and have backends for nwfilter and openflow. Each rule it translate to needed rules.. -- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru
Hi Gang, Daniel Added a little Script, linked to /etc/libvirt/qemu<https://github.com/PurePeople/archipv6/blob/master/qemu.py> to be able to add some flows to my openvswitch. Script ran to my liking (IANAP), and then adding it to /etc/libvirt/hooks... Hang. Only after a lot of looking around (in the last lines of the Hooks page), aha! Don't do this! don't call libvirt in your hook script... :-( What I want: the vnets and their mac addresses for the started VM, otherwise there wouldn't be a need for any of this script, but yeah... nope... Is there _any_ way I can get the vnet list for the star{t,ted) vm with the corresponding mac address of the tap interface ? I figure that the only way to know the mac addr of the VM would be to query libvirt, so I'm at a loss here. Note, of course, this script ran perfectly from command line, until it went into /etc/libvirt/hooks/qemu ;-) da-mn! Can this eventually be called from somewhere else? Since libvirt has everything figured out in the 'start' stage, can't I receive more than just 4 parameters ? Any ideas? ;-) Greetz Jan On Wed, Nov 13, 2013 at 7:39 AM, Vasiliy Tolstov <v.tolstov@selfip.ru>wrote:
2013/11/12 Laine Stump <laine@laine.org>:
Because it's terribly inefficient.
You may, on the other hand, view it as "clever", because it is able to work around deficiencies in the individual components to make something that works at all. It certainly is true, though, that a lot of cycles are being wasted on each packet's trip through all that network linkage, and it would sure be nice if that waste could be avoided.
May be the best way to create minimal daemon that able to talk via rpc with libvirt and have backends for nwfilter and openflow. Each rule it translate to needed rules..
-- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru jabber: vase@selfip.ru
participants (5)
-
Daniel P. Berrange -
Jan De Landtsheer -
Laine Stump -
Nicolas Sebrecht -
Vasiliy Tolstov