Hi! The following set of patches add network filtering (ACL) extensions to libvirt and enable network traffic filtering for VMs using ebtables and, depending on the networking technology being used (tap, but not macvtap), also iptables. Usage of either is optional and controlled through filters that a VM is referencing. The ebtables-level filtering is based on the XML derived from the CIM network slide 10 (filtering) from the DMTF website (http://www.dmtf.org/standards/cim/cim_schema_v2230/CIM_Network.pdf). The XML we derived from this was discussed on the list before. On the ebtables level we currently handle filtering of IPv4 and ARP traffic. The iptables-level filtering is based on similar XML where XML nodes described the particular protocol to filter for. Its extensions enable the filtering of traffic using iptables for tcp, udp, icmp, igmp, sctp and 'all' types of traffic. This list of protocols maps to the features supported by iptables and only excludes protocols like 'esp' and 'ah'. Currently only bridging mode is supported and based on availability of the physdev match. The filtering framework adds new libvirt virsh commands for managing the filters. The 5 new commands are: - virsh nwfilter-list - virsh nwfilter-dumpxml <name of filter> - virsh nwfilter-define <name of file containing filter desc.> - virsh nwfilter-undefine <name of filter> - virsh nwfilter-edit <name of filter> Above commands are similar to commands for already existing pools and as such much of the code directly related to the above commands could be borrowed from other drivers. The network filters can either contain rules using the above mentioned XML or contain references to other filters in order to build more complex filters that form some sort of filter tree or can contain both. An example for a filter referencing other filters would be this one here: <filter name='demofilter4' chain='root'> <uuid>66f62d1d-34c1-1421-824f-c62d5ee5e8b6</uuid> <filterref filter='no-mac-spoofing'/> <filterref filter='no-mac-broadcast'/> <filterref filter='no-arp-spoofing'/> <filterref filter='allow-dhcp'> <parameter name='DHCPSERVER' value='10.0.0.1'/> </filterref> <filterref filter='no-other-l2-traffic'/> <filterref filter='recv-only-vm-ipaddress'/> <filterref filter='recv-only-vm-macaddress'/> <filterref filter='l3-test'/> </filter> A filter containing actual rules would look like this: <filter name='no-mac-broadcast' chain='ipv4'> <uuid>ffe2ccd6-edec-7360-1852-6b5ccb553234</uuid> <rule action='drop' direction='out' priority='500'> <mac dstmacaddr='ff:ff:ff:ff:ff:ff'/> </rule> </filter> The filter XML now also holds a priority attribute in the rule. This provides control over the ordering of the applied ebtables/iptables rules beyond their appearance in the XML. The domain XML has been extended to reference a top level filter from within each <interface> XML node. A valid reference to such a top level filter looks like this: <interface type='bridge'> <source bridge='static'/> <filterref filter='demofilter4'> <parameter name='IP' value='9.59.241.151'/> </filterref> </interface> In this XML a parameter IP is passed for instantiation of the referenced filters, that may require the availability of this parameter. In the above case the IP parameter's value describes the value of the IP address of the VM and allows to enable those filters to be instantiated that require this 'IP' variable. If a filter requires a parameter that is not provided, the VM will not start or the interface will not attach to a running VM. Any names of parameters can be provided for instantiation of filters and their names and values only need to pass a regular expression test. Currently only MAC and IP addresses and port numbers can be replaced with variables inside the filter XML. In a subsequent patch we will be adding capability to allow users to omit the IP parameter (only) and enable libvirt to learn the IP address of the VM and have it instantiate the filter once it knows it. While virtual machines are running, it is possible to update their filters. For that all running VMs' filter 'trees' are traversed to detect whether the updated filter is referenced by the VM. If so, its ebtables/iptable rules are applied. If one of the VMs' update fails allupdates are rolled back and the filter XML update is rejected. One comment about the instantiation of the rules: Since the XML allows to create nearly any possible combination of parameters to ebtables or iptables commands, I haven't used the ebtables or iptables wrappers. Instead, I am writing ebtables/iptables command into a buffer, add command line options to each one of them as described in the rule's XML, write the buffer into a file and run it as a script. For those commands that are not allowed to fail I am using the following format to run them: cmd="ebtables <some options>" r=`${cmd}` if [ $? -ne 0 ]; then echo "Failure in command ${cmd}." exit 1 fi cmd="..." [...] If one of the command fails in such a batch, the libvirt code is going pick up the error code '1', tear down anything previously established and report an error back. The actual error message shown above is currently not reported back, but can be later on with some changes to the commands running external programs that need to read the script's stdout. One comment to patch 13: It currently #include's a .c file into a .c file only for the reason so I don't have to change too much code once I change code in the underlying patch. So this has to be changed. The patch series works without patch 13, but then only supports ebtables. The patches apply to the current tip. They pass 'make syntax-check' and have been frequently run in valgrind for memory leak checks. The order in which I apply the patches is as follows: add_recursive_locks.diff add_build_support.diff add_public_api.diff add_internal_api.diff impl_pub_api.diff def_wire_protocol_format.diff impl_rpc_client.c impl_srv_dispatch.diff add_virsh_support.diff add_xml_parsing.diff add_qemu_support.diff impl_driver.diff add_iptables_support.diff Looking forward to your feedback on the patches. Thanks and regards, Stefan and Gerhard -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

On Thu, Mar 11, 2010 at 08:06:04AM -0500, Stefan Berger wrote: > Hi! > > The following set of patches add network filtering (ACL) extensions to > libvirt and enable network traffic filtering for VMs using ebtables

> depending on the networking technology being used (tap, but not > macvtap), also iptables. Usage of either is optional and controlled > through filters that a VM is referencing. > > The ebtables-level filtering is based on the XML derived from the CIM > network slide 10 (filtering) from the DMTF website > (http://www.dmtf.org/standards/cim/cim_schema_v2230/CIM_Network.pdf). > The XML we derived from this was discussed on the list before. On the > ebtables level we currently handle filtering of IPv4 and ARP traffic. It is planned to cover IPv6 too, either at ebtables or ip6tables level ?

I've not looked at the actual code in detail yet, but the public API, virsh commands, XML etc all looks generally good to me. I'll try and get you a detailed code review friday/monday once I get through my current work items.

> One comment about the instantiation of the rules: Since the XML allows > to create nearly any possible combination of parameters to ebtables or > iptables commands, I haven't used the ebtables or iptables wrappers. > Instead, I am writing ebtables/iptables command into a buffer, add > command line options to each one of them as described in the rule's

> write the buffer into a file and run it as a script. For those

> that are not allowed to fail I am using the following format to run > them: > > cmd="ebtables <some options>" > r=`${cmd}` > if [ $? -ne 0 ]; then > echo "Failure in command ${cmd}." > exit 1 > fi > > cmd="..." > [...] > > If one of the command fails in such a batch, the libvirt code is going > pick up the error code '1', tear down anything previously established > and report an error back. The actual error message shown above is > currently not reported back, but can be later on with some changes to > the commands running external programs that need to read the script's > stdout. I understand why you can't use the ebtables/iptables APIs we currently have there, but I'm not much of a fan of using the shell script. Isn't it just as easy to directly call virRun() with each set of ARGV to be exec'd ?

Regards, Daniel -- |: Red Hat, Engineering, London -o-

|: http://libvirt.org -o- http://virt-manager.org -o-

|: http://autobuild.org -o-

|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B

Please respond to "Daniel P. Berrange" On Wed, Mar 17, 2010 at 10:53:37AM -0400, Stefan Berger wrote: > "Daniel P. Berrange" <berrange(a)redhat.com&gt; wrote on 03/17/2010

> AM: > > > > I hadn't thought about calling that function... I would want to call a

> function that can handle something like bash scripts, i.e., multiple > concatenated fragments as those shown above just to be more

Is it really more efficient ? If you need to run 20 ebtables commands, then using bash does 1 fork/exec for bash & bash then does another 20 fork/exec for ebtables. Alternatively just use virRun() for each ebtables command you just still have 20 fork/execs, without using bash. > If virRun() can handle that and $? for example would be treated there

> the return value (which I think is bash-dependent), I'd be happy to

> it as well. I'd think just call virRun once for each ebtables command - virRun gives you back the exit status of the command

Regards, Daniel -- |: Red Hat, Engineering, London -o-

|: http://libvirt.org -o- http://virt-manager.org -o-

|: http://autobuild.org -o-

|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B