"Daniel P. Berrange" <berrange@redhat.com> wrote on 03/25/2010 07:37:14 AM:
> > Please respond to "Daniel P. Berrange"
> > On Tue, Mar 23, 2010 at 10:54:06AM -0400, stefanb@us.ibm.com wrote: > > One comment about the instantiation of the rules: Since the XML allows > > to create nearly any possible combination of parameters to ebtables or > > iptables commands, I haven't used the ebtables or iptables wrappers. > > Instead, I am writing ebtables/iptables command into a buffer, add > > command line options to each one of them as described in the rule's XML, > > write the buffer into a file and run it as a script. For those commands > > that are not allowed to fail I am using the following format to run > > them: > > > > cmd="ebtables <some options>" > > r=`${cmd}` > > if [ $? -ne 0 ]; then > > echo "Failure in command ${cmd}." > > exit 1 > > fi > > > > cmd="..." > > [...] > > > > If one of the command fails in such a batch, the libvirt code is going > > pick up the error code '1', tear down anything previously established > > and report an error back. The actual error message shown above is > > currently not reported back, but can be later on with some changes to > > the commands running external programs that need to read the script's > > stdout. > > I've tried out this patch series, but not entirely sure whether it is > working correctly, since there are lots of errors in the libvirt debug > logs. >

The errors in the debugging log are due to attempts to cleanup user-defined tables
that do not exist. Though attempting to clean them up gets the system into a well
known start state. The errors could be avoided with a sequence of checking for
existence of the user-defined chain first and then try to flush and delete them,
but it's faster not to do so.
> I have a guest > > <interface type='network'> > <mac address='52:54:00:4d:c8:5f'/> > <source network='default'/> > <filterref filter='demofilter4'> > <parameter name='IP' value='9.59.241.151'/> > </filterref> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > </interface> > > And demofilter4 is setup as: > > <filter name='demofilter4' chain='root'> > <uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid> > <filterref filter='no-mac-spoofing'/> > <filterref filter='no-mac-broadcast'/> > <filterref filter='no-arp-spoofing'/> > <filterref filter='allow-dhcp'> > <parameter name='DHCPSERVER' value='10.0.0.1'/> > </filterref> > </filter> > > The filter it references here are the XML examples you sent out previously > > The guest starts without error, and I get a lot of things in the ebtables > 'nat' table, but nothing in the 'filter' table - is that expected ?

Yes, this is correct. The nat table handles the filtering correctly and with the
PREROUTING chain we can filter for traffic coming from the VM (that becomes
incoming to the host - prefix letter 'I') and in the POSTROUTING chain
traffic going to the VM (outgoing from the host - prefix letter 'O').
> > # ebtables -t nat -L > Bridge table: nat > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -i vnet0 -j I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -o vnet0 -j O-vnet0 > > Bridge chain: I-vnet0, entries: 2, policy: ACCEPT > -p IPv4 -j I-vnet0-ipv4 > -p ARP -j I-vnet0-arp > > Bridge chain: O-vnet0, entries: 2, policy: ACCEPT > -p IPv4 -j O-vnet0-ipv4 > -p ARP -j O-vnet0-arp > > Bridge chain: I-vnet0-ipv4, entries: 3, policy: ACCEPT > -s ! 52:54:0:4d:c8:5f -j DROP > -p IPv4 --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp -- > ip-sport 68 --ip-dport 67 -j ACCEPT > -d Broadcast -j DROP > > Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT > -p IPv4 --ip-src 10.0.0.1 --ip-proto udp --ip-sport 67 --ip-dport 68-j ACCEPT > > Bridge chain: I-vnet0-arp, entries: 5, policy: ACCEPT > -p ARP --arp-mac-src ! 52:54:0:4d:c8:5f -j DROP > -p ARP --arp-ip-src ! 9.59.241.151 -j DROP > -p ARP --arp-op Request -j ACCEPT > -p ARP --arp-op Reply -j ACCEPT > -j DROP > > Bridge chain: O-vnet0-arp, entries: 5, policy: ACCEPT > -p ARP --arp-op Reply --arp-mac-dst ! 52:54:0:4d:c8:5f -j DROP > -p ARP --arp-ip-dst ! 9.59.241.151 -j DROP > -p ARP --arp-op Request -j ACCEPT > -p ARP --arp-op Reply -j ACCEPT > -j DROP > > > > The libvirt logs show a bunch of shell erorrs - see the attachment to this > mail. Are these errors to be expected, or are they actual errors ?

They are due to attempted cleanups that try to cleanup user-defined tables that did not exist on your system,
however. As said, this gets the system into a well known start-out state so that those user-defined ebtables
tables can be established with rules. As seen above, every interface gets a 'tree' of user-defined tables.
Following the chain identifier in the above XML

<filter name='demofilter4' chain='root'> <uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid> [...]

which can be either left out entirely for meaning of 'root', or be 'root' explicitly or have the values 'arp' or 'ipv4'.

The patterns of the user-defined ebtables tables is due to 32 byte size restrictions (including trailing 0) as follows:

<one letter prefix>-<name of the interface>-<filter chain identifier>:

This then leads to

I-vnet0 and
I-vnet0-ipv4 etc.

as shown above.

However, the tricky part are updates of filters. You can, using virsh-define, update any filter while it is being used by running
VMs. For updates I decided not to compare the current filter tree against the new one, calculating 'diffs'
for the rules and remove rules that don't exist in the new one along with user defined tables that may not exist anymore
etc., which would be much more complicated. Instead, I am completely rebuilding the filter tree. So, a temporary filter
tree is established with user-defined names following the above pattern while the current filter tree is still active.
However, the prefixes are different. So the temporary filter tree would have user-defined tables named

J-vnet0 and
J-vnet0-ipv4 etc.

where all the rules are established. Then, while the 'old' filter tree is active, the new one is anchored, leading to
entries like this one:

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j I-vnet0
-i vnet0 -j J-vnet0

Yes, temporarily you would have two parallel filter trees for an interface, but the old one is still active. Then
comes the switch-over to the new tree where the old filter tree is un-anchored, leading to this situation:

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j J-vnet0
Now all possible chains starting with prefix I-vnet0-<XYZ> are flushed and deleted.
In the subsequent step I rename the filter tree starting with J-<vnet0>-<XYZ> to I-<vnet0>-<XYZ> leading to

Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j I-vnet0

which is where we want to be so that the next update can do the same procedure again.

I have made some changes to the core patch (part 11) that split up the application of new filter rules into
3 different (interface) functions, for application of new rules, switch over + tear down of old, or
tear down of new in case an error happened.
> > > Also, I'm wondering if it is possible to include 'libvirt-' in the custom > ebtables/iptables chains that we create ? I think that would be useful > for sysadmins who might be surprised at where all these rules are coming > from. From the code it looks like we have a fairly low size limit on the > possible chain names, but I think at least for the first two chains we > ought to be able to get 'libvirt-' prefix in there. Or perhaps instead > of adding the per-VIF chains straight to PREROUTING/POSTROUTING, have > an unconditional jump to a libvirt specific chain. > > So either have > > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -i vnet0 -j libvirt-I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -o vnet0 -j libvirt-O-vnet0 > > > > Or the slightly more complex > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -j libvirt-I > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -j libvirt-O > > Bridge chain: libvirt-I, entries: 1, policy: ACCEPT > -i vnet0 -j I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: libvirt-O, entries: 1, policy: ACCEPT > -o vnet0 -j O-vnet0

Both is possible with the first one resulting in being more efficient.

I'll try to make that change.

Regards,
Stefan

> > > Regards, > Daniel > -- > |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/:|> |:http://libvirt.org-o-http://virt-manager.org-o-http://deltacloud.org:|> |:http://autobuild.org -o- http://search.cpan.org/~danberr/:|> |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| > [attachment "filter.log" deleted by Stefan Berger/Watson/IBM]