Hi, we've experienced some issues with starting lots of KVM based VM's with libvirt. Since I couldn't find any clues on the libvirt mailing list, I'm posting the way I fixed the issues. When starting a VM, /var/log/messages was spammed with the following message: xt_physdev: using --physdev-out in the OUTPUT, FORWARD and POSTROUTING chains for non-bridged traffic is not supported anymore. With each extra VM I start, the messages get amplified exponentially. This results in longer starting times every new VM, relative the the previously started VM. When I ran a test with starting 100 equal VM's, the first VM started in about 2 seconds, the 100th VM took 48 seconds to start. I'm running a vanilla 3.7.1 kernel, but I have the same issue on VM hosts with kernel 3.2.28 or 3.2.0, running libvirt 0.9.12 and 0.9.8 respectively. Looking into the warning, it seemed that iptables need an extra argument, --physdev-is-bridged, in commands like: iptables -A libvirt-out -m physdev --physdev-is-bridged --physdev-out vnet99 -g FP-vnet99 I patched the libvirt source (version 1.0.0) to test whether this works or not: --- src/nwfilter/nwfilter_ebiptables_driver.c.orig 2013-01-16 10:51:43.000000000 +0100 +++ src/nwfilter/nwfilter_ebiptables_driver.c 2013-01-16 10:52:07.000000000 +0100 @@ -166,7 +166,7 @@ snprintf(buf, sizeof(buf), "%c%c-%s", prefix[0], prefix[1], ifname) #define PHYSDEV_IN "--physdev-in" -#define PHYSDEV_OUT "--physdev-out" +#define PHYSDEV_OUT "--physdev-is-bridged --physdev-out" static const char *m_state_out_str = "-m state --state NEW,ESTABLISHED"; static const char *m_state_in_str = "-m state --state ESTABLISHED"; The warnings in /var/log/messages are gone and running the test again proved the 100th VM started in 3.8 seconds. It suprises me I'm the first to mention this problem on the libvirt mailing list and I wondering if I'm doing something wrong. Until then, this fix helps me a lot! Reinier Schoof -- TransIP BV | https://www.transip.nl/