[libvirt-users] It takes long time to start kvm virtual machine with nwfilter in docker container.
1. It takes minutes to start the virtual machine when I add "filterref" to libvirt.xml and run command "virsh start vm1". It also takes minutes to destroy the virtual machine. <interface type="bridge"> <mac address="fa:16:3e:fa:f7:94"/> <target dev="tap69e948b0-bf"/> <source bridge="br02"/> <model type="virtio"/> <filterref filter="no-arp-spoofing"> <parameter name="IP" value="192.168.2.2"/> <parameter name="MAC" value="fa:16:3e:fa:f7:94"/> </filterref> </interface> 2. I found some logs in /var/log/libvirt/libvirtd.log 2019-11-04 03:46:18.495+0000: 15257: info : virFirewallApplyRule:815 : Applying rule '/usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i tap69e948b0-bf -j libvirt-J-tap69e948b0-bf' 2019-11-04 03:46:18.495+0000: 15257: debug : virCommandRunAsync:2499 : About to run /usr/sbin/ebtables --concurrent -t nat -D PREROUTING -i tap69e948b0-bf -j libvirt-J-tap69e948b0-bf 2019-11-04 03:46:18.496+0000: 15257: debug : virFileClose:111 : Closed fd 24 2019-11-04 03:46:18.496+0000: 15257: debug : virFileClose:111 : Closed fd 26 2019-11-04 03:46:18.496+0000: 15257: debug : virFileClose:111 : Closed fd 29 2019-11-04 03:46:18.496+0000: 15257: debug : virCommandRunAsync:2502 : Command result 0, with PID 16291 2019-11-04 03:46:19.242+0000: 15257: debug : virCommandRun:2350 : Result invalid value 255, stdout: '' stderr: '2019-11-04 03:46:19.239+0000: 16291: debug : virFileClose:111 : Closed fd 26 2019-11-04 03:46:19.239+0000: 16291: debug : virFileClose:111 : Closed fd 29 2019-11-04 03:46:19.239+0000: 16291: debug : virFileClose:111 : Closed fd 24 Illegal target name 'libvirt-J-tap69e948b0-bf'. ' 2019-11-04 03:46:19.242+0000: 15257: debug : virFirewallApplyRuleDirect:704 : Ignoring error running command 2019-11-04 03:46:19.242+0000: 15257: debug : virFileClose:111 : Closed fd 25 2019-11-04 03:46:19.243+0000: 15257: debug : virFileClose:111 : Closed fd 28 2019-11-04 03:46:19.243+0000: 15257: info : virFirewallApplyRule:815 : Applying rule '/usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o tap69e948b0-bf -j libvirt-P-tap69e948b0-bf' 2019-11-04 03:46:19.243+0000: 15257: debug : virCommandRunAsync:2499 : About to run /usr/sbin/ebtables --concurrent -t nat -D POSTROUTING -o tap69e948b0-bf -j libvirt-P-tap69e948b0-bf 2019-11-04 03:46:19.243+0000: 15257: debug : virFileClose:111 : Closed fd 24 2019-11-04 03:46:19.243+0000: 15257: debug : virFileClose:111 : Closed fd 26 2019-11-04 03:46:19.244+0000: 15257: debug : virFileClose:111 : Closed fd 29 2019-11-04 03:46:19.244+0000: 15257: debug : virCommandRunAsync:2502 : Command result 0, with PID 16292 2019-11-04 03:46:19.990+0000: 15257: debug : virCommandRun:2350 : Result invalid value 255, stdout: '' stderr: '2019-11-04 03:46:19.986+0000: 16292: debug : virFileClose:111 : Closed fd 26 2019-11-04 03:46:19.986+0000: 16292: debug : virFileClose:111 : Closed fd 29 2019-11-04 03:46:19.986+0000: 16292: debug : virFileClose:111 : Closed fd 24 Illegal target name 'libvirt-P-tap69e948b0-bf'. ' 2019-11-04 03:46:19.990+0000: 15257: debug : virFirewallApplyRuleDirect:704 : Ignoring error running command 2019-11-04 03:46:19.990+0000: 15257: debug : virFileClose:111 : Closed fd 25 2019-11-04 03:46:19.990+0000: 15257: debug : virFileClose:111 : Closed fd 28 3. It works fine on hosts. 4. It works fine in other host's libvirt container. How can I solve this problem? Best regards, John
On Mon, Nov 04, 2019 at 04:55:07PM +0800, John Y. wrote:
1. It takes minutes to start the virtual machine when I add "filterref" to libvirt.xml and run command "virsh start vm1". It also takes minutes to destroy the virtual machine.
You don't mention which version of libvirt you have... My guess is that your docker container has set an enourmous ulimit for max files. On a normal host it is 1024, but on docker I've seen it default to 1 million. This impacts libvirt when it spawns processes, which is common with nwfilter in particular, because we must close all open file handles. In v5.6.0 we added code to let libvirt use /proc/$PID/fd to close file handles, which is massively faster when ulimits are high. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Thank you for your reply. According to what you said, I set open files to 1024 and solved the problem. Thanks! ------- Sorry for not mentioning the version information. The version of libvirt is 4.5.0. Regards, John Daniel P. Berrangé <berrange@redhat.com> 于2019年11月4日周一 下午6:04写道：
On Mon, Nov 04, 2019 at 04:55:07PM +0800, John Y. wrote:
1. It takes minutes to start the virtual machine when I add "filterref" to libvirt.xml and run command "virsh start vm1". It also takes minutes to destroy the virtual machine.
You don't mention which version of libvirt you have...
My guess is that your docker container has set an enourmous ulimit for max files. On a normal host it is 1024, but on docker I've seen it default to 1 million.
This impacts libvirt when it spawns processes, which is common with nwfilter in particular, because we must close all open file handles. In v5.6.0 we added code to let libvirt use /proc/$PID/fd to close file handles, which is massively faster when ulimits are high.
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (2)
-
Daniel P. Berrangé -
John Y.