[libvirt-users] Best practice for custom iptables rules
Hi, I'm using libvirt to manage some VMs on a CentOS host, and I need some custom iptables rules to always be in place for some communications to happen, e.g. between the VMs and the outside world in both directions. Some of these rules need to be at the top of the iptables chain, otherwise the default rules added by libvirt would block the communications I need. So I cannot just add the rules in /etc/sysconfig/iptables, because libvirt adds its own rules _before_ the rules contained in this config file. I was looking at filters, but maybe not every rule can be made into a filter? Specifically, I need a rule for the POSTROUTING chain in the "nat" table. Can it be added through filters? Also, regarding the "iptables restart problem" described in the last paragraph at <http://libvirt.org/firewall.html>, is there really no acceptable way to make libvirt add its rules back automatically upon iptables/network restart? Thanks for any info. Marco -- 01
On 01/08/2014 01:43 PM, ZeroUno wrote:
Hi, I'm using libvirt to manage some VMs on a CentOS host, and I need some custom iptables rules to always be in place for some communications to happen, e.g. between the VMs and the outside world in both directions.
Some of these rules need to be at the top of the iptables chain, otherwise the default rules added by libvirt would block the communications I need. So I cannot just add the rules in /etc/sysconfig/iptables, because libvirt adds its own rules _before_ the rules contained in this config file.
I was looking at filters, but maybe not every rule can be made into a filter? Specifically, I need a rule for the POSTROUTING chain in the "nat" table. Can it be added through filters?
Correct. nwfilter can't add rules to the nat table.
Also, regarding the "iptables restart problem" described in the last paragraph at <http://libvirt.org/firewall.html>, is there really no acceptable way to make libvirt add its rules back automatically upon iptables/network restart?
Take a look at this, it may help you: http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections (Recently libvirt gained the ability for an application to register functions that will be called when a network is defined/undefined/started/stopped, but using that would require an application to be running which registered the necessary callback functions; not nearly as simple as stuffing a shell script into /etc/libvirt/hooks (should we do that? Or are the shell script hooks considered passe with the advent of event callbacks?))
Il 08/01/14 16:17, Laine Stump ha scritto:
On 01/08/2014 01:43 PM, ZeroUno wrote:
Also, regarding the "iptables restart problem" described in the last paragraph at <http://libvirt.org/firewall.html>, is there really no acceptable way to make libvirt add its rules back automatically upon iptables/network restart?
Take a look at this, it may help you:
http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
Uhm, apart from the fact that the page clearly states this is a "hack", so it's far from being a best practice (although surely easy and interesting!), AFAICT this might help with adding rules to the NAT table, which was the first part of my question, but does not help with the network restart issue because hook scripts are only called upon libvirt events: libvirt daemon start/stop, guest start/stop... Did I understand correctly?
(Recently libvirt gained the ability for an application to register functions that will be called when a network is defined/undefined/started/stopped, but using that would require an application to be running which registered the necessary callback functions; not nearly as simple as stuffing a shell script into
Indeed, looks like this would be overkill for my needs. Thank you! -- 01
Il 09/01/14 11:38, ZeroUno ha scritto:
Il 08/01/14 16:17, Laine Stump ha scritto:
http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
interesting!), AFAICT this might help with adding rules to the NAT table, which was the first part of my question, but does not help with
...also, it appears that the hook script /etc/libvirt/hooks/daemon to be called when the libvirt daemon is started is actually called _before_ libvirt adds its own iptables rules, because I am not able to insert my custom rule at the top of the chain. Maybe I might use the qemu script which is called each time a guest is started/stopped, by inserting some checks to prevent duplicates, but it becomes even more "hackish"... :) -- 01
On 01/09/2014 02:07 PM, ZeroUno wrote:
Il 09/01/14 11:38, ZeroUno ha scritto:
Il 08/01/14 16:17, Laine Stump ha scritto:
http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
interesting!), AFAICT this might help with adding rules to the NAT table, which was the first part of my question, but does not help with
...also, it appears that the hook script /etc/libvirt/hooks/daemon to be called when the libvirt daemon is started is actually called _before_ libvirt adds its own iptables rules, because I am not able to insert my custom rule at the top of the chain.
Maybe I might use the qemu script which is called each time a guest is started/stopped, by inserting some checks to prevent duplicates, but it becomes even more "hackish"... :)
Interesting point, and one which reinforces the idea that a network event hook script might be a nice thing to have (although adding in callout to an externally-created shell script always has security implications, especially for a process running as root).
...also, it appears that the hook script /etc/libvirt/hooks/daemon to be called when the libvirt daemon is started is actually called _before_ libvirt adds its own iptables rules, because I am not able to insert my custom rule at the top of the chain.
how about this daemon hook script? #!/bin/bash # insert_rule() { sleep 2 iptables -t nat -D CUSTOM_RULE iptables -t nat -I CUSTOM_RULE } case $2 in start|reload) insert_rule >/dev/null 2>&1 & ;; *) : ;; esac
Il 13/01/14 04:06, Gao Yongwei ha scritto:
how about this daemon hook script?
#!/bin/bash # insert_rule() { sleep 2 iptables -t nat -D CUSTOM_RULE iptables -t nat -I CUSTOM_RULE } [...]
Thanks, I already tried inserting a delay with "sleep" but it didn't change anything, as the hook script is not processed in parallel with other operations: libvirt waits until the hook script has been completed, before proceeding with the creation of its own iptables rules. -- 01
Thanks, I already tried inserting a delay with "sleep" but it didn't change anything, as the hook script is not processed in parallel with other operations: libvirt waits until the hook script has been completed, before proceeding with the creation of its own iptables rules.
plz take a closer look at my script, and have a real try with it.
Il 13/01/14 12:06, Gao Yongwei ha scritto:
plz take a closer look at my script, and have a real try with it.
Plz next time share with me that I'm missing the "&" ;). So _that_ was the suggestion, and it actually works, thanks! Now, generally speaking, we just need a way to do it which will not be called "a hack" ;), and a similar hook for network events. -- 01
On 01/09/2014 12:38 PM, ZeroUno wrote:
Il 08/01/14 16:17, Laine Stump ha scritto:
On 01/08/2014 01:43 PM, ZeroUno wrote:
Also, regarding the "iptables restart problem" described in the last paragraph at <http://libvirt.org/firewall.html>, is there really no acceptable way to make libvirt add its rules back automatically upon iptables/network restart?
Take a look at this, it may help you:
http://wiki.libvirt.org/page/Networking#Forwarding_Incoming_Connections
Uhm, apart from the fact that the page clearly states this is a "hack", so it's far from being a best practice (although surely easy and interesting!),
you asked for "best", not "ideal" :-) Aside from eliminating all use of libvirt-created networks and instead manually setting up the bridge, iptables rules, and dnsmasq yourself in the host system config, that really is currently the "best" way of achieving what you want.
AFAICT this might help with adding rules to the NAT table, which was the first part of my question, but does not help with the network restart issue because hook scripts are only called upon libvirt events: libvirt daemon start/stop, guest start/stop...
Did I understand correctly?
Correct. The problem the paragraph that you referenced is referring to is caused by the lack of a central authority/controller for managing iptables rule addition/removal requests from multiple applications/services. and that's not a problem that libvirt is able to solve by itself. But that same paragraph also tells you how to have the iptables service signal libvirt to reload its iptables rules. Alternately, the better solution to this problem is firewalld - if your system uses firewalld, then libvirt is listening for firewalld events on dbus, and will automatically reload its rules anytime firewalld restarts.
(Recently libvirt gained the ability for an application to register functions that will be called when a network is defined/undefined/started/stopped, but using that would require an application to be running which registered the necessary callback functions; not nearly as simple as stuffing a shell script into
Indeed, looks like this would be overkill for my needs.
Thank you!
Il 09/01/14 13:40, Laine Stump ha scritto:
you asked for "best", not "ideal" :-) Aside from eliminating all use of
;)
solve by itself. But that same paragraph also tells you how to have the iptables service signal libvirt to reload its iptables rules.
Sorry, what do you mean? I'm not able to find such an indication in that page... -- 01
On 01/10/2014 06:02 PM, ZeroUno wrote:
Il 09/01/14 13:40, Laine Stump ha scritto:
you asked for "best", not "ideal" :-) Aside from eliminating all use of
;)
solve by itself. But that same paragraph also tells you how to have the iptables service signal libvirt to reload its iptables rules.
Sorry, what do you mean? I'm not able to find such an indication in that page...
Hmm, I guess you're right - the final paragraph of http://libvirt.org/firewall.html doesn't tell you *how* to do that, it just tells you that you need to. Depending on your Linux distro and version, you could do this with a local modification to the script that starts/stops the iptables service - e.g. /usr/libexec/iptables/iptables.init when systemd is in use, or /etc/init.d/iptables for for initscripts. Of course this is also a hack, as it's liable to be overwritten when the iptables package is updated :-(
Hi libvirt: Id like some feedback : I've written up how I create and add new disks to my running VMs (for development and testing so not worried about high performance io or anything like that..): http://jayunit100.blogspot.com/2014/01/adding-new-virtual-disks-to-running.h... I don't like this solution though because it directly references qemu. Is there a more "virt" Style abstraction I can use to create new virtual disk images for adding storage on the fly to my VMs? Also general comments would be welcome... I'm new to dev oriented virtualization with virt and want to use it with he right idioms if possible.
participants (4)
-
Gao Yongwei -
Jay Vyas -
Laine Stump -
ZeroUno