[libvirt] Fwd: libvirtd and listen_addr
Hello, got no reaction on this, I try again :-) -------- Message original -------- Sujet: libvirtd and listen_addr Date : Sat, 01 Jan 2011 11:33:55 +0100 De : Daniel Huhardeaux <tech@tootai.net> Pour : libvir-list@redhat.com Hi, at first, best wishes to all for this new year. We have a request: would it be possible to start libvirtd listen_addr *after* the VM network is up? We have VPN through different libvirt+kvm servers and want to virsh connect to the private VM network address of each server instead of his VPN IP. The problem is that libvirt doesn't start with unknown IP address. If network would be already up, libvirt could listen on this address. Regards libvirtd (libvirt) 0.8.3 - Debian stable + backports -- Daniel
On 06/01/2011, at 8:26 PM, Daniel Huhardeaux wrote:
Hello,
got no reaction on this, I try again :-)
Hmmm, just thought of a workaround if it helps. :) You already know that when libvirtd starts, it automatically starts the virtual networks that it has been told to. But, if you then shut libvirtd down, they're left running. So... my thought of a dodgy workaround is this... after libvirtd starts... and the virtual networks have started... restart libvirtd. I *think* that would then let libvirt bind to the new ip addresses. Reckon it's worth trying? Regards and best wishes, Justin Clift
-------- Message original -------- Sujet: libvirtd and listen_addr Date : Sat, 01 Jan 2011 11:33:55 +0100 De : Daniel Huhardeaux <tech@tootai.net> Pour : libvir-list@redhat.com
Hi,
at first, best wishes to all for this new year.
We have a request: would it be possible to start libvirtd listen_addr *after* the VM network is up?
We have VPN through different libvirt+kvm servers and want to virsh connect to the private VM network address of each server instead of his VPN IP. The problem is that libvirt doesn't start with unknown IP address. If network would be already up, libvirt could listen on this address.
Regards
libvirtd (libvirt) 0.8.3 - Debian stable + backports -- Daniel
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Le 06/01/2011 17:24, Justin Clift a écrit :
On 06/01/2011, at 8:26 PM, Daniel Huhardeaux wrote:
Hello,
got no reaction on this, I try again :-)
Hmmm, just thought of a workaround if it helps. :)
You already know that when libvirtd starts, it automatically starts the virtual networks that it has been told to.
But, if you then shut libvirtd down, they're left running.
So... my thought of a dodgy workaround is this... after libvirtd starts... and the virtual networks have started... restart libvirtd.
I *think* that would then let libvirt bind to the new ip addresses.
Reckon it's worth trying?
That's how I discover the problem ;-) Everything was running fine until ... I had to reboot the server! VMs are started automatically but libvirtd will not start. In your doggy workaround ;-) libvirtd will NOT start because of the missing listen_addr. That's why I was thinking that libvirt start networks *before* it take care of listen_addr or perhaps listen_addr shouldn't cancel the start of libvirt (delay it) until network is up and then check if listen_addr is in error or not. -- Daniel
On Thu, Jan 06, 2011 at 06:00:24PM +0100, Daniel Huhardeaux wrote:
Le 06/01/2011 17:24, Justin Clift a écrit :
On 06/01/2011, at 8:26 PM, Daniel Huhardeaux wrote:
Hello,
got no reaction on this, I try again :-)
Hmmm, just thought of a workaround if it helps. :)
You already know that when libvirtd starts, it automatically starts the virtual networks that it has been told to.
But, if you then shut libvirtd down, they're left running.
So... my thought of a dodgy workaround is this... after libvirtd starts... and the virtual networks have started... restart libvirtd.
I *think* that would then let libvirt bind to the new ip addresses.
Reckon it's worth trying?
That's how I discover the problem ;-)
Everything was running fine until ... I had to reboot the server! VMs are started automatically but libvirtd will not start.
In your doggy workaround ;-) libvirtd will NOT start because of the missing listen_addr.
That's why I was thinking that libvirt start networks *before* it take care of listen_addr or perhaps listen_addr shouldn't cancel the start of libvirt (delay it) until network is up and then check if listen_addr is in error or not.
In general this approach isn't going to work. In the very near future, libvirt itself won't be listen()ing on sockets itself. Instead SystemD will do the listening and pass the pre-opened server socket to libvirt at startup time. In addition, we delibrately listen on all sockets very early in startup, so that libvirtd can go into the background quickly. This lets system bootup continue without delays, while the potentially very slow auto-startup of VMs, networks, etc is done in the background by libvirtd. The key is that by listen() on the sockets immediately, but not accept()ing connections until autostart is complete, clients like virsh can immediately connect to libvirt and block on autostart. We can't reverse the startup order as you suggest without breaking this key feature. An alternative approach without the chicken & egg problem is to listen on all addresses, and use the firewall to restrict where you can connect from. Regards, Daniel
On Thu, Jan 06, 2011 at 05:13:41PM +0000, Daniel P. Berrange wrote:
On Thu, Jan 06, 2011 at 06:00:24PM +0100, Daniel Huhardeaux wrote:
Le 06/01/2011 17:24, Justin Clift a écrit :
On 06/01/2011, at 8:26 PM, Daniel Huhardeaux wrote:
Hello,
got no reaction on this, I try again :-)
Hmmm, just thought of a workaround if it helps. :)
You already know that when libvirtd starts, it automatically starts the virtual networks that it has been told to.
But, if you then shut libvirtd down, they're left running.
So... my thought of a dodgy workaround is this... after libvirtd starts... and the virtual networks have started... restart libvirtd.
I *think* that would then let libvirt bind to the new ip addresses.
Reckon it's worth trying?
That's how I discover the problem ;-)
Everything was running fine until ... I had to reboot the server! VMs are started automatically but libvirtd will not start.
In your doggy workaround ;-) libvirtd will NOT start because of the missing listen_addr.
That's why I was thinking that libvirt start networks *before* it take care of listen_addr or perhaps listen_addr shouldn't cancel the start of libvirt (delay it) until network is up and then check if listen_addr is in error or not.
In general this approach isn't going to work. In the very near future, libvirt itself won't be listen()ing on sockets itself. Instead SystemD will do the listening and pass the pre-opened server socket to libvirt at startup time.
In addition, we delibrately listen on all sockets very early in startup, so that libvirtd can go into the background quickly. This lets system bootup continue without delays, while the potentially very slow auto-startup of VMs, networks, etc is done in the background by libvirtd. The key is that by listen() on the sockets immediately, but not accept()ing connections until autostart is complete, clients like virsh can immediately connect to libvirt and block on autostart. We can't reverse the startup order as you suggest without breaking this key feature.
An alternative approach without the chicken & egg problem is to listen on all addresses, and use the firewall to restrict where you can connect from.
Or make libvirtd listen on 127.0.0.1, and then use iptables to setup a local port forwarding from 10.0.0.1 to 127.0.0.1 on the libvirt port(s). Daniel
Le 06/01/2011 18:58, Daniel P. Berrange a écrit : [...]
Or make libvirtd listen on 127.0.0.1, and then use iptables to setup a local port forwarding from 10.0.0.1 to 127.0.0.1 on the libvirt port(s).
For the archives, iptables is not the solution as kernel doesn't accept to send out of 127.0.0.0 packets from 127.0.0.1 IP. Solution is to use TCP redirection softwares like redir, stone or rinetd (non exhaustive list). It works perfect. Thanks for having pointed us to a solution. -- Daniel
On 07/01/2011, at 4:00 AM, Daniel Huhardeaux wrote:
Le 06/01/2011 17:24, Justin Clift a écrit :
On 06/01/2011, at 8:26 PM, Daniel Huhardeaux wrote:
Hello,
got no reaction on this, I try again :-)
Hmmm, just thought of a workaround if it helps. :)
You already know that when libvirtd starts, it automatically starts the virtual networks that it has been told to.
But, if you then shut libvirtd down, they're left running.
So... my thought of a dodgy workaround is this... after libvirtd starts... and the virtual networks have started... restart libvirtd.
I *think* that would then let libvirt bind to the new ip addresses.
Reckon it's worth trying?
That's how I discover the problem ;-)
Everything was running fine until ... I had to reboot the server! VMs are started automatically but libvirtd will not start.
In your doggy workaround ;-) libvirtd will NOT start because of the missing listen_addr.
That's why I was thinking that libvirt start networks *before* it take care of listen_addr or perhaps listen_addr shouldn't cancel the start of libvirt (delay it) until network is up and then check if listen_addr is in error or not.
Hmmm, in your libvirtd.conf file, what does the line say where you have "listen_addr"? I'm thinking it should be something like: listen_addr = "0.0.0.0" Which I *think* tells it to bind to everything it can. ?
Le 06/01/2011 18:16, Justin Clift a écrit : [...]
Hmmm, in your libvirtd.conf file, what does the line say where you have "listen_addr"?
I'm thinking it should be something like:
listen_addr = "0.0.0.0"
Which I *think* tells it to bind to everything it can.
?
Yes but that's not what I want. Let's say we have 3 servers each of them with VMs and all connected through OpenVPN server 1: VMs Net 10.0.1.1 (IP of server virbr0) VMs from .1.11 to ... server 2: VMs Net 10.0.2.1 (IP of server virbr0) VMs from .2.11 to ... server 3: VMs Net 10.0.3.1 (IP of server virbr0) VMs from .3.11 to ... Having OpenVPN running, each VM -or other host running OpenVPN- can reach each other. So what I want, for security reason, is that listen_addr of each server is *only* 10.0.[1|2|3].1 which is transparent and independant of other network settings (public addresses, localnet, other VPN, ...). Hope I clarify my needs :-) -- Daniel
On 07/01/2011, at 4:39 AM, Daniel Huhardeaux wrote:
Le 06/01/2011 18:16, Justin Clift a écrit :
[...]
Hmmm, in your libvirtd.conf file, what does the line say where you have "listen_addr"?
I'm thinking it should be something like:
listen_addr = "0.0.0.0"
Which I *think* tells it to bind to everything it can.
?
Yes but that's not what I want. Let's say we have 3 servers each of them with VMs and all connected through OpenVPN
server 1: VMs Net 10.0.1.1 (IP of server virbr0) VMs from .1.11 to ... server 2: VMs Net 10.0.2.1 (IP of server virbr0) VMs from .2.11 to ... server 3: VMs Net 10.0.3.1 (IP of server virbr0) VMs from .3.11 to ...
Having OpenVPN running, each VM -or other host running OpenVPN- can reach each other. So what I want, for security reason, is that listen_addr of each server is *only* 10.0.[1|2|3].1 which is transparent and independant of other network settings (public addresses, localnet, other VPN, ...).
Hope I clarify my needs :-)
Heh, yeah. I think Daniel Berrange's approach of using firewall rules to control the access is probably the most rugged... My only other thought, and probably pretty fragile unless you put good scripting around it, is to still do the restarting thing, but change the listen_addr entry before the restart. Though, I think it'll be an interesting (bad) situation if you want to change or restart one of the virtual networks when libvirtd is attached to it. Can see problems with that. ;)
Le 06/01/2011 18:48, Justin Clift a écrit :
On 07/01/2011, at 4:39 AM, Daniel Huhardeaux wrote:
Le 06/01/2011 18:16, Justin Clift a écrit :
[...]
Hmmm, in your libvirtd.conf file, what does the line say where you have "listen_addr"?
I'm thinking it should be something like:
listen_addr = "0.0.0.0"
Which I *think* tells it to bind to everything it can.
?
Yes but that's not what I want. Let's say we have 3 servers each of them with VMs and all connected through OpenVPN
server 1: VMs Net 10.0.1.1 (IP of server virbr0) VMs from .1.11 to ... server 2: VMs Net 10.0.2.1 (IP of server virbr0) VMs from .2.11 to ... server 3: VMs Net 10.0.3.1 (IP of server virbr0) VMs from .3.11 to ...
Having OpenVPN running, each VM -or other host running OpenVPN- can reach each other. So what I want, for security reason, is that listen_addr of each server is *only* 10.0.[1|2|3].1 which is transparent and independant of other network settings (public addresses, localnet, other VPN, ...).
Hope I clarify my needs :-)
Heh, yeah. I think Daniel Berrange's approach of using firewall rules to control the access is probably the most rugged...
What I've done yet is to listen the VPN IP. If VPN fail to start for a reason or another I will face the same problem. Another thing is that the VPN IP is not always reachable from other servers. Will follow the advice to use iptables. Thanks to all for your time. -- Daniel
participants (3)
-
Daniel Huhardeaux -
Daniel P. Berrange -
Justin Clift