[libvirt] mass create vm errors
Hi again =). I have another problem then testing libvirt with massive vm start: 2015-07-18 08:25:21.687+0000: 36893: error : qemuMonitorIO:750 : internal error: early end of file from monitor: possible problem: 2015-07-18T08:25:21.586487Z qemu-system-x86_64: -vnc [::]:4,password: Failed to start VNC server: Failed to bind socket: Address already in use As i understand, when libvirt try to detect free port for vnc it fails, because another process already binds to it. How can i avoid this errors? Does it fixed in never libvirt releases? libvirt 1.216 qemu 2.4.0-rc0 -- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru
On 18.07.2015 10:29, Vasiliy Tolstov wrote:
Hi again =). I have another problem then testing libvirt with massive vm start: 2015-07-18 08:25:21.687+0000: 36893: error : qemuMonitorIO:750 : internal error: early end of file from monitor: possible problem: 2015-07-18T08:25:21.586487Z qemu-system-x86_64: -vnc [::]:4,password: Failed to start VNC server: Failed to bind socket: Address already in use
As i understand, when libvirt try to detect free port for vnc it fails, because another process already binds to it. How can i avoid this errors? Does it fixed in never libvirt releases?
libvirt 1.216 qemu 2.4.0-rc0
There has not been much movement in that particular area since 1.2.16. There's inherent race though: by the time that libvirt detects that a certain port is free and qemu binds to it. During this window another process may just allocate the port which will result in the error message you're seeing. One of the solutions may be to use static ports, or make sure that other programs don't interfere with the range that's reserved for graphical sessions (the range can be configured too in qemu.conf). Michal
2015-08-03 10:01 GMT+03:00 Michal Privoznik <mprivozn@redhat.com>:
There has not been much movement in that particular area since 1.2.16. There's inherent race though: by the time that libvirt detects that a certain port is free and qemu binds to it. During this window another process may just allocate the port which will result in the error message you're seeing.
This is very bad thing in case of mass start , for example when node dies i have 100-150 vm started at the same time, and sometimes they running on single host. Does it possible to create lock for this port when port passed to qemu, so another process when detect new port don't use it?
One of the solutions may be to use static ports, or make sure that other programs don't interfere with the range that's reserved for graphical sessions (the range can be configured too in qemu.conf). But if i specify port ranges in qemu.conf, how this solve my issue? as i understand libvirt again randomly allocated port for vm...
-- Vasiliy Tolstov, e-mail: v.tolstov@selfip.ru
On 04.08.2015 14:31, Vasiliy Tolstov wrote:
2015-08-03 10:01 GMT+03:00 Michal Privoznik <mprivozn@redhat.com>:
There has not been much movement in that particular area since 1.2.16. There's inherent race though: by the time that libvirt detects that a certain port is free and qemu binds to it. During this window another process may just allocate the port which will result in the error message you're seeing.
This is very bad thing in case of mass start , for example when node dies i have 100-150 vm started at the same time, and sometimes they running on single host. Does it possible to create lock for this port when port passed to qemu, so another process when detect new port don't use it?
No. It would require change in other apps so that they use the lock file.
One of the solutions may be to use static ports, or make sure that other programs don't interfere with the range that's reserved for graphical sessions (the range can be configured too in qemu.conf). But if i specify port ranges in qemu.conf, how this solve my issue? as i understand libvirt again randomly allocated port for vm...
No, libvirt does not randomly allocate port. In fact, libvirt does not allocate port at all. It merely checks on domain startup that the graphics port is free. Or it searches for next free port within the range set in qemu.conf (in case of autoport). Then, the port number is put on qemu command line and qemu binds to it. However during this window other apps may step in and allocate the port. But if you choose a different range, unused one, you may be more successful. BTW: to view which app has allocated the 5904 port you can use: 'netstat -ltnp | grep 5904'. Of course, the internal bitmap of free and taken ports is protected by a mutex, so libvirt should not assign the same port to two domains. Michal
participants (2)
-
Michal Privoznik -
Vasiliy Tolstov