We're hitting an assert whenever we try to create an HVM instance under Xen via libvirtd. System is running on Gentoo, package information as follows: app-emulation/xen-4.5.0 USE="api debug flask hvm pam pygrub python qemu screen" app-emulation/xen-tools-4.5.0 USE="api debug flask hvm pam pygrub python qemu screen" app-emulation/libvirt-1.2.11-r2:0/1.2.11 USE="caps libvirtd lvm macvtap nls qemu udev vepa virtualbox xen" The following commands are run in parallel: vmmachine ~ # libvirtd --listen 2015-01-22 16:33:13.596+0000: 2620: info : libvirt version: 1.2.11 2015-01-22 16:33:13.596+0000: 2620: error : udevGetDMIData:1607 : Failed to get udev device for syspath '/sys/devices/virtual/dmi/id' or '/sys/class/dmi/id' libvirtd: libxl_fork.c:350: sigchld_installhandler_core: Assertion `((void)"application must negotiate with libxl about SIGCHLD", !(sigchld_saved_action.sa_flags & 4) && (sigchld_saved_action.__sigaction_handler.sa_handler == ((__sighandler_t) 0) || sigchld_saved_action.__sigaction_handler.sa_handler == ((__sighandler_t) 1)))' failed. Aborted

After some debugging we found what was causing of the assert. In our configuration we have two kernels to boot, one is a pv-linux for Xen dom0 and another just a normal linux kernel. We have libvirt built with both Xen and vbox support. When running with Xen, the libxl driver is used so it ends calling libxenlight who doesn't want any SIGCHLD handler set. Normally this is the case but, since we have vbox support in libvirt the vbox driver loads some of the vbox libs and one of them sets a SIGCHLD handler. When libxenlight checks if there is any handler for SIGCHLD it finds that one and fails. Here is the backtrace from when vbox is setting the handler: (gdb) bt #0 0x00007ffff71d8250 in sigaction () from /lib64/libpthread.so.0 #1 0x00007fffedbf3716 in ?? () from /usr/lib64/virtualbox/VBoxRT.so #2 0x00007fffedbf3960 in ?? () from /usr/lib64/virtualbox/VBoxRT.so #3 0x00007fffedeea485 in VBoxGetCAPIFunctions () from /usr/lib64/virtualbox/VBoxXPCOMC.so #4 0x00007fffee37d7ca in ?? () from /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_network.so #5 0x00007fffee37d995 in VBoxCGlueInit () from /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_network.so #6 0x00007fffee321e7b in vboxNetworkRegister () from /usr/lib64/libvirt/connection-driver/libvirt_driver_vbox_network.so #7 0x00007ffff755bbe8 in virDriverLoadModule () from /usr/lib64/libvirt.so.0 #8 0x0000555555568714 in ?? () #9 0x000055555556ab86 in ?? () #10 0x00007ffff6e54aa5 in __libc_start_main () from /lib64/libc.so.6 #11 0x0000555555567f59 in ?? () After building libvirt without vbox support the assert disappeared.

On Fri, Jan 23, 2015 at 5:14 AM, Michal Privoznik <mprivozn(a)redhat.com <mailto:mprivozn@redhat.com>> wrote: On 22.01.2015 17:49, CloudPatch Staff wrote: > We're hitting an assert whenever we try to create an HVM instance under > Xen via libvirtd. > > System is running on Gentoo, package information as follows: > > app-emulation/xen-4.5.0 USE="api debug flask hvm pam pygrub python qemu > screen" > app-emulation/xen-tools-4.5.0 USE="api debug flask hvm pam pygrub python > qemu screen" > app-emulation/libvirt-1.2.11-r2:0/1.2.11 USE="caps libvirtd lvm macvtap > nls qemu udev vepa virtualbox xen" > > The following commands are run in parallel: > > vmmachine ~ # libvirtd --listen > 2015-01-22 16:33:13.596+0000: 2620: info : libvirt version: 1.2.11 > 2015-01-22 16:33:13.596+0000: 2620: error : udevGetDMIData:1607 : Failed > to get udev device for syspath '/sys/devices/virtual/dmi/id' or > '/sys/class/dmi/id' > libvirtd: libxl_fork.c:350: sigchld_installhandler_core: Assertion > `((void)"application must negotiate with libxl about SIGCHLD", > !(sigchld_saved_action.sa_flags & 4) && > (sigchld_saved_action.__sigaction_handler.sa_handler == > ((__sighandler_t) 0) || > sigchld_saved_action.__sigaction_handler.sa_handler == ((__sighandler_t) > 1)))' failed. > Aborted Interesting. Can you attach a debugger so we can see stacktrace? Michal

Hi all, On 26.01.2015 10:52, Michal Privoznik wrote: > [CC'ing vbox-dev list] > > On 23.01.2015 21:42, CloudPatch Staff wrote: >> After some debugging we found what was causing of the assert. In our >> configuration we have >> two kernels to boot, one is a pv-linux for Xen dom0 and another just a >> normal linux kernel. >> We have libvirt built with both Xen and vbox support. When running with >> Xen, the libxl >> driver is used so it ends calling libxenlight who doesn't want any >> SIGCHLD handler set. >> Normally this is the case but, since we have vbox support in libvirt the >> vbox driver loads >> some of the vbox libs and one of them sets a SIGCHLD handler. When >> libxenlight checks >> if there is any handler for SIGCHLD it finds that one and fails. >> >> Here is the backtrace from when vbox is setting the handler: >> >> (gdb) bt >> #0 0x00007ffff71d8250 in sigaction () from /lib64/libpthread.so.0 >> #1 0x00007fffedbf3716 in ?? () from /usr/lib64/virtualbox/VBoxRT.so >> #2 0x00007fffedbf3960 in ?? () from /usr/lib64/virtualbox/VBoxRT.so >> #3 0x00007fffedeea485 in VBoxGetCAPIFunctions () from >> /usr/lib64/virtualbox/VBoxXPCOMC.so [...] That's https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Main/cbinding/VBox... - initializing the runtime our code uses which abstracts platform specifics. The rationale why the runtime installs a dummy signal handler is that if SIGCHLD is set to be ignored (that's the default) then POSIX compliant waitpid() won't work. It's mentioned in the wait(2) man page on my linux system, see the notes about the ECHILD error code. It's really excessive that some other code prevents using the full functionality. VirtualBox checks if the SIGCHLD handler is already set. If the signal isn't ignored it happily continues without touching SIGCHLD. So if libvirt wants to establish a sane default it would also work. To state the obvious: running an API client with crippled waitpid() can lead to extremely strange behavior. XPCOM creates worker threads on demand and expects to be able to wait on their termination. >> After building libvirt without vbox support the assert disappeared. > > So what are you saying is that vbox interferes with libxenlight? While I > see why vbox library wants SIGCHLD handler, maybe they became fault > tolerant meanwhile. Or? Hope the explanations help resolving the conflict. It's not that we want to sabotage anyone else. Klaus > >> >> >> On Fri, Jan 23, 2015 at 5:14 AM, Michal Privoznik <mprivozn(a)redhat.com >> <mailto:mprivozn@redhat.com>> wrote: >> >> On 22.01.2015 17:49, CloudPatch Staff wrote: >> > We're hitting an assert whenever we try to create an HVM instance under >> > Xen via libvirtd. >> > >> > System is running on Gentoo, package information as follows: >> > >> > app-emulation/xen-4.5.0 USE="api debug flask hvm pam pygrub python qemu >> > screen" >> > app-emulation/xen-tools-4.5.0 USE="api debug flask hvm pam pygrub >> python >> > qemu screen" >> > app-emulation/libvirt-1.2.11-r2:0/1.2.11 USE="caps libvirtd lvm macvtap >> > nls qemu udev vepa virtualbox xen" >> > >> > The following commands are run in parallel: >> > >> > vmmachine ~ # libvirtd --listen >> > 2015-01-22 16:33:13.596+0000: 2620: info : libvirt version: 1.2.11 >> > 2015-01-22 16:33:13.596+0000: 2620: error : udevGetDMIData:1607 : >> Failed >> > to get udev device for syspath '/sys/devices/virtual/dmi/id' or >> > '/sys/class/dmi/id' >> > libvirtd: libxl_fork.c:350: sigchld_installhandler_core: Assertion >> > `((void)"application must negotiate with libxl about SIGCHLD", >> > !(sigchld_saved_action.sa_flags & 4) && >> > (sigchld_saved_action.__sigaction_handler.sa_handler == >> > ((__sighandler_t) 0) || >> > sigchld_saved_action.__sigaction_handler.sa_handler == >> ((__sighandler_t) >> > 1)))' failed. >> > Aborted >> >> Interesting. Can you attach a debugger so we can see stacktrace? >> >> Michal >> >> > > Michal > -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list