Re: [Libvirt-cim] [PATCH] DO NOT PUSH UPSTREAM

DV, Its not mirrored to RH yet, I am working on it. In the meantime, here is the description of the defect. ******************************************** virInitialize leads to segfault when unloading libvirt-cim providers. Contact Information = Sharad Mishra/snmishra(a)us.ibm.com, ---uname output--- Linux elm3b193.beaverton.ibm.com 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux Machine Type = IBM system x blade, machine type - 8853, model - MC1 ---Debugger--- A debugger is not configured ---Steps to Reproduce--- 1. Install libvirt and libvirt-cim 2 start libvirtd 3. start tog-pegasus 4. run wbemcli command to enum instance on KVM_ComputerSystem class. This is just a way to load the provider to memory. All we need to do is run virConnectOpen() and virConnectClose() in this provider. So in this step all that is required is for virConnectOpen and virConnectClose to run. 5. Stop tog-pegasus. A core is gererated. ---System Management Component Data--- Userspace tool common name: libvirt/libvirt-cim/tog-pegasus System management type: IMM The userspace tool has the following bit modes: 64 Userspace rpm: libvirt-0.6.3-33.el5, libvirt-cim-0.5.14-1 and tog-pegasus-2.9.2-2.el5 Userspace tool obtained from project website: na *Additional Instructions for Sharad Mishra/snmishra(a)us.ibm.com, : -Post a private note with access information to the machine that the bug is occuring on. -Attach ltrace and strace of userspace application. -Attach screen shoot of System Managment Utility Here is the gist of investigation done so far --- Here is the gist of his investigation - I've looked into the libvirt source code and the reason for issues with the unload of libvirt-cim providers is there (and cimprovider -r forces an unload, just as cimserver shutdown does). virConnectOpen() does call virInitialize() which then will initialize the gcrypt library after initializes the thread setup ->virThreadInitialize(), creates the error reporting structure ->virErrorInitialize() and initialize a random number generator ->virRandomInitialize(). The issue is hidden in virErrorInitialize(). It calls virErrorInitialize() which uses the wrapper function virThreadLocalInit(&virLastErr, virLastErrFreeData);. virThreadLocalInit() calls pthread_create() with a destructor function (virLastErrFreeData()) which will be called on thread exits. Trouble here is that pthread_key_delete() is never called in libvirt, but next time the system runs through the key list and libvirt has been unloaded the system will jump to an already unloaded function (the pointer were virLastErrFreeData() was loaded formerly) and consequently crash the process. Fixing this will not be easy, but a start would be a libvirt function on its externalize interface that allows cleanup short before library unload, or alternatively libvirt does the thread key delete on library unload. Trace output from gdb - /var/log/messages: Aug 6 11:35:58 SYX-e41f13fe5558 cimserver[21602]: OpenPegasus stopped. Aug 6 11:35:59 SYX-e41f13fe5558 kernel: cimserver[21376]: segfault at 00002aaaab165f00 rip 00002aaaab165f00 rsp 00000000457d40b8 error 14 call stack trace: command: gdb -batch -x /tmp/gdb.27236.cmd /usr/sbin/cimserver /var/log/core/core.25567 time: Sat Aug 6 11:35:59 UTC 2011 stdout/stderr: ----------------------------------------------------------------- [New Thread 25567] Core was generated by `/usr/sbin/cimserver'. Program terminated with signal 11, Segmentation fault. #0 0x00002aaaab4f6f00 in ?? () #0 0x00002aaaab4f6f00 in ?? () #1 0x00002ab4fb3c6ad9 in __nptl_deallocate_tsd () from /lib64/libpthread.so.0 #2 0x00002ab4fb3c774b in start_thread () from /lib64/libpthread.so.0 #3 0x00002ab4fc07af6d in clone () from /lib64/libc.so.6 ----------------------------------------------------------------- exit_code=0 ----------------------------------------------------------------- Regards, Sharad Mishra Open Virtualization Linux Technology Center IBM libvirt-cim-bounces(a)redhat.com wrote on 08/30/2011 01:43:06 AM: <libvirt-cim(a)redhat.com&gt; <libvirt-cim(a)redhat.com&gt;