On 24.03.2014 21:45, Brian Rak wrote: > I'm seeing a very weird (and somewhat reproducable) crash in > setSchedulerParameters. The backtrace looks like this: > > *** glibc detected *** python2.7: free(): invalid pointer: > 0x000000000152bc48 *** > ======= Backtrace: ========= > /lib64/libc.so.6(+0x76166)[0x7faa9e991166] > /usr/lib64/python2.7/site-packages/libvirtmod.so(virFree+0x29)[0x7faa9887bfe9] > > > /lib/libvirt.so.0(virTypedParamsClear+0x54)[0x7faa98342fe4] > /lib/libvirt.so.0(virTypedParamsFree+0x1e)[0x7faa9834302e] > /usr/lib64/python2.7/site-packages/libvirtmod.so(+0x1b4dc)[0x7faa9886c4dc] > > /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5629)[0x7faa9f63a129] > > /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x68e8)[0x7faa9f63b3e8] > > /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x8ae)[0x7faa9f63bd5e] > /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7faa9f63be72] > /usr/lib64/libpython2.7.so.1.0(+0xff25c)[0x7faa9f65625c] > /usr/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x90)[0x7faa9f656330] > /usr/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xef)[0x7faa9f6578cf] > > > /usr/lib64/libpython2.7.so.1.0(Py_Main+0xc56)[0x7faa9f6693f6] > /lib64/libc.so.6(__libc_start_main+0xfd)[0x7faa9e939d1d] > python2.7[0x400649] > > > I am using: > Python2.7 > Python-Libvirt 1.2.2 with the "[PATCH libvirt-python 1/2] > setPyVirTypedParameter: Copy full field name" patch. > Libvirt 1.2.1 Is the following patch applied as well? commit 69c4600d61fa74c4977d2471a29fb73f0fe5edb0 Author: Michal Privoznik <mprivozn(a)redhat.com&gt; AuthorDate: Tue Mar 18 09:20:00 2014 +0100 Commit: Michal Privoznik <mprivozn(a)redhat.com&gt; CommitDate: Tue Mar 18 14:43:10 2014 +0100 setPyVirTypedParameter: free whole return variable on error The @ret value is built in a loop. However, if in one iteration there's an error, we should free all the fields built so far. For instance, if there's an error and the previous item was type of VIR_TYPED_PARAM_STRING we definitely must free it. Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com&gt; Can you install debuginfo so we see the full stack trace? Michal

On 3/27/2014 10:54 AM, Brian Rak wrote: > > On 3/25/2014 6:50 AM, Michal Privoznik wrote: >> On 24.03.2014 21:45, Brian Rak wrote: >>> I'm seeing a very weird (and somewhat reproducable) crash in >>> setSchedulerParameters. The backtrace looks like this: >>> >>> *** glibc detected *** python2.7: free(): invalid pointer: >>> 0x000000000152bc48 *** >>> ======= Backtrace: ========= >>> /lib64/libc.so.6(+0x76166)[0x7faa9e991166] >>> /usr/lib64/python2.7/site-packages/libvirtmod.so(virFree+0x29)[0x7faa9887bfe9] >>> >>> >>> /lib/libvirt.so.0(virTypedParamsClear+0x54)[0x7faa98342fe4] >>> /lib/libvirt.so.0(virTypedParamsFree+0x1e)[0x7faa9834302e] >>> /usr/lib64/python2.7/site-packages/libvirtmod.so(+0x1b4dc)[0x7faa9886c4dc] >>> >>> /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5629)[0x7faa9f63a129] >>> >>> /usr/lib64/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x68e8)[0x7faa9f63b3e8] >>> >>> /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x8ae)[0x7faa9f63bd5e] >>> /usr/lib64/libpython2.7.so.1.0(PyEval_EvalCode+0x32)[0x7faa9f63be72] >>> /usr/lib64/libpython2.7.so.1.0(+0xff25c)[0x7faa9f65625c] >>> /usr/lib64/libpython2.7.so.1.0(PyRun_FileExFlags+0x90)[0x7faa9f656330] >>> /usr/lib64/libpython2.7.so.1.0(PyRun_SimpleFileExFlags+0xef)[0x7faa9f6578cf] >>> >>> >>> /usr/lib64/libpython2.7.so.1.0(Py_Main+0xc56)[0x7faa9f6693f6] >>> /lib64/libc.so.6(__libc_start_main+0xfd)[0x7faa9e939d1d] >>> python2.7[0x400649] >>> >>> >>> I am using: >>> Python2.7 >>> Python-Libvirt 1.2.2 with the "[PATCH libvirt-python 1/2] >>> setPyVirTypedParameter: Copy full field name" patch. >>> Libvirt 1.2.1 >> >> Is the following patch applied as well? >> >> commit 69c4600d61fa74c4977d2471a29fb73f0fe5edb0 >> Author: Michal Privoznik <mprivozn(a)redhat.com&gt; >> AuthorDate: Tue Mar 18 09:20:00 2014 +0100 >> Commit: Michal Privoznik <mprivozn(a)redhat.com&gt; >> CommitDate: Tue Mar 18 14:43:10 2014 +0100 >> >> setPyVirTypedParameter: free whole return variable on error >> >> The @ret value is built in a loop. However, if in one iteration >> there's an error, we should free all the fields built so far. For >> instance, if there's an error and the previous item was >> type of VIR_TYPED_PARAM_STRING we definitely must free it. >> >> Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com&gt; >> >> >> Can you install debuginfo so we see the full stack trace? >> >> Michal > Unfortunately, even with that patch I'm still seeing crashes. > > Apparently, this crash only happens when running normally (it doesn't > crash when running under gdb). I created a core dump, and have what > might be a better call stack: > > (gdb) where > #0 0x00007f96047cd925 in raise () from /lib64/libc.so.6 > #1 0x00007f96047cf105 in abort () from /lib64/libc.so.6 > #2 0x00007f960480b837 in __libc_message () from /lib64/libc.so.6 > #3 0x00007f9604811166 in malloc_printerr () from /lib64/libc.so.6 > #4 0x00007f95fe6fbff9 in virFree (ptrptr=0x2463918) at > libvirt-utils.c:119 > #5 0x00007f95fe1c2fe4 in virTypedParamsClear (params=<value optimized > out>, nparams=5) at util/virtypedparam.c:1147 > #6 0x00007f95fe1c302e in virTypedParamsFree (params=0x2463800, > nparams=<value optimized out>) at util/virtypedparam.c:1166 > #7 0x00007f95fe6ec4ec in libvirt_virDomainSetSchedulerParameters > (self=<value optimized out>, args=<value optimized out>) at > libvirt-override.c:975 > #8 0x00007f96054ba129 in call_function (f=<value optimized out>, > throwflag=<value optimized out>) at > /usr/src/debug/Python-2.7.6/Python/ceval.c:4098 > ... truncated > > > This patch corrects the issue. new_params is not guaranteed to have > the same number of parameters as params, so we can't use nparams here > to free the object. > > diff --git a/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c > b/python27-libvirt/libvi > index f8fafa7..9725870 100755 > --- a/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c > +++ b/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c > @@ -913,7 +913,7 @@ libvirt_virDomainSetSchedulerParameters(PyObject > *self ATTRIBUTE_UNUSED, > int i_retval; > int nparams = 0; > Py_ssize_t size = 0; > - virTypedParameterPtr params, new_params = NULL; > + virTypedParameterPtr params = NULL, new_params = NULL; > > if (!PyArg_ParseTuple(args, (char > *)"OO:virDomainSetScedulerParameters", > &pyobj_domain, &info)) > @@ -972,7 +972,7 @@ libvirt_virDomainSetSchedulerParameters(PyObject > *self ATTRIBUTE_UNUSED, > > cleanup: > virTypedParamsFree(params, nparams); > - virTypedParamsFree(new_params, nparams); > + virTypedParamsFree(new_params, size); > return ret; > } > > > I am not sure if the 'params = NULL' change is necessary, but that > seemed like a potential bug anyway. Turns out this issue occurs a bunch of other places as well. Better patch, fixes all the ones I see: diff --git a/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c b/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c --- a/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c +++ b/python27-libvirt/libvirt-python-1.2.2/libvirt-override.c @@ -913,7 +913,7 @@ libvirt_virDomainSetSchedulerParameters(PyObject *self ATTRIBUTE_UNUSED, int i_retval; int nparams = 0; Py_ssize_t size = 0; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OO:virDomainSetScedulerParameters", &pyobj_domain, &info)) @@ -972,7 +972,7 @@ libvirt_virDomainSetSchedulerParameters(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; } @@ -1063,7 +1063,7 @@ libvirt_virDomainSetBlkioParameters(PyObject *self ATTRIBUTE_UNUSED, int nparams = 0; Py_ssize_t size = 0; unsigned int flags; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OOi:virDomainSetBlkioParameters", @@ -1122,7 +1122,7 @@ libvirt_virDomainSetBlkioParameters(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; } @@ -1183,7 +1183,7 @@ libvirt_virDomainSetMemoryParameters(PyObject *self ATTRIBUTE_UNUSED, int nparams = 0; Py_ssize_t size = 0; unsigned int flags; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OOi:virDomainSetMemoryParameters", @@ -1242,7 +1242,7 @@ libvirt_virDomainSetMemoryParameters(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; } @@ -1303,7 +1303,7 @@ libvirt_virDomainSetNumaParameters(PyObject *self ATTRIBUTE_UNUSED, int nparams = 0; Py_ssize_t size = 0; unsigned int flags; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OOi:virDomainSetNumaParameters", @@ -1362,7 +1362,7 @@ libvirt_virDomainSetNumaParameters(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; } @@ -1424,7 +1424,7 @@ libvirt_virDomainSetInterfaceParameters(PyObject *self ATTRIBUTE_UNUSED, Py_ssize_t size = 0; unsigned int flags; const char *device = NULL; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OzOi:virDomainSetInterfaceParameters", @@ -1487,7 +1487,7 @@ libvirt_virDomainSetInterfaceParameters(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; } @@ -4785,7 +4785,7 @@ libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, Py_ssize_t size = 0; const char *disk; unsigned int flags; - virTypedParameterPtr params, new_params = NULL; + virTypedParameterPtr params = NULL, new_params = NULL; if (!PyArg_ParseTuple(args, (char *)"OzOi:virDomainSetBlockIoTune", &pyobj_domain, &disk, &info, &flags)) @@ -4843,7 +4843,7 @@ libvirt_virDomainSetBlockIoTune(PyObject *self ATTRIBUTE_UNUSED, cleanup: virTypedParamsFree(params, nparams); - virTypedParamsFree(new_params, nparams); + virTypedParamsFree(new_params, size); return ret; }