11:06 a.m.
Another segfault, again after calling list in virsh after a domain failed to
start:
=====================
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffef5fe710 (LWP 30490)]
0x00007ffff7cd5cfd in virDomainObjListCountActive (payload=0x7fffdc006ef0,
name=<value optimized out>, data=0x7fffef5fdb0c) at conf/domain_conf.c:6769
6769 if (virDomainObjIsActive(obj))
Missing separate debuginfos, use: debuginfo-install glibc-2.11.2-1.x86_64 nss-
softokn-freebl-3.12.6-2.fc12.1.x86_64 openssl-1.0.0a-1.fc12.x86_64
(gdb)
(gdb) bt
#0 0x00007ffff7cd5cfd in virDomainObjListCountActive (payload=0x7fffdc006ef0,
name=<value optimized out>, data=0x7fffef5fdb0c) at conf/domain_conf.c:6769
#1 0x00007ffff7cc06ca in virHashForEach (table=0x6f9820, iter=0x7ffff7cd5ce0
<virDomainObjListCountActive>, data=<value optimized out>) at util/hash.c:495
#2 0x00007ffff7cd5224 in virDomainObjListNumOfDomains (doms=<value optimized
out>, active=<value optimized out>) at conf/domain_conf.c:6788
#3 0x0000000000438418 in qemudNumDomains (conn=<value optimized out>) at
qemu/qemu_driver.c:4260
#4 0x00007ffff7d05989 in virConnectNumOfDomains (conn=0x7fffe4000e50) at
libvirt.c:1903
#5 0x0000000000422d2c in remoteDispatchNumOfDomains (server=<value optimized
out>, client=<value optimized out>, conn=0x7fffe4000e50, hdr=<value optimized
out>, rerr=0x7fffef5fdc70,
args=<value optimized out>, ret=0x7fffef5fdbc0) at remote.c:2905
#6 0x0000000000426bc1 in remoteDispatchClientCall (server=<value optimized
out>, client=0x7ffff0053a90, msg=0x7ffff0012240) at dispatch.c:506
#7 0x0000000000426f73 in remoteDispatchClientRequest (server=0x6e3cd0,
client=0x7ffff0053a90, msg=0x7ffff0012240) at dispatch.c:388
#8 0x0000000000417ed8 in qemudWorker (data=0x7ffff0000920) at libvirtd.c:1568
#9 0x0000003818c06a3a in start_thread () from /lib64/libpthread.so.0
#10 0x00000038188de77d in clone () from /lib64/libc.so.6
#11 0x0000000000000000 in ?? ()
(gdb) info locals
obj = 0x7fffdc006ef0
(gdb) inspect *obj
$2 = {lock = {lock = {__data = {__lock = 0, __count = 0, __owner = 825373486,
__nusers = 1701060666, __kind = 543651170, __spins = 1769349178, __list =
{__prev = 0x7463656e6e6f4372,
__next = 0x6d6f44664f6d754e}}, __size =
"\000\000\000\000\000\000\000\000.321: debug : virConnectNumOfDom", __align =
0}}, refs = 1936615777, pid = 959983930, state = 540680242,
autostart = 1, persistent = 1, def = 0x656666663778303d, newDef =
0xa30356530303034, snapshots = {objs = 0x70797420313d7200}, current_snapshot =
0x7461747320303d65,
privateData = 0x72657320303d7375, privateDataFreeFunc = 0x3d6c6169}
(gdb) print data
$4 = (void *) 0x7fffef5fdb0c
(gdb) x 0x7fffef5fdb0c
0x7fffef5fdb0c: 0x00000016
(gdb) print obj->def
$6 = (virDomainDefPtr) 0x656666663778303d
(gdb) print *(obj->def)
Cannot access memory at address 0x656666663778303d
(gdb) up
#1 0x00007ffff7cc06ca in virHashForEach (table=0x6f9820, iter=0x7ffff7cd5ce0
<virDomainObjListCountActive>, data=<value optimized out>) at util/hash.c:495
495 iter(entry->payload, entry->name, data);
(gdb) info locals
entry = 0x7fffdc00e860
i = <value optimized out>
count = <value optimized out>
(gdb) print entry
$7 = (virHashEntry *) 0x7fffdc00e860
(gdb) print *entry
$8 = {next = 0x0, name = 0x7fffdc00ea90 "654c9839-db0e-
ab95-5fad-7c91d9e7c9c4", payload = 0x7fffdc006ef0, valid = 1}
(gdb) print {virDomainObj} entry->payload
$9 = {lock = {lock = {__data = {__lock = 0, __count = 0, __owner = 825373486,
__nusers = 1701060666, __kind = 543651170, __spins = 1769349178, __list =
{__prev = 0x7463656e6e6f4372,
__next = 0x6d6f44664f6d754e}}, __size =
"\000\000\000\000\000\000\000\000.321: debug : virConnectNumOfDom", __align =
0}}, refs = 1936615777, pid = 959983930, state = 540680242,
autostart = 1, persistent = 1, def = 0x656666663778303d, newDef =
0xa30356530303034, snapshots = {objs = 0x70797420313d7200}, current_snapshot =
0x7461747320303d65,
privateData = 0x72657320303d7375, privateDataFreeFunc = 0x3d6c6169}
(gdb) print entry->payload
$10 = (void *) 0x7fffdc006ef0
(gdb) print {virDomainDef} 0xa30356530303034
Cannot access memory at address 0xa30356530303034
(gdb) print {virDomainDef} 0x656666663778303d
Cannot access memory at address 0x656666663778303d
(gdb) up
#2 0x00007ffff7cd5224 in virDomainObjListNumOfDomains (doms=<value optimized
out>, active=<value optimized out>) at conf/domain_conf.c:6788
6788 virHashForEach(doms->objs, virDomainObjListCountActive,
&count);
(gdb) info locals
count = 22
=====================
(I have tried to piece together some more information about what happened with
what little gdb skills I possess... The less interesting bits, where I
struggled to get useful information out of gdb, are cut out.)
It looks like the real problem is that the def and newDef pointers of the last
virDomainObj point to unallocated memory, making libvirtd crash in
static inline int virDomainObjIsActive(virDomainObjPtr dom), where it calls
return dom->def->id != -1;.
Guido