Many moons ago, I wanted to make locking more debuggable from logs.
An idea appeared that this might be better off with SystemTap. I let
it rot for a while and some time back I had a deadlock that I wanted
to try it on. So I started scripting it and here's the work (in
progress).
Well, the idea is simple. For each mutex that is about to get locked,
append it's symname and pid that locked it into the list A, when it
get's locked move it to list B and when it's unlocked, remove it.
Then whenever the user presses ^C, the script prints all the locks
that were locked (with their particular backtraces) and locks that are
waiting to be locked. This could be enhanced that it would print both
pieces of information only for the locks that are waiting to be
acquired. But I don't really care what color the shed will have.
Here are some problems I'd love to get any help with:
- I cannot run it against built git version, two problems with that:
- When I'm not root, staprun needs setuid bit set and that is, of
course, incompatible with the needed LD_LIBRARY_PATH.
- Even when I am root, I'm unable to get any backtrace that goes
beyond virLogMutex() no matter how I'm loading the debug symbols.
- When running with installed libvirtd, I am not getting as nice
backtrace as I would in gdb (but that's probably the only way how
systemtap will get any backtrace so it's better than nothing). And
I'm unable to get line numbers even with addr2line.
I'd be interested in hearing how this works for others, or any hints
on how to make the script so good that I could post it for the
inclusion upstream. Any best practices about systemtap scripts are
welcomed as well.
_______________________________________________________
(*) Not quite, I just wanted you to notice this mail ;)
Martin Kletzander (2):
lock-debug.stp
DO NOT APPLY: deadlock test
daemon/libvirtd.c | 1 +
examples/systemtap/lock-debug.stp | 115 ++++++++++++++++++++++++++++++++++++++
2 files changed, 116 insertions(+)
create mode 100644 examples/systemtap/lock-debug.stp
--
2.1.0