[libvirt-users] libvirt hang on domain attemp to create domain

good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml <?xml version="1.0" encoding="UTF-8"?> <domain type="kvm"> <tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os> <type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/> </os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model"> <topology sockets="1" cores="1" threads="1"/> </cpu> <clock offset="localtime"/> <devices> <graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix"> <source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/> </channel> </devices> </domain> i have this calls on stack 0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop 2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40 what i am doing wrong ?

On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml
<?xml version="1.0" encoding="UTF-8"?> <domain type="kvm"> <tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os> <type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/> </os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model"> <topology sockets="1" cores="1" threads="1"/> </cpu> <clock offset="localtime"/> <devices> <graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix"> <source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/> </channel> </devices> </domain>
i have this calls on stack
0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd to provide a reply message. Are you able to get libvirtd debug logs so we can see what's going on? http://wiki.libvirt.org/page/DebugLogs BTW, what's the libvirt version you are running? Michal
2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40
what i am doing wrong ?
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

В письме от Вторник, 18-июн-2013 12:10:41 пользователь Michal Privoznik написал:
On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml
<?xml version="1.0" encoding="UTF-8"?> <domain type="kvm">
<tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os>
<type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/>
</os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model">
<topology sockets="1" cores="1" threads="1"/>
</cpu> <clock offset="localtime"/> <devices>
<graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix">
<source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/>
</channel>
</devices>
</domain>
i have this calls on stack
0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd to provide a reply message. Are you able to get libvirtd debug logs so we can see what's going on?
http://wiki.libvirt.org/page/DebugLogs
BTW, what's the libvirt version you are running?
Michal
2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40
what i am doing wrong ?
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
i have libvirt version 1.0.6, debug log in attachment.

On 18.06.2013 22:21, Александр wrote:
В письме от Вторник, 18-июн-2013 12:10:41 пользователь Michal Privoznik написал:
On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml
<?xml version="1.0" encoding="UTF-8"?> <domain type="kvm">
<tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os>
<type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/>
</os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model">
<topology sockets="1" cores="1" threads="1"/>
</cpu> <clock offset="localtime"/> <devices>
<graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix">
<source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/>
</channel>
</devices>
</domain>
i have this calls on stack
0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd to provide a reply message. Are you able to get libvirtd debug logs so we can see what's going on?
http://wiki.libvirt.org/page/DebugLogs
BTW, what's the libvirt version you are running?
Michal
2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40
what i am doing wrong ?
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
i have libvirt version 1.0.6, debug log in attachment.
Yuck, this is 1.0.6; Well, from the logs it seems like the daemon is stuck. Can you attach to the libvirtd while it's being stuck and run 't a a bt' and post the result? I wonder if we have a deadlock somewhere. Michal

В письме от Среда, 19-июн-2013 09:35:27 пользователь Michal Privoznik написал:
On 18.06.2013 22:21, Александр wrote:
В письме от Вторник, 18-июн-2013 12:10:41 пользователь Michal Privoznik
написал:
On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml
<?xml version="1.0" encoding="UTF-8"?> <domain type="kvm">
<tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os>
<type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/>
</os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model">
<topology sockets="1" cores="1" threads="1"/>
</cpu> <clock offset="localtime"/> <devices>
<graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix">
<source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/>
</channel>
</devices>
</domain>
i have this calls on stack
0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd to provide a reply message. Are you able to get libvirtd debug logs so we can see what's going on?
http://wiki.libvirt.org/page/DebugLogs
BTW, what's the libvirt version you are running?
Michal
2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40
what i am doing wrong ?
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
i have libvirt version 1.0.6, debug log in attachment.
Yuck, this is 1.0.6; Well, from the logs it seems like the daemon is stuck. Can you attach to the libvirtd while it's being stuck and run 't a a bt' and post the result? I wonder if we have a deadlock somewhere.
Michal
if i correctly understand you, you need debugger backtrace after libvirtd stuck ?, if so, here is bt from gdb (gdb) bt #0 0x00007f00c1ae876d in poll () from /lib64/libc.so.6 #1 0x00007f00c203cfcb in virEventPollRunOnce () from /usr/lib64/libvirt.so.0 #2 0x00007f00c203ba4d in virEventRunDefaultImpl () from /usr/lib64/libvirt.so.0 #3 0x00007f00c214a5cd in virNetServerRun () from /usr/lib64/libvirt.so.0 #4 0x00007f00c2aea08a in main () also maybe build options means..., here is my build options * app-emulation/libvirt ::gentoo 1.0.3-r2 1.0.5.1-r3 1.0.5.2(~) 1.0.6(~)* (9999)K {:0} ::installed 1.0.6 {:0} app-emulation/libvirt-1.0.6:0::installed (world) Description C toolkit to manipulate virtual machines Homepage http://www.libvirt.org/ From repositories gentoo Installed time Tue Jun 18 07:18:08 EEST 2013 Installed using paludis-1.4.0 License LGPL-2.1 Use flags USE (-audit) (-avahi) (caps) (-firewalld) (fuse) (iscsi) (libvirtd) (lvm) (-lxc) (macvtap) (nfs) (nls) (-numa) (-openvz) (-parted) (pcap) (-phyp) (policykit) (python) (qemu) (-rbd) (sasl) (-selinux) (-systemd) (udev) (-uml) (vepa) (-virt-network) (virtualbox) (-xen) python_single_target (-python2_5) (-python2_6) (python2_7) python_targets (-python2_5) (-python2_6) (python2_7) Build Options -trace Trace actions executed by the package (very noisy, for debugging broken builds only)

On 19.06.2013 21:24, Александр wrote:
В письме от Среда, 19-июн-2013 09:35:27 пользователь Michal Privoznik написал:
On 18.06.2013 22:21, Александр wrote:
В письме от Вторник, 18-июн-2013 12:10:41 пользователь Michal Privoznik
написал:
On 18.06.2013 07:14, Александр wrote:
good day, i am trying to create domain via virDomainDefineXML. and my program hanging, also i failed to caonnect to libvirt daemon with virt-manager after this, only killing libvirtd via SIGKILL and restarting it solveing hang (i mean after restart i able to connect with virt-manager), i using following domain xml
<?xml version="1.0" encoding="UTF-8"?> <domain type="kvm">
<tittle>Automatically created domain with default configuration</tittle> <name>vm1</name> <description>no description</description> <os>
<type>hvm</type> <boot dev="cdrom"/> <boot dev="hd"/>
</os> <vcpu>1</vcpu> <memory unit="MB">512</memory> <cpu model="host-model">
<topology sockets="1" cores="1" threads="1"/>
</cpu> <clock offset="localtime"/> <devices>
<graphics type="vnc" autoport="yes" password="admin"/> <channel type="unix">
<source mode="bind" path="/var/run/vm1.sock"/> <target type="virtio" name="org.guest_agent.0"/>
</channel>
</devices>
</domain>
i have this calls on stack
0 0x00007ffff48c1750 __poll_nocancel 1 0x00007ffff649e5ca virNetClientIOEventLoop
This is just probably a client waiting for the corresponding libvirtd to provide a reply message. Are you able to get libvirtd debug logs so we can see what's going on?
http://wiki.libvirt.org/page/DebugLogs
BTW, what's the libvirt version you are running?
Michal
2 0x00007ffff649ed96 virNetClientSendInternal 3 0x00007ffff64a0443 virNetClientSendWithReply 4 0x00007ffff64a0daa virNetClientProgramCall 5 0x00007ffff6478566 callFull.isra.2 6 0x00007ffff6487fa2 remoteDomainDefineXML 7 0x00007ffff6449d2f virDomainDefineXML 8 0x0000000000426941 virt_domain::create /home/sss/install/git/work/avirt/vm_pool/vm_pool/libvirt.cpp 295 9 0x000000000040c1a9 main /home/sss/install/git/work/avirt/vm_pool/vm_pool/main.cpp 40
what i am doing wrong ?
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
i have libvirt version 1.0.6, debug log in attachment.
Yuck, this is 1.0.6; Well, from the logs it seems like the daemon is stuck. Can you attach to the libvirtd while it's being stuck and run 't a a bt' and post the result? I wonder if we have a deadlock somewhere.
Michal
if i correctly understand you, you need debugger backtrace after libvirtd stuck ?, if so, here is bt from gdb
You understand correctly. However, simple 'bt' doesn't show what's going on in other threads - libvirtd is multi threaded.
(gdb) bt #0 0x00007f00c1ae876d in poll () from /lib64/libc.so.6 #1 0x00007f00c203cfcb in virEventPollRunOnce () from /usr/lib64/libvirt.so.0 #2 0x00007f00c203ba4d in virEventRunDefaultImpl () from /usr/lib64/libvirt.so.0 #3 0x00007f00c214a5cd in virNetServerRun () from /usr/lib64/libvirt.so.0 #4 0x00007f00c2aea08a in main ()
So all we see here is the libvirtd event loop. That's why we need 't a a bt' which is 'thread apply all backtrace' which will run 'bt' over all threads. Michal
participants (2)
-
Michal Privoznik
-
Александр