FYI, I have found a way to terminate instance:
virsh schedinfo <domain> --set vcpu_quota=20000

After this, shutdown, reboot and destroy all works. Of course, this disables the vcpu throttling. It was set to 5000, which may cause the CPU stall problem in guest.

Thanks,
Yuanle

On Sat, Feb 8, 2014 at 11:34 AM, sylecn <sylecn@gmail.com> wrote:

Hi,

I'm using libvirt/kvm with openstack. When vm has CPU stall, the instance can't be destroyed. I have asked the question on openstack user mailing list and received no answers in weeks.

Here is one example:

    # virsh destroy instance-00000085
    error: Failed to destroy domain instance-00000085
    error: Failed to terminate process 61222 with SIGKILL: Device or resource busy

    # lsof -p 61222
    # ps -f -p 61222
    UID        PID PPID C STIME TTY          TIME CMD
    107      61222     1 68 Jan24 ?        9-23:06:19 [qemu-system-x86] <defunct>

The log in /var/log/libvirt/libvirtd.log:
    2014-02-08 03:21:31.597+0000: 2177: error : virProcessKillPainfully:315 : Failed to terminate process 61222 with SIGKILL: Device or resource busy

Restart libvirt-bin service did not solve the problem.

virsh shutdown/reboot just hang. Maybe that is because the CPU in vm is stalled.

I wonder if there is some way to destroy (or rebuild) dead instance without rebooting the hypervisor. Any suggestions is welcome.

The software I am running:
hypervisor os: ubuntu 12.04.3
hypervisor kernel: 3.8.0-35-generic
vm os: ubuntu 12.04
vm kernel: 3.2.0-56-virtual

openstack: havana
libvirt: 1.1.1-0ubuntu8~cloud2

Thanks,
Yuanle