9:48 a.m.
[moderator note: .pngs were stripped to avoid overwhelming the mail
server and list recipients with 1M of data]
-------- Forwarded Message --------
Date: Wed, 30 Mar 2016 11:25:09 +0200
Message-ID:
<20160330112509.Horde.0Oad-ZfkzdZnouXf-VkEmw1(a)webmailperso.univ-brest.fr>
From: villeneu(a)kassis.univ-brest.fr
To: Franky Van Liedekerke <liedekef(a)telenet.be>
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] VM crash and lock manager
References:
<20160329192740.Horde.BZXrmDPzMBshASPvOs_x4Q6(a)webmailperso.univ-brest.fr>
<20160329231311.1ae867a1(a)telenet.be>
In-Reply-To: <20160329231311.1ae867a1(a)telenet.be>
---
Franky Van Liedekerke <liedekef(a)telenet.be> a écrit :
On Tue, 29 Mar 2016 19:27:40 +0200
villeneu(a)kassis.univ-brest.fr wrote:
> Hello
>
> I changed my hypervisors ( 8) from centos 6.0 to fedora 23 and I
> added locks to prevent starting VM on multiple nodes, and it's work
> well.
>
> Since I changed the OS some old VMs ( centos 5.x and 6.x ) with very
> old kernel seem very instable and crash very often ( after one or two
> days ). Before they never have these problems.
>
> I saw, perhaps the reason is the dev is too slow ??? see
screenshot
> joined
>
> A major problem with these crashs is that the VMs couldn't be
> destroyed by the virsh command, the qemu process is notified as
> defunct by the ps command.
>
> with virsh destroy VM
> I often get
>
> Failed to terminate process xxx with SIGTERM: Device or resource busy
> and the VM is still in the list
>
> If I try to remove all the file associated with the VM in /var/run or
> /var/lib....channel .. It doesn't give results The
> VM is still running as defunct and after a libvirt restart , libvirt
> and virsh command are pending.
>
> With sanlock it's a problem because the is never release and it's
> impossible to restart the VM. I can't remove the VM in the sanlock
> list. I tried kill commands, rm ... but without a real success.
>
> I tried many command with sanlock perhaps I misunderstood some
> commands but I never succeed in removing lock on the VMs.
>
> sanlock client rem_lockspace -r
>
__LIBVIRT__DISKS__:b90b9c61e2d6413077205907ffb3281a:/var/lib/libvirt/images/POOL_ADMIN/sanlock/b90b9c61e2d6413077205907ffb3281a:0:4
> -p 9383
>
> So I'm trying to use lock_manager to replace the sanlock but
> lock_manager doesn't have tools to retrieve the lock / VM name in
> case of failure or crash.
>
> My questions are, is there a way to release or to force to release a
> lock with sanlock even if the qemu process is defunc and a way to
> restart libvirt in normal usage ?
>
> Perhaps by the API it`s possible to forece the release the lock ?
>
> Is there tools to retreive or calculate SHA256 lock for a VM with the
> lock_manager lock. I saw it's the lock is a SHA256 hash of the path
> and the VM name but I nerver get the good hash when I try by the hand.
>
> Is there tools to retreive or calculate SHA256 the lock for a VM with
> the lock_manager lock. I saw that the lock is a SHA256 hash of the
> path and the VM name but I never get the good hash when I try by the
> hand.
>
> Thanks.
> Sorry for my poor english ... I 'am french
> Michel
It seems the lock held by sanlock isn't being released, so the vm can't
be shut down properly. Maybe some selinux issue?
I've been using virtlockd without troubles for more than a year already
(without selinux). For the locks, see if the lslocks command helps you.
Franky
_______________________________________________
libvirt-users mailing list
libvirt-users@redhat.comhttps://www.redhat.com/mailman/listinfo/libvirt-users
I do not use selinux too, I preffered sanlock at the beginning because
sanlock comes with tools to remove locks. But when VMs crash abnormaly or
freeze ( see the sreenshot I missed to attach it yesterday ) there is
no way to unlease or remove the locks, they are presents even if I kill
the process qemu and I can't restart the VM because the lock is still here.
So I try now lock-manager, but do you know exactly how the lock is
calculate . In case of problem I want to be able to retrieve the lock
coupled with the VM.
I have seen that it is a hash SHA256 of the path and the name, but if a try
to do by the hand a sha256 of the complete path of the VM I don't have
the correct value ( the same value as lock_manager ).
Any Ideas ?
Thanks
crash1.png
crash2.png
3190
days inactive
3190
days old
0 comments
1 participants
participants (1)
-
Eric Blake