8:23 a.m.
On Jun 30, 2017 6:22 AM, "Peter Krempa" <pkrempa(a)redhat.com> wrote:
> On Fri, Jun 30, 2017 at 12:05:47 +0200, Peter Krempa wrote:
> > On Thu, Jun 22, 2017 at 11:02:41 -0400, Doug Hughes wrote:
> > > [...]
> > > > virsh blockcommit $vm $DISK --active --pivot
> > > > > Normally this works
fine, though something went wrong on the 20th.
> > > something happened to make the blockcommit fail, but the -snap file
got
> > > deleted (note to self - check return code from blockcommit command!)
> > > > > So now I'm in a
state. The domain i still running. but it's running
off
> In fact, it's way simpler. If libvirt still knows
about the overlay
> image (this is necessary only so that it can say the proper things to
> qemu) you can re-do the block commit:
> $ virsh list
> Id Name State
> ----------------------------------------------------
> 3 fedora23 running
> $ virsh snapshot-create-as --disk-only --no-metadata
fedora23
> Domain snapshot 1498817916 created
> $ virsh domblklist fedora23
> Target Source
> ------------------------------------------------
> vda /var/lib/libvirt/images/fedora23.1498817916
> hda -
> $ rm /var/lib/libvirt/images/fedora23.1498817916
> $ ls /var/lib/libvirt/images/fedora23.1498817916
> ls: cannot access '/var/lib/libvirt/images/fedora23.1498817916': No such
file or directory
> $ virsh blockcommit --active --pivot fedora23 vda
> Successfully pivoted
> $ virsh domblklist fedora23
> Target Source
> ------------------------------------------------
> vda /var/lib/libvirt/images/fedora23.qcow2
> hda -
Thanks for the reply! The original image is still there. Only the 1st and
only top-level snapshot is deleted.
the blockcommit fails though:
[root@vm1 ~]# virsh blockcommit serv1r2 vda --active --pivot
error: block copy still active: disk 'vda' already in active block job
> > the -snapshot that is in the xml. I googled around for how
to recover
a
> > blockcommit from a deleted snapshot, but didn't find
anything.
(pointers
> > > welcome)
8:38 a.m.
New subject: [libvirt-users] recovering from deleted snapshot
On Fri, Jun 30, 2017 at 09:23:29 -0400, Doug Hughes wrote:
On Jun 30, 2017 6:22 AM, "Peter Krempa"
<pkrempa(a)redhat.com> wrote:
> On Fri, Jun 30, 2017 at 12:05:47 +0200, Peter Krempa wrote:
> > On Thu, Jun 22, 2017 at 11:02:41 -0400, Doug Hughes wrote:
[...]
file or directory
> $ virsh blockcommit --active --pivot fedora23 vda
>
> Successfully pivoted
> $ virsh domblklist fedora23
> Target Source
> ------------------------------------------------
> vda /var/lib/libvirt/images/fedora23.qcow2
> hda -
>
Thanks for the reply! The original image is still there. Only the 1st and
only top-level snapshot is deleted.
the blockcommit fails though:
[root@vm1 ~]# virsh blockcommit serv1r2 vda --active --pivot
error: block copy still active: disk 'vda' already in active block job
What does 'virsh blockjob serv1r2 vda' report?
If it's something like this:
$ virsh blockjob fedora23 vda
Active Block Commit: [100 %]
Then you just need to finalize the block job by:
virsh blockjob --pivot serv1r2 vda
If that's not the case, please post what virsh blockjob is reporting and
the <disk> part of the XML.
In case where libvirt falsely thinks that the job did not finish I'll
guide you how to recover it, since restarting libvirtd would make
libvirt clear the block job busy flag, but libvirt would lose the
metadata.
9:41 a.m.
New subject: [libvirt-users] recovering from deleted snapshot
sorry about top post, tablet mail client...
brilliant! that did it! Any idea what happened to cause it to partially fail and get into
this state?
oot@vm1 ~]# virsh blockjob serv1r2 vda
Active Block Commit: [100 %]
[root@vm1 ~]# virsh blockjob --pivot serv1r2 vda
[root@vm1 ~]#
[root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
-rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:39 /var/lib/libvirt/images/serv1r2.qcow2
[root@vm1 ~]# date
Fri Jun 30 10:39:12 EDT 2017
[root@vm1 ~]# date
Fri Jun 30 10:39:51 EDT 2017
[root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
-rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:40 /var/lib/libvirt/images/serv1r2.qcow2
[root@vm1 ~]#
Sent from my android device.
-----Original Message-----
From: Peter Krempa <pkrempa(a)redhat.com>
To: Doug Hughes <doug.hughes(a)keystonenap.com>
Cc: libvirt-users(a)redhat.com
Sent: Fri, 30 Jun 2017 9:38
Subject: Re: [libvirt-users] recovering from deleted snapshot
On Fri, Jun 30, 2017 at 09:23:29 -0400, Doug Hughes wrote:
On Jun 30, 2017 6:22 AM, "Peter Krempa"
<pkrempa(a)redhat.com> wrote:
> On Fri, Jun 30, 2017 at 12:05:47 +0200, Peter Krempa wrote:
> > On Thu, Jun 22, 2017 at 11:02:41 -0400, Doug Hughes wrote:
[...]
file or directory
> $ virsh blockcommit --active --pivot fedora23 vda
>
> Successfully pivoted
> $ virsh domblklist fedora23
> Target Source
> ------------------------------------------------
> vda /var/lib/libvirt/images/fedora23.qcow2
> hda -
>
Thanks for the reply! The original image is still there. Only the 1st and
only top-level snapshot is deleted.
the blockcommit fails though:
[root@vm1 ~]# virsh blockcommit serv1r2 vda --active --pivot
error: block copy still active: disk 'vda' already in active block job
What does 'virsh blockjob serv1r2 vda' report?
If it's something like this:
$ virsh blockjob fedora23 vda
Active Block Commit: [100 %]
Then you just need to finalize the block job by:
virsh blockjob --pivot serv1r2 vda
If that's not the case, please post what virsh blockjob is reporting and
the <disk> part of the XML.
In case where libvirt falsely thinks that the job did not finish I'll
guide you how to recover it, since restarting libvirtd would make
libvirt clear the block job busy flag, but libvirt would lose the
metadata.
9:55 a.m.
New subject: [libvirt-users] recovering from deleted snapshot
On Fri, Jun 30, 2017 at 10:41:25 -0400, doug.hughes(a)keystonenap.com wrote:
sorry about top post, tablet mail client...
brilliant! that did it! Any idea what happened to cause it to partially fail and get into
this state?
virsh blockcommit --pivot is actually 3 operations in sequence:
1) initiate the commit job
2) wait till it finishes
3) initiate the pivot job (this is synchronous, so the wait till it
finises is itegrated in the API call)
Apparently in your case the waiting part failed for some reason and thus
the job was not finished, lingering in the synchronised state.
The unfortunate part is that you don't know which operation failed.
Unfortunately it does not seem that we have wa waiting version of 'virsh
blockjob' that could be used to do the waiting. You can do the polling
manualy though.
The third operation can be split out by default, since you can specify
--wait instead of --pivot for 'virsh blockcommit' and it will finish
after step 2 above.
Also one of the cases which would help is that libvirt could actually
remove the file after commit, which would avoid the original problem,
since the file would still be there.
oot@vm1 ~]# virsh blockjob serv1r2 vda
Active Block Commit: [100 %]
[root@vm1 ~]# virsh blockjob --pivot serv1r2 vda
[root@vm1 ~]#
[root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
-rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:39 /var/lib/libvirt/images/serv1r2.qcow2
[root@vm1 ~]# date
Fri Jun 30 10:39:12 EDT 2017
[root@vm1 ~]# date
Fri Jun 30 10:39:51 EDT 2017
[root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
-rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:40 /var/lib/libvirt/images/serv1r2.qcow2
[root@vm1 ~]#
9:59 a.m.
New subject: [libvirt-users] recovering from deleted snapshot
Yes, I find it annoying that I have to manually remove the no-nos bless
leftover snapshot file. That'd be a great feature. In the mean time I will
look at adding this additional error handling in my script. Thanks much!
On Jun 30, 2017 10:55 AM, "Peter Krempa" <pkrempa(a)redhat.com> wrote:
On Fri, Jun 30, 2017 at 10:41:25 -0400, doug.hughes(a)keystonenap.com
wrote:
> sorry about top post, tablet mail client...
>
> brilliant! that did it! Any idea what happened to cause it to partially
fail and get into this state?
virsh blockcommit --pivot is actually 3 operations in sequence:
1) initiate the commit job
2) wait till it finishes
3) initiate the pivot job (this is synchronous, so the wait till it
finises is itegrated in the API call)
Apparently in your case the waiting part failed for some reason and thus
the job was not finished, lingering in the synchronised state.
The unfortunate part is that you don't know which operation failed.
Unfortunately it does not seem that we have wa waiting version of 'virsh
blockjob' that could be used to do the waiting. You can do the polling
manualy though.
The third operation can be split out by default, since you can specify
--wait instead of --pivot for 'virsh blockcommit' and it will finish
after step 2 above.
Also one of the cases which would help is that libvirt could actually
remove the file after commit, which would avoid the original problem,
since the file would still be there.
> oot@vm1 ~]# virsh blockjob serv1r2 vda
> Active Block Commit: [100 %]
>
> [root@vm1 ~]# virsh blockjob --pivot serv1r2 vda
>
> [root@vm1 ~]#
> [root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
> -rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:39 /var/lib/libvirt/images/
serv1r2.qcow2
> [root@vm1 ~]# date
> Fri Jun 30 10:39:12 EDT 2017
> [root@vm1 ~]# date
> Fri Jun 30 10:39:51 EDT 2017
> [root@vm1 ~]# ls -l /var/lib/libvirt/images/serv1r2.qcow2
> -rw-r--r--. 1 qemu qemu 54200696832 Jun 30 10:40 /var/lib/libvirt/images/
serv1r2.qcow2
> [root@vm1 ~]#
2683
days inactive
2683
days old
4 comments
3 participants
participants (3)
-
Doug Hughes -
doug.hughes＠keystonenap.com
-
Peter Krempa