[libvirt] vm snapshot multi-disk
Hi all, I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create followed by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of? In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks like it is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same. # virsh snapshot-create 7 Domain snapshot 1436792720 created # virsh snapshot-list 7 Name Creation Time State ------------------------------------------------------------ 1436792720 2015-07-13 06:05:20 -0700 running # virsh domblklist 7 Target Source ------------------------------------------------ vda /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5 vdb /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c # qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 1436792720 173M 2015-07-13 06:05:20 00:01:10.938 # qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 1436792720 0 2015-07-13 06:05:20 00:01:10.938 # virsh snapshot-delete 7 1436792720 Domain snapshot 1436792720 deleted # qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5 # qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
Oh, I almost forgot to mention the versions: libvirt 1.2.8-16.0.1.el7_1.2.x86_64 qemu 2.1.2-23.el7_1.1.x86_64 Also, I'm unclear if the domain snapshot feature is orchestrated by libvirt, or something that is simply called into qemu to take care of. Please forgive me if this is a qemu issue. On Mon, Jul 13, 2015 at 2:35 PM, Marcus <shadowsor@gmail.com> wrote:
Hi all,
I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create followed by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of?
In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks like it is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same.
# virsh snapshot-create 7
Domain snapshot 1436792720 created
# virsh snapshot-list 7
Name Creation Time State
------------------------------------------------------------
1436792720 2015-07-13 06:05:20 -0700 running
# virsh domblklist 7
Target Source
------------------------------------------------
vda /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
vdb /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 173M 2015-07-13 06:05:20 00:01:10.938
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
# virsh snapshot-delete 7 1436792720
Domain snapshot 1436792720 deleted
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
I'm guessing this is an issue with qemu, as the code indicates that libvirt relies on the qemu monitor when a vm is active, via qemuMonitorDeleteSnapshot. If I'm barking up the wrong tree, someone please let me know. Otherwise I'll follow up with qemu-devel. On Mon, Jul 13, 2015 at 2:42 PM, Marcus <shadowsor@gmail.com> wrote:
Oh, I almost forgot to mention the versions:
libvirt 1.2.8-16.0.1.el7_1.2.x86_64
qemu 2.1.2-23.el7_1.1.x86_64
Also, I'm unclear if the domain snapshot feature is orchestrated by libvirt, or something that is simply called into qemu to take care of. Please forgive me if this is a qemu issue.
On Mon, Jul 13, 2015 at 2:35 PM, Marcus <shadowsor@gmail.com> wrote:
Hi all,
I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create followed by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of?
In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks like it is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same.
# virsh snapshot-create 7
Domain snapshot 1436792720 created
# virsh snapshot-list 7
Name Creation Time State
------------------------------------------------------------
1436792720 2015-07-13 06:05:20 -0700 running
# virsh domblklist 7
Target Source
------------------------------------------------
vda /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
vdb /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 173M 2015-07-13 06:05:20 00:01:10.938
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
# virsh snapshot-delete 7 1436792720
Domain snapshot 1436792720 deleted
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a5
# qemu-img snapshot -l /mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
On 7/14/2015 at 05:42 AM, in message <CALFpzo551ioR=w5ASF7JXmtP8EgQSP2au_O6awmyofz9MfG3vg@mail.gmail.com>, Marcus <shadowsor@gmail.com> wrote: Oh, I almost forgot to mention the versions:
libvirt 1.2.8-16.0.1.el7_1.2.x86_64
qemu 2.1.2-23.el7_1.1.x86_64
Just FYI, tested on my machine, libvirt 1.2.5, qemu 2.0.0, didn't see this problem. Both disk snapshots are deleted.
Also, I'm unclear if the domain snapshot feature is orchestralted by libvirt, or something that is simply called into qemu to take care of. Please forgive me if this is a qemu issue.
On Mon, Jul 13, 2015 at 2:35 PM, Marcus <shadowsor@gmail.com> wrote:
Hi all,
I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create followed by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of?
In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks like it is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same.
# virsh snapshot-create 7
Domain snapshot 1436792720 created
# virsh snapshot-list 7
Name Creation Time State
------------------------------------------------------------
1436792720 2015-07-13 06:05:20 -0700 running
# virsh domblklist 7
Target Source
------------------------------------------------
vda
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a 5
vdb
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86 c
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a 5
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 173M 2015-07-13 06:05:20 00:01:10.938
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86 c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
# virsh snapshot-delete 7 1436792720
Domain snapshot 1436792720 deleted
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a 5
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86 c
Snapshot list:
ID TAG VM SIZE DATE VM CLOCK
1 1436792720 0 2015-07-13 06:05:20 00:01:10.938
I tracked it down. Found it was introduced in 1.7.0 and patched recently in qemu-kvm. Your version might have the fix backported, qemu-kvm-ev 2.1.2 from redhat/centos/fedora doesn't. Libvirt sends a 'delvm <snapshotid>' to the qemu monitor via HMP, and Qemu, instead of looping over all disks, repeatedly looped over the first one. Thanks for looking at it. https://bugzilla.redhat.com/show_bug.cgi?id=1243102 commit af957387547b05ed6dc4d84c10cca42700a7aeda Author: Zhang Haoyu <zhanghy@sangfor.com> Date: Mon Sep 29 16:38:02 2014 +0800 snapshot: fix referencing wrong variable in while loop in do_delvm The while loop variabal is "bs1", but "bs" is always passed to bdrv_snapshot_delete_by_id_or_name. Broken in commit a89d89d, v1.7.0. On Wed, Jul 15, 2015 at 7:26 PM, Chun Yan Liu <cyliu@suse.com> wrote:
On 7/14/2015 at 05:42 AM, in message <CALFpzo551ioR=w5ASF7JXmtP8EgQSP2au_O6awmyofz9MfG3vg@mail.gmail.com>, Marcus <shadowsor@gmail.com> wrote: Oh, I almost forgot to mention the versions:
libvirt 1.2.8-16.0.1.el7_1.2.x86_64
qemu 2.1.2-23.el7_1.1.x86_64
Just FYI, tested on my machine, libvirt 1.2.5, qemu 2.0.0, didn't see this problem. Both disk snapshots are deleted.
Also, I'm unclear if the domain snapshot feature is orchestralted by libvirt, or something that is simply called into qemu to take care of. Please forgive me if this is a qemu issue.
On Mon, Jul 13, 2015 at 2:35 PM, Marcus <shadowsor@gmail.com> wrote:
Hi all,
I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create
followed
by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of?
In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks like it is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html ), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same.
# virsh snapshot-create 7
Domain snapshot 1436792720 created
# virsh snapshot-list 7
Name Creation Time State
------------------------------------------------------------
1436792720 2015-07-13 06:05:20 -0700 running
# virsh domblklist 7
Target Source
------------------------------------------------
vda
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
vdb
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 173M 2015-07-13 06:05:20
00:01:10.938
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 0 2015-07-13 06:05:20
00:01:10.938
# virsh snapshot-delete 7 1436792720
Domain snapshot 1436792720 deleted
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 0 2015-07-13 06:05:20
00:01:10.938
well, not *that* recently, I guess... On Wed, Jul 15, 2015 at 10:51 PM, Marcus <shadowsor@gmail.com> wrote:
I tracked it down. Found it was introduced in 1.7.0 and patched recently in qemu-kvm. Your version might have the fix backported, qemu-kvm-ev 2.1.2 from redhat/centos/fedora doesn't.
Libvirt sends a 'delvm <snapshotid>' to the qemu monitor via HMP, and Qemu, instead of looping over all disks, repeatedly looped over the first one.
Thanks for looking at it.
https://bugzilla.redhat.com/show_bug.cgi?id=1243102
commit af957387547b05ed6dc4d84c10cca42700a7aeda Author: Zhang Haoyu <zhanghy@sangfor.com> Date: Mon Sep 29 16:38:02 2014 +0800
snapshot: fix referencing wrong variable in while loop in do_delvm
The while loop variabal is "bs1", but "bs" is always passed to bdrv_snapshot_delete_by_id_or_name. Broken in commit a89d89d, v1.7.0.
On Wed, Jul 15, 2015 at 7:26 PM, Chun Yan Liu <cyliu@suse.com> wrote:
On 7/14/2015 at 05:42 AM, in message <CALFpzo551ioR=w5ASF7JXmtP8EgQSP2au_O6awmyofz9MfG3vg@mail.gmail.com>, Marcus <shadowsor@gmail.com> wrote: Oh, I almost forgot to mention the versions:
libvirt 1.2.8-16.0.1.el7_1.2.x86_64
qemu 2.1.2-23.el7_1.1.x86_64
Just FYI, tested on my machine, libvirt 1.2.5, qemu 2.0.0, didn't see this problem. Both disk snapshots are deleted.
Also, I'm unclear if the domain snapshot feature is orchestralted by libvirt, or something that is simply called into qemu to take care of. Please forgive me if this is a qemu issue.
On Mon, Jul 13, 2015 at 2:35 PM, Marcus <shadowsor@gmail.com> wrote:
Hi all,
I've recently been toying with VM snapshots, and have ran into an issue. Given a VM with multiple disks, it seems a snapshot-create
by a snapshot-delete will only remove the qcow2 snapshot for the first disk (or perhaps just the disk that contains the memory), not all of the disk snapshots it created. Is this something people are aware of?
In searching around, I found a bug report where snapshot-creates would fail due to the qcow2 snapshot ids being inconsistent. That looks
followed like it
is patched for 2.4 qemu ( http://lists.nongnu.org/archive/html/qemu-devel/2015-03/msg04963.html ), this bug would trigger that one by leaving IDs around that are inconsistent between member disks, but is not the same.
# virsh snapshot-create 7
Domain snapshot 1436792720 created
# virsh snapshot-list 7
Name Creation Time State
------------------------------------------------------------
1436792720 2015-07-13 06:05:20 -0700 running
# virsh domblklist 7
Target Source
------------------------------------------------
vda
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
vdb
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 173M 2015-07-13 06:05:20
00:01:10.938
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 0 2015-07-13 06:05:20
00:01:10.938
# virsh snapshot-delete 7 1436792720
Domain snapshot 1436792720 deleted
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/e4d6e885-1382-40bc-890b-ad9c8b51a7a
5
# qemu-img snapshot -l
/mnt/2a270ef3-f389-37a4-942f-380bed9f70aa/7033e4c6-5f59-4325-b7e0-ae191e12e86
c
Snapshot list:
ID TAG VM SIZE DATE VM
CLOCK
1 1436792720 0 2015-07-13 06:05:20
00:01:10.938
participants (2)
-
Chun Yan Liu -
Marcus