12:09 p.m.
Simplify how we approach bitmap handling to increase robustness and
understanding of the code. Additionally document intricacies of how we
approach handling of bitmaps so that in cases when users decide to
do snapshots outside of libvirt they can re-create the bitmaps so that
they will still work with backups (requires new qemu-img).
I'll also follow up documenting implicationsof blockjobs on bitmaps
later.
To fetch this including a commit which allows testing:
git fetch https://gitlab.com/pipo.sk/libvirt.git checkpoint-rewrite-and-document
This doesn't require any special qemu patches.
Note that this shares many patches from my previous posting which tried
to make use of 'block-dirty-bitmap-populate', but keeps explicit bitmap
tracking for now. We'll be able to revisit use of the blockjob if the
qemu work will continue somehow.
Peter Krempa (32):
qemu: backup: Split up code traversing checkpoint list looking for
bitmaps
qemu: backup: Fix backup of disk skipped in an intermediate checkpoint
conf: backup: Store incremental backup checkpoint name per-disk
qemu: backup: Move fetching of checkpoint list for incremental backup
qemublocktest: Add 'empty' test case for bitmaps
qemublocktest: Add 'empty' case for incremental backup test
qemublocktest: Add 'empty' case for checkpoint deletion
qemublocktest: Add 'empty' case for blockcopy bitmap handling test
qemublocktest: Add 'empty' case for checkpoint bitmap handling
qemublocktest: Disable testcases for all bitmap handling
qemublocktest: Delete 'synthetic' bitmap test cases
qemublocktest: Extract printing of nodename list
qemu: checkpoint: Don't chain bitmaps for checkpoints
qemublocktest: Replace 'basic' bitmap detection test case data
qemublocktest: Replace 'snapshots' bitmap detection test case data
qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps
qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots'
cases
qemublocktest: Add new 'synthetic' bitmap detection and validation
test case
qemu: checkpoint: Don't merge checkpoints during deletion
qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to
TEST_CHECKPOINT_DELETE
qemublocktest: Re-introduce testing of checkpoint deletion
qemu: block: Add universal helper for merging dirty bitmaps for all
scenarios
qemu: backup: Rewrite backup bitmap handling to the new bitmap
semantics
qemublocktest: Add 'basic' tests for backup bitmap handling
qemublocktest: Add 'snapshots' tests for backup bitmap handling
qemu: Rewrite bitmap handling for block commit
qemublocktest: Add 'basic' tests for commit bitmap handling
qemublocktest: Add 'snapshots' tests for block commit bitmap handling
qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job
private data
qemu: Rewrite bitmap handling for block copy
qemublocktest: Add test cases for handling bitmaps during block-copy
kbase: Add document outlining internals of incremental backup in qemu
docs/kbase.html.in | 3 +
docs/kbase/incrementalbackupinternals.rst | 210 +++
src/conf/backup_conf.c | 8 +
src/conf/backup_conf.h | 1 +
src/qemu/qemu_backup.c | 199 +--
src/qemu/qemu_backup.h | 12 +-
src/qemu/qemu_block.c | 500 +++---
src/qemu/qemu_block.h | 21 +-
src/qemu/qemu_blockjob.c | 109 +-
src/qemu/qemu_blockjob.h | 3 -
src/qemu/qemu_checkpoint.c | 174 +--
src/qemu/qemu_checkpoint.h | 1 -
src/qemu/qemu_domain.c | 22 -
src/qemu/qemu_driver.c | 70 +-
tests/qemublocktest.c | 254 ++-
.../backupmerge/basic-deep-out.json | 35 +-
.../backupmerge/basic-flat-out.json | 23 +-
.../backupmerge/basic-intermediate-out.json | 23 +-
.../backupmerge/empty-out.json | 3 +
.../backupmerge/snapshot-deep-out.json | 38 -
.../backupmerge/snapshot-flat-out.json | 6 -
.../snapshot-intermediate-out.json | 14 -
.../backupmerge/snapshots-deep-out.json | 41 +
.../backupmerge/snapshots-flat-out.json | 25 +
.../snapshots-intermediate-out.json | 29 +
tests/qemublocktestdata/bitmap/basic.json | 229 +--
tests/qemublocktestdata/bitmap/basic.out | 8 +-
tests/qemublocktestdata/bitmap/empty.json | 70 +
tests/qemublocktestdata/bitmap/empty.out | 1 +
.../bitmap/snapshots-synthetic-broken.json | 837 ----------
.../bitmap/snapshots-synthetic-broken.out | 14 -
.../snapshots-synthetic-checkpoint.json | 827 ----------
.../bitmap/snapshots-synthetic-checkpoint.out | 13 -
tests/qemublocktestdata/bitmap/snapshots.json | 1382 +++++++----------
tests/qemublocktestdata/bitmap/snapshots.out | 13 +-
tests/qemublocktestdata/bitmap/synthetic.json | 606 ++++++--
tests/qemublocktestdata/bitmap/synthetic.out | 19 +-
.../bitmapblockcommit/basic-1-2 | 54 +-
.../bitmapblockcommit/basic-1-3 | 54 +-
.../bitmapblockcommit/basic-2-3 | 1 -
.../qemublocktestdata/bitmapblockcommit/empty | 1 +
.../bitmapblockcommit/snapshots-1-2 | 76 +-
.../bitmapblockcommit/snapshots-1-3 | 77 +-
.../bitmapblockcommit/snapshots-1-4 | 99 +-
.../bitmapblockcommit/snapshots-1-5 | 103 +-
.../bitmapblockcommit/snapshots-2-3 | 48 +-
.../bitmapblockcommit/snapshots-2-4 | 58 +-
.../bitmapblockcommit/snapshots-2-5 | 62 +-
.../bitmapblockcommit/snapshots-3-4 | 34 +-
.../bitmapblockcommit/snapshots-3-5 | 42 +-
.../bitmapblockcommit/snapshots-4-5 | 17 -
.../snapshots-synthetic-broken-1-2 | 57 -
.../snapshots-synthetic-broken-1-3 | 112 --
.../snapshots-synthetic-broken-1-4 | 119 --
.../snapshots-synthetic-broken-1-5 | 119 --
.../snapshots-synthetic-broken-2-3 | 89 --
.../snapshots-synthetic-broken-2-4 | 96 --
.../snapshots-synthetic-broken-2-5 | 96 --
.../snapshots-synthetic-broken-3-4 | 27 -
.../snapshots-synthetic-broken-3-5 | 27 -
.../snapshots-synthetic-broken-4-5 | 20 -
.../bitmapblockcopy/basic-deep-out.json | 53 +-
.../bitmapblockcopy/basic-shallow-out.json | 53 +-
.../bitmapblockcopy/empty-deep-out.json | 0
.../bitmapblockcopy/empty-shallow-out.json | 0
.../bitmapblockcopy/snapshots-deep-out.json | 95 +-
.../snapshots-shallow-out.json | 98 +-
.../checkpointdelete/basic-current-out.json | 20 -
.../basic-intermediate1-out.json | 13 -
.../basic-intermediate2-out.json | 13 -
.../basic-intermediate3-out.json | 13 -
.../checkpointdelete/empty-out.json | 1 +
.../snapshots-current-out.json | 20 -
.../snapshots-intermediate1-out.json | 20 +-
.../snapshots-intermediate2-out.json | 42 +-
.../snapshots-intermediate3-out.json | 43 -
.../snapshots-noparent-out.json | 15 +
...hots-synthetic-checkpoint-current-out.json | 29 -
...ynthetic-checkpoint-intermediate1-out.json | 31 -
...ynthetic-checkpoint-intermediate2-out.json | 34 -
...ynthetic-checkpoint-intermediate3-out.json | 61 -
...ots-synthetic-checkpoint-noparent-out.json | 27 -
.../synthetic-current-out.json | 9 +
.../synthetic-intermediate1-out.json | 11 +
.../synthetic-intermediate2-out.json | 11 +
.../synthetic-intermediate3-out.json | 19 +
.../synthetic-noparent-out.json | 11 +
.../blockjob-blockdev-in.xml | 4 -
88 files changed, 2927 insertions(+), 5150 deletions(-)
create mode 100644 docs/kbase/incrementalbackupinternals.rst
create mode 100644 tests/qemublocktestdata/backupmerge/empty-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-deep-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-flat-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
create mode 100644 tests/qemublocktestdata/bitmap/empty.json
create mode 100644 tests/qemublocktestdata/bitmap/empty.out
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/empty
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/empty-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json
--
2.26.2
12:09 p.m.
New subject: [PATCH 01/32] qemu: backup: Split up code traversing checkpoint list looking for bitmaps
The algorithm is getting quite complex. Split out the lookup of range of
backing chain storage sources and bitmaps contained in them which
correspond to one checkpoint.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 111 ++++++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 50 deletions(-)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index b317b841cd..69df6c14c6 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -173,72 +173,83 @@ qemuBackupDiskDataCleanup(virDomainObjPtr vm,
}
+static int
+qemuBackupGetBitmapMergeRange(virStorageSourcePtr from,
+ const char *bitmapname,
+ virJSONValuePtr *actions,
+ virStorageSourcePtr *to,
+ const char *diskdst,
+ virHashTablePtr blockNamedNodeData)
+{
+ g_autoptr(virJSONValue) ret = virJSONValueNewArray();
+ virStorageSourcePtr tmpsrc = NULL;
+ virStorageSourcePtr n;
+ bool foundbitmap = false;
+
+ for (n = from; virStorageSourceIsBacking(n); n = n->backingStore) {
+ qemuBlockNamedNodeDataBitmapPtr bitmap = NULL;
+
+ if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n,
+ bitmapname)))
+ break;
+
+ foundbitmap = true;
+
+ if (bitmap->inconsistent) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("bitmap '%s' for image '%s%u' is
inconsistent"),
+ bitmap->name, diskdst, n->id);
+ return -1;
+ }
+
+ if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(ret,
+ n->nodeformat,
+ bitmapname) < 0)
+ return -1;
+
+ tmpsrc = n;
+ }
+
+ if (!foundbitmap) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("failed to find bitmap '%s' in image
'%s%u'"),
+ bitmapname, diskdst, from->id);
+ return -1;
+ }
+
+ *actions = g_steal_pointer(&ret);
+ *to = tmpsrc;
+
+ return 0;
+}
+
+
virJSONValuePtr
qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr *incremental,
virStorageSourcePtr backingChain,
virHashTablePtr blockNamedNodeData,
const char *diskdst)
{
- qemuBlockNamedNodeDataBitmapPtr bitmap;
g_autoptr(virJSONValue) ret = NULL;
size_t incridx = 0;
+ virStorageSourcePtr n = backingChain;
ret = virJSONValueNewArray();
- if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- backingChain,
- incremental[0]->name))) {
- virReportError(VIR_ERR_INVALID_ARG,
- _("failed to find bitmap '%s' in image
'%s%u'"),
- incremental[0]->name, diskdst, backingChain->id);
- return NULL;
- }
+ for (incridx = 0; incremental[incridx]; incridx++) {
+ g_autoptr(virJSONValue) tmp = virJSONValueNewArray();
+ virStorageSourcePtr tmpsrc = NULL;
- while (bitmap) {
- if (bitmap->inconsistent) {
- virReportError(VIR_ERR_INVALID_ARG,
- _("bitmap '%s' for image '%s%u' is
inconsistent"),
- bitmap->name, diskdst, backingChain->id);
+ if (qemuBackupGetBitmapMergeRange(n, incremental[incridx]->name,
+ &tmp, &tmpsrc, diskdst,
+ blockNamedNodeData) < 0)
return NULL;
- }
- if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(ret,
-
backingChain->nodeformat,
- bitmap->name) < 0)
+ if (virJSONValueArrayConcat(ret, tmp) < 0)
return NULL;
- if (backingChain->backingStore &&
- (bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
-
backingChain->backingStore,
-
incremental[incridx]->name))) {
- backingChain = backingChain->backingStore;
- continue;
- }
-
- if (incremental[incridx + 1]) {
- if ((bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- backingChain,
- incremental[incridx +
1]->name))) {
- incridx++;
- continue;
- }
-
- if (backingChain->backingStore &&
- (bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
-
backingChain->backingStore,
- incremental[incridx +
1]->name))) {
- incridx++;
- backingChain = backingChain->backingStore;
- continue;
- }
-
- virReportError(VIR_ERR_INVALID_ARG,
- _("failed to find bitmap '%s' in image
'%s%u'"),
- incremental[incridx]->name, diskdst, backingChain->id);
- return NULL;
- } else {
- break;
- }
+ n = tmpsrc;
}
return g_steal_pointer(&ret);
--
2.26.2
1:15 p.m.
New subject: [PATCH 01/32] qemu: backup: Split up code traversing checkpoint list looking for bitmaps
On 6/15/20 12:09 PM, Peter Krempa wrote:
The algorithm is getting quite complex. Split out the lookup of range
of
backing chain storage sources and bitmaps contained in them which
correspond to one checkpoint.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 111 ++++++++++++++++++++++-------------------
1 file changed, 61 insertions(+), 50 deletions(-)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index b317b841cd..69df6c14c6 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -173,72 +173,83 @@ qemuBackupDiskDataCleanup(virDomainObjPtr vm,
}
+static int
+qemuBackupGetBitmapMergeRange(virStorageSourcePtr from,
+ const char *bitmapname,
+ virJSONValuePtr *actions,
+ virStorageSourcePtr *to,
+ const char *diskdst,
+ virHashTablePtr blockNamedNodeData)
+{
+ g_autoptr(virJSONValue) ret = virJSONValueNewArray();
Naming a pointer 'ret' when you will be returning an int seems awkward.
+ virStorageSourcePtr tmpsrc = NULL;
+ virStorageSourcePtr n;
+ bool foundbitmap = false;
+
+ for (n = from; virStorageSourceIsBacking(n); n = n->backingStore) {
+ qemuBlockNamedNodeDataBitmapPtr bitmap = NULL;
+
+ if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n,
+ bitmapname)))
+ break;
+
+ foundbitmap = true;
+
+ if (bitmap->inconsistent) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("bitmap '%s' for image '%s%u' is
inconsistent"),
+ bitmap->name, diskdst, n->id);
+ return -1;
+ }
+
+ if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(ret,
+ n->nodeformat,
+ bitmapname) < 0)
+ return -1;
+
+ tmpsrc = n;
+ }
+
+ if (!foundbitmap) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("failed to find bitmap '%s' in image
'%s%u'"),
+ bitmapname, diskdst, from->id);
+ return -1;
+ }
+
+ *actions = g_steal_pointer(&ret);
Maybe s/ret/act/.
Otherwise, this looks like a sane refactoring.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 02/32] qemu: backup: Fix backup of disk skipped in an intermediate checkpoint
If a disk is not captured by one of the intermediate checkpoints the
code would fail, but we can easily calculate the bitmaps to merge
correctly by skipping over checkpoints which don't describe the disk.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 24 ++++++++++++++++++++++++
tests/qemublocktest.c | 6 ++++++
2 files changed, 30 insertions(+)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index 69df6c14c6..686ed57e82 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -240,6 +240,30 @@ qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr
*incremental,
for (incridx = 0; incremental[incridx]; incridx++) {
g_autoptr(virJSONValue) tmp = virJSONValueNewArray();
virStorageSourcePtr tmpsrc = NULL;
+ virDomainCheckpointDefPtr chkdef = (virDomainCheckpointDefPtr)
incremental[incridx];
+ bool checkpoint_has_disk = false;
+ size_t i;
+
+ for (i = 0; i < chkdef->ndisks; i++) {
+ if (STRNEQ_NULLABLE(diskdst, chkdef->disks[i].name))
+ continue;
+
+ if (chkdef->disks[i].type == VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP)
+ checkpoint_has_disk = true;
+
+ break;
+ }
+
+ if (!checkpoint_has_disk) {
+ if (!incremental[incridx + 1]) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("disk '%s' not found in checkpoint
'%s'"),
+ diskdst, incremental[incridx]->name);
+ return NULL;
+ }
+
+ continue;
+ }
if (qemuBackupGetBitmapMergeRange(n, incremental[incridx]->name,
&tmp, &tmpsrc, diskdst,
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 0cdedb9ad4..f00d2ff129 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -727,6 +727,12 @@ testQemuBackupGetIncrementalMoment(const char *name)
if (!(checkpoint = virDomainCheckpointDefNew()))
abort();
+ checkpoint->disks = g_new0(virDomainCheckpointDiskDef, 1);
+ checkpoint->ndisks = 1;
+
+ checkpoint->disks[0].name = g_strdup("testdisk");
+ checkpoint->disks[0].type = VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP;
+
checkpoint->parent.name = g_strdup(name);
return (virDomainMomentDefPtr) checkpoint;
--
2.26.2
1:21 p.m.
New subject: [PATCH 02/32] qemu: backup: Fix backup of disk skipped in an intermediate checkpoint
On 6/15/20 12:09 PM, Peter Krempa wrote:
If a disk is not captured by one of the intermediate checkpoints the
code would fail, but we can easily calculate the bitmaps to merge
correctly by skipping over checkpoints which don't describe the disk.
If I'm understanding the setup, you can hit this by having two disks, do
a checkpoint on both A+B, then another checkpoint on just B, then
another checkpoint on both disks again; requesting a differential backup
from the first checkpoint now has to realize that no bitmap was created
for A during the second checkpoint, but that's not fatal
Visually:
time-> ----C1-------------C2-------------C3-------now
A :bitmap1 :bitmap3
B :bitmap1 :bitmap2 :bitmap3
the differential from C1-now merges bitmap1+3 for disk A, and
bitmap1+2+3 for disk B.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 24 ++++++++++++++++++++++++
tests/qemublocktest.c | 6 ++++++
2 files changed, 30 insertions(+)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index 69df6c14c6..686ed57e82 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -240,6 +240,30 @@ qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr
*incremental,
for (incridx = 0; incremental[incridx]; incridx++) {
g_autoptr(virJSONValue) tmp = virJSONValueNewArray();
virStorageSourcePtr tmpsrc = NULL;
+ virDomainCheckpointDefPtr chkdef = (virDomainCheckpointDefPtr)
incremental[incridx];
+ bool checkpoint_has_disk = false;
+ size_t i;
+
+ for (i = 0; i < chkdef->ndisks; i++) {
+ if (STRNEQ_NULLABLE(diskdst, chkdef->disks[i].name))
+ continue;
+
+ if (chkdef->disks[i].type == VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP)
+ checkpoint_has_disk = true;
+
+ break;
+ }
+
+ if (!checkpoint_has_disk) {
+ if (!incremental[incridx + 1]) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("disk '%s' not found in checkpoint
'%s'"),
+ diskdst, incremental[incridx]->name);
+ return NULL;
+ }
+
+ continue;
+ }
if (qemuBackupGetBitmapMergeRange(n, incremental[incridx]->name,
&tmp, &tmpsrc, diskdst,
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 0cdedb9ad4..f00d2ff129 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -727,6 +727,12 @@ testQemuBackupGetIncrementalMoment(const char *name)
if (!(checkpoint = virDomainCheckpointDefNew()))
abort();
+ checkpoint->disks = g_new0(virDomainCheckpointDiskDef, 1);
+ checkpoint->ndisks = 1;
+
+ checkpoint->disks[0].name = g_strdup("testdisk");
+ checkpoint->disks[0].type = VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP;
+
I'm not quite sure how this test change matches the code change above,
but the code change looked reasonable.
checkpoint->parent.name = g_strdup(name);
return (virDomainMomentDefPtr) checkpoint;
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
9:01 a.m.
New subject: [PATCH 02/32] qemu: backup: Fix backup of disk skipped in an intermediate checkpoint
On Wed, Jun 17, 2020 at 13:21:04 -0500, Eric Blake wrote:
On 6/15/20 12:09 PM, Peter Krempa wrote:
> If a disk is not captured by one of the intermediate checkpoints the
> code would fail, but we can easily calculate the bitmaps to merge
> correctly by skipping over checkpoints which don't describe the disk.
[...]
> diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
> index 0cdedb9ad4..f00d2ff129 100644
> --- a/tests/qemublocktest.c
> +++ b/tests/qemublocktest.c
> @@ -727,6 +727,12 @@ testQemuBackupGetIncrementalMoment(const char *name)
> if (!(checkpoint = virDomainCheckpointDefNew()))
> abort();
>
> + checkpoint->disks = g_new0(virDomainCheckpointDiskDef, 1);
> + checkpoint->ndisks = 1;
> +
> + checkpoint->disks[0].name = g_strdup("testdisk");
> + checkpoint->disks[0].type = VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP;
> +
I'm not quite sure how this test change matches the code change above, but
the code change looked reasonable.
This doesn't really test the change but allows previous tests to work,
by actually creating the metadata for a disk named 'testdisk' in the
fake checkpoint object which was not needed for backup before.
12:09 p.m.
New subject: [PATCH 03/32] conf: backup: Store incremental backup checkpoint name per-disk
In preparation to allow heterogenous backups store the 'incremental'
field per-disk and fill it by default from the per-backup field.
Having this will be important once we'll want to allow incremental
backup working while hotplugging a new disk.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/conf/backup_conf.c | 8 ++++++++
src/conf/backup_conf.h | 1 +
2 files changed, 9 insertions(+)
diff --git a/src/conf/backup_conf.c b/src/conf/backup_conf.c
index c0c6f25d10..92106d8aaa 100644
--- a/src/conf/backup_conf.c
+++ b/src/conf/backup_conf.c
@@ -72,6 +72,7 @@ virDomainBackupDefFree(virDomainBackupDefPtr def)
virDomainBackupDiskDefPtr disk = def->disks + i;
g_free(disk->name);
+ g_free(disk->incremental);
g_free(disk->exportname);
g_free(disk->exportbitmap);
virObjectUnref(disk->store);
@@ -505,5 +506,12 @@ virDomainBackupAlignDisks(virDomainBackupDefPtr def,
}
}
+ for (i = 0; i < def->ndisks; i++) {
+ virDomainBackupDiskDefPtr backupdisk = &def->disks[i];
+
+ if (def->incremental && !backupdisk->incremental)
+ backupdisk->incremental = g_strdup(def->incremental);
+ }
+
return 0;
}
diff --git a/src/conf/backup_conf.h b/src/conf/backup_conf.h
index b5685317c5..172eb1cf1c 100644
--- a/src/conf/backup_conf.h
+++ b/src/conf/backup_conf.h
@@ -51,6 +51,7 @@ typedef virDomainBackupDiskDef *virDomainBackupDiskDefPtr;
struct _virDomainBackupDiskDef {
char *name; /* name matching the <target dev='...' of the domain */
virTristateBool backup; /* whether backup is requested */
+ char *incremental; /* name of the starting point checkpoint of an incremental backup
*/
char *exportname; /* name of the NBD export for pull mode backup */
char *exportbitmap; /* name of the bitmap exposed in NBD for pull mode backup */
--
2.26.2
2:57 p.m.
New subject: [PATCH 03/32] conf: backup: Store incremental backup checkpoint name per-disk
On 6/15/20 12:09 PM, Peter Krempa wrote:
In preparation to allow heterogenous backups store the
'incremental'
field per-disk and fill it by default from the per-backup field.
Having this will be important once we'll want to allow incremental
backup working while hotplugging a new disk.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/conf/backup_conf.c | 8 ++++++++
src/conf/backup_conf.h | 1 +
2 files changed, 9 insertions(+)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 04/32] qemu: backup: Move fetching of checkpoint list for incremental backup
Fetch the checkpoint list for every disk specifically based on the new
per-disk 'incremental' field.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 108 ++++++++++++++++++++---------------------
1 file changed, 52 insertions(+), 56 deletions(-)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index 686ed57e82..495da86ff0 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -173,6 +173,50 @@ qemuBackupDiskDataCleanup(virDomainObjPtr vm,
}
+/**
+ * qemuBackupBeginCollectIncrementalCheckpoints:
+ * @vm: domain object
+ * @incrFrom: name of checkpoint representing starting point of incremental backup
+ *
+ * Returns a NULL terminated list of pointers to checkpoint definitions in
+ * chronological order starting from the 'current' checkpoint until reaching
+ * @incrFrom.
+ */
+static virDomainMomentDefPtr *
+qemuBackupBeginCollectIncrementalCheckpoints(virDomainObjPtr vm,
+ const char *incrFrom)
+{
+ virDomainMomentObjPtr n = virDomainCheckpointGetCurrent(vm->checkpoints);
+ g_autofree virDomainMomentDefPtr *incr = NULL;
+ size_t nincr = 0;
+
+ while (n) {
+ virDomainMomentDefPtr def = n->def;
+
+ if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
+ return NULL;
+
+ if (STREQ(def->name, incrFrom)) {
+ def = NULL;
+ if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
+ return NULL;
+
+ return g_steal_pointer(&incr);
+ }
+
+ if (!n->def->parent_name)
+ break;
+
+ n = virDomainCheckpointFindByName(vm->checkpoints,
n->def->parent_name);
+ }
+
+ virReportError(VIR_ERR_OPERATION_INVALID,
+ _("could not locate checkpoint '%s' for incremental
backup"),
+ incrFrom);
+ return NULL;
+}
+
+
static int
qemuBackupGetBitmapMergeRange(virStorageSourcePtr from,
const char *bitmapname,
@@ -334,11 +378,11 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
struct qemuBackupDiskData *dd,
virJSONValuePtr actions,
bool pull,
- virDomainMomentDefPtr *incremental,
virHashTablePtr blockNamedNodeData,
virQEMUDriverConfigPtr cfg)
{
qemuDomainObjPrivatePtr priv = vm->privateData;
+ g_autofree virDomainMomentDefPtr *incremental = NULL;
/* set data structure */
dd->backupdisk = backupdisk;
@@ -363,7 +407,7 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
* pull mode:
* both: original disk
*/
- if (pull || (incremental && dd->store->format >=
VIR_STORAGE_FILE_BACKING)) {
+ if (pull || (dd->backupdisk->incremental && dd->store->format
>= VIR_STORAGE_FILE_BACKING)) {
dd->backingStore = dd->domdisk->src;
} else {
dd->backingStore = dd->terminator = virStorageSourceNew();
@@ -375,7 +419,10 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
if (qemuDomainPrepareStorageSourceBlockdev(NULL, dd->store, priv, cfg) < 0)
return -1;
- if (incremental) {
+ if (dd->backupdisk->incremental) {
+ if (!(incremental = qemuBackupBeginCollectIncrementalCheckpoints(vm,
dd->backupdisk->incremental)))
+ return -1;
+
if (dd->backupdisk->exportbitmap)
dd->incrementalBitmap = g_strdup(dd->backupdisk->exportbitmap);
else
@@ -444,7 +491,6 @@ qemuBackupDiskPrepareDataOnePull(virJSONValuePtr actions,
static ssize_t
qemuBackupDiskPrepareData(virDomainObjPtr vm,
virDomainBackupDefPtr def,
- virDomainMomentDefPtr *incremental,
virHashTablePtr blockNamedNodeData,
virJSONValuePtr actions,
virQEMUDriverConfigPtr cfg,
@@ -467,8 +513,7 @@ qemuBackupDiskPrepareData(virDomainObjPtr vm,
ndisks++;
if (qemuBackupDiskPrepareDataOne(vm, backupdisk, dd, actions, pull,
- incremental, blockNamedNodeData,
- cfg) < 0)
+ blockNamedNodeData, cfg) < 0)
goto error;
if (pull) {
@@ -625,50 +670,6 @@ qemuBackupBeginPullExportDisks(virDomainObjPtr vm,
}
-/**
- * qemuBackupBeginCollectIncrementalCheckpoints:
- * @vm: domain object
- * @incrFrom: name of checkpoint representing starting point of incremental backup
- *
- * Returns a NULL terminated list of pointers to checkpoint definitions in
- * chronological order starting from the 'current' checkpoint until reaching
- * @incrFrom.
- */
-static virDomainMomentDefPtr *
-qemuBackupBeginCollectIncrementalCheckpoints(virDomainObjPtr vm,
- const char *incrFrom)
-{
- virDomainMomentObjPtr n = virDomainCheckpointGetCurrent(vm->checkpoints);
- g_autofree virDomainMomentDefPtr *incr = NULL;
- size_t nincr = 0;
-
- while (n) {
- virDomainMomentDefPtr def = n->def;
-
- if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
- return NULL;
-
- if (STREQ(def->name, incrFrom)) {
- def = NULL;
- if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
- return NULL;
-
- return g_steal_pointer(&incr);
- }
-
- if (!n->def->parent_name)
- break;
-
- n = virDomainCheckpointFindByName(vm->checkpoints,
n->def->parent_name);
- }
-
- virReportError(VIR_ERR_OPERATION_INVALID,
- _("could not locate checkpoint '%s' for incremental
backup"),
- incrFrom);
- return NULL;
-}
-
-
void
qemuBackupJobTerminate(virDomainObjPtr vm,
qemuDomainJobStatus jobstatus)
@@ -802,7 +803,6 @@ qemuBackupBegin(virDomainObjPtr vm,
bool pull = false;
virDomainMomentObjPtr chk = NULL;
g_autoptr(virDomainCheckpointDef) chkdef = NULL;
- g_autofree virDomainMomentDefPtr *incremental = NULL;
g_autoptr(virJSONValue) actions = NULL;
struct qemuBackupDiskData *dd = NULL;
ssize_t ndd = 0;
@@ -870,10 +870,6 @@ qemuBackupBegin(virDomainObjPtr vm,
if (virDomainBackupAlignDisks(def, vm->def, suffix) < 0)
goto endjob;
- if (def->incremental &&
- !(incremental = qemuBackupBeginCollectIncrementalCheckpoints(vm,
def->incremental)))
- goto endjob;
-
actions = virJSONValueNewArray();
/* The 'chk' checkpoint must be rolled back if the transaction command
@@ -887,7 +883,7 @@ qemuBackupBegin(virDomainObjPtr vm,
if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm, QEMU_ASYNC_JOB_BACKUP)))
goto endjob;
- if ((ndd = qemuBackupDiskPrepareData(vm, def, incremental, blockNamedNodeData,
+ if ((ndd = qemuBackupDiskPrepareData(vm, def, blockNamedNodeData,
actions, cfg, &dd)) <= 0) {
if (ndd == 0) {
virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
--
2.26.2
3:14 p.m.
New subject: [PATCH 04/32] qemu: backup: Move fetching of checkpoint list for incremental backup
On 6/15/20 12:09 PM, Peter Krempa wrote:
Fetch the checkpoint list for every disk specifically based on the
new
per-disk 'incremental' field.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 108 ++++++++++++++++++++---------------------
1 file changed, 52 insertions(+), 56 deletions(-)
@@ -363,7 +407,7 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
* pull mode:
* both: original disk
*/
- if (pull || (incremental && dd->store->format >=
VIR_STORAGE_FILE_BACKING)) {
+ if (pull || (dd->backupdisk->incremental && dd->store->format
>= VIR_STORAGE_FILE_BACKING)) {
Long line, if you want to break at the &&.
dd->backingStore = dd->domdisk->src;
} else {
dd->backingStore = dd->terminator = virStorageSourceNew();
@@ -375,7 +419,10 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
if (qemuDomainPrepareStorageSourceBlockdev(NULL, dd->store, priv, cfg) < 0)
return -1;
- if (incremental) {
+ if (dd->backupdisk->incremental) {
+ if (!(incremental = qemuBackupBeginCollectIncrementalCheckpoints(vm,
dd->backupdisk->incremental)))
Another long line, but here it is harder to suggest a short alternative.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 05/32] qemublocktest: Add 'empty' test case for bitmaps
Add test data for an image without bitmaps.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 ++
tests/qemublocktestdata/bitmap/empty.json | 70 +++++++++++++++++++++++
tests/qemublocktestdata/bitmap/empty.out | 1 +
3 files changed, 75 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmap/empty.json
create mode 100644 tests/qemublocktestdata/bitmap/empty.out
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index f00d2ff129..8d613d7cac 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1292,6 +1292,8 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_BITMAP_DETECT("empty");
+
TEST_BITMAP_DETECT("basic");
TEST_BITMAP_DETECT("synthetic");
TEST_BITMAP_DETECT("snapshots");
@@ -1360,6 +1362,8 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_BITMAP_VALIDATE("empty", "a", false);
+
TEST_BITMAP_VALIDATE("basic", "a", true);
TEST_BITMAP_VALIDATE("basic", "b", true);
TEST_BITMAP_VALIDATE("basic", "c", true);
diff --git a/tests/qemublocktestdata/bitmap/empty.json
b/tests/qemublocktestdata/bitmap/empty.json
new file mode 100644
index 0000000000..ec43b25f0d
--- /dev/null
+++ b/tests/qemublocktestdata/bitmap/empty.json
@@ -0,0 +1,70 @@
+[
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 10485760,
+ "filename": "/tmp/pull4.qcow2",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 200704,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "lazy-refcounts": false,
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/pull4.qcow2",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 197120,
+ "filename": "/tmp/pull4.qcow2",
+ "format": "file",
+ "actual-size": 200704,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/pull4.qcow2",
+ "encryption_key_missing": false
+ }
+]
diff --git a/tests/qemublocktestdata/bitmap/empty.out
b/tests/qemublocktestdata/bitmap/empty.out
new file mode 100644
index 0000000000..3787cbd354
--- /dev/null
+++ b/tests/qemublocktestdata/bitmap/empty.out
@@ -0,0 +1 @@
+libvirt-1-format:
--
2.26.2
3:17 p.m.
New subject: [PATCH 05/32] qemublocktest: Add 'empty' test case for bitmaps
On 6/15/20 12:09 PM, Peter Krempa wrote:
Add test data for an image without bitmaps.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 ++
tests/qemublocktestdata/bitmap/empty.json | 70 +++++++++++++++++++++++
tests/qemublocktestdata/bitmap/empty.out | 1 +
3 files changed, 75 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmap/empty.json
create mode 100644 tests/qemublocktestdata/bitmap/empty.out
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 06/32] qemublocktest: Add 'empty' case for incremental backup test
Use the new test data when calculating incremental backup operations. As
incremental backup fails with no bitmap the test code is modified to
allow testing this case too.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 19 ++++++++++---------
.../backupmerge/empty-out.json | 1 +
2 files changed, 11 insertions(+), 9 deletions(-)
create mode 100644 tests/qemublocktestdata/backupmerge/empty-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 8d613d7cac..9e54c254e8 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -793,17 +793,16 @@ testQemuBackupIncrementalBitmapCalculate(const void *opaque)
incremental = testQemuBackupGetIncremental(data->incremental);
- if (!(mergebitmaps = qemuBackupDiskPrepareOneBitmapsChain(incremental,
- data->chain,
- nodedata,
- "testdisk"))) {
- VIR_TEST_VERBOSE("failed to calculate merged bitmaps");
- return -1;
+ if ((mergebitmaps = qemuBackupDiskPrepareOneBitmapsChain(incremental,
+ data->chain,
+ nodedata,
+ "testdisk"))) {
+ if (!(actual = virJSONValueToString(mergebitmaps, true)))
+ return -1;
+ } else {
+ actual = g_strdup("NULL\n");
}
- if (!(actual = virJSONValueToString(mergebitmaps, true)))
- return -1;
-
return virTestCompareToFile(actual, expectpath);
}
@@ -1312,6 +1311,8 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_BACKUP_BITMAP_CALCULATE("empty", bitmapSourceChain, "a",
"empty");
+
TEST_BACKUP_BITMAP_CALCULATE("basic-flat", bitmapSourceChain,
"current", "basic");
TEST_BACKUP_BITMAP_CALCULATE("basic-intermediate", bitmapSourceChain,
"d", "basic");
TEST_BACKUP_BITMAP_CALCULATE("basic-deep", bitmapSourceChain,
"a", "basic");
diff --git a/tests/qemublocktestdata/backupmerge/empty-out.json
b/tests/qemublocktestdata/backupmerge/empty-out.json
new file mode 100644
index 0000000000..7951defec1
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/empty-out.json
@@ -0,0 +1 @@
+NULL
--
2.26.2
3:21 p.m.
New subject: [PATCH 06/32] qemublocktest: Add 'empty' case for incremental backup test
On 6/15/20 12:09 PM, Peter Krempa wrote:
Use the new test data when calculating incremental backup operations.
As
incremental backup fails with no bitmap the test code is modified to
allow testing this case too.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 19 ++++++++++---------
.../backupmerge/empty-out.json | 1 +
2 files changed, 11 insertions(+), 9 deletions(-)
create mode 100644 tests/qemublocktestdata/backupmerge/empty-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 07/32] qemublocktest: Add 'empty' case for checkpoint deletion
Use the new test data for checkpoint deletion testing. This test also
requires modification of the internals to allow checking for test
failure.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 13 +++++++------
.../checkpointdelete/empty-out.json | 1 +
2 files changed, 8 insertions(+), 6 deletions(-)
create mode 100644 tests/qemublocktestdata/checkpointdelete/empty-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 9e54c254e8..1821c227d5 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -851,14 +851,13 @@ testQemuCheckpointDeleteMerge(const void *opaque)
data->parentbitmap,
actions,
"testdisk",
- &reopenimages) < 0) {
- VIR_TEST_VERBOSE("failed to generate checkpoint delete
transaction\n");
- return -1;
+ &reopenimages) >= 0) {
+ if (virJSONValueToBuffer(actions, &buf, true) < 0)
+ return -1;
+ } else {
+ virBufferAddLit(&buf, "NULL\n");
}
- if (virJSONValueToBuffer(actions, &buf, true) < 0)
- return -1;
-
if (reopenimages) {
virBufferAddLit(&buf, "reopen nodes:\n");
@@ -1333,6 +1332,8 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_CHECKPOINT_DELETE_MERGE("empty", "a", NULL,
"empty");
+
TEST_CHECKPOINT_DELETE_MERGE("basic-noparent", "a", NULL,
"basic");
TEST_CHECKPOINT_DELETE_MERGE("basic-intermediate1", "b",
"a", "basic");
TEST_CHECKPOINT_DELETE_MERGE("basic-intermediate2", "c",
"b", "basic");
diff --git a/tests/qemublocktestdata/checkpointdelete/empty-out.json
b/tests/qemublocktestdata/checkpointdelete/empty-out.json
new file mode 100644
index 0000000000..7951defec1
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/empty-out.json
@@ -0,0 +1 @@
+NULL
--
2.26.2
3:22 p.m.
New subject: [PATCH 07/32] qemublocktest: Add 'empty' case for checkpoint deletion
On 6/15/20 12:09 PM, Peter Krempa wrote:
Use the new test data for checkpoint deletion testing. This test
also
requires modification of the internals to allow checking for test
failure.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 13 +++++++------
.../checkpointdelete/empty-out.json | 1 +
2 files changed, 8 insertions(+), 6 deletions(-)
create mode 100644 tests/qemublocktestdata/checkpointdelete/empty-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 08/32] qemublocktest: Add 'empty' case for blockcopy bitmap handling test
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 3 +++
tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json | 0
tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json | 0
3 files changed, 3 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 1821c227d5..2ddfab72eb 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1408,6 +1408,9 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_BITMAP_BLOCKCOPY("empty-shallow", true, "empty");
+ TEST_BITMAP_BLOCKCOPY("empty-deep", false, "empty");
+
TEST_BITMAP_BLOCKCOPY("basic-shallow", true, "basic");
TEST_BITMAP_BLOCKCOPY("basic-deep", false, "basic");
diff --git a/tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json
b/tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json
b/tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json
new file mode 100644
index 0000000000..e69de29bb2
--
2.26.2
3:23 p.m.
New subject: [PATCH 08/32] qemublocktest: Add 'empty' case for blockcopy bitmap handling test
On 6/15/20 12:09 PM, Peter Krempa wrote:
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 3 +++
tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json | 0
tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json | 0
3 files changed, 3 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/empty-shallow-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 09/32] qemublocktest: Add 'empty' case for checkpoint bitmap handling
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 2 ++
tests/qemublocktestdata/bitmapblockcommit/empty | 2 ++
2 files changed, 4 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/empty
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 2ddfab72eb..d66e169fd8 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1430,6 +1430,8 @@ mymain(void)
ret = -1; \
} while (0)
+ TEST_BITMAP_BLOCKCOMMIT("empty", 1, 2, "empty");
+
TEST_BITMAP_BLOCKCOMMIT("basic-1-2", 1, 2, "basic");
TEST_BITMAP_BLOCKCOMMIT("basic-1-3", 1, 3, "basic");
TEST_BITMAP_BLOCKCOMMIT("basic-2-3", 2, 3, "basic");
diff --git a/tests/qemublocktestdata/bitmapblockcommit/empty
b/tests/qemublocktestdata/bitmapblockcommit/empty
new file mode 100644
index 0000000000..bfc58f994e
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/empty
@@ -0,0 +1,2 @@
+pre job bitmap disable:
+merge bitmpas:
--
2.26.2
3:52 p.m.
New subject: [PATCH 09/32] qemublocktest: Add 'empty' case for checkpoint bitmap handling
On 6/15/20 12:09 PM, Peter Krempa wrote:
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 2 ++
tests/qemublocktestdata/bitmapblockcommit/empty | 2 ++
2 files changed, 4 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/empty
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 10/32] qemublocktest: Disable testcases for all bitmap handling
Upcoming patches are going to rewrite and semantically modify how
bitmaps are handled during blockjobs. This is possible as incremental
backup is not yet fully enabled.
As the changes are going to be incompatible with any current test data
remove all test cases for bitmap handling during checkpoint deletion,
incremental backups, block commit, block copy, and bitmap validation
operations.
The tests will be gradually added back later after the code and
test-data is refactored.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 94 -------------
.../backupmerge/basic-deep-out.json | 22 ---
.../backupmerge/basic-flat-out.json | 6 -
.../backupmerge/basic-intermediate-out.json | 10 --
.../backupmerge/snapshot-deep-out.json | 38 -----
.../backupmerge/snapshot-flat-out.json | 6 -
.../snapshot-intermediate-out.json | 14 --
.../bitmapblockcommit/basic-1-2 | 119 ----------------
.../bitmapblockcommit/basic-1-3 | 119 ----------------
.../bitmapblockcommit/basic-2-3 | 2 -
.../bitmapblockcommit/snapshots-1-2 | 49 -------
.../bitmapblockcommit/snapshots-1-3 | 76 ----------
.../bitmapblockcommit/snapshots-1-4 | 126 -----------------
.../bitmapblockcommit/snapshots-1-5 | 130 -----------------
.../bitmapblockcommit/snapshots-2-3 | 49 -------
.../bitmapblockcommit/snapshots-2-4 | 99 -------------
.../bitmapblockcommit/snapshots-2-5 | 103 --------------
.../bitmapblockcommit/snapshots-3-4 | 72 ----------
.../bitmapblockcommit/snapshots-3-5 | 76 ----------
.../bitmapblockcommit/snapshots-4-5 | 33 -----
.../snapshots-synthetic-broken-1-2 | 57 --------
.../snapshots-synthetic-broken-1-3 | 112 ---------------
.../snapshots-synthetic-broken-1-4 | 119 ----------------
.../snapshots-synthetic-broken-1-5 | 119 ----------------
.../snapshots-synthetic-broken-2-3 | 89 ------------
.../snapshots-synthetic-broken-2-4 | 96 -------------
.../snapshots-synthetic-broken-2-5 | 96 -------------
.../snapshots-synthetic-broken-3-4 | 27 ----
.../snapshots-synthetic-broken-3-5 | 27 ----
.../snapshots-synthetic-broken-4-5 | 20 ---
.../bitmapblockcopy/basic-deep-out.json | 117 ---------------
.../bitmapblockcopy/basic-shallow-out.json | 117 ---------------
.../bitmapblockcopy/snapshots-deep-out.json | 133 ------------------
.../snapshots-shallow-out.json | 48 -------
.../checkpointdelete/basic-current-out.json | 29 ----
.../basic-intermediate1-out.json | 22 ---
.../basic-intermediate2-out.json | 22 ---
.../basic-intermediate3-out.json | 22 ---
.../checkpointdelete/basic-noparent-out.json | 9 --
.../snapshots-current-out.json | 29 ----
.../snapshots-intermediate1-out.json | 24 ----
.../snapshots-intermediate2-out.json | 62 --------
.../snapshots-intermediate3-out.json | 61 --------
.../snapshots-noparent-out.json | 27 ----
...hots-synthetic-checkpoint-current-out.json | 29 ----
...ynthetic-checkpoint-intermediate1-out.json | 31 ----
...ynthetic-checkpoint-intermediate2-out.json | 34 -----
...ynthetic-checkpoint-intermediate3-out.json | 61 --------
...ots-synthetic-checkpoint-noparent-out.json | 27 ----
49 files changed, 2909 deletions(-)
delete mode 100644 tests/qemublocktestdata/backupmerge/basic-deep-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/basic-flat-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-deep-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-flat-out.json
delete mode 100644 tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-2
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-3
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-2-3
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
delete mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
delete mode 100644
tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
delete mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
delete mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
delete mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
delete mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/basic-current-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
delete mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json
delete mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index d66e169fd8..d2f105486e 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1312,14 +1312,6 @@ mymain(void)
TEST_BACKUP_BITMAP_CALCULATE("empty", bitmapSourceChain, "a",
"empty");
- TEST_BACKUP_BITMAP_CALCULATE("basic-flat", bitmapSourceChain,
"current", "basic");
- TEST_BACKUP_BITMAP_CALCULATE("basic-intermediate", bitmapSourceChain,
"d", "basic");
- TEST_BACKUP_BITMAP_CALCULATE("basic-deep", bitmapSourceChain,
"a", "basic");
-
- TEST_BACKUP_BITMAP_CALCULATE("snapshot-flat", bitmapSourceChain,
"current", "snapshots");
- TEST_BACKUP_BITMAP_CALCULATE("snapshot-intermediate", bitmapSourceChain,
"d", "snapshots");
- TEST_BACKUP_BITMAP_CALCULATE("snapshot-deep", bitmapSourceChain,
"a", "snapshots");
-
#define TEST_CHECKPOINT_DELETE_MERGE(testname, delbmp, parbmp, named) \
do { \
checkpointdeletedata.name = testname; \
@@ -1334,24 +1326,6 @@ mymain(void)
TEST_CHECKPOINT_DELETE_MERGE("empty", "a", NULL,
"empty");
- TEST_CHECKPOINT_DELETE_MERGE("basic-noparent", "a", NULL,
"basic");
- TEST_CHECKPOINT_DELETE_MERGE("basic-intermediate1", "b",
"a", "basic");
- TEST_CHECKPOINT_DELETE_MERGE("basic-intermediate2", "c",
"b", "basic");
- TEST_CHECKPOINT_DELETE_MERGE("basic-intermediate3", "d",
"c", "basic");
- TEST_CHECKPOINT_DELETE_MERGE("basic-current", "current",
"d", "basic");
-
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-noparent", "a", NULL,
"snapshots");
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-intermediate1", "b",
"a", "snapshots");
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-intermediate2", "c",
"b", "snapshots");
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-intermediate3", "d",
"c", "snapshots");
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-current", "current",
"d", "snapshots");
-
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-synthetic-checkpoint-noparent",
"a", NULL, "snapshots-synthetic-checkpoint");
-
TEST_CHECKPOINT_DELETE_MERGE("snapshots-synthetic-checkpoint-intermediate1",
"b", "a", "snapshots-synthetic-checkpoint");
-
TEST_CHECKPOINT_DELETE_MERGE("snapshots-synthetic-checkpoint-intermediate2",
"c", "b", "snapshots-synthetic-checkpoint");
-
TEST_CHECKPOINT_DELETE_MERGE("snapshots-synthetic-checkpoint-intermediate3",
"d", "c", "snapshots-synthetic-checkpoint");
- TEST_CHECKPOINT_DELETE_MERGE("snapshots-synthetic-checkpoint-current",
"current", "d", "snapshots-synthetic-checkpoint");
-
#define TEST_BITMAP_VALIDATE(testname, bitmap, rc) \
do { \
blockbitmapvalidatedata.name = testname; \
@@ -1366,36 +1340,6 @@ mymain(void)
TEST_BITMAP_VALIDATE("empty", "a", false);
- TEST_BITMAP_VALIDATE("basic", "a", true);
- TEST_BITMAP_VALIDATE("basic", "b", true);
- TEST_BITMAP_VALIDATE("basic", "c", true);
- TEST_BITMAP_VALIDATE("basic", "d", true);
- TEST_BITMAP_VALIDATE("basic", "current", true);
-
- TEST_BITMAP_VALIDATE("snapshots", "a", true);
- TEST_BITMAP_VALIDATE("snapshots", "b", true);
- TEST_BITMAP_VALIDATE("snapshots", "c", true);
- TEST_BITMAP_VALIDATE("snapshots", "d", true);
- TEST_BITMAP_VALIDATE("snapshots", "current", true);
-
- TEST_BITMAP_VALIDATE("synthetic", "a", false);
- TEST_BITMAP_VALIDATE("synthetic", "b", true);
- TEST_BITMAP_VALIDATE("synthetic", "c", true);
- TEST_BITMAP_VALIDATE("synthetic", "d", true);
- TEST_BITMAP_VALIDATE("synthetic", "current", true);
-
- TEST_BITMAP_VALIDATE("snapshots-synthetic-checkpoint", "a",
true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-checkpoint", "b",
true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-checkpoint", "c",
true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-checkpoint", "d",
true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-checkpoint", "current",
true);
-
- TEST_BITMAP_VALIDATE("snapshots-synthetic-broken", "a", false);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-broken", "b", true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-broken", "c", true);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-broken", "d", false);
- TEST_BITMAP_VALIDATE("snapshots-synthetic-broken", "current",
true);
-
#define TEST_BITMAP_BLOCKCOPY(testname, shllw, ndf) \
do { \
blockbitmapblockcopydata.name = testname; \
@@ -1411,12 +1355,6 @@ mymain(void)
TEST_BITMAP_BLOCKCOPY("empty-shallow", true, "empty");
TEST_BITMAP_BLOCKCOPY("empty-deep", false, "empty");
- TEST_BITMAP_BLOCKCOPY("basic-shallow", true, "basic");
- TEST_BITMAP_BLOCKCOPY("basic-deep", false, "basic");
-
- TEST_BITMAP_BLOCKCOPY("snapshots-shallow", true, "snapshots");
- TEST_BITMAP_BLOCKCOPY("snapshots-deep", false, "snapshots");
-
#define TEST_BITMAP_BLOCKCOMMIT(testname, topimg, baseimg, ndf) \
do {\
@@ -1432,38 +1370,6 @@ mymain(void)
TEST_BITMAP_BLOCKCOMMIT("empty", 1, 2, "empty");
- TEST_BITMAP_BLOCKCOMMIT("basic-1-2", 1, 2, "basic");
- TEST_BITMAP_BLOCKCOMMIT("basic-1-3", 1, 3, "basic");
- TEST_BITMAP_BLOCKCOMMIT("basic-2-3", 2, 3, "basic");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-1-2", 1, 2, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-1-3", 1, 3, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-1-4", 1, 4, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-1-5", 1, 5, "snapshots");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-2-3", 2, 3, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-2-4", 2, 4, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-2-5", 2, 5, "snapshots");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-3-4", 3, 4, "snapshots");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-3-5", 3, 5, "snapshots");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-4-5", 4, 5, "snapshots");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-1-2", 1, 2,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-1-3", 1, 3,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-1-4", 1, 4,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-1-5", 1, 5,
"snapshots-synthetic-broken");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-2-3", 2, 3,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-2-4", 2, 4,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-2-5", 2, 5,
"snapshots-synthetic-broken");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-3-4", 3, 4,
"snapshots-synthetic-broken");
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-3-5", 3, 5,
"snapshots-synthetic-broken");
-
- TEST_BITMAP_BLOCKCOMMIT("snapshots-synthetic-broken-4-5", 4, 5,
"snapshots-synthetic-broken");
-
cleanup:
qemuTestDriverFree(&driver);
VIR_FREE(capslatest_x86_64);
diff --git a/tests/qemublocktestdata/backupmerge/basic-deep-out.json
b/tests/qemublocktestdata/backupmerge/basic-deep-out.json
deleted file mode 100644
index 28c3d16259..0000000000
--- a/tests/qemublocktestdata/backupmerge/basic-deep-out.json
+++ /dev/null
@@ -1,22 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- },
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-1-format",
- "name": "c"
- },
- {
- "node": "libvirt-1-format",
- "name": "b"
- },
- {
- "node": "libvirt-1-format",
- "name": "a"
- }
-]
diff --git a/tests/qemublocktestdata/backupmerge/basic-flat-out.json
b/tests/qemublocktestdata/backupmerge/basic-flat-out.json
deleted file mode 100644
index b89252e284..0000000000
--- a/tests/qemublocktestdata/backupmerge/basic-flat-out.json
+++ /dev/null
@@ -1,6 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
-]
diff --git a/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
b/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
deleted file mode 100644
index 0dffcafd5f..0000000000
--- a/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
+++ /dev/null
@@ -1,10 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- },
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
-]
diff --git a/tests/qemublocktestdata/backupmerge/snapshot-deep-out.json
b/tests/qemublocktestdata/backupmerge/snapshot-deep-out.json
deleted file mode 100644
index 526fc8d55b..0000000000
--- a/tests/qemublocktestdata/backupmerge/snapshot-deep-out.json
+++ /dev/null
@@ -1,38 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- },
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "b"
- },
- {
- "node": "libvirt-3-format",
- "name": "a"
- },
- {
- "node": "libvirt-4-format",
- "name": "a"
- },
- {
- "node": "libvirt-5-format",
- "name": "a"
- }
-]
diff --git a/tests/qemublocktestdata/backupmerge/snapshot-flat-out.json
b/tests/qemublocktestdata/backupmerge/snapshot-flat-out.json
deleted file mode 100644
index b89252e284..0000000000
--- a/tests/qemublocktestdata/backupmerge/snapshot-flat-out.json
+++ /dev/null
@@ -1,6 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
-]
diff --git a/tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json
b/tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json
deleted file mode 100644
index 537d776ec6..0000000000
--- a/tests/qemublocktestdata/backupmerge/snapshot-intermediate-out.json
+++ /dev/null
@@ -1,14 +0,0 @@
-[
- {
- "node": "libvirt-1-format",
- "name": "current"
- },
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
b/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
deleted file mode 100644
index 8eeb4c3a11..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
+++ /dev/null
@@ -1,119 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "a",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
b/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
deleted file mode 100644
index 71b48e31a5..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
+++ /dev/null
@@ -1,119 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "a",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
b/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
deleted file mode 100644
index bfc58f994e..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
+++ /dev/null
@@ -1,2 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
deleted file mode 100644
index 0015b9ceb3..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
+++ /dev/null
@@ -1,49 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-2-format",
- "name": "d"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
deleted file mode 100644
index 5691b408aa..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
+++ /dev/null
@@ -1,76 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-3-format",
- "name": "c"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
deleted file mode 100644
index 454001531a..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
+++ /dev/null
@@ -1,126 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-4-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
deleted file mode 100644
index 2fd43d7917..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
+++ /dev/null
@@ -1,130 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- },
- {
- "node": "libvirt-4-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
deleted file mode 100644
index d719a90bd7..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
+++ /dev/null
@@ -1,49 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-3-format",
- "name": "c"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "d",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
deleted file mode 100644
index 9e37962344..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
+++ /dev/null
@@ -1,99 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-4-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "d",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
deleted file mode 100644
index d6b20a5d05..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
+++ /dev/null
@@ -1,103 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- },
- {
- "node": "libvirt-4-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "d",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
deleted file mode 100644
index b96e8910d7..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
+++ /dev/null
@@ -1,72 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-4-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "c",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
deleted file mode 100644
index 9570c34c40..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
+++ /dev/null
@@ -1,76 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- },
- {
- "node": "libvirt-4-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "c",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
deleted file mode 100644
index 7e1020d96e..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
+++ /dev/null
@@ -1,33 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-4-format",
- "name": "a"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
deleted file mode 100644
index 463120d442..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-2
+++ /dev/null
@@ -1,57 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- },
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- },
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
deleted file mode 100644
index fec6f95dd1..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-3
+++ /dev/null
@@ -1,112 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
deleted file mode 100644
index 697230f67b..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-4
+++ /dev/null
@@ -1,119 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
deleted file mode 100644
index 6bf1f0da64..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-1-5
+++ /dev/null
@@ -1,119 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
deleted file mode 100644
index f202bb94b1..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-3
+++ /dev/null
@@ -1,89 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-3-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
deleted file mode 100644
index 864cc9041b..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-4
+++ /dev/null
@@ -1,96 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
deleted file mode 100644
index 4c5d8dbe80..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-2-5
+++ /dev/null
@@ -1,96 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "oa",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "oa",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "oa"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "ob",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "ob",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "ob"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
deleted file mode 100644
index 367a930a74..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-4
+++ /dev/null
@@ -1,27 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-4-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-4-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
deleted file mode 100644
index 0062ec140c..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-3-5
+++ /dev/null
@@ -1,27 +0,0 @@
-pre job bitmap disable:
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-5-format",
- "name": "b",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-5-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
deleted file mode 100644
index b1f10a8a24..0000000000
--- a/tests/qemublocktestdata/bitmapblockcommit/snapshots-synthetic-broken-4-5
+++ /dev/null
@@ -1,20 +0,0 @@
-pre job bitmap disable:
-[
- {
- "type": "block-dirty-bitmap-disable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-merge bitmpas:
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
b/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
deleted file mode 100644
index 4ed2b97e95..0000000000
--- a/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
+++ /dev/null
@@ -1,117 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "a",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
b/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
deleted file mode 100644
index 4ed2b97e95..0000000000
--- a/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
+++ /dev/null
@@ -1,117 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "a",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
b/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
deleted file mode 100644
index 5456553d78..0000000000
--- a/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
+++ /dev/null
@@ -1,133 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "a",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "a"
- },
- {
- "node": "libvirt-4-format",
- "name": "a"
- },
- {
- "node": "libvirt-5-format",
- "name": "a"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- },
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- },
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
b/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
deleted file mode 100644
index ddd47f7ee1..0000000000
--- a/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
+++ /dev/null
@@ -1,48 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "current",
- "persistent": true,
- "disabled": false,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "current",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "mirror-format-node",
- "name": "d",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "mirror-format-node",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
deleted file mode 100644
index 1b607567e8..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
+++ /dev/null
@@ -1,29 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "current"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
deleted file mode 100644
index eccb7ed15f..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
+++ /dev/null
@@ -1,22 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "b"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
deleted file mode 100644
index de40e4b5b0..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
+++ /dev/null
@@ -1,22 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "c"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
deleted file mode 100644
index b5d85f43f0..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
+++ /dev/null
@@ -1,22 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
deleted file mode 100644
index e87382fdb4..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
+++ /dev/null
@@ -1,9 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "a"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
deleted file mode 100644
index 1b607567e8..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
+++ /dev/null
@@ -1,29 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "current"
- }
- }
-]
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
deleted file mode 100644
index c9bda3a17a..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
+++ /dev/null
@@ -1,24 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- }
-]
-reopen nodes:
-libvirt-3-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
deleted file mode 100644
index 8a0e3f2cff..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
+++ /dev/null
@@ -1,62 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-2-format",
- "name": "c"
- }
- },
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-3-format",
- "name": "c"
- }
- }
-]
-reopen nodes:
-libvirt-3-format
-libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
deleted file mode 100644
index 211bc40baf..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
+++ /dev/null
@@ -1,61 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-1-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- },
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-2-format",
- "name": "c"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-2-format",
- "name": "d"
- }
- }
-]
-reopen nodes:
-libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
deleted file mode 100644
index f750f44da2..0000000000
--- a/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
+++ /dev/null
@@ -1,27 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-3-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-4-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-reopen nodes:
-libvirt-5-format
-libvirt-4-format
-libvirt-3-format
diff --git
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json
deleted file mode 100644
index 1b607567e8..0000000000
---
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-current-out.json
+++ /dev/null
@@ -1,29 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "d",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "current"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "current"
- }
- }
-]
diff --git
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json
deleted file mode 100644
index d7e6d18637..0000000000
---
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate1-out.json
+++ /dev/null
@@ -1,31 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-3-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-3-format",
- "target": "a",
- "bitmaps": [
- {
- "node": "libvirt-3-format",
- "name": "b"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-3-format",
- "name": "b"
- }
- }
-]
-reopen nodes:
-libvirt-3-format
diff --git
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json
deleted file mode 100644
index cfbff010c2..0000000000
---
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate2-out.json
+++ /dev/null
@@ -1,34 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-2-format",
- "name": "b",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "b",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "c"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-2-format",
- "name": "c"
- }
- }
-]
-reopen nodes:
-libvirt-2-format
diff --git
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json
deleted file mode 100644
index 211bc40baf..0000000000
---
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-intermediate3-out.json
+++ /dev/null
@@ -1,61 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-add",
- "data": {
- "node": "libvirt-1-format",
- "name": "c",
- "persistent": true,
- "disabled": true,
- "granularity": 65536
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-1-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-1-format",
- "name": "d"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-1-format",
- "name": "d"
- }
- },
- {
- "type": "block-dirty-bitmap-enable",
- "data": {
- "node": "libvirt-2-format",
- "name": "c"
- }
- },
- {
- "type": "block-dirty-bitmap-merge",
- "data": {
- "node": "libvirt-2-format",
- "target": "c",
- "bitmaps": [
- {
- "node": "libvirt-2-format",
- "name": "d"
- }
- ]
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-2-format",
- "name": "d"
- }
- }
-]
-reopen nodes:
-libvirt-2-format
diff --git
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json
deleted file mode 100644
index f750f44da2..0000000000
---
a/tests/qemublocktestdata/checkpointdelete/snapshots-synthetic-checkpoint-noparent-out.json
+++ /dev/null
@@ -1,27 +0,0 @@
-[
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-3-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-4-format",
- "name": "a"
- }
- },
- {
- "type": "block-dirty-bitmap-remove",
- "data": {
- "node": "libvirt-5-format",
- "name": "a"
- }
- }
-]
-reopen nodes:
-libvirt-5-format
-libvirt-4-format
-libvirt-3-format
--
2.26.2
3:58 p.m.
New subject: [PATCH 10/32] qemublocktest: Disable testcases for all bitmap handling
On 6/15/20 12:09 PM, Peter Krempa wrote:
Upcoming patches are going to rewrite and semantically modify how
bitmaps are handled during blockjobs. This is possible as incremental
backup is not yet fully enabled.
As the changes are going to be incompatible with any current test data
remove all test cases for bitmap handling during checkpoint deletion,
incremental backups, block commit, block copy, and bitmap validation
operations.
The tests will be gradually added back later after the code and
test-data is refactored.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
49 files changed, 2909 deletions(-)
And this is the point where we have to commit on which interface we will
stick to once we allow incremental backups by default rather than just
by opting in. With your idea of keeping multiple bitmaps active, we may
need to tweak qemu to be more efficient at how it stores bitmap
information (for example, instead of having each guest write modify the
in-memory representation of all active bitmaps, which only get flushed
to disk when closing the guest, we could instead have qemu store guest
writes into a single in-memory bitmap, then merge that into all active
bitmaps when flushing) - but that's for qemu to worry about. In the
meantime, if your n ew proposed bitmap handling is easier to manage in
libvirt, even if it costs more work in current qemu, I think we can live
with that.
I'm reluctant to ack this patch until I've seen the rest of the series
adding support back in, but unless something major turns up in the rest
of the series, the approach of a clean slate followed by a new
implementation is reasonable enough.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 11/32] qemublocktest: Delete 'synthetic' bitmap test cases
They will be replaced by a different set which will test scenarios
relevant for the new semantics.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 3 -
.../bitmap/snapshots-synthetic-broken.json | 837 ------------------
.../bitmap/snapshots-synthetic-broken.out | 14 -
.../snapshots-synthetic-checkpoint.json | 827 -----------------
.../bitmap/snapshots-synthetic-checkpoint.out | 13 -
tests/qemublocktestdata/bitmap/synthetic.json | 118 ---
tests/qemublocktestdata/bitmap/synthetic.out | 6 -
7 files changed, 1818 deletions(-)
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json
delete mode 100644 tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out
delete mode 100644 tests/qemublocktestdata/bitmap/synthetic.json
delete mode 100644 tests/qemublocktestdata/bitmap/synthetic.out
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index d2f105486e..28cbaeba38 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1293,10 +1293,7 @@ mymain(void)
TEST_BITMAP_DETECT("empty");
TEST_BITMAP_DETECT("basic");
- TEST_BITMAP_DETECT("synthetic");
TEST_BITMAP_DETECT("snapshots");
- TEST_BITMAP_DETECT("snapshots-synthetic-checkpoint");
- TEST_BITMAP_DETECT("snapshots-synthetic-broken");
#define TEST_BACKUP_BITMAP_CALCULATE(testname, source, incrbackup, named) \
do { \
diff --git a/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json
b/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json
deleted file mode 100644
index 8cf14d4baa..0000000000
--- a/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.json
+++ /dev/null
@@ -1,837 +0,0 @@
-[
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename":
"/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911550",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "in-use",
- "auto"
- ],
- "name": "current",
- "granularity": 65536
- },
- {
- "flags": [
- "in-use"
- ],
- "name": "d",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911540",
- "backing-filename": "/tmp/pull4.1575911540",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-format",
- "backing_file_depth": 4,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911540",
- "dirty-bitmaps": [
- {
- "name": "d",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911550",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-2-format",
- "backing_file_depth": 3,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911527",
- "dirty-bitmaps": [
- {
- "name": "c",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "oa",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "ob",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "d",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "inconsistent": true,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911540",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-2-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-3-format",
- "backing_file_depth": 2,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911522",
- "dirty-bitmaps": [
- {
- "name": "a",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "b",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 459264,
- "filename": "/tmp/pull4.1575911527",
- "format": "file",
- "actual-size": 217088,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-3-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-4-format",
- "backing_file_depth": 1,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.qcow2",
- "dirty-bitmaps": [
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.1575911522",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-4-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-5-format",
- "backing_file_depth": 0,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "dirty-bitmaps": [
- {
- "name": "a",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.qcow2",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-5-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- }
-]
diff --git a/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out
b/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out
deleted file mode 100644
index ad24a580f1..0000000000
--- a/tests/qemublocktestdata/bitmap/snapshots-synthetic-broken.out
+++ /dev/null
@@ -1,14 +0,0 @@
-libvirt-1-format:
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-2-format:
- c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- oa: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- ob: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- d: record:1 busy:0 persist:1 inconsist:1 gran:65536 dirty:0
-libvirt-3-format:
- a: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-4-format:
-libvirt-5-format:
- a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
diff --git a/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json
b/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json
deleted file mode 100644
index 25cc150d67..0000000000
--- a/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.json
+++ /dev/null
@@ -1,827 +0,0 @@
-[
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename":
"/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911550",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "in-use",
- "auto"
- ],
- "name": "current",
- "granularity": 65536
- },
- {
- "flags": [
- "in-use"
- ],
- "name": "d",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911540",
- "backing-filename": "/tmp/pull4.1575911540",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-format",
- "backing_file_depth": 4,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911540",
- "dirty-bitmaps": [
- {
- "name": "d",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911550",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-2-format",
- "backing_file_depth": 3,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911527",
- "dirty-bitmaps": [
- {
- "name": "c",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "d",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911540",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-2-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-3-format",
- "backing_file_depth": 2,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911522",
- "dirty-bitmaps": [
- {
- "name": "a",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "b",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 459264,
- "filename": "/tmp/pull4.1575911527",
- "format": "file",
- "actual-size": 217088,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-3-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-4-format",
- "backing_file_depth": 1,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.qcow2",
- "dirty-bitmaps": [
- {
- "name": "a",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.1575911522",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-4-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-5-format",
- "backing_file_depth": 0,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "dirty-bitmaps": [
- {
- "name": "a",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.qcow2",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-5-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- }
-]
diff --git a/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out
b/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out
deleted file mode 100644
index 0270657001..0000000000
--- a/tests/qemublocktestdata/bitmap/snapshots-synthetic-checkpoint.out
+++ /dev/null
@@ -1,13 +0,0 @@
-libvirt-1-format:
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-2-format:
- c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-3-format:
- a: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-4-format:
- a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
-libvirt-5-format:
- a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
diff --git a/tests/qemublocktestdata/bitmap/synthetic.json
b/tests/qemublocktestdata/bitmap/synthetic.json
deleted file mode 100644
index 56882bd615..0000000000
--- a/tests/qemublocktestdata/bitmap/synthetic.json
+++ /dev/null
@@ -1,118 +0,0 @@
-[
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 200704,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-format",
- "backing_file_depth": 0,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "dirty-bitmaps": [
- {
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "d",
- "recording": false,
- "persistent": false,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "c",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 1234,
- "count": 0
- },
- {
- "name": "b",
- "recording": false,
- "persistent": true,
- "busy": true,
- "status": "disabled",
- "granularity": 65536,
- "count": 21314
- },
- {
- "name": "a",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "inconsistent": true,
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- },
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 197120,
- "filename": "/tmp/pull4.qcow2",
- "format": "file",
- "actual-size": 200704,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- }
-]
diff --git a/tests/qemublocktestdata/bitmap/synthetic.out
b/tests/qemublocktestdata/bitmap/synthetic.out
deleted file mode 100644
index e8041c5b77..0000000000
--- a/tests/qemublocktestdata/bitmap/synthetic.out
+++ /dev/null
@@ -1,6 +0,0 @@
-libvirt-1-format:
- current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- d: record:0 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
- c: record:0 busy:0 persist:1 inconsist:0 gran:1234 dirty:0
- b: record:0 busy:1 persist:1 inconsist:0 gran:65536 dirty:21314
- a: record:0 busy:0 persist:1 inconsist:1 gran:65536 dirty:0
--
2.26.2
4:36 p.m.
New subject: [PATCH 11/32] qemublocktest: Delete 'synthetic' bitmap test cases
On 6/15/20 12:09 PM, Peter Krempa wrote:
They will be replaced by a different set which will test scenarios
relevant for the new semantics.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 3 -
.../bitmap/snapshots-synthetic-broken.json | 837 ------------------
.../bitmap/snapshots-synthetic-broken.out | 14 -
.../snapshots-synthetic-checkpoint.json | 827 -----------------
.../bitmap/snapshots-synthetic-checkpoint.out | 13 -
tests/qemublocktestdata/bitmap/synthetic.json | 118 ---
tests/qemublocktestdata/bitmap/synthetic.out | 6 -
7 files changed, 1818 deletions(-)
Same comment as for the previous patch - if we get to the end of the
series without major issues, then this temporary deletion to make the
rewrite easier is fine.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:09 p.m.
New subject: [PATCH 12/32] qemublocktest: Extract printing of nodename list
There will be multiple places where we'll need to print nodenames from a
GSList of virStorageSource for testing purposes. Extract the code into a
function.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 27 ++++++++++++++++++---------
1 file changed, 18 insertions(+), 9 deletions(-)
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 28cbaeba38..0ed9b99bc4 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -647,6 +647,23 @@ testQemuDetectBitmaps(const void *opaque)
}
+static void
+testQemuBitmapListPrint(const char *title,
+ GSList *next,
+ virBufferPtr buf)
+{
+ if (!next)
+ return;
+
+ virBufferAsprintf(buf, "%s\n", title);
+
+ for (; next; next = next->next) {
+ virStorageSourcePtr src = next->data;
+ virBufferAsprintf(buf, "%s\n", src->nodeformat);
+ }
+}
+
+
static virStorageSourcePtr
testQemuBackupIncrementalBitmapCalculateGetFakeImage(size_t idx)
{
@@ -829,7 +846,6 @@ testQemuCheckpointDeleteMerge(const void *opaque)
g_autoptr(virHashTable) nodedata = NULL;
g_autoptr(GSList) reopenimages = NULL;
g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
- GSList *tmp;
expectpath = g_strdup_printf("%s/%s%s-out.json", abs_srcdir,
checkpointDeletePrefix, data->name);
@@ -858,14 +874,7 @@ testQemuCheckpointDeleteMerge(const void *opaque)
virBufferAddLit(&buf, "NULL\n");
}
- if (reopenimages) {
- virBufferAddLit(&buf, "reopen nodes:\n");
-
- for (tmp = reopenimages; tmp; tmp = tmp->next) {
- virStorageSourcePtr src = tmp->data;
- virBufferAsprintf(&buf, "%s\n", src->nodeformat);
- }
- }
+ testQemuBitmapListPrint("reopen nodes:", reopenimages, &buf);
actual = virBufferContentAndReset(&buf);
--
2.26.2
4:41 p.m.
New subject: [PATCH 12/32] qemublocktest: Extract printing of nodename list
On 6/15/20 12:09 PM, Peter Krempa wrote:
There will be multiple places where we'll need to print nodenames
from a
GSList of virStorageSource for testing purposes. Extract the code into a
function.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 27 ++++++++++++++++++---------
1 file changed, 18 insertions(+), 9 deletions(-)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 13/32] qemu: checkpoint: Don't chain bitmaps for checkpoints
Chaining bitmaps for checkpoints (disabling the active one and creating
a new) severely overcomplicated all operations in regards to bitmaps.
Specifically it requires us re-matching the on-disk state to the
internal metadata and in case of merging during block jobs it makes it
almost impossible to cover all corner cases.
Since the checkpoints and incremental backups were not yet enabled,
let's change the design to keep one bitmap per checkpoint. In case of
layered snapshots this will be filled in by using dirty-bitmap-populate.
Finally the main reason for this unnecessary complexity was the fear
that qemu's performance could degrade. In the end I think that
addressing the performance issue will be better done in qemu (e.g by
keeping an internal bitmap updated with changes and merging it
periodically back to the real bitmaps. QEMU writes out changes to disk
at shutdown so consistency is not a problem).
Removing the relationships between bitmaps frees us from complex
handling and also makes all the surrounding code more robust as one
broken bitmap doesn't necessarily invalidate whole chains of backups.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_checkpoint.c | 21 +--------------------
1 file changed, 1 insertion(+), 20 deletions(-)
diff --git a/src/qemu/qemu_checkpoint.c b/src/qemu/qemu_checkpoint.c
index b71b4a7d14..9b4f3ad396 100644
--- a/src/qemu/qemu_checkpoint.c
+++ b/src/qemu/qemu_checkpoint.c
@@ -456,7 +456,6 @@ qemuCheckpointPrepare(virQEMUDriverPtr driver,
static int
qemuCheckpointAddActions(virDomainObjPtr vm,
virJSONValuePtr actions,
- virDomainMomentObjPtr old_current,
virDomainCheckpointDefPtr def)
{
size_t i;
@@ -464,7 +463,6 @@ qemuCheckpointAddActions(virDomainObjPtr vm,
for (i = 0; i < def->ndisks; i++) {
virDomainCheckpointDiskDef *chkdisk = &def->disks[i];
virDomainDiskDefPtr domdisk = virDomainDiskByTarget(vm->def,
chkdisk->name);
- virDomainCheckpointDiskDef *parentchkdisk = NULL;
/* checkpoint definition validator mandates that the corresponding
* domdisk should exist */
@@ -475,23 +473,6 @@ qemuCheckpointAddActions(virDomainObjPtr vm,
if (qemuMonitorTransactionBitmapAdd(actions, domdisk->src->nodeformat,
chkdisk->bitmap, true, false, 0) < 0)
return -1;
-
- /* We only want one active bitmap for a disk along the
- * checkpoint chain, then later differential backups will
- * merge the bitmaps (only one active) between the bounding
- * checkpoint and the leaf checkpoint. If the same disks are
- * involved in each checkpoint, this search terminates in one
- * iteration; but it is also possible to have to search
- * further than the immediate parent to find another
- * checkpoint with a bitmap on the same disk. */
- if ((parentchkdisk = qemuCheckpointFindActiveDiskInParent(vm, old_current,
- chkdisk->name))) {
-
- if (qemuMonitorTransactionBitmapDisable(actions,
- domdisk->src->nodeformat,
- parentchkdisk->bitmap) < 0)
- return -1;
- }
}
return 0;
}
@@ -540,7 +521,7 @@ qemuCheckpointCreateCommon(virQEMUDriverPtr driver,
tmpactions = virJSONValueNewArray();
- if (qemuCheckpointAddActions(vm, tmpactions, parent, *def) < 0)
+ if (qemuCheckpointAddActions(vm, tmpactions, *def) < 0)
return -1;
if (!(*chk = virDomainCheckpointAssignDef(vm->checkpoints, *def)))
--
2.26.2
9:02 a.m.
New subject: [PATCH 13/32] qemu: checkpoint: Don't chain bitmaps for checkpoints
On 6/15/20 12:10 PM, Peter Krempa wrote:
Chaining bitmaps for checkpoints (disabling the active one and
creating
a new) severely overcomplicated all operations in regards to bitmaps.
Specifically it requires us re-matching the on-disk state to the
internal metadata and in case of merging during block jobs it makes it
almost impossible to cover all corner cases.
Since the checkpoints and incremental backups were not yet enabled,
let's change the design to keep one bitmap per checkpoint. In case of
layered snapshots this will be filled in by using dirty-bitmap-populate.
Finally the main reason for this unnecessary complexity was the fear
that qemu's performance could degrade. In the end I think that
addressing the performance issue will be better done in qemu (e.g by
keeping an internal bitmap updated with changes and merging it
periodically back to the real bitmaps. QEMU writes out changes to disk
at shutdown so consistency is not a problem).
Removing the relationships between bitmaps frees us from complex
handling and also makes all the surrounding code more robust as one
broken bitmap doesn't necessarily invalidate whole chains of backups.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_checkpoint.c | 21 +--------------------
1 file changed, 1 insertion(+), 20 deletions(-)
Definitely looks simpler. In graphics, pre-patch we had:
time --C1-----C2-----C3-----now
:b1....:
:b2....:
:b3....
where the differential from C1 is b1+b2+b3, the differential from C2 is
b2+b3, the incremental from C3 is b3.
post-patch, we have:
time --C1-----C2-----C3-----now
:b1...................
:b2............
:b3....
where the differential from C1 is b1, the differential from C2 is b2,
the incremental from C3 is b3.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 14/32] qemublocktest: Replace 'basic' bitmap detection test case data
Use test data which conforms to the new semantics which changed in the
previous patch.
The test data was created by the same set of commands as originally in
commit 9aac9d5bdab039a50de2d8c627b3a1f1578ed471
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktestdata/bitmap/basic.json | 229 +++++++++++-----------
tests/qemublocktestdata/bitmap/basic.out | 8 +-
2 files changed, 119 insertions(+), 118 deletions(-)
diff --git a/tests/qemublocktestdata/bitmap/basic.json
b/tests/qemublocktestdata/bitmap/basic.json
index 9d418b1a37..718106bf99 100644
--- a/tests/qemublocktestdata/bitmap/basic.json
+++ b/tests/qemublocktestdata/bitmap/basic.json
@@ -1,117 +1,118 @@
[
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 200704,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-format",
- "backing_file_depth": 0,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "dirty-bitmaps": [
- {
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "d",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "c",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "b",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "a",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
- }
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.qcow2",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 200704,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "dirty-flag": false
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 197120,
- "filename": "/tmp/pull4.qcow2",
- "format": "file",
- "actual-size": 200704,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- }
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "dirty-bitmaps": [
+ {
+ "name": "current",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "c",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "b",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 197120,
+ "filename": "/tmp/bitmaps.qcow2",
+ "format": "file",
+ "actual-size": 200704,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ }
]
diff --git a/tests/qemublocktestdata/bitmap/basic.out
b/tests/qemublocktestdata/bitmap/basic.out
index 539f7d9bad..5c4c35b3f0 100644
--- a/tests/qemublocktestdata/bitmap/basic.out
+++ b/tests/qemublocktestdata/bitmap/basic.out
@@ -1,6 +1,6 @@
libvirt-1-format:
current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- b: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- a: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
--
2.26.2
9:07 a.m.
New subject: [PATCH 14/32] qemublocktest: Replace 'basic' bitmap detection test case data
On 6/15/20 12:10 PM, Peter Krempa wrote:
Use test data which conforms to the new semantics which changed in
the
previous patch.
The test data was created by the same set of commands as originally in
commit 9aac9d5bdab039a50de2d8c627b3a1f1578ed471
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktestdata/bitmap/basic.json | 229 +++++++++++-----------
tests/qemublocktestdata/bitmap/basic.out | 8 +-
2 files changed, 119 insertions(+), 118 deletions(-)
- "dirty-bitmaps": [
- {
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
- },
- {
- "name": "d",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
The old code showed "recording":false for past bitmaps;
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
The new output is built with a newer qemu that understands different
compression types (doesn't change correctness of the patch), and...
+ "dirty-bitmaps": [
+ {
+ "name": "current",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
...shows that even the past bitmaps are still active. If qemu has
performance problems with multiple active bitmaps, I agree that
addressing that in qemu is reasonable.
+++ b/tests/qemublocktestdata/bitmap/basic.out
@@ -1,6 +1,6 @@
libvirt-1-format:
current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- b: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- a: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
And this reflects that libvirt is parsing the qemu output according to
libvirt's new paradigm.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 15/32] qemublocktest: Replace 'snapshots' bitmap detection test case data
Use test data which conforms to the new semantics which changed in the
previous patch.
The test data was created by the same set of commands as originally in
commit 0b27b655b1bac480186ce80457113cd5dc34e6a1
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktestdata/bitmap/snapshots.json | 1382 +++++++----------
tests/qemublocktestdata/bitmap/snapshots.out | 13 +-
2 files changed, 588 insertions(+), 807 deletions(-)
diff --git a/tests/qemublocktestdata/bitmap/snapshots.json
b/tests/qemublocktestdata/bitmap/snapshots.json
index 87e77ad408..2a20aea812 100644
--- a/tests/qemublocktestdata/bitmap/snapshots.json
+++ b/tests/qemublocktestdata/bitmap/snapshots.json
@@ -1,836 +1,612 @@
[
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename":
"/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 104857600,
+ "filename": "/tmp/bitmaps.1591792950",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 225280,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "current",
+ "granularity": 65536
+ },
+ {
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "c",
+ "granularity": 65536
},
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911550",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "in-use",
- "auto"
- ],
- "name": "current",
- "granularity": 65536
- },
- {
- "flags": [
- "in-use"
- ],
- "name": "d",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
+ {
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "b",
+ "granularity": 65536
},
- "full-backing-filename": "/tmp/pull4.1575911540",
- "backing-filename": "/tmp/pull4.1575911540",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-format",
- "backing_file_depth": 4,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911540",
- "dirty-bitmaps": [
{
- "name": "d",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
},
{
- "name": "current",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "d",
+ "granularity": 65536
}
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1591792938",
+ "backing-filename": "/tmp/bitmaps.1591792938",
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1591792938",
+ "dirty-bitmaps": [
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "b",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "c",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "current",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1591792950",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 590336,
+ "filename": "/tmp/bitmaps.1591792950",
+ "format": "file",
+ "actual-size": 225280,
+ "dirty-flag": false
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911550",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-1-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911550",
- "encryption_key_missing": false
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
+ "file": "/tmp/bitmaps.1591792950",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 104857600,
+ "filename": "/tmp/bitmaps.1591792938",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 221184,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "d",
+ "granularity": 65536
},
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911540",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 212992,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "d",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "c",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
},
- "full-backing-filename": "/tmp/pull4.1575911527",
- "backing-filename": "/tmp/pull4.1575911527",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-2-format",
- "backing_file_depth": 3,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911527",
- "dirty-bitmaps": [
{
- "name": "c",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "b",
+ "granularity": 65536
},
{
- "name": "d",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "c",
+ "granularity": 65536
}
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1591792930",
+ "backing-filename": "/tmp/bitmaps.1591792930",
+ "dirty-flag": false
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 393728,
- "filename": "/tmp/pull4.1575911540",
- "format": "file",
- "actual-size": 212992,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-2-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911540",
- "encryption_key_missing": false
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-2-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1591792930",
+ "dirty-bitmaps": [
+ {
+ "name": "c",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "b",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911527",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 217088,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "c",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "b",
- "granularity": 65536
- },
- {
- "flags": [
-
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.1575911522",
- "backing-filename": "/tmp/pull4.1575911522",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-3-format",
- "backing_file_depth": 2,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.1575911522",
- "dirty-bitmaps": [
+ "file": "/tmp/bitmaps.1591792938",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 524800,
+ "filename": "/tmp/bitmaps.1591792938",
+ "format": "file",
+ "actual-size": 221184,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-2-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1591792938",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 104857600,
+ "filename": "/tmp/bitmaps.1591792930",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 217088,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
{
- "name": "a",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "c",
+ "granularity": 65536
},
{
- "name": "b",
- "recording": false,
- "persistent": true,
- "busy": false,
- "status": "disabled",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "b",
+ "granularity": 65536
},
{
- "name": "c",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
}
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1591792925",
+ "backing-filename": "/tmp/bitmaps.1591792925",
+ "dirty-flag": false
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 459264,
- "filename": "/tmp/pull4.1575911527",
- "format": "file",
- "actual-size": 217088,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-3-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911527",
- "encryption_key_missing": false
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-3-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1591792925",
+ "dirty-bitmaps": [
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "b",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "c",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "backing-image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "backing-filename-format": "qcow2",
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.1575911522",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "full-backing-filename": "/tmp/pull4.qcow2",
- "backing-filename": "/tmp/pull4.qcow2",
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-4-format",
- "backing_file_depth": 1,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "backing_file": "/tmp/pull4.qcow2",
- "dirty-bitmaps": [
+ "file": "/tmp/bitmaps.1591792930",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 459264,
+ "filename": "/tmp/bitmaps.1591792930",
+ "format": "file",
+ "actual-size": 217088,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-3-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1591792930",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 104857600,
+ "filename": "/tmp/bitmaps.1591792925",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 208896,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
{
- "name": "a",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
}
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.qcow2",
+ "backing-filename": "/tmp/bitmaps.qcow2",
+ "dirty-flag": false
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.1575911522",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-4-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.1575911522",
- "encryption_key_missing": false
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-4-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.qcow2",
+ "dirty-bitmaps": [
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 10485760,
- "filename": "/tmp/pull4.qcow2",
- "cluster-size": 65536,
- "format": "qcow2",
- "actual-size": 208896,
- "format-specific": {
- "type": "qcow2",
- "data": {
- "compat": "1.1",
- "lazy-refcounts": false,
- "bitmaps": [
- {
- "flags": [
- "auto"
- ],
- "name": "a",
- "granularity": 65536
- }
- ],
- "refcount-bits": 16,
- "corrupt": false
- }
- },
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": true,
- "node-name": "libvirt-5-format",
- "backing_file_depth": 0,
- "drv": "qcow2",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "dirty-bitmaps": [
+ "file": "/tmp/bitmaps.1591792925",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 328192,
+ "filename": "/tmp/bitmaps.1591792925",
+ "format": "file",
+ "actual-size": 208896,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-4-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1591792925",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 104857600,
+ "filename": "/tmp/bitmaps.qcow2",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 208896,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
{
- "name": "a",
- "recording": true,
- "persistent": true,
- "busy": false,
- "status": "active",
- "granularity": 65536,
- "count": 0
+ "flags": [
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
}
- ],
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-5-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "dirty-bitmaps": [
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 328192,
+ "filename": "/tmp/bitmaps.qcow2",
+ "format": "file",
+ "actual-size": 208896,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-5-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
},
- {
- "iops_rd": 0,
- "detect_zeroes": "off",
- "image": {
- "virtual-size": 328192,
- "filename": "/tmp/pull4.qcow2",
- "format": "file",
- "actual-size": 208896,
- "dirty-flag": false
- },
- "iops_wr": 0,
- "ro": false,
- "node-name": "libvirt-5-storage",
- "backing_file_depth": 0,
- "drv": "file",
- "iops": 0,
- "bps_wr": 0,
- "write_threshold": 0,
- "encrypted": false,
- "bps": 0,
- "bps_rd": 0,
- "cache": {
- "no-flush": false,
- "direct": false,
- "writeback": true
- },
- "file": "/tmp/pull4.qcow2",
- "encryption_key_missing": false
- }
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ }
]
diff --git a/tests/qemublocktestdata/bitmap/snapshots.out
b/tests/qemublocktestdata/bitmap/snapshots.out
index 5dafe946cf..24ca27e4d8 100644
--- a/tests/qemublocktestdata/bitmap/snapshots.out
+++ b/tests/qemublocktestdata/bitmap/snapshots.out
@@ -1,12 +1,17 @@
libvirt-1-format:
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
libvirt-2-format:
- c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
libvirt-3-format:
- a: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
- b: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
libvirt-4-format:
a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
--
2.26.2
9:58 a.m.
New subject: [PATCH 15/32] qemublocktest: Replace 'snapshots' bitmap detection test case data
On 6/15/20 12:10 PM, Peter Krempa wrote:
Use test data which conforms to the new semantics which changed in
the
previous patch.
Well, a recent patch (the previous patch only touched tests, rather than
changing semantics)
The test data was created by the same set of commands as originally in
commit 0b27b655b1bac480186ce80457113cd5dc34e6a1
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktestdata/bitmap/snapshots.json | 1382 +++++++----------
tests/qemublocktestdata/bitmap/snapshots.out | 13 +-
2 files changed, 588 insertions(+), 807 deletions(-)
+++ b/tests/qemublocktestdata/bitmap/snapshots.out
@@ -1,12 +1,17 @@
libvirt-1-format:
- d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
So, where the old paradigm stitched bitmaps across snapshots (no need to
add a bitmap to a new snapshot layer if it was already inactive in the
old layer), the new paradigm says that every reachable checkpoint must
have an active bitmap in the current layer, in addition to whatever
bitmaps it had in earlier layers.
Presumably, upcoming patches will tweak libvirt to actually create these
bitmaps as part of creating an external snapshot, and I also see how you
could utilize qemu adding block-dirty-bitmap-populate as a convenient
way for re-creating such bitmaps after the fact (any bitmap that exists
in the backing chain should basically mirror the allocation of the
current layer in the chain).
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
10:13 a.m.
New subject: [PATCH 15/32] qemublocktest: Replace 'snapshots' bitmap detection test case data
On Thu, Jun 18, 2020 at 09:58:35 -0500, Eric Blake wrote:
On 6/15/20 12:10 PM, Peter Krempa wrote:
> Use test data which conforms to the new semantics which changed in the
> previous patch.
Well, a recent patch (the previous patch only touched tests, rather than
changing semantics)
>
> The test data was created by the same set of commands as originally in
> commit 0b27b655b1bac480186ce80457113cd5dc34e6a1
>
> Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
> ---
> tests/qemublocktestdata/bitmap/snapshots.json | 1382 +++++++----------
> tests/qemublocktestdata/bitmap/snapshots.out | 13 +-
> 2 files changed, 588 insertions(+), 807 deletions(-)
> +++ b/tests/qemublocktestdata/bitmap/snapshots.out
> @@ -1,12 +1,17 @@
> libvirt-1-format:
> - d: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
> + d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
> + a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
> + b: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
> + c: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
> current: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
So, where the old paradigm stitched bitmaps across snapshots (no need to add
a bitmap to a new snapshot layer if it was already inactive in the old
layer), the new paradigm says that every reachable checkpoint must have an
active bitmap in the current layer, in addition to whatever bitmaps it had
in earlier layers.
Presumably, upcoming patches will tweak libvirt to actually create these
bitmaps as part of creating an external snapshot, and I also see how you
could utilize qemu adding block-dirty-bitmap-populate as a convenient way
for re-creating such bitmaps after the fact (any bitmap that exists in the
backing chain should basically mirror the allocation of the current layer in
the chain).
They are actually already being created by
'qemuDomainSnapshotDiskBitmapsPropagate',
and that code is still valid. But I've noticed that the comment needs to
be fixed probably.
12:10 p.m.
New subject: [PATCH 16/32] qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps
Reject duplicates and other problematic bitmaps according to the new
semantics of bitmap use in libvirt.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 38 +++++++++++++++++++++++---------------
1 file changed, 23 insertions(+), 15 deletions(-)
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 6f9c7071c9..83e3df9601 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -2850,41 +2850,49 @@ qemuBlockGetNamedNodeData(virDomainObjPtr vm,
/**
* qemuBlockBitmapChainIsValid:
*
- * Validates that the backing chain of @src contains proper consistent bitmap
- * data for a chain of bitmaps named @bitmapname.
+ * Validates that the backing chain of @src contains bitmaps which libvirt will
+ * consider as properly corresponding to a checkpoint named @bitmapname.
*
- * A valid chain:
- * 1) bitmaps of same name are in a consecutive subset of images without gap
- * 2) don't have any inconsistent bitmaps
+ * The bitmaps need to:
+ * 1) start from the top image @src
+ * 2) must be present in consecutive layers
+ * 3) all must be active, persistent and not incosistent
*/
bool
qemuBlockBitmapChainIsValid(virStorageSourcePtr src,
const char *bitmapname,
virHashTablePtr blockNamedNodeData)
{
- qemuBlockNamedNodeDataBitmapPtr bitmap;
virStorageSourcePtr n;
- bool chain_started = false;
+ bool found = false;
bool chain_ended = false;
- for (n = src; n; n = n->backingStore) {
- if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData, n,
bitmapname))) {
- if (chain_started)
- chain_ended = true;
+ for (n = src; virStorageSourceIsBacking(n); n = n->backingStore) {
+ qemuBlockNamedNodeDataBitmapPtr bitmap;
+
+ if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n, bitmapname))) {
+ /* rule 1, must start from top */
+ if (!found)
+ return false;
+
+ chain_ended = true;
continue;
}
+ /* rule 2, no-gaps */
if (chain_ended)
return false;
- chain_started = true;
-
- if (bitmap->inconsistent)
+ /* rule 3 */
+ if (bitmap->inconsistent || !bitmap->persistent || !bitmap->recording)
return false;
+
+ found = true;
}
- return chain_started;
+ return found;
}
--
2.26.2
11:08 a.m.
New subject: [PATCH 16/32] qemuBlockBitmapChainIsValid: Adjust to new semantics of bitmaps
On 6/15/20 12:10 PM, Peter Krempa wrote:
Reject duplicates and other problematic bitmaps according to the new
semantics of bitmap use in libvirt.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 38 +++++++++++++++++++++++---------------
1 file changed, 23 insertions(+), 15 deletions(-)
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 6f9c7071c9..83e3df9601 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -2850,41 +2850,49 @@ qemuBlockGetNamedNodeData(virDomainObjPtr vm,
/**
* qemuBlockBitmapChainIsValid:
*
- * Validates that the backing chain of @src contains proper consistent bitmap
- * data for a chain of bitmaps named @bitmapname.
+ * Validates that the backing chain of @src contains bitmaps which libvirt will
+ * consider as properly corresponding to a checkpoint named @bitmapname.
*
- * A valid chain:
- * 1) bitmaps of same name are in a consecutive subset of images without gap
- * 2) don't have any inconsistent bitmaps
+ * The bitmaps need to:
+ * 1) start from the top image @src
+ * 2) must be present in consecutive layers
+ * 3) all must be active, persistent and not incosistent
inconsistent
Otherwise,
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 17/32] qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases
Now that we've updated both the test data and the validator to new
semantics we can start testing again.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 0ed9b99bc4..6d1bf3f250 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1346,6 +1346,18 @@ mymain(void)
TEST_BITMAP_VALIDATE("empty", "a", false);
+ TEST_BITMAP_VALIDATE("basic", "a", true);
+ TEST_BITMAP_VALIDATE("basic", "b", true);
+ TEST_BITMAP_VALIDATE("basic", "c", true);
+ TEST_BITMAP_VALIDATE("basic", "d", true);
+ TEST_BITMAP_VALIDATE("basic", "current", true);
+
+ TEST_BITMAP_VALIDATE("snapshots", "a", true);
+ TEST_BITMAP_VALIDATE("snapshots", "b", true);
+ TEST_BITMAP_VALIDATE("snapshots", "c", true);
+ TEST_BITMAP_VALIDATE("snapshots", "d", true);
+ TEST_BITMAP_VALIDATE("snapshots", "current", true);
+
#define TEST_BITMAP_BLOCKCOPY(testname, shllw, ndf) \
do { \
blockbitmapblockcopydata.name = testname; \
--
2.26.2
11:11 a.m.
New subject: [PATCH 17/32] qemublocktest: Re-add bitmap validation for 'basic' and 'snapshots' cases
On 6/15/20 12:10 PM, Peter Krempa wrote:
Now that we've updated both the test data and the validator to
new
semantics we can start testing again.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 18/32] qemublocktest: Add new 'synthetic' bitmap detection and validation test case
Based on the 'snapshots' example with manual tweaks to introduce
inactive, transient, inconsistent and duplicate bitmaps in various parts
of the chain to excercise detection and new validation code.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 11 +
tests/qemublocktestdata/bitmap/synthetic.json | 506 ++++++++++++++++++
tests/qemublocktestdata/bitmap/synthetic.out | 15 +
3 files changed, 532 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmap/synthetic.json
create mode 100644 tests/qemublocktestdata/bitmap/synthetic.out
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 6d1bf3f250..47e9f0f363 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1303,6 +1303,7 @@ mymain(void)
TEST_BITMAP_DETECT("basic");
TEST_BITMAP_DETECT("snapshots");
+ TEST_BITMAP_DETECT("synthetic");
#define TEST_BACKUP_BITMAP_CALCULATE(testname, source, incrbackup, named) \
do { \
@@ -1358,6 +1359,16 @@ mymain(void)
TEST_BITMAP_VALIDATE("snapshots", "d", true);
TEST_BITMAP_VALIDATE("snapshots", "current", true);
+ TEST_BITMAP_VALIDATE("synthetic", "a", false);
+ TEST_BITMAP_VALIDATE("synthetic", "b", false);
+ TEST_BITMAP_VALIDATE("synthetic", "c", false);
+ TEST_BITMAP_VALIDATE("synthetic", "d", false);
+ TEST_BITMAP_VALIDATE("synthetic", "current", false);
+ TEST_BITMAP_VALIDATE("synthetic", "top-ok", true);
+ TEST_BITMAP_VALIDATE("synthetic", "top-inactive", false);
+ TEST_BITMAP_VALIDATE("synthetic", "top-transient", false);
+ TEST_BITMAP_VALIDATE("synthetic", "top-inactive-transient",
false);
+
#define TEST_BITMAP_BLOCKCOPY(testname, shllw, ndf) \
do { \
blockbitmapblockcopydata.name = testname; \
diff --git a/tests/qemublocktestdata/bitmap/synthetic.json
b/tests/qemublocktestdata/bitmap/synthetic.json
new file mode 100644
index 0000000000..3712c8e5fc
--- /dev/null
+++ b/tests/qemublocktestdata/bitmap/synthetic.json
@@ -0,0 +1,506 @@
+[
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.1590749073",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 208896,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "in-use",
+ "auto"
+ ],
+ "name": "current",
+ "granularity": 65536
+ }
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1590749012",
+ "backing-filename": "/tmp/bitmaps.1590749012",
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1590749012",
+ "dirty-bitmaps": [
+ {
+ "name": "current",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "inconsistent": true,
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "top-ok",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "top-inactive",
+ "recording": false,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "top-transient",
+ "recording": true,
+ "persistent": false,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "top-transient-inactive",
+ "recording": false,
+ "persistent": false,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590749073",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 328192,
+ "filename": "/tmp/bitmaps.1590749073",
+ "format": "file",
+ "actual-size": 208896,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-1-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590749073",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.1590749012",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 208896,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "d",
+ "granularity": 65536
+ }
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1590748995",
+ "backing-filename": "/tmp/bitmaps.1590748995",
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-2-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1590748995",
+ "dirty-bitmaps": [
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590749012",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 328192,
+ "filename": "/tmp/bitmaps.1590749012",
+ "format": "file",
+ "actual-size": 208896,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-2-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590749012",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.1590748995",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 212992,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "c",
+ "granularity": 65536
+ },
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "b",
+ "granularity": 65536
+ }
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.1590748992",
+ "backing-filename": "/tmp/bitmaps.1590748992",
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-3-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.1590748992",
+ "dirty-bitmaps": [
+ {
+ "name": "b",
+ "recording": true,
+ "persistent": false,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "c",
+ "recording": false,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ },
+ {
+ "name": "d",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590748995",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 393728,
+ "filename": "/tmp/bitmaps.1590748995",
+ "format": "file",
+ "actual-size": 212992,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-3-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590748995",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "backing-filename-format": "qcow2",
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.1590748992",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 200704,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "full-backing-filename": "/tmp/bitmaps.qcow2",
+ "backing-filename": "/tmp/bitmaps.qcow2",
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-4-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "backing_file": "/tmp/bitmaps.qcow2",
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590748992",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 197120,
+ "filename": "/tmp/bitmaps.1590748992",
+ "format": "file",
+ "actual-size": 200704,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-4-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.1590748992",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 10485760,
+ "filename": "/tmp/bitmaps.qcow2",
+ "cluster-size": 65536,
+ "format": "qcow2",
+ "actual-size": 208896,
+ "format-specific": {
+ "type": "qcow2",
+ "data": {
+ "compat": "1.1",
+ "compression-type": "zlib",
+ "lazy-refcounts": false,
+ "bitmaps": [
+ {
+ "flags": [
+ "auto"
+ ],
+ "name": "a",
+ "granularity": 65536
+ }
+ ],
+ "refcount-bits": 16,
+ "corrupt": false
+ }
+ },
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": true,
+ "node-name": "libvirt-5-format",
+ "backing_file_depth": 0,
+ "drv": "qcow2",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "dirty-bitmaps": [
+ {
+ "name": "a",
+ "recording": true,
+ "persistent": true,
+ "busy": false,
+ "status": "active",
+ "granularity": 65536,
+ "count": 0
+ }
+ ],
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ },
+ {
+ "iops_rd": 0,
+ "detect_zeroes": "off",
+ "image": {
+ "virtual-size": 328192,
+ "filename": "/tmp/bitmaps.qcow2",
+ "format": "file",
+ "actual-size": 208896,
+ "dirty-flag": false
+ },
+ "iops_wr": 0,
+ "ro": false,
+ "node-name": "libvirt-5-storage",
+ "backing_file_depth": 0,
+ "drv": "file",
+ "iops": 0,
+ "bps_wr": 0,
+ "write_threshold": 0,
+ "encrypted": false,
+ "bps": 0,
+ "bps_rd": 0,
+ "cache": {
+ "no-flush": false,
+ "direct": false,
+ "writeback": true
+ },
+ "file": "/tmp/bitmaps.qcow2",
+ "encryption_key_missing": false
+ }
+]
diff --git a/tests/qemublocktestdata/bitmap/synthetic.out
b/tests/qemublocktestdata/bitmap/synthetic.out
new file mode 100644
index 0000000000..cde7228e01
--- /dev/null
+++ b/tests/qemublocktestdata/bitmap/synthetic.out
@@ -0,0 +1,15 @@
+libvirt-1-format:
+ current: record:1 busy:0 persist:1 inconsist:1 gran:65536 dirty:0
+ top-ok: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ top-inactive: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ top-transient: record:1 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+ top-transient-inactive: record:0 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+libvirt-2-format:
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+libvirt-3-format:
+ b: record:1 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+ c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+libvirt-4-format:
+libvirt-5-format:
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
--
2.26.2
11:21 a.m.
New subject: [PATCH 18/32] qemublocktest: Add new 'synthetic' bitmap detection and validation test case
On 6/15/20 12:10 PM, Peter Krempa wrote:
Based on the 'snapshots' example with manual tweaks to
introduce
inactive, transient, inconsistent and duplicate bitmaps in various parts
of the chain to excercise detection and new validation code.
exercise
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 11 +
tests/qemublocktestdata/bitmap/synthetic.json | 506 ++++++++++++++++++
tests/qemublocktestdata/bitmap/synthetic.out | 15 +
3 files changed, 532 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmap/synthetic.json
create mode 100644 tests/qemublocktestdata/bitmap/synthetic.out
+++ b/tests/qemublocktestdata/bitmap/synthetic.out
@@ -0,0 +1,15 @@
+libvirt-1-format:
+ current: record:1 busy:0 persist:1 inconsist:1 gran:65536 dirty:0
+ top-ok: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ top-inactive: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ top-transient: record:1 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+ top-transient-inactive: record:0 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+libvirt-2-format:
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+libvirt-3-format:
+ b: record:1 busy:0 persist:0 inconsist:0 gran:65536 dirty:0
+ c: record:0 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+ d: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
+libvirt-4-format:
+libvirt-5-format:
+ a: record:1 busy:0 persist:1 inconsist:0 gran:65536 dirty:0
Looks like a pretty decent set of awkward setups; libvirt should not
ever produce these, but being robust to whatever happened to the qcow2
files externally to libvirt is worthwhile.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 19/32] qemu: checkpoint: Don't merge checkpoints during deletion
Now that we've switched to the simple handling, the first thing that can
be massively simplified is checkpoint deletion. We now need to only go
through the backing chain and find the appropriately named bitmaps and
delete them, no complex lookups or merging.
Note that compared to other functions this deletes the bitmap in all
layers compared to others where we expect only exactly 1 bitmap of a
name in the backing chain to prevent potential problems.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_checkpoint.c | 153 ++++++-------------------------------
src/qemu/qemu_checkpoint.h | 1 -
tests/qemublocktest.c | 7 +-
3 files changed, 25 insertions(+), 136 deletions(-)
diff --git a/src/qemu/qemu_checkpoint.c b/src/qemu/qemu_checkpoint.c
index 9b4f3ad396..c24d97443c 100644
--- a/src/qemu/qemu_checkpoint.c
+++ b/src/qemu/qemu_checkpoint.c
@@ -105,140 +105,41 @@ qemuCheckpointWriteMetadata(virDomainObjPtr vm,
}
-/**
- * qemuCheckpointFindActiveDiskInParent:
- * @vm: domain object
- * @from: starting moment object
- * @diskname: name (target) of the disk to find
- *
- * Find the first checkpoint starting from @from continuing through parents
- * of the checkpoint which describes disk @diskname. Return the pointer to the
- * definition of the disk.
- */
-static virDomainCheckpointDiskDef *
-qemuCheckpointFindActiveDiskInParent(virDomainObjPtr vm,
- virDomainMomentObjPtr from,
- const char *diskname)
-{
- virDomainMomentObjPtr parent = from;
- virDomainCheckpointDefPtr parentdef = NULL;
- size_t i;
-
- while (parent) {
- parentdef = virDomainCheckpointObjGetDef(parent);
-
- for (i = 0; i < parentdef->ndisks; i++) {
- virDomainCheckpointDiskDef *chkdisk = &parentdef->disks[i];
-
- if (STRNEQ(chkdisk->name, diskname))
- continue;
-
- /* currently inspected checkpoint doesn't describe the disk,
- * continue into parent checkpoint */
- if (chkdisk->type != VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP)
- break;
-
- return chkdisk;
- }
-
- parent = virDomainCheckpointFindByName(vm->checkpoints,
- parentdef->parent.parent_name);
- }
-
- return NULL;
-}
-
-
int
qemuCheckpointDiscardDiskBitmaps(virStorageSourcePtr src,
virHashTablePtr blockNamedNodeData,
const char *delbitmap,
- const char *parentbitmap,
virJSONValuePtr actions,
const char *diskdst,
GSList **reopenimages)
{
- virStorageSourcePtr n = src;
+ virStorageSourcePtr n;
+ bool found = false;
/* find the backing chain entry with bitmap named '@delbitmap' */
- while (n) {
- qemuBlockNamedNodeDataBitmapPtr tmp;
-
- if ((tmp = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- n, delbitmap))) {
- break;
- }
-
- n = n->backingStore;
- }
-
- if (!n) {
- virReportError(VIR_ERR_INTERNAL_ERROR,
- _("bitmap '%s' not found in backing chain of
'%s'"),
- delbitmap, diskdst);
- return -1;
- }
-
- while (n) {
- qemuBlockNamedNodeDataBitmapPtr srcbitmap;
-
- if (!(srcbitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- n, delbitmap)))
- break;
-
- /* For the actual checkpoint deletion we will merge any bitmap into the
- * bitmap of the parent checkpoint (@parentbitmap) or for any image
- * where the parent checkpoint bitmap is not present we must rename
- * the bitmap of the deleted checkpoint into the bitmap of the parent
- * checkpoint as qemu can't currently take the allocation map and turn
- * it into a bitmap and thus we wouldn't be able to do a backup. */
- if (parentbitmap) {
- qemuBlockNamedNodeDataBitmapPtr dstbitmap;
- g_autoptr(virJSONValue) arr = NULL;
-
- dstbitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- n, parentbitmap);
-
- if (dstbitmap) {
- if (srcbitmap->recording && !dstbitmap->recording) {
- if (qemuMonitorTransactionBitmapEnable(actions,
- n->nodeformat,
- dstbitmap->name) < 0)
- return -1;
- }
-
- } else {
- if (qemuMonitorTransactionBitmapAdd(actions,
- n->nodeformat,
- parentbitmap,
- true,
- !srcbitmap->recording,
- srcbitmap->granularity) < 0)
- return -1;
- }
+ for (n = src; virStorageSourceIsBacking(n); n = n->backingStore) {
+ qemuBlockNamedNodeDataBitmapPtr bitmapdata;
- arr = virJSONValueNewArray();
-
- if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(arr,
- n->nodeformat,
- srcbitmap->name) <
0)
- return -1;
+ if (!(bitmapdata = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n, delbitmap)))
+ continue;
- if (qemuMonitorTransactionBitmapMerge(actions,
- n->nodeformat,
- parentbitmap, &arr) < 0)
- return -1;
- }
+ found = true;
if (qemuMonitorTransactionBitmapRemove(actions,
n->nodeformat,
- srcbitmap->name) < 0)
+ bitmapdata->name) < 0)
return -1;
if (n != src)
*reopenimages = g_slist_prepend(*reopenimages, n);
+ }
- n = n->backingStore;
+ if (!found) {
+ virReportError(VIR_ERR_INTERNAL_ERROR,
+ _("bitmap '%s' not found in backing chain of
'%s'"),
+ delbitmap, diskdst);
+ return -1;
}
return 0;
@@ -247,8 +148,7 @@ qemuCheckpointDiscardDiskBitmaps(virStorageSourcePtr src,
static int
qemuCheckpointDiscardBitmaps(virDomainObjPtr vm,
- virDomainCheckpointDefPtr chkdef,
- virDomainMomentObjPtr parent)
+ virDomainCheckpointDefPtr chkdef)
{
qemuDomainObjPrivatePtr priv = vm->privateData;
virQEMUDriverPtr driver = priv->driver;
@@ -268,8 +168,6 @@ qemuCheckpointDiscardBitmaps(virDomainObjPtr vm,
for (i = 0; i < chkdef->ndisks; i++) {
virDomainCheckpointDiskDef *chkdisk = &chkdef->disks[i];
virDomainDiskDefPtr domdisk = virDomainDiskByTarget(vm->def,
chkdisk->name);
- virDomainCheckpointDiskDef *parentchkdisk = NULL;
- const char *parentbitmap = NULL;
/* domdisk can be missing e.g. when it was unplugged */
if (!domdisk)
@@ -278,15 +176,8 @@ qemuCheckpointDiscardBitmaps(virDomainObjPtr vm,
if (chkdisk->type != VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP)
continue;
- /* If any ancestor checkpoint has a bitmap for the same
- * disk, then this bitmap must be merged to the
- * ancestor. */
- if ((parentchkdisk = qemuCheckpointFindActiveDiskInParent(vm, parent,
- chkdisk->name)))
- parentbitmap = parentchkdisk->bitmap;
-
if (qemuCheckpointDiscardDiskBitmaps(domdisk->src, blockNamedNodeData,
- chkdisk->bitmap, parentbitmap,
+ chkdisk->bitmap,
actions, domdisk->dst,
&reopenimages) < 0)
return -1;
@@ -334,7 +225,6 @@ qemuCheckpointDiscard(virQEMUDriverPtr driver,
bool update_parent,
bool metadata_only)
{
- virDomainMomentObjPtr parent = NULL;
g_autoptr(virQEMUDriverConfig) cfg = virQEMUDriverGetConfig(driver);
g_autofree char *chkFile = NULL;
bool chkcurrent = chk == virDomainCheckpointGetCurrent(vm->checkpoints);
@@ -350,14 +240,17 @@ qemuCheckpointDiscard(virQEMUDriverPtr driver,
if (!metadata_only) {
virDomainCheckpointDefPtr chkdef = virDomainCheckpointObjGetDef(chk);
- parent = virDomainCheckpointFindByName(vm->checkpoints,
- chk->def->parent_name);
- if (qemuCheckpointDiscardBitmaps(vm, chkdef, parent) < 0)
+ if (qemuCheckpointDiscardBitmaps(vm, chkdef) < 0)
return -1;
}
if (chkcurrent) {
+ virDomainMomentObjPtr parent = NULL;
+
virDomainCheckpointSetCurrent(vm->checkpoints, NULL);
+ parent = virDomainCheckpointFindByName(vm->checkpoints,
+ chk->def->parent_name);
+
if (update_parent && parent) {
virDomainCheckpointSetCurrent(vm->checkpoints, parent);
if (qemuCheckpointWriteMetadata(vm, parent,
diff --git a/src/qemu/qemu_checkpoint.h b/src/qemu/qemu_checkpoint.h
index cf1e9e46cb..0d267a188c 100644
--- a/src/qemu/qemu_checkpoint.h
+++ b/src/qemu/qemu_checkpoint.h
@@ -76,7 +76,6 @@ int
qemuCheckpointDiscardDiskBitmaps(virStorageSourcePtr src,
virHashTablePtr blockNamedNodeData,
const char *delbitmap,
- const char *parentbitmap,
virJSONValuePtr actions,
const char *diskdst,
GSList **reopenimages);
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 47e9f0f363..336ff4b3ef 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -830,7 +830,6 @@ struct testQemuCheckpointDeleteMergeData {
const char *name;
virStorageSourcePtr chain;
const char *deletebitmap;
- const char *parentbitmap;
const char *nodedatafile;
};
@@ -864,7 +863,6 @@ testQemuCheckpointDeleteMerge(const void *opaque)
if (qemuCheckpointDiscardDiskBitmaps(data->chain,
nodedata,
data->deletebitmap,
- data->parentbitmap,
actions,
"testdisk",
&reopenimages) >= 0) {
@@ -1319,19 +1317,18 @@ mymain(void)
TEST_BACKUP_BITMAP_CALCULATE("empty", bitmapSourceChain, "a",
"empty");
-#define TEST_CHECKPOINT_DELETE_MERGE(testname, delbmp, parbmp, named) \
+#define TEST_CHECKPOINT_DELETE_MERGE(testname, delbmp, named) \
do { \
checkpointdeletedata.name = testname; \
checkpointdeletedata.chain = bitmapSourceChain; \
checkpointdeletedata.deletebitmap = delbmp; \
- checkpointdeletedata.parentbitmap = parbmp; \
checkpointdeletedata.nodedatafile = named; \
if (virTestRun("checkpoint delete " testname, \
testQemuCheckpointDeleteMerge, &checkpointdeletedata) < 0)
\
ret = -1; \
} while (0)
- TEST_CHECKPOINT_DELETE_MERGE("empty", "a", NULL,
"empty");
+ TEST_CHECKPOINT_DELETE_MERGE("empty", "a", "empty");
#define TEST_BITMAP_VALIDATE(testname, bitmap, rc) \
do { \
--
2.26.2
11:30 a.m.
New subject: [PATCH 19/32] qemu: checkpoint: Don't merge checkpoints during deletion
On 6/15/20 12:10 PM, Peter Krempa wrote:
Now that we've switched to the simple handling, the first thing
that can
be massively simplified is checkpoint deletion. We now need to only go
through the backing chain and find the appropriately named bitmaps and
delete them, no complex lookups or merging.
Note that compared to other functions this deletes the bitmap in all
layers compared to others where we expect only exactly 1 bitmap of a
name in the backing chain to prevent potential problems.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_checkpoint.c | 153 ++++++-------------------------------
src/qemu/qemu_checkpoint.h | 1 -
tests/qemublocktest.c | 7 +-
3 files changed, 25 insertions(+), 136 deletions(-)
No kidding about the simplification.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 20/32] qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE
Also rename the helper struct and function.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 336ff4b3ef..82297d0a8b 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -826,7 +826,7 @@ testQemuBackupIncrementalBitmapCalculate(const void *opaque)
static const char *checkpointDeletePrefix =
"qemublocktestdata/checkpointdelete/";
-struct testQemuCheckpointDeleteMergeData {
+struct testQemuCheckpointDeleteData {
const char *name;
virStorageSourcePtr chain;
const char *deletebitmap;
@@ -835,9 +835,9 @@ struct testQemuCheckpointDeleteMergeData {
static int
-testQemuCheckpointDeleteMerge(const void *opaque)
+testQemuCheckpointDelete(const void *opaque)
{
- const struct testQemuCheckpointDeleteMergeData *data = opaque;
+ const struct testQemuCheckpointDeleteData *data = opaque;
g_autofree char *actual = NULL;
g_autofree char *expectpath = NULL;
g_autoptr(virJSONValue) actions = NULL;
@@ -1037,7 +1037,7 @@ mymain(void)
struct testJSONtoJSONData jsontojsondata;
struct testQemuImageCreateData imagecreatedata;
struct testQemuBackupIncrementalBitmapCalculateData backupbitmapcalcdata;
- struct testQemuCheckpointDeleteMergeData checkpointdeletedata;
+ struct testQemuCheckpointDeleteData checkpointdeletedata;
struct testQemuBlockBitmapValidateData blockbitmapvalidatedata;
struct testQemuBlockBitmapBlockcopyData blockbitmapblockcopydata;
struct testQemuBlockBitmapBlockcommitData blockbitmapblockcommitdata;
@@ -1317,18 +1317,18 @@ mymain(void)
TEST_BACKUP_BITMAP_CALCULATE("empty", bitmapSourceChain, "a",
"empty");
-#define TEST_CHECKPOINT_DELETE_MERGE(testname, delbmp, named) \
+#define TEST_CHECKPOINT_DELETE(testname, delbmp, named) \
do { \
checkpointdeletedata.name = testname; \
checkpointdeletedata.chain = bitmapSourceChain; \
checkpointdeletedata.deletebitmap = delbmp; \
checkpointdeletedata.nodedatafile = named; \
if (virTestRun("checkpoint delete " testname, \
- testQemuCheckpointDeleteMerge, &checkpointdeletedata) < 0)
\
+ testQemuCheckpointDelete, &checkpointdeletedata) < 0) \
ret = -1; \
} while (0)
- TEST_CHECKPOINT_DELETE_MERGE("empty", "a", "empty");
+ TEST_CHECKPOINT_DELETE("empty", "a", "empty");
#define TEST_BITMAP_VALIDATE(testname, bitmap, rc) \
do { \
--
2.26.2
1:20 p.m.
New subject: [PATCH 20/32] qemublocktest: Rename TEST_CHECKPOINT_DELETE_MERGE to TEST_CHECKPOINT_DELETE
On 6/15/20 12:10 PM, Peter Krempa wrote:
Also rename the helper struct and function.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
Mechanical.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 21/32] qemublocktest: Re-introduce testing of checkpoint deletion
Exercise the now arguably simpler checkpoint deletion code on the
'basic', 'snapshots', and 'synthetic' test data sets.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 18 ++++++++
.../checkpointdelete/basic-current-out.json | 9 ++++
.../basic-intermediate1-out.json | 9 ++++
.../basic-intermediate2-out.json | 9 ++++
.../basic-intermediate3-out.json | 9 ++++
.../checkpointdelete/basic-noparent-out.json | 9 ++++
.../snapshots-current-out.json | 9 ++++
.../snapshots-intermediate1-out.json | 26 ++++++++++++
.../snapshots-intermediate2-out.json | 26 ++++++++++++
.../snapshots-intermediate3-out.json | 18 ++++++++
.../snapshots-noparent-out.json | 42 +++++++++++++++++++
.../synthetic-current-out.json | 9 ++++
.../synthetic-intermediate1-out.json | 11 +++++
.../synthetic-intermediate2-out.json | 11 +++++
.../synthetic-intermediate3-out.json | 19 +++++++++
.../synthetic-noparent-out.json | 11 +++++
16 files changed, 245 insertions(+)
create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-current-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json
create mode 100644
tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json
create mode 100644 tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 82297d0a8b..9e142a8851 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1330,6 +1330,24 @@ mymain(void)
TEST_CHECKPOINT_DELETE("empty", "a", "empty");
+ TEST_CHECKPOINT_DELETE("basic-noparent", "a",
"basic");
+ TEST_CHECKPOINT_DELETE("basic-intermediate1", "b",
"basic");
+ TEST_CHECKPOINT_DELETE("basic-intermediate2", "c",
"basic");
+ TEST_CHECKPOINT_DELETE("basic-intermediate3", "d",
"basic");
+ TEST_CHECKPOINT_DELETE("basic-current", "current",
"basic");
+
+ TEST_CHECKPOINT_DELETE("snapshots-noparent", "a",
"snapshots");
+ TEST_CHECKPOINT_DELETE("snapshots-intermediate1", "b",
"snapshots");
+ TEST_CHECKPOINT_DELETE("snapshots-intermediate2", "c",
"snapshots");
+ TEST_CHECKPOINT_DELETE("snapshots-intermediate3", "d",
"snapshots");
+ TEST_CHECKPOINT_DELETE("snapshots-current", "current",
"snapshots");
+
+ TEST_CHECKPOINT_DELETE("synthetic-noparent", "a",
"synthetic");
+ TEST_CHECKPOINT_DELETE("synthetic-intermediate1", "b",
"synthetic");
+ TEST_CHECKPOINT_DELETE("synthetic-intermediate2", "c",
"synthetic");
+ TEST_CHECKPOINT_DELETE("synthetic-intermediate3", "d",
"synthetic");
+ TEST_CHECKPOINT_DELETE("synthetic-current", "current",
"synthetic");
+
#define TEST_BITMAP_VALIDATE(testname, bitmap, rc) \
do { \
blockbitmapvalidatedata.name = testname; \
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
new file mode 100644
index 0000000000..6ed1b63b66
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/basic-current-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "current"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
new file mode 100644
index 0000000000..e1dd4920cd
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/basic-intermediate1-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "b"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
new file mode 100644
index 0000000000..5638532d35
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/basic-intermediate2-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "c"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
new file mode 100644
index 0000000000..8c56b0e689
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/basic-intermediate3-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "d"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
new file mode 100644
index 0000000000..e87382fdb4
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/basic-noparent-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "a"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
new file mode 100644
index 0000000000..6ed1b63b66
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/snapshots-current-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "current"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
new file mode 100644
index 0000000000..1cde09131e
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate1-out.json
@@ -0,0 +1,26 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "b"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "b"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ }
+]
+reopen nodes:
+libvirt-3-format
+libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
new file mode 100644
index 0000000000..6fcd7ee0d1
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate2-out.json
@@ -0,0 +1,26 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "c"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "c"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ }
+]
+reopen nodes:
+libvirt-3-format
+libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
new file mode 100644
index 0000000000..2977dc3042
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/snapshots-intermediate3-out.json
@@ -0,0 +1,18 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "d"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ }
+]
+reopen nodes:
+libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
new file mode 100644
index 0000000000..e4a6a413a3
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/snapshots-noparent-out.json
@@ -0,0 +1,42 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "a"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "a"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "a"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "a"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "a"
+ }
+ }
+]
+reopen nodes:
+libvirt-5-format
+libvirt-4-format
+libvirt-3-format
+libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json
b/tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json
new file mode 100644
index 0000000000..6ed1b63b66
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/synthetic-current-out.json
@@ -0,0 +1,9 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-1-format",
+ "name": "current"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json
b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json
new file mode 100644
index 0000000000..6eb9716aaa
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate1-out.json
@@ -0,0 +1,11 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ }
+]
+reopen nodes:
+libvirt-3-format
diff --git a/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json
b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json
new file mode 100644
index 0000000000..92b849cfc9
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate2-out.json
@@ -0,0 +1,11 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ }
+]
+reopen nodes:
+libvirt-3-format
diff --git a/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json
b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json
new file mode 100644
index 0000000000..466df52007
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/synthetic-intermediate3-out.json
@@ -0,0 +1,19 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "d"
+ }
+ }
+]
+reopen nodes:
+libvirt-3-format
+libvirt-2-format
diff --git a/tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json
b/tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json
new file mode 100644
index 0000000000..0479815f36
--- /dev/null
+++ b/tests/qemublocktestdata/checkpointdelete/synthetic-noparent-out.json
@@ -0,0 +1,11 @@
+[
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "a"
+ }
+ }
+]
+reopen nodes:
+libvirt-5-format
--
2.26.2
1:41 p.m.
New subject: [PATCH 21/32] qemublocktest: Re-introduce testing of checkpoint deletion
On 6/15/20 12:10 PM, Peter Krempa wrote:
Exercise the now arguably simpler checkpoint deletion code on the
'basic', 'snapshots', and 'synthetic' test data sets.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 18 ++++++++
.../checkpointdelete/basic-current-out.json | 9 ++++
.../basic-intermediate1-out.json | 9 ++++
.../basic-intermediate2-out.json | 9 ++++
.../basic-intermediate3-out.json | 9 ++++
.../checkpointdelete/basic-noparent-out.json | 9 ++++
.../snapshots-current-out.json | 9 ++++
.../snapshots-intermediate1-out.json | 26 ++++++++++++
.../snapshots-intermediate2-out.json | 26 ++++++++++++
.../snapshots-intermediate3-out.json | 18 ++++++++
.../snapshots-noparent-out.json | 42 +++++++++++++++++++
.../synthetic-current-out.json | 9 ++++
.../synthetic-intermediate1-out.json | 11 +++++
.../synthetic-intermediate2-out.json | 11 +++++
.../synthetic-intermediate3-out.json | 19 +++++++++
.../synthetic-noparent-out.json | 11 +++++
Looks reasonable; you're deleting the bitmap across all files in the
chain where it appears, and don't have to mess around with which bitmaps
to re-enable or merge into.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 22/32] qemu: block: Add universal helper for merging dirty bitmaps for all scenarios
Add a function which allows to merge bitmaps according to the new
semantics and will allow to replace all the specific ad-hoc functions
currently in use for 'backup', 'block commit', 'block copy' and
will
also be usable in the future for 'block pull' and non-shared storage
migration.
The semantics are a bit quirky for the 'backup' case but these quirks
are documented and will prevent us from having two slightly different
algorithms.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 172 ++++++++++++++++++++++++++++++++++++++++++
src/qemu/qemu_block.h | 10 +++
2 files changed, 182 insertions(+)
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 83e3df9601..86f8410ce7 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -2847,6 +2847,178 @@ qemuBlockGetNamedNodeData(virDomainObjPtr vm,
}
+/**
+ * qemuBlockGetBitmapMergeActionsGetBitmaps:
+ *
+ * Collect a list of bitmaps which need to be handled in
+ * qemuBlockGetBitmapMergeActions. The list contains only valid bitmaps in the
+ * sub-chain which is being processed.
+ *
+ * Note that the returned GSList contains bitmap names string pointers borrowed
+ * from @blockNamedNodeData so they must not be freed.
+ */
+static GSList *
+qemuBlockGetBitmapMergeActionsGetBitmaps(virStorageSourcePtr topsrc,
+ const char *bitmapname,
+ virHashTablePtr blockNamedNodeData)
+{
+ g_autoptr(GSList) ret = NULL;
+ qemuBlockNamedNodeDataPtr entry;
+ size_t i;
+
+ /* for now it doesn't make sense to consider bitmaps which are not present
+ * in @topsrc as we can't recreate a bitmap for a layer if it's missing */
+
+ if (!(entry = virHashLookup(blockNamedNodeData, topsrc->nodeformat)))
+ return NULL;
+
+ for (i = 0; i < entry->nbitmaps; i++) {
+ qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
+
+ if (bitmapname &&
+ STRNEQ(bitmapname, bitmap->name))
+ continue;
+
+ if (!qemuBlockBitmapChainIsValid(topsrc, bitmap->name, blockNamedNodeData))
+ continue;
+
+ ret = g_slist_prepend(ret, bitmap->name);
+ }
+
+ return g_steal_pointer(&ret);
+}
+
+
+/**
+ * qemuBlockGetBitmapMergeActions:
+ * @topsrc: top of the chain to merge bitmaps in
+ * @basesrc: bottom of the chain to merge bitmaps in (NULL for full chain)
+ * @target: destination storage source of the merge (may be part of original chain)
+ * @bitmapname: name of bitmap to perform the merge (NULL for all bitmaps)
+ * @dstbitmapname: name of destination bitmap of the merge (see below for caveats)
+ * @writebitmapsrc: storage source corresponding to the node containing the write
temporary bitmap
+ * @actions: returns actions for a 'transaction' QMP command for executing the
merge
+ * @blockNamedNodeData: hash table filled with qemuBlockNamedNodeData
+ *
+ * Calculate handling of dirty block bitmaps between @topsrc and @basesrc. If
+ * @basesrc is NULL the end of the chain is considered. @target is the destination
+ * storage source definition of the merge and may or may not be part of the
+ * merged chain.
+ *
+ * Specifically the merging algorithm ensures that each considered bitmap is
+ * merged with the appropriate bitmaps so that it properly describes
+ * the state of dirty blocks when looked at from @topsrc based on the depth
+ * of the backing chain where the bitmap is placed.
+ *
+ * If @bitmapname is non-NULL only bitmaps with that name are handled, otherwise
+ * all bitmaps are considered.
+ *
+ * If @dstbitmap is non-NULL everything is merged into a bitmap with that name,
+ * otherwise each bitmap is merged into a bitmap with the same name into @target.
+ * Additionally if @dstbitmap is non-NULL the target bitmap is created as
'inactive'
+ * and 'transient' as a special case for the backup operation.
+ *
+ * If @writebitmapsrc is non-NULL, the 'libvirt-tmp-activewrite' bitmap from
+ * given node is merged along with others. This bitmap corresponds to the writes
+ * which occured between an active layer job finished and the rest of the bitmap
+ * merging.
+ *
+ * If the bitmap is not valid somehow (see qemuBlockBitmapChainIsValid) given
+ * bitmap is silently skipped, so callers must ensure that given bitmap is valid
+ * if they care about it.
+ *
+ * The resulting 'transacion' QMP command actions are filled in and returned via
+ * @actions.
+ *
+ * Note that @actions may be NULL if no merging is required.
+ */
+int
+qemuBlockGetBitmapMergeActions(virStorageSourcePtr topsrc,
+ virStorageSourcePtr basesrc,
+ virStorageSourcePtr target,
+ const char *bitmapname,
+ const char *dstbitmapname,
+ virStorageSourcePtr writebitmapsrc,
+ virJSONValuePtr *actions,
+ virHashTablePtr blockNamedNodeData)
+{
+ g_autoptr(virJSONValue) act = virJSONValueNewArray();
+ virStorageSourcePtr n;
+
+ g_autoptr(GSList) bitmaps = NULL;
+ GSList *next;
+
+ if (!(bitmaps = qemuBlockGetBitmapMergeActionsGetBitmaps(topsrc, bitmapname,
+ blockNamedNodeData)))
+ return 0;
+
+ for (next = bitmaps; next; next = next->next) {
+ const char *curbitmap = next->data;
+ const char *mergebitmapname = dstbitmapname;
+ bool mergebitmappersistent = false;
+ bool mergebitmapdisabled = true;
+ g_autoptr(virJSONValue) merge = virJSONValueNewArray();
+ unsigned long long granularity = 0;
+ qemuBlockNamedNodeDataBitmapPtr bitmap;
+
+ /* explicitly named destinations mean that we want a temporary
+ * disabled bitmap only, so undo the default for non-explicit cases */
+ if (!mergebitmapname) {
+ mergebitmapname = curbitmap;
+ mergebitmappersistent = true;
+ mergebitmapdisabled = false;
+ }
+
+ for (n = topsrc; virStorageSourceIsBacking(n) && n != basesrc; n =
n->backingStore) {
+ if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n, curbitmap)))
+ continue;
+
+ if (granularity == 0)
+ granularity = bitmap->granularity;
+
+ if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(merge,
+ n->nodeformat,
+ bitmap->name) <
0)
+ return -1;
+ }
+
+ if (dstbitmapname ||
+ !(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ target, curbitmap))) {
+
+ if (qemuMonitorTransactionBitmapAdd(act,
+ target->nodeformat,
+ mergebitmapname,
+ mergebitmappersistent,
+ mergebitmapdisabled,
+ granularity) < 0)
+ return -1;
+ }
+
+ if (writebitmapsrc &&
+ qemuMonitorTransactionBitmapMergeSourceAddBitmap(merge,
+
writebitmapsrc->nodeformat,
+
"libvirt-tmp-activewrite") < 0)
+ return -1;
+
+ if (qemuMonitorTransactionBitmapMerge(act, target->nodeformat,
+ mergebitmapname, &merge) < 0)
+ return -1;
+ }
+
+ if (writebitmapsrc &&
+ qemuMonitorTransactionBitmapRemove(act, writebitmapsrc->nodeformat,
+ "libvirt-tmp-activewrite") < 0)
+ return -1;
+
+ if (virJSONValueArraySize(act) > 0)
+ *actions = g_steal_pointer(&act);
+
+ return 0;
+}
+
+
/**
* qemuBlockBitmapChainIsValid:
*
diff --git a/src/qemu/qemu_block.h b/src/qemu/qemu_block.h
index 2ad2ce1a1f..0f5b4cfaa9 100644
--- a/src/qemu/qemu_block.h
+++ b/src/qemu/qemu_block.h
@@ -221,6 +221,16 @@ virHashTablePtr
qemuBlockGetNamedNodeData(virDomainObjPtr vm,
qemuDomainAsyncJob asyncJob);
+int
+qemuBlockGetBitmapMergeActions(virStorageSourcePtr topsrc,
+ virStorageSourcePtr basesrc,
+ virStorageSourcePtr target,
+ const char *bitmapname,
+ const char *dstbitmapname,
+ virStorageSourcePtr writebitmapsrc,
+ virJSONValuePtr *actions,
+ virHashTablePtr blockNamedNodeData);
+
bool
qemuBlockBitmapChainIsValid(virStorageSourcePtr src,
const char *bitmapname,
--
2.26.2
4:19 p.m.
New subject: [PATCH 22/32] qemu: block: Add universal helper for merging dirty bitmaps for all scenarios
On 6/15/20 12:10 PM, Peter Krempa wrote:
Add a function which allows to merge bitmaps according to the new
Another reminder about "allows to $VERB" being non-idiomatic :)
s/allows to merge/allows merging/
semantics and will allow to replace all the specific ad-hoc functions
s/allow to replace/allow replacing/
currently in use for 'backup', 'block commit',
'block copy' and will
also be usable in the future for 'block pull' and non-shared storage
migration.
The semantics are a bit quirky for the 'backup' case but these quirks
are documented and will prevent us from having two slightly different
algorithms.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 172 ++++++++++++++++++++++++++++++++++++++++++
src/qemu/qemu_block.h | 10 +++
2 files changed, 182 insertions(+)
I'll have to read the later patches that start using this in place of
their current ad-hoc usage, but in isolation, the new function looks
reasonable.
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 83e3df9601..86f8410ce7 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -2847,6 +2847,178 @@ qemuBlockGetNamedNodeData(virDomainObjPtr vm,
}
+/**
+ * qemuBlockGetBitmapMergeActionsGetBitmaps:
+ *
+ * Collect a list of bitmaps which need to be handled in
+ * qemuBlockGetBitmapMergeActions. The list contains only valid bitmaps in the
+ * sub-chain which is being processed.
+ *
+ * Note that the returned GSList contains bitmap names string pointers borrowed
+ * from @blockNamedNodeData so they must not be freed.
+ */
+static GSList *
+qemuBlockGetBitmapMergeActionsGetBitmaps(virStorageSourcePtr topsrc,
+ const char *bitmapname,
+ virHashTablePtr blockNamedNodeData)
+{
+ g_autoptr(GSList) ret = NULL;
+ qemuBlockNamedNodeDataPtr entry;
+ size_t i;
+
+ /* for now it doesn't make sense to consider bitmaps which are not present
+ * in @topsrc as we can't recreate a bitmap for a layer if it's missing */
I'm guessing qemu adding block-dirty-bitmap-populate will somewhat help
here (if it's missing in top, but present in a backing layer, the fix is
to populate the bitmap in the top layer according to the allocation map).
+
+ if (!(entry = virHashLookup(blockNamedNodeData, topsrc->nodeformat)))
+ return NULL;
+
+ for (i = 0; i < entry->nbitmaps; i++) {
+ qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
+
+ if (bitmapname &&
+ STRNEQ(bitmapname, bitmap->name))
+ continue;
+
+ if (!qemuBlockBitmapChainIsValid(topsrc, bitmap->name, blockNamedNodeData))
+ continue;
+
+ ret = g_slist_prepend(ret, bitmap->name);
+ }
+
+ return g_steal_pointer(&ret);
+}
+
+
+/**
+ * qemuBlockGetBitmapMergeActions:
+ * @topsrc: top of the chain to merge bitmaps in
+ * @basesrc: bottom of the chain to merge bitmaps in (NULL for full chain)
+ * @target: destination storage source of the merge (may be part of original chain)
+ * @bitmapname: name of bitmap to perform the merge (NULL for all bitmaps)
+ * @dstbitmapname: name of destination bitmap of the merge (see below for caveats)
+ * @writebitmapsrc: storage source corresponding to the node containing the write
temporary bitmap
+ * @actions: returns actions for a 'transaction' QMP command for executing the
merge
+ * @blockNamedNodeData: hash table filled with qemuBlockNamedNodeData
+ *
+ * Calculate handling of dirty block bitmaps between @topsrc and @basesrc. If
+ * @basesrc is NULL the end of the chain is considered. @target is the destination
+ * storage source definition of the merge and may or may not be part of the
+ * merged chain.
+ *
+ * Specifically the merging algorithm ensures that each considered bitmap is
+ * merged with the appropriate bitmaps so that it properly describes
+ * the state of dirty blocks when looked at from @topsrc based on the depth
+ * of the backing chain where the bitmap is placed.
+ *
+ * If @bitmapname is non-NULL only bitmaps with that name are handled, otherwise
+ * all bitmaps are considered.
+ *
+ * If @dstbitmap is non-NULL everything is merged into a bitmap with that name,
+ * otherwise each bitmap is merged into a bitmap with the same name into @target.
+ * Additionally if @dstbitmap is non-NULL the target bitmap is created as
'inactive'
+ * and 'transient' as a special case for the backup operation.
+ *
+ * If @writebitmapsrc is non-NULL, the 'libvirt-tmp-activewrite' bitmap from
+ * given node is merged along with others. This bitmap corresponds to the writes
+ * which occured between an active layer job finished and the rest of the bitmap
occurred
+ * merging.
+ *
+ * If the bitmap is not valid somehow (see qemuBlockBitmapChainIsValid) given
+ * bitmap is silently skipped, so callers must ensure that given bitmap is valid
+ * if they care about it.
+ *
+ * The resulting 'transacion' QMP command actions are filled in and returned via
transaction
+ * @actions.
+ *
+ * Note that @actions may be NULL if no merging is required.
+ */
+int
+qemuBlockGetBitmapMergeActions(virStorageSourcePtr topsrc,
+ virStorageSourcePtr basesrc,
+ virStorageSourcePtr target,
+ const char *bitmapname,
+ const char *dstbitmapname,
+ virStorageSourcePtr writebitmapsrc,
+ virJSONValuePtr *actions,
+ virHashTablePtr blockNamedNodeData)
Looks a bit odd to have an output parameter (actions) prior to an input
parameter (blockNamedNodeData), but not fatal.
+{
+ g_autoptr(virJSONValue) act = virJSONValueNewArray();
+ virStorageSourcePtr n;
+
+ g_autoptr(GSList) bitmaps = NULL;
+ GSList *next;
+
+ if (!(bitmaps = qemuBlockGetBitmapMergeActionsGetBitmaps(topsrc, bitmapname,
+ blockNamedNodeData)))
+ return 0;
Look up one or all bitmaps in the top layer (based on whether bitmapname
is set),
+
+ for (next = bitmaps; next; next = next->next) {
+ const char *curbitmap = next->data;
+ const char *mergebitmapname = dstbitmapname;
+ bool mergebitmappersistent = false;
+ bool mergebitmapdisabled = true;
+ g_autoptr(virJSONValue) merge = virJSONValueNewArray();
+ unsigned long long granularity = 0;
+ qemuBlockNamedNodeDataBitmapPtr bitmap;
+
+ /* explicitly named destinations mean that we want a temporary
+ * disabled bitmap only, so undo the default for non-explicit cases */
+ if (!mergebitmapname) {
+ mergebitmapname = curbitmap;
+ mergebitmappersistent = true;
+ mergebitmapdisabled = false;
+ }
+
+ for (n = topsrc; virStorageSourceIsBacking(n) && n != basesrc; n =
n->backingStore) {
+ if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ n, curbitmap)))
+ continue;
+
+ if (granularity == 0)
+ granularity = bitmap->granularity;
+
+ if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(merge,
+ n->nodeformat,
+ bitmap->name) <
0)
+ return -1;
+ }
So this loop merges the same bitmap name along all disks in the backing
chain where it is present,
+
+ if (dstbitmapname ||
+ !(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
+ target, curbitmap))) {
+
+ if (qemuMonitorTransactionBitmapAdd(act,
+ target->nodeformat,
+ mergebitmapname,
+ mergebitmappersistent,
+ mergebitmapdisabled,
+ granularity) < 0)
+ return -1;
+ }
if a destination was set or if the bitmap does not exist in the target
layer, then we add in a bitmap for the destination,
+
+ if (writebitmapsrc &&
+ qemuMonitorTransactionBitmapMergeSourceAddBitmap(merge,
+
writebitmapsrc->nodeformat,
+
"libvirt-tmp-activewrite") < 0)
+ return -1;
if a temporary bitmap was needed, we add that in too,
+
+ if (qemuMonitorTransactionBitmapMerge(act, target->nodeformat,
+ mergebitmapname, &merge) < 0)
+ return -1;
and repeat a merge inside the single transaction for each bitmap involved.
+ }
+
+ if (writebitmapsrc &&
+ qemuMonitorTransactionBitmapRemove(act, writebitmapsrc->nodeformat,
+ "libvirt-tmp-activewrite") < 0)
+ return -1;
The transaction also removes any temporary bitmap. You only do this
once instead of per-bitmap; I'm guessing that means that writebitmapsrc
can only be used when bitmapname is non-NULL. Should there be a check
for that?
+
+ if (virJSONValueArraySize(act) > 0)
+ *actions = g_steal_pointer(&act);
+
+ return 0;
+}
+
+
Otherwise, it seems to make sense as a generalized tool. We'll see how
later patches use it.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 23/32] qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics
Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc
iplementatio of bitmap merging for backup. The new approach is simpler
and also more robust in case some of the bitmaps break as they remove
the dependancy on the whole chain of bitmaps working.
The new approach also allows backups if a snapshot is created outside of
libvirt.
Additionally the code is greatly simplified.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 220 +++---------------
src/qemu/qemu_backup.h | 12 +-
tests/qemublocktest.c | 86 ++-----
.../backupmerge/empty-out.json | 4 +-
4 files changed, 63 insertions(+), 259 deletions(-)
diff --git a/src/qemu/qemu_backup.c b/src/qemu/qemu_backup.c
index 495da86ff0..7c337846dc 100644
--- a/src/qemu/qemu_backup.c
+++ b/src/qemu/qemu_backup.c
@@ -173,199 +173,58 @@ qemuBackupDiskDataCleanup(virDomainObjPtr vm,
}
-/**
- * qemuBackupBeginCollectIncrementalCheckpoints:
- * @vm: domain object
- * @incrFrom: name of checkpoint representing starting point of incremental backup
- *
- * Returns a NULL terminated list of pointers to checkpoint definitions in
- * chronological order starting from the 'current' checkpoint until reaching
- * @incrFrom.
- */
-static virDomainMomentDefPtr *
-qemuBackupBeginCollectIncrementalCheckpoints(virDomainObjPtr vm,
- const char *incrFrom)
-{
- virDomainMomentObjPtr n = virDomainCheckpointGetCurrent(vm->checkpoints);
- g_autofree virDomainMomentDefPtr *incr = NULL;
- size_t nincr = 0;
-
- while (n) {
- virDomainMomentDefPtr def = n->def;
-
- if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
- return NULL;
-
- if (STREQ(def->name, incrFrom)) {
- def = NULL;
- if (VIR_APPEND_ELEMENT_COPY(incr, nincr, def) < 0)
- return NULL;
-
- return g_steal_pointer(&incr);
- }
-
- if (!n->def->parent_name)
- break;
-
- n = virDomainCheckpointFindByName(vm->checkpoints,
n->def->parent_name);
- }
-
- virReportError(VIR_ERR_OPERATION_INVALID,
- _("could not locate checkpoint '%s' for incremental
backup"),
- incrFrom);
- return NULL;
-}
-
-
-static int
-qemuBackupGetBitmapMergeRange(virStorageSourcePtr from,
- const char *bitmapname,
- virJSONValuePtr *actions,
- virStorageSourcePtr *to,
- const char *diskdst,
- virHashTablePtr blockNamedNodeData)
+int
+qemuBackupDiskPrepareOneBitmapsChain(virStorageSourcePtr backingChain,
+ virStorageSourcePtr targetsrc,
+ const char *targetbitmap,
+ const char *incremental,
+ virJSONValuePtr actions,
+ virHashTablePtr blockNamedNodeData)
{
- g_autoptr(virJSONValue) ret = virJSONValueNewArray();
- virStorageSourcePtr tmpsrc = NULL;
- virStorageSourcePtr n;
- bool foundbitmap = false;
+ g_autoptr(virJSONValue) tmpactions = NULL;
- for (n = from; virStorageSourceIsBacking(n); n = n->backingStore) {
- qemuBlockNamedNodeDataBitmapPtr bitmap = NULL;
-
- if (!(bitmap = qemuBlockNamedNodeDataGetBitmapByName(blockNamedNodeData,
- n,
- bitmapname)))
- break;
-
- foundbitmap = true;
-
- if (bitmap->inconsistent) {
- virReportError(VIR_ERR_INVALID_ARG,
- _("bitmap '%s' for image '%s%u' is
inconsistent"),
- bitmap->name, diskdst, n->id);
- return -1;
- }
-
- if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(ret,
- n->nodeformat,
- bitmapname) < 0)
- return -1;
-
- tmpsrc = n;
- }
-
- if (!foundbitmap) {
- virReportError(VIR_ERR_INVALID_ARG,
- _("failed to find bitmap '%s' in image
'%s%u'"),
- bitmapname, diskdst, from->id);
+ if (qemuBlockGetBitmapMergeActions(backingChain, NULL, targetsrc,
+ incremental, targetbitmap, NULL,
+ &tmpactions,
+ blockNamedNodeData) < 0)
return -1;
- }
- *actions = g_steal_pointer(&ret);
- *to = tmpsrc;
+ if (tmpactions &&
+ virJSONValueArrayConcat(actions, tmpactions) < 0)
+ return -1;
return 0;
}
-virJSONValuePtr
-qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr *incremental,
- virStorageSourcePtr backingChain,
- virHashTablePtr blockNamedNodeData,
- const char *diskdst)
-{
- g_autoptr(virJSONValue) ret = NULL;
- size_t incridx = 0;
- virStorageSourcePtr n = backingChain;
-
- ret = virJSONValueNewArray();
-
- for (incridx = 0; incremental[incridx]; incridx++) {
- g_autoptr(virJSONValue) tmp = virJSONValueNewArray();
- virStorageSourcePtr tmpsrc = NULL;
- virDomainCheckpointDefPtr chkdef = (virDomainCheckpointDefPtr)
incremental[incridx];
- bool checkpoint_has_disk = false;
- size_t i;
-
- for (i = 0; i < chkdef->ndisks; i++) {
- if (STRNEQ_NULLABLE(diskdst, chkdef->disks[i].name))
- continue;
-
- if (chkdef->disks[i].type == VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP)
- checkpoint_has_disk = true;
-
- break;
- }
-
- if (!checkpoint_has_disk) {
- if (!incremental[incridx + 1]) {
- virReportError(VIR_ERR_INVALID_ARG,
- _("disk '%s' not found in checkpoint
'%s'"),
- diskdst, incremental[incridx]->name);
- return NULL;
- }
-
- continue;
- }
-
- if (qemuBackupGetBitmapMergeRange(n, incremental[incridx]->name,
- &tmp, &tmpsrc, diskdst,
- blockNamedNodeData) < 0)
- return NULL;
-
- if (virJSONValueArrayConcat(ret, tmp) < 0)
- return NULL;
-
- n = tmpsrc;
- }
-
- return g_steal_pointer(&ret);
-}
-
-
static int
qemuBackupDiskPrepareOneBitmaps(struct qemuBackupDiskData *dd,
virJSONValuePtr actions,
- virDomainMomentDefPtr *incremental,
virHashTablePtr blockNamedNodeData)
{
- g_autoptr(virJSONValue) mergebitmaps = NULL;
- g_autoptr(virJSONValue) mergebitmapsstore = NULL;
-
- if (!(mergebitmaps = qemuBackupDiskPrepareOneBitmapsChain(incremental,
- dd->domdisk->src,
- blockNamedNodeData,
- dd->domdisk->dst)))
- return -1;
-
- if (!(mergebitmapsstore = virJSONValueCopy(mergebitmaps)))
- return -1;
-
- if (qemuMonitorTransactionBitmapAdd(actions,
- dd->domdisk->src->nodeformat,
- dd->incrementalBitmap,
- false,
- true, 0) < 0)
- return -1;
-
- if (qemuMonitorTransactionBitmapMerge(actions,
- dd->domdisk->src->nodeformat,
- dd->incrementalBitmap,
- &mergebitmaps) < 0)
+ if (!qemuBlockBitmapChainIsValid(dd->domdisk->src,
+ dd->backupdisk->incremental,
+ blockNamedNodeData)) {
+ virReportError(VIR_ERR_INVALID_ARG,
+ _("missing or broken bitmap '%s' for disk
'%s'"),
+ dd->backupdisk->incremental, dd->domdisk->dst);
return -1;
+ }
- if (qemuMonitorTransactionBitmapAdd(actions,
- dd->store->nodeformat,
- dd->incrementalBitmap,
- false,
- true, 0) < 0)
+ if (qemuBackupDiskPrepareOneBitmapsChain(dd->domdisk->src,
+ dd->domdisk->src,
+ dd->incrementalBitmap,
+ dd->backupdisk->incremental,
+ actions,
+ blockNamedNodeData) < 0)
return -1;
- if (qemuMonitorTransactionBitmapMerge(actions,
- dd->store->nodeformat,
- dd->incrementalBitmap,
- &mergebitmapsstore) < 0)
+ if (qemuBackupDiskPrepareOneBitmapsChain(dd->domdisk->src,
+ dd->store,
+ dd->incrementalBitmap,
+ dd->backupdisk->incremental,
+ actions,
+ blockNamedNodeData) < 0)
return -1;
return 0;
@@ -382,7 +241,6 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
virQEMUDriverConfigPtr cfg)
{
qemuDomainObjPrivatePtr priv = vm->privateData;
- g_autofree virDomainMomentDefPtr *incremental = NULL;
/* set data structure */
dd->backupdisk = backupdisk;
@@ -420,16 +278,12 @@ qemuBackupDiskPrepareDataOne(virDomainObjPtr vm,
return -1;
if (dd->backupdisk->incremental) {
- if (!(incremental = qemuBackupBeginCollectIncrementalCheckpoints(vm,
dd->backupdisk->incremental)))
- return -1;
-
if (dd->backupdisk->exportbitmap)
dd->incrementalBitmap = g_strdup(dd->backupdisk->exportbitmap);
else
dd->incrementalBitmap = g_strdup_printf("backup-%s",
dd->domdisk->dst);
- if (qemuBackupDiskPrepareOneBitmaps(dd, actions, incremental,
- blockNamedNodeData) < 0)
+ if (qemuBackupDiskPrepareOneBitmaps(dd, actions, blockNamedNodeData) < 0)
return -1;
}
@@ -883,8 +737,8 @@ qemuBackupBegin(virDomainObjPtr vm,
if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm, QEMU_ASYNC_JOB_BACKUP)))
goto endjob;
- if ((ndd = qemuBackupDiskPrepareData(vm, def, blockNamedNodeData,
- actions, cfg, &dd)) <= 0) {
+ if ((ndd = qemuBackupDiskPrepareData(vm, def, blockNamedNodeData, actions,
+ cfg, &dd)) <= 0) {
if (ndd == 0) {
virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
_("no disks selected for backup"));
diff --git a/src/qemu/qemu_backup.h b/src/qemu/qemu_backup.h
index b19c3bf1c9..075fde709b 100644
--- a/src/qemu/qemu_backup.h
+++ b/src/qemu/qemu_backup.h
@@ -53,8 +53,10 @@ qemuBackupGetJobInfoStats(virQEMUDriverPtr driver,
qemuDomainJobInfoPtr jobInfo);
/* exported for testing */
-virJSONValuePtr
-qemuBackupDiskPrepareOneBitmapsChain(virDomainMomentDefPtr *incremental,
- virStorageSourcePtr backingChain,
- virHashTablePtr blockNamedNodeData,
- const char *diskdst);
+int
+qemuBackupDiskPrepareOneBitmapsChain(virStorageSourcePtr backingChain,
+ virStorageSourcePtr targetsrc,
+ const char *targetbitmap,
+ const char *incremental,
+ virJSONValuePtr actions,
+ virHashTablePtr blockNamedNodeData);
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 9e142a8851..30662f6f79 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -716,65 +716,6 @@ testQemuBitmapGetFakeChainEntry(virStorageSourcePtr src,
}
-typedef virDomainMomentDefPtr testMomentList;
-
-static void
-testMomentListFree(testMomentList *list)
-{
- testMomentList *tmp = list;
-
- if (!list)
- return;
-
- while (*tmp) {
- virObjectUnref(*tmp);
- tmp++;
- }
-
- g_free(list);
-}
-
-G_DEFINE_AUTOPTR_CLEANUP_FUNC(testMomentList, testMomentListFree);
-
-static virDomainMomentDefPtr
-testQemuBackupGetIncrementalMoment(const char *name)
-{
- virDomainCheckpointDefPtr checkpoint = NULL;
-
- if (!(checkpoint = virDomainCheckpointDefNew()))
- abort();
-
- checkpoint->disks = g_new0(virDomainCheckpointDiskDef, 1);
- checkpoint->ndisks = 1;
-
- checkpoint->disks[0].name = g_strdup("testdisk");
- checkpoint->disks[0].type = VIR_DOMAIN_CHECKPOINT_TYPE_BITMAP;
-
- checkpoint->parent.name = g_strdup(name);
-
- return (virDomainMomentDefPtr) checkpoint;
-}
-
-
-static virDomainMomentDefPtr *
-testQemuBackupGetIncremental(const char *incFrom)
-{
- const char *checkpoints[] = {"current", "d", "c",
"b", "a"};
- virDomainMomentDefPtr *incr;
- size_t i;
-
- incr = g_new0(virDomainMomentDefPtr, G_N_ELEMENTS(checkpoints) + 1);
-
- for (i = 0; i < G_N_ELEMENTS(checkpoints); i++) {
- incr[i] = testQemuBackupGetIncrementalMoment(checkpoints[i]);
-
- if (STREQ(incFrom, checkpoints[i]))
- break;
- }
-
- return incr;
-}
-
static const char *backupDataPrefix = "qemublocktestdata/backupmerge/";
struct testQemuBackupIncrementalBitmapCalculateData {
@@ -791,10 +732,10 @@ testQemuBackupIncrementalBitmapCalculate(const void *opaque)
const struct testQemuBackupIncrementalBitmapCalculateData *data = opaque;
g_autoptr(virJSONValue) nodedatajson = NULL;
g_autoptr(virHashTable) nodedata = NULL;
- g_autoptr(virJSONValue) mergebitmaps = NULL;
- g_autofree char *actual = NULL;
+ g_autoptr(virJSONValue) actions = virJSONValueNewArray();
g_autofree char *expectpath = NULL;
- g_autoptr(testMomentList) incremental = NULL;
+ g_autoptr(virStorageSource) target = NULL;
+ g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
expectpath = g_strdup_printf("%s/%s%s-out.json", abs_srcdir,
backupDataPrefix, data->name);
@@ -808,19 +749,24 @@ testQemuBackupIncrementalBitmapCalculate(const void *opaque)
return -1;
}
- incremental = testQemuBackupGetIncremental(data->incremental);
+ if (!(target = virStorageSourceNew()))
+ return -1;
- if ((mergebitmaps = qemuBackupDiskPrepareOneBitmapsChain(incremental,
- data->chain,
- nodedata,
- "testdisk"))) {
- if (!(actual = virJSONValueToString(mergebitmaps, true)))
+ target->nodeformat = g_strdup_printf("target_node");
+
+ if (qemuBackupDiskPrepareOneBitmapsChain(data->chain,
+ target,
+ "target-bitmap-name",
+ data->incremental,
+ actions,
+ nodedata) >= 0) {
+ if (virJSONValueToBuffer(actions, &buf, true) < 0)
return -1;
} else {
- actual = g_strdup("NULL\n");
+ virBufferAddLit(&buf, "NULL\n");
}
- return virTestCompareToFile(actual, expectpath);
+ return virTestCompareToFile(virBufferCurrentContent(&buf), expectpath);
}
diff --git a/tests/qemublocktestdata/backupmerge/empty-out.json
b/tests/qemublocktestdata/backupmerge/empty-out.json
index 7951defec1..41b42e677b 100644
--- a/tests/qemublocktestdata/backupmerge/empty-out.json
+++ b/tests/qemublocktestdata/backupmerge/empty-out.json
@@ -1 +1,3 @@
-NULL
+[
+
+]
--
2.26.2
4:41 p.m.
New subject: [PATCH 23/32] qemu: backup: Rewrite backup bitmap handling to the new bitmap semantics
On 6/15/20 12:10 PM, Peter Krempa wrote:
Reuse qemuBlockGetBitmapMergeActions which allows to remove the
ad-hoc
allows the removal of the
iplementatio of bitmap merging for backup. The new approach is
simpler
implementation
and also more robust in case some of the bitmaps break as they
remove
the dependancy on the whole chain of bitmaps working.
dependency
The new approach also allows backups if a snapshot is created outside of
libvirt.
Additionally the code is greatly simplified.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_backup.c | 220 +++---------------
src/qemu/qemu_backup.h | 12 +-
tests/qemublocktest.c | 86 ++-----
.../backupmerge/empty-out.json | 4 +-
4 files changed, 63 insertions(+), 259 deletions(-)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 24/32] qemublocktest: Add 'basic' tests for backup bitmap handling
The 'basic' case is just a single backing store layer containing the
bitmaps so we just copy the bitmaps over to the backup bitmap.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 +++
.../backupmerge/basic-deep-out.json | 25 +++++++++++++++++++
.../backupmerge/basic-flat-out.json | 25 +++++++++++++++++++
.../backupmerge/basic-intermediate-out.json | 25 +++++++++++++++++++
4 files changed, 79 insertions(+)
create mode 100644 tests/qemublocktestdata/backupmerge/basic-deep-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/basic-flat-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 30662f6f79..1300e0cbe9 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1263,6 +1263,10 @@ mymain(void)
TEST_BACKUP_BITMAP_CALCULATE("empty", bitmapSourceChain, "a",
"empty");
+ TEST_BACKUP_BITMAP_CALCULATE("basic-flat", bitmapSourceChain,
"current", "basic");
+ TEST_BACKUP_BITMAP_CALCULATE("basic-intermediate", bitmapSourceChain,
"d", "basic");
+ TEST_BACKUP_BITMAP_CALCULATE("basic-deep", bitmapSourceChain,
"a", "basic");
+
#define TEST_CHECKPOINT_DELETE(testname, delbmp, named) \
do { \
checkpointdeletedata.name = testname; \
diff --git a/tests/qemublocktestdata/backupmerge/basic-deep-out.json
b/tests/qemublocktestdata/backupmerge/basic-deep-out.json
new file mode 100644
index 0000000000..ff77af789b
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/basic-deep-out.json
@@ -0,0 +1,25 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/backupmerge/basic-flat-out.json
b/tests/qemublocktestdata/backupmerge/basic-flat-out.json
new file mode 100644
index 0000000000..4637bbc377
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/basic-flat-out.json
@@ -0,0 +1,25 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
b/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
new file mode 100644
index 0000000000..f2f3b3f568
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
@@ -0,0 +1,25 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ }
+ ]
+ }
+ }
+]
--
2.26.2
4:57 p.m.
New subject: [PATCH 24/32] qemublocktest: Add 'basic' tests for backup bitmap handling
On 6/15/20 12:10 PM, Peter Krempa wrote:
The 'basic' case is just a single backing store layer
containing the
bitmaps so we just copy the bitmaps over to the backup bitmap.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 +++
.../backupmerge/basic-deep-out.json | 25 +++++++++++++++++++
.../backupmerge/basic-flat-out.json | 25 +++++++++++++++++++
.../backupmerge/basic-intermediate-out.json | 25 +++++++++++++++++++
4 files changed, 79 insertions(+)
create mode 100644 tests/qemublocktestdata/backupmerge/basic-deep-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/basic-flat-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/basic-intermediate-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 25/32] qemublocktest: Add 'snapshots' tests for backup bitmap handling
The 'snapshots' case has multiple layers so we need to make sure that
the bitmaps are merged with the appropriate temporary bitmaps formatted
from the allocation bitmap for any backing chain layer above.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 ++
.../backupmerge/snapshots-deep-out.json | 41 +++++++++++++++++++
.../backupmerge/snapshots-flat-out.json | 25 +++++++++++
.../snapshots-intermediate-out.json | 29 +++++++++++++
4 files changed, 99 insertions(+)
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 1300e0cbe9..fd6dff82f9 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1267,6 +1267,10 @@ mymain(void)
TEST_BACKUP_BITMAP_CALCULATE("basic-intermediate", bitmapSourceChain,
"d", "basic");
TEST_BACKUP_BITMAP_CALCULATE("basic-deep", bitmapSourceChain,
"a", "basic");
+ TEST_BACKUP_BITMAP_CALCULATE("snapshots-flat", bitmapSourceChain,
"current", "snapshots");
+ TEST_BACKUP_BITMAP_CALCULATE("snapshots-intermediate", bitmapSourceChain,
"d", "snapshots");
+ TEST_BACKUP_BITMAP_CALCULATE("snapshots-deep", bitmapSourceChain,
"a", "snapshots");
+
#define TEST_CHECKPOINT_DELETE(testname, delbmp, named) \
do { \
checkpointdeletedata.name = testname; \
diff --git a/tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
b/tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
new file mode 100644
index 0000000000..d678507b85
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
@@ -0,0 +1,41 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "a"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
b/tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
new file mode 100644
index 0000000000..4637bbc377
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
@@ -0,0 +1,25 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
b/tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
new file mode 100644
index 0000000000..30d8bcd8b7
--- /dev/null
+++ b/tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
@@ -0,0 +1,29 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "target_node",
+ "name": "target-bitmap-name",
+ "persistent": false,
+ "disabled": true,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "target_node",
+ "target": "target-bitmap-name",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ ]
+ }
+ }
+]
--
2.26.2
5:03 p.m.
New subject: [PATCH 25/32] qemublocktest: Add 'snapshots' tests for backup bitmap handling
On 6/15/20 12:10 PM, Peter Krempa wrote:
The 'snapshots' case has multiple layers so we need to make
sure that
the bitmaps are merged with the appropriate temporary bitmaps formatted
from the allocation bitmap for any backing chain layer above.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 ++
.../backupmerge/snapshots-deep-out.json | 41 +++++++++++++++++++
.../backupmerge/snapshots-flat-out.json | 25 +++++++++++
.../snapshots-intermediate-out.json | 29 +++++++++++++
4 files changed, 99 insertions(+)
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-deep-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-flat-out.json
create mode 100644 tests/qemublocktestdata/backupmerge/snapshots-intermediate-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 26/32] qemu: Rewrite bitmap handling for block commit
Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc
implementatio of bitmap merging for block commit. The new approach is
way simpler and more robust and also allows us to get rid of the
disabling of bitmaps done prior to the start as we actually do want to
update the bitmaps in the base.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 203 +-----------------
src/qemu/qemu_block.h | 11 +-
src/qemu/qemu_blockjob.c | 25 ++-
src/qemu/qemu_driver.c | 56 +----
tests/qemublocktest.c | 18 +-
.../qemublocktestdata/bitmapblockcommit/empty | 1 -
6 files changed, 42 insertions(+), 272 deletions(-)
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index 86f8410ce7..e958fb71fa 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -3209,117 +3209,7 @@ qemuBlockBitmapsHandleBlockcopy(virStorageSourcePtr src,
/**
* @topsrc: virStorageSource representing 'top' of the job
* @basesrc: virStorageSource representing 'base' of the job
- * @blockNamedNodeData: hash table containing data about bitmaps
- * @actions: filled with arguments for a 'transaction' command
- * @disabledBitmapsBase: filled with a list of bitmap names which must be disabled
- *
- * Prepares data for correctly handling bitmaps during the start of a commit
- * job. The bitmaps in the 'base' image must be disabled, so that the writes
- * done by the blockjob don't dirty the enabled bitmaps.
- *
- * @actions and @disabledBitmapsBase are untouched if no bitmaps need
- * to be disabled.
- */
-int
-qemuBlockBitmapsHandleCommitStart(virStorageSourcePtr topsrc,
- virStorageSourcePtr basesrc,
- virHashTablePtr blockNamedNodeData,
- virJSONValuePtr *actions,
- char ***disabledBitmapsBase)
-{
- g_autoptr(virJSONValue) act = virJSONValueNewArray();
- VIR_AUTOSTRINGLIST bitmaplist = NULL;
- size_t curbitmapstr = 0;
- qemuBlockNamedNodeDataPtr entry;
- bool disable_bitmaps = false;
- size_t i;
-
- if (!(entry = virHashLookup(blockNamedNodeData, basesrc->nodeformat)))
- return 0;
-
- bitmaplist = g_new0(char *, entry->nbitmaps + 1);
-
- for (i = 0; i < entry->nbitmaps; i++) {
- qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
-
- if (!bitmap->recording || bitmap->inconsistent ||
- !qemuBlockBitmapChainIsValid(topsrc, bitmap->name, blockNamedNodeData))
- continue;
-
- disable_bitmaps = true;
-
- if (qemuMonitorTransactionBitmapDisable(act, basesrc->nodeformat,
- bitmap->name) < 0)
- return -1;
-
- bitmaplist[curbitmapstr++] = g_strdup(bitmap->name);
- }
-
- if (disable_bitmaps) {
- *actions = g_steal_pointer(&act);
- *disabledBitmapsBase = g_steal_pointer(&bitmaplist);
- }
-
- return 0;
-}
-
-
-struct qemuBlockBitmapsHandleCommitData {
- bool skip;
- bool create;
- bool enable;
- const char *basenode;
- virJSONValuePtr merge;
- unsigned long long granularity;
- bool persistent;
-};
-
-
-static void
-qemuBlockBitmapsHandleCommitDataFree(void *opaque)
-{
- struct qemuBlockBitmapsHandleCommitData *data = opaque;
-
- virJSONValueFree(data->merge);
- g_free(data);
-}
-
-
-static int
-qemuBlockBitmapsHandleCommitFinishIterate(void *payload,
- const void *entryname,
- void *opaque)
-{
- struct qemuBlockBitmapsHandleCommitData *data = payload;
- const char *bitmapname = entryname;
- virJSONValuePtr actions = opaque;
-
- if (data->skip)
- return 0;
-
- if (data->create) {
- if (qemuMonitorTransactionBitmapAdd(actions, data->basenode, bitmapname,
- data->persistent, !data->enable,
- data->granularity) < 0)
- return -1;
- } else {
- if (data->enable &&
- qemuMonitorTransactionBitmapEnable(actions, data->basenode, bitmapname)
< 0)
- return -1;
- }
-
- if (data->merge &&
- qemuMonitorTransactionBitmapMerge(actions, data->basenode, bitmapname,
- &data->merge) < 0)
- return -1;
-
- return 0;
-}
-
-
-/**
- * @topsrc: virStorageSource representing 'top' of the job
- * @basesrc: virStorageSource representing 'base' of the job
+ * @active: commit job is an active layer block-commit
* @blockNamedNodeData: hash table containing data about bitmaps
* @actions: filled with arguments for a 'transaction' command
* @disabledBitmapsBase: bitmap names which were disabled
@@ -3332,95 +3222,20 @@ qemuBlockBitmapsHandleCommitFinishIterate(void *payload,
int
qemuBlockBitmapsHandleCommitFinish(virStorageSourcePtr topsrc,
virStorageSourcePtr basesrc,
+ bool active,
virHashTablePtr blockNamedNodeData,
- virJSONValuePtr *actions,
- char **disabledBitmapsBase)
+ virJSONValuePtr *actions)
{
- g_autoptr(virJSONValue) act = virJSONValueNewArray();
- virStorageSourcePtr n;
- qemuBlockNamedNodeDataPtr entry;
- g_autoptr(virHashTable) commitdata = NULL;
- struct qemuBlockBitmapsHandleCommitData *bitmapdata;
- size_t i;
-
- commitdata = virHashNew(qemuBlockBitmapsHandleCommitDataFree);
-
- for (n = topsrc; n != basesrc; n = n->backingStore) {
- if (!(entry = virHashLookup(blockNamedNodeData, n->nodeformat)))
- continue;
-
- for (i = 0; i < entry->nbitmaps; i++) {
- qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
-
- if (!(bitmapdata = virHashLookup(commitdata, bitmap->name))) {
- bitmapdata = g_new0(struct qemuBlockBitmapsHandleCommitData, 1);
+ virStorageSourcePtr writebitmapsrc = NULL;
- /* we must mirror the state of the topmost bitmap and merge
- * everything else */
- bitmapdata->create = true;
- bitmapdata->enable = bitmap->recording;
- bitmapdata->basenode = basesrc->nodeformat;
- bitmapdata->merge = virJSONValueNewArray();
- bitmapdata->granularity = bitmap->granularity;
- bitmapdata->persistent = bitmap->persistent;
+ if (active)
+ writebitmapsrc = basesrc;
- if (virHashAddEntry(commitdata, bitmap->name, bitmapdata) < 0) {
- qemuBlockBitmapsHandleCommitDataFree(bitmapdata);
- return -1;
- }
- }
-
- if (bitmap->inconsistent ||
- !qemuBlockBitmapChainIsValid(topsrc, bitmap->name,
blockNamedNodeData))
- bitmapdata->skip = true;
-
- if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(bitmapdata->merge,
- n->nodeformat,
- bitmap->name) <
0)
- return -1;
- }
- }
-
- if ((entry = virHashLookup(blockNamedNodeData, basesrc->nodeformat))) {
- /* note that all bitmaps in 'base' were disabled when commit was started
*/
- for (i = 0; i < entry->nbitmaps; i++) {
- qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
-
- if ((bitmapdata = virHashLookup(commitdata, bitmap->name))) {
- bitmapdata->create = false;
- } else {
- if (disabledBitmapsBase) {
- char **disabledbitmaps;
-
- for (disabledbitmaps = disabledBitmapsBase; *disabledbitmaps;
disabledbitmaps++) {
- if (STREQ(*disabledbitmaps, bitmap->name)) {
- bitmapdata = g_new0(struct qemuBlockBitmapsHandleCommitData,
1);
-
- bitmapdata->create = false;
- bitmapdata->enable = true;
- bitmapdata->basenode = basesrc->nodeformat;
- bitmapdata->granularity = bitmap->granularity;
- bitmapdata->persistent = bitmap->persistent;
-
- if (virHashAddEntry(commitdata, bitmap->name, bitmapdata)
< 0) {
- qemuBlockBitmapsHandleCommitDataFree(bitmapdata);
- return -1;
- }
-
- break;
- }
- }
- }
- }
- }
- }
-
- if (virHashForEach(commitdata, qemuBlockBitmapsHandleCommitFinishIterate, act) <
0)
+ if (qemuBlockGetBitmapMergeActions(topsrc, basesrc, basesrc, NULL, NULL,
+ writebitmapsrc, actions,
+ blockNamedNodeData) < 0)
return -1;
- if (virJSONValueArraySize(act) > 0)
- *actions = g_steal_pointer(&act);
-
return 0;
}
diff --git a/src/qemu/qemu_block.h b/src/qemu/qemu_block.h
index 0f5b4cfaa9..24b87e79db 100644
--- a/src/qemu/qemu_block.h
+++ b/src/qemu/qemu_block.h
@@ -243,19 +243,12 @@ qemuBlockBitmapsHandleBlockcopy(virStorageSourcePtr src,
bool shallow,
virJSONValuePtr *actions);
-int
-qemuBlockBitmapsHandleCommitStart(virStorageSourcePtr topsrc,
- virStorageSourcePtr basesrc,
- virHashTablePtr blockNamedNodeData,
- virJSONValuePtr *actions,
- char ***disabledBitmapsBase);
-
int
qemuBlockBitmapsHandleCommitFinish(virStorageSourcePtr topsrc,
virStorageSourcePtr basesrc,
+ bool active,
virHashTablePtr blockNamedNodeData,
- virJSONValuePtr *actions,
- char **disabledBitmapsBase);
+ virJSONValuePtr *actions);
int
qemuBlockReopenReadWrite(virDomainObjPtr vm,
diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c
index 17dc08476b..8e81b142f4 100644
--- a/src/qemu/qemu_blockjob.c
+++ b/src/qemu/qemu_blockjob.c
@@ -1069,6 +1069,7 @@ qemuBlockJobProcessEventCompletedCommitBitmaps(virDomainObjPtr vm,
qemuDomainObjPrivatePtr priv = vm->privateData;
g_autoptr(virHashTable) blockNamedNodeData = NULL;
g_autoptr(virJSONValue) actions = NULL;
+ bool active = job->type == QEMU_BLOCKJOB_TYPE_ACTIVE_COMMIT;
if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_BLOCKDEV_REOPEN))
return 0;
@@ -1078,16 +1079,18 @@ qemuBlockJobProcessEventCompletedCommitBitmaps(virDomainObjPtr
vm,
if (qemuBlockBitmapsHandleCommitFinish(job->data.commit.top,
job->data.commit.base,
+ active,
blockNamedNodeData,
- &actions,
- job->data.commit.disabledBitmapsBase) <
0)
+ &actions) < 0)
return 0;
if (!actions)
return 0;
- if (qemuBlockReopenReadWrite(vm, job->data.commit.base, asyncJob) < 0)
- return -1;
+ if (!active) {
+ if (qemuBlockReopenReadWrite(vm, job->data.commit.base, asyncJob) < 0)
+ return -1;
+ }
if (qemuDomainObjEnterMonitorAsync(priv->driver, vm, asyncJob) < 0)
return -1;
@@ -1097,8 +1100,10 @@ qemuBlockJobProcessEventCompletedCommitBitmaps(virDomainObjPtr vm,
if (qemuDomainObjExitMonitor(priv->driver, vm) < 0)
return -1;
- if (qemuBlockReopenReadOnly(vm, job->data.commit.base, asyncJob) < 0)
- return -1;
+ if (!active) {
+ if (qemuBlockReopenReadOnly(vm, job->data.commit.base, asyncJob) < 0)
+ return -1;
+ }
return 0;
}
@@ -1268,6 +1273,9 @@ qemuBlockJobProcessEventCompletedActiveCommit(virQEMUDriverPtr
driver,
job->disk->src = job->data.commit.base;
job->disk->src->readonly = job->data.commit.top->readonly;
+ if (qemuBlockJobProcessEventCompletedCommitBitmaps(vm, job, asyncJob) < 0)
+ return;
+
qemuBlockJobEventProcessConcludedRemoveChain(driver, vm, asyncJob,
job->data.commit.top);
if (job->data.commit.deleteCommittedImages)
@@ -1339,6 +1347,7 @@ qemuBlockJobProcessEventFailedActiveCommit(virQEMUDriverPtr driver,
virDomainObjPtr vm,
qemuBlockJobDataPtr job)
{
+ g_autoptr(virJSONValue) actions = virJSONValueNewArray();
virDomainDiskDefPtr disk = job->disk;
VIR_DEBUG("active commit job '%s' on VM '%s' failed",
job->name, vm->def->name);
@@ -1346,6 +1355,10 @@ qemuBlockJobProcessEventFailedActiveCommit(virQEMUDriverPtr
driver,
if (!disk)
return;
+ ignore_value(qemuMonitorTransactionBitmapRemove(actions,
disk->mirror->nodeformat,
+
"libvirt-tmp-activewrite"));
+
+
/* Ideally, we would make the backing chain read only again (yes, SELinux
* can do that using different labels). But that is not implemented yet and
* not leaking security driver metadata is more important. */
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 88517ba6a7..6256046762 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -17312,9 +17312,9 @@ qemuDomainBlockPivot(virQEMUDriverPtr driver,
case QEMU_BLOCKJOB_TYPE_COPY:
if (blockdev && !job->jobflagsmissing) {
- g_autoptr(virHashTable) blockNamedNodeData = NULL;
bool shallow = job->jobflags & VIR_DOMAIN_BLOCK_COPY_SHALLOW;
bool reuse = job->jobflags & VIR_DOMAIN_BLOCK_COPY_REUSE_EXT;
+ g_autoptr(virHashTable) blockNamedNodeData = NULL;
if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm,
QEMU_ASYNC_JOB_NONE)))
return -1;
@@ -17352,16 +17352,15 @@ qemuDomainBlockPivot(virQEMUDriverPtr driver,
* the bitmaps if it wasn't present thus must skip this */
if (blockdev &&
virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_BLOCKDEV_REOPEN)) {
- g_autoptr(virHashTable) blockNamedNodeData = NULL;
- if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm,
QEMU_ASYNC_JOB_NONE)))
- return -1;
+ actions = virJSONValueNewArray();
- if (qemuBlockBitmapsHandleCommitFinish(job->data.commit.top,
- job->data.commit.base,
- blockNamedNodeData,
- &actions,
-
job->data.commit.disabledBitmapsBase) < 0)
+ if (qemuMonitorTransactionBitmapAdd(actions,
+ job->data.commit.base->nodeformat,
+ "libvirt-tmp-activewrite",
+ false,
+ false,
+ 0) < 0)
return -1;
}
@@ -18482,7 +18481,6 @@ qemuDomainBlockCommit(virDomainPtr dom,
const char *nodebase = NULL;
bool persistjob = false;
bool blockdev = false;
- g_autoptr(virJSONValue) bitmapDisableActions = NULL;
VIR_AUTOSTRINGLIST bitmapDisableList = NULL;
virCheckFlags(VIR_DOMAIN_BLOCK_COMMIT_SHALLOW |
@@ -18647,27 +18645,6 @@ qemuDomainBlockCommit(virDomainPtr dom,
goto endjob;
}
- if (blockdev &&
- virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_BLOCKDEV_REOPEN)) {
- g_autoptr(virHashTable) blockNamedNodeData = NULL;
- if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm, QEMU_ASYNC_JOB_NONE)))
- goto endjob;
-
- if (qemuBlockBitmapsHandleCommitStart(topSource, baseSource,
- blockNamedNodeData,
- &bitmapDisableActions,
- &bitmapDisableList) < 0)
- goto endjob;
-
- /* if we don't have terminator on 'base' we can't reopen it */
- if (bitmapDisableActions && !baseSource->backingStore) {
- virReportError(VIR_ERR_OPERATION_UNSUPPORTED,
- _("can't handle bitmaps on unterminated backing image
'%s'"),
- base);
- goto endjob;
- }
- }
-
if (!(job = qemuBlockJobDiskNewCommit(vm, disk, top_parent, topSource,
baseSource, &bitmapDisableList,
flags & VIR_DOMAIN_BLOCK_COMMIT_DELETE,
@@ -18692,23 +18669,6 @@ qemuDomainBlockCommit(virDomainPtr dom,
!(backingPath = qemuBlockGetBackingStoreString(baseSource, false)))
goto endjob;
- if (bitmapDisableActions) {
- int rc;
-
- if (qemuBlockReopenReadWrite(vm, baseSource, QEMU_ASYNC_JOB_NONE) < 0)
- goto endjob;
-
- qemuDomainObjEnterMonitor(driver, vm);
- rc = qemuMonitorTransaction(priv->mon, &bitmapDisableActions);
- if (qemuDomainObjExitMonitor(driver, vm) < 0)
- goto endjob;
-
- if (qemuBlockReopenReadOnly(vm, baseSource, QEMU_ASYNC_JOB_NONE) < 0)
- goto endjob;
-
- if (rc < 0)
- goto endjob;
- }
} else {
device = job->name;
}
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index fd6dff82f9..5be98b9320 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -928,12 +928,11 @@ testQemuBlockBitmapBlockcommit(const void *opaque)
g_autofree char *actual = NULL;
g_autofree char *expectpath = NULL;
- g_autoptr(virJSONValue) actionsDisable = NULL;
g_autoptr(virJSONValue) actionsMerge = NULL;
g_autoptr(virJSONValue) nodedatajson = NULL;
g_autoptr(virHashTable) nodedata = NULL;
g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
- VIR_AUTOSTRINGLIST bitmapsDisable = NULL;
+ bool active = data->top == data->chain;
expectpath = g_strdup_printf("%s/%s%s", abs_srcdir,
blockcommitPrefix, data->name);
@@ -947,20 +946,10 @@ testQemuBlockBitmapBlockcommit(const void *opaque)
return -1;
}
- if (qemuBlockBitmapsHandleCommitStart(data->top, data->base, nodedata,
- &actionsDisable, &bitmapsDisable) <
0)
- return -1;
-
- virBufferAddLit(&buf, "pre job bitmap disable:\n");
-
- if (actionsDisable &&
- virJSONValueToBuffer(actionsDisable, &buf, true) < 0)
- return -1;
-
virBufferAddLit(&buf, "merge bitmpas:\n");
- if (qemuBlockBitmapsHandleCommitFinish(data->top, data->base, nodedata,
- &actionsMerge, bitmapsDisable) < 0)
+ if (qemuBlockBitmapsHandleCommitFinish(data->top, data->base, active,
nodedata,
+ &actionsMerge) < 0)
return -1;
if (actionsMerge &&
@@ -1357,6 +1346,7 @@ mymain(void)
#define TEST_BITMAP_BLOCKCOMMIT(testname, topimg, baseimg, ndf) \
do {\
blockbitmapblockcommitdata.name = testname; \
+ blockbitmapblockcommitdata.chain = bitmapSourceChain; \
blockbitmapblockcommitdata.top =
testQemuBitmapGetFakeChainEntry(bitmapSourceChain, topimg); \
blockbitmapblockcommitdata.base =
testQemuBitmapGetFakeChainEntry(bitmapSourceChain, baseimg); \
blockbitmapblockcommitdata.nodedatafile = ndf; \
diff --git a/tests/qemublocktestdata/bitmapblockcommit/empty
b/tests/qemublocktestdata/bitmapblockcommit/empty
index bfc58f994e..9260011852 100644
--- a/tests/qemublocktestdata/bitmapblockcommit/empty
+++ b/tests/qemublocktestdata/bitmapblockcommit/empty
@@ -1,2 +1 @@
-pre job bitmap disable:
merge bitmpas:
--
2.26.2
6:56 p.m.
New subject: [PATCH 26/32] qemu: Rewrite bitmap handling for block commit
On 6/15/20 12:10 PM, Peter Krempa wrote:
Reuse qemuBlockGetBitmapMergeActions which allows to remove the
ad-hoc
s/allows to remove/allows the removal of/
implementatio of bitmap merging for block commit. The new approach is
implementation
way simpler and more robust and also allows us to get rid of the
disabling of bitmaps done prior to the start as we actually do want to
update the bitmaps in the base.
What if the commit fails, but we have merged in portions of the bitmap?
In practice, this merely means that a future attempt to use that bitmap
in the backing file will now copy more data than previously necessary;
what's more, if we try a commit again (if the failure was transient),
we'll be setting those same bits again. So yeah, I don't think it hurts
to not disable the bitmaps in the backing files.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 203 +-----------------
src/qemu/qemu_block.h | 11 +-
src/qemu/qemu_blockjob.c | 25 ++-
src/qemu/qemu_driver.c | 56 +----
tests/qemublocktest.c | 18 +-
.../qemublocktestdata/bitmapblockcommit/empty | 1 -
6 files changed, 42 insertions(+), 272 deletions(-)
@@ -3332,95 +3222,20 @@
qemuBlockBitmapsHandleCommitFinishIterate(void *payload,
int
qemuBlockBitmapsHandleCommitFinish(virStorageSourcePtr topsrc,
virStorageSourcePtr basesrc,
+ bool active,
virHashTablePtr blockNamedNodeData,
- virJSONValuePtr *actions,
- char **disabledBitmapsBase)
+ virJSONValuePtr *actions)
{
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
6:10 a.m.
New subject: [PATCH 26/32] qemu: Rewrite bitmap handling for block commit
On Thu, Jun 18, 2020 at 18:56:15 -0500, Eric Blake wrote:
On 6/15/20 12:10 PM, Peter Krempa wrote:
> Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc
s/allows to remove/allows the removal of/
> implementatio of bitmap merging for block commit. The new approach is
implementation
> way simpler and more robust and also allows us to get rid of the
> disabling of bitmaps done prior to the start as we actually do want to
> update the bitmaps in the base.
What if the commit fails, but we have merged in portions of the bitmap? In
practice, this merely means that a future attempt to use that bitmap in the
backing file will now copy more data than previously necessary; what's more,
if we try a commit again (if the failure was transient), we'll be setting
those same bits again. So yeah, I don't think it hurts to not disable the
bitmaps in the backing files.
Well actually the bitmaps are merged only after the commit has finished
so we update them only once it's done. In the meanwhile all of them
which were present in the image before will get auto-updated with the
commited blocks which is IMO also a good thing.
Regardless, from the full chain point of view it will still look correct
and after a commit failure, the partial chain between the comitted
images is not valid anyways so we can't do much.
12:10 p.m.
New subject: [PATCH 27/32] qemublocktest: Add 'basic' tests for commit bitmap handling
In the 'basic case we have few bitmaps in only the top layer. Simulate
commit into the backing of the top layer and also 2 levels deep.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 +
.../bitmapblockcommit/basic-1-2 | 145 ++++++++++++++++++
.../bitmapblockcommit/basic-1-3 | 145 ++++++++++++++++++
.../bitmapblockcommit/basic-2-3 | 1 +
4 files changed, 295 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-2-3
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 5be98b9320..1a8ce3b73d 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1358,6 +1358,10 @@ mymain(void)
TEST_BITMAP_BLOCKCOMMIT("empty", 1, 2, "empty");
+ TEST_BITMAP_BLOCKCOMMIT("basic-1-2", 1, 2, "basic");
+ TEST_BITMAP_BLOCKCOMMIT("basic-1-3", 1, 3, "basic");
+ TEST_BITMAP_BLOCKCOMMIT("basic-2-3", 2, 3, "basic");
+
cleanup:
qemuTestDriverFree(&driver);
VIR_FREE(capslatest_x86_64);
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
b/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
new file mode 100644
index 0000000000..d393d31044
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
@@ -0,0 +1,145 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
b/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
new file mode 100644
index 0000000000..3fa902c354
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/basic-1-3
@@ -0,0 +1,145 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
b/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
new file mode 100644
index 0000000000..9260011852
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/basic-2-3
@@ -0,0 +1 @@
+merge bitmpas:
--
2.26.2
9:36 a.m.
New subject: [PATCH 27/32] qemublocktest: Add 'basic' tests for commit bitmap handling
On 6/15/20 12:10 PM, Peter Krempa wrote:
In the 'basic case we have few bitmaps in only the top layer.
Simulate
stray or missing '
commit into the backing of the top layer and also 2 levels deep.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 4 +
.../bitmapblockcommit/basic-1-2 | 145 ++++++++++++++++++
.../bitmapblockcommit/basic-1-3 | 145 ++++++++++++++++++
.../bitmapblockcommit/basic-2-3 | 1 +
4 files changed, 295 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/basic-2-3
+++ b/tests/qemublocktestdata/bitmapblockcommit/basic-1-2
@@ -0,0 +1,145 @@
+merge bitmpas:
bitmaps (fixing the source of the typo probably has ripple effects in
expected output, and might even be in a prior patch...)
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
Hmm, so you are merging the 'libvirt-tmp-activewrite' bitmap into bitmap
(libvirt-2-format,a)...
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
...and (libvirt-2-format,b) simultaneously. But that still makes sense:
if libvirt-tmp-activewrite contains bits for everything that has changed
since a certain point in time, and both bitmap a and b will be left
active when the operation is done, they will both need those same bits
merged in.
It took me a while to see why you were reusing a bitmap as a source to
more than one destination, but it looks correct. With the typo fixed,
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 28/32] qemublocktest: Add 'snapshots' tests for block commit bitmap handling
Simulate commit between all the combinations of layers in the
'snapshots' case to see whether the code merges the correct bitmaps with
the correct depth of temporary bitmaps.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 14 ++
.../bitmapblockcommit/snapshots-1-2 | 105 +++++++++++
.../bitmapblockcommit/snapshots-1-3 | 131 ++++++++++++++
.../bitmapblockcommit/snapshots-1-4 | 163 +++++++++++++++++
.../bitmapblockcommit/snapshots-1-5 | 167 ++++++++++++++++++
.../bitmapblockcommit/snapshots-2-3 | 65 +++++++
.../bitmapblockcommit/snapshots-2-4 | 97 ++++++++++
.../bitmapblockcommit/snapshots-2-5 | 101 +++++++++++
.../bitmapblockcommit/snapshots-3-4 | 62 +++++++
.../bitmapblockcommit/snapshots-3-5 | 66 +++++++
.../bitmapblockcommit/snapshots-4-5 | 16 ++
11 files changed, 987 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 1a8ce3b73d..4a737e19d3 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1362,6 +1362,20 @@ mymain(void)
TEST_BITMAP_BLOCKCOMMIT("basic-1-3", 1, 3, "basic");
TEST_BITMAP_BLOCKCOMMIT("basic-2-3", 2, 3, "basic");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-1-2", 1, 2, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-1-3", 1, 3, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-1-4", 1, 4, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-1-5", 1, 5, "snapshots");
+
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-2-3", 2, 3, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-2-4", 2, 4, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-2-5", 2, 5, "snapshots");
+
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-3-4", 3, 4, "snapshots");
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-3-5", 3, 5, "snapshots");
+
+ TEST_BITMAP_BLOCKCOMMIT("snapshots-4-5", 4, 5, "snapshots");
+
cleanup:
qemuTestDriverFree(&driver);
VIR_FREE(capslatest_x86_64);
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
new file mode 100644
index 0000000000..7d5b0a4402
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
@@ -0,0 +1,105 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-2-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-2-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
new file mode 100644
index 0000000000..17a020ba20
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
@@ -0,0 +1,131 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
new file mode 100644
index 0000000000..972d30b2e1
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
@@ -0,0 +1,163 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
new file mode 100644
index 0000000000..3cda5b4132
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
@@ -0,0 +1,167 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
new file mode 100644
index 0000000000..cb952eeb06
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
@@ -0,0 +1,65 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-3-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-3-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
new file mode 100644
index 0000000000..a540ca42e3
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
@@ -0,0 +1,97 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
new file mode 100644
index 0000000000..8ea6036a33
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
@@ -0,0 +1,101 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
new file mode 100644
index 0000000000..736d4fd9df
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
@@ -0,0 +1,62 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-4-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-4-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
new file mode 100644
index 0000000000..207d82e6b6
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
@@ -0,0 +1,66 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "libvirt-5-format",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ }
+ ]
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
b/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
new file mode 100644
index 0000000000..4a8e6c5cff
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
@@ -0,0 +1,16 @@
+merge bitmpas:
+[
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "libvirt-5-format",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ }
+ ]
+ }
+ }
+]
--
2.26.2
9:40 a.m.
New subject: [PATCH 28/32] qemublocktest: Add 'snapshots' tests for block commit bitmap handling
On 6/15/20 12:10 PM, Peter Krempa wrote:
Simulate commit between all the combinations of layers in the
'snapshots' case to see whether the code merges the correct bitmaps with
the correct depth of temporary bitmaps.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 14 ++
.../bitmapblockcommit/snapshots-1-2 | 105 +++++++++++
.../bitmapblockcommit/snapshots-1-3 | 131 ++++++++++++++
.../bitmapblockcommit/snapshots-1-4 | 163 +++++++++++++++++
.../bitmapblockcommit/snapshots-1-5 | 167 ++++++++++++++++++
.../bitmapblockcommit/snapshots-2-3 | 65 +++++++
.../bitmapblockcommit/snapshots-2-4 | 97 ++++++++++
.../bitmapblockcommit/snapshots-2-5 | 101 +++++++++++
.../bitmapblockcommit/snapshots-3-4 | 62 +++++++
.../bitmapblockcommit/snapshots-3-5 | 66 +++++++
.../bitmapblockcommit/snapshots-4-5 | 16 ++
11 files changed, 987 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-2
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-1-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-3
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-2-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-4
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-3-5
create mode 100644 tests/qemublocktestdata/bitmapblockcommit/snapshots-4-5
Lots of combinations, but that serves for good regression prevention.
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 29/32] qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data
New semantics of the bitmap handling don't need this. Remove the field
and all uses of it including the status XML.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_blockjob.c | 44 -------------------
src/qemu/qemu_blockjob.h | 3 --
src/qemu/qemu_domain.c | 22 ----------
src/qemu/qemu_driver.c | 3 +-
.../blockjob-blockdev-in.xml | 4 --
5 files changed, 1 insertion(+), 75 deletions(-)
diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c
index 8e81b142f4..58ba3e6e96 100644
--- a/src/qemu/qemu_blockjob.c
+++ b/src/qemu/qemu_blockjob.c
@@ -84,11 +84,6 @@ qemuBlockJobDataDisposeJobdata(qemuBlockJobDataPtr job)
virObjectUnref(job->data.backup.store);
g_free(job->data.backup.bitmap);
}
-
- if (job->type == QEMU_BLOCKJOB_TYPE_COMMIT ||
- job->type == QEMU_BLOCKJOB_TYPE_ACTIVE_COMMIT) {
- virStringListFree(job->data.commit.disabledBitmapsBase);
- }
}
@@ -290,7 +285,6 @@ qemuBlockJobDiskNewCommit(virDomainObjPtr vm,
virStorageSourcePtr topparent,
virStorageSourcePtr top,
virStorageSourcePtr base,
- char ***disabledBitmapsBase,
bool delete_imgs,
unsigned int jobflags)
{
@@ -316,7 +310,6 @@ qemuBlockJobDiskNewCommit(virDomainObjPtr vm,
job->data.commit.top = top;
job->data.commit.base = base;
job->data.commit.deleteCommittedImages = delete_imgs;
- job->data.commit.disabledBitmapsBase = g_steal_pointer(disabledBitmapsBase);
job->jobflags = jobflags;
if (qemuBlockJobRegister(job, vm, disk, true) < 0)
@@ -1369,40 +1362,6 @@ qemuBlockJobProcessEventFailedActiveCommit(virQEMUDriverPtr
driver,
}
-static void
-qemuBlockJobProcessEventFailedCommitCommon(virDomainObjPtr vm,
- qemuBlockJobDataPtr job,
- qemuDomainAsyncJob asyncJob)
-{
- qemuDomainObjPrivatePtr priv = vm->privateData;
- g_autoptr(virJSONValue) actions = virJSONValueNewArray();
- char **disabledBitmaps = job->data.commit.disabledBitmapsBase;
-
- if (!disabledBitmaps || !*disabledBitmaps)
- return;
-
- for (; *disabledBitmaps; disabledBitmaps++) {
- qemuMonitorTransactionBitmapEnable(actions,
- job->data.commit.base->nodeformat,
- *disabledBitmaps);
- }
-
- if (qemuBlockReopenReadWrite(vm, job->data.commit.base, asyncJob) < 0)
- return;
-
- if (qemuDomainObjEnterMonitorAsync(priv->driver, vm, asyncJob) < 0)
- return;
-
- qemuMonitorTransaction(priv->mon, &actions);
-
- if (qemuDomainObjExitMonitor(priv->driver, vm) < 0)
- return;
-
- if (qemuBlockReopenReadOnly(vm, job->data.commit.base, asyncJob) < 0)
- return;
-}
-
-
static void
qemuBlockJobProcessEventConcludedCreate(virQEMUDriverPtr driver,
virDomainObjPtr vm,
@@ -1510,8 +1469,6 @@ qemuBlockJobEventProcessConcludedTransition(qemuBlockJobDataPtr
job,
case QEMU_BLOCKJOB_TYPE_COMMIT:
if (success)
qemuBlockJobProcessEventCompletedCommit(driver, vm, job, asyncJob);
- else
- qemuBlockJobProcessEventFailedCommitCommon(vm, job, asyncJob);
break;
case QEMU_BLOCKJOB_TYPE_ACTIVE_COMMIT:
@@ -1519,7 +1476,6 @@ qemuBlockJobEventProcessConcludedTransition(qemuBlockJobDataPtr
job,
qemuBlockJobProcessEventCompletedActiveCommit(driver, vm, job, asyncJob);
} else {
qemuBlockJobProcessEventFailedActiveCommit(driver, vm, job);
- qemuBlockJobProcessEventFailedCommitCommon(vm, job, asyncJob);
}
break;
diff --git a/src/qemu/qemu_blockjob.h b/src/qemu/qemu_blockjob.h
index 19498b5bd8..9f73a3547c 100644
--- a/src/qemu/qemu_blockjob.h
+++ b/src/qemu/qemu_blockjob.h
@@ -88,8 +88,6 @@ struct _qemuBlockJobCommitData {
virStorageSourcePtr top;
virStorageSourcePtr base;
bool deleteCommittedImages;
- char **disabledBitmapsBase; /* a NULL-terminated list of bitmap names which
- were disabled in @base for the commit job */
};
@@ -187,7 +185,6 @@ qemuBlockJobDiskNewCommit(virDomainObjPtr vm,
virStorageSourcePtr topparent,
virStorageSourcePtr top,
virStorageSourcePtr base,
- char ***disabledBitmapsBase,
bool delete_imgs,
unsigned int jobflags);
diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
index 2dad823a86..0548e9c2e6 100644
--- a/src/qemu/qemu_domain.c
+++ b/src/qemu/qemu_domain.c
@@ -2623,7 +2623,6 @@ qemuDomainPrivateBlockJobFormatCommit(qemuBlockJobDataPtr job,
virBufferPtr buf)
{
g_auto(virBuffer) disabledBitmapsBuf = VIR_BUFFER_INIT_CHILD(buf);
- char **bitmaps = job->data.commit.disabledBitmapsBase;
if (job->data.commit.base)
virBufferAsprintf(buf, "<base node='%s'/>\n",
job->data.commit.base->nodeformat);
@@ -2637,9 +2636,6 @@ qemuDomainPrivateBlockJobFormatCommit(qemuBlockJobDataPtr job,
if (job->data.commit.deleteCommittedImages)
virBufferAddLit(buf, "<deleteCommittedImages/>\n");
- while (bitmaps && *bitmaps)
- virBufferEscapeString(&disabledBitmapsBuf, "<bitmap
name='%s'/>\n", *(bitmaps++));
-
virXMLFormatElement(buf, "disabledBaseBitmaps", NULL,
&disabledBitmapsBuf);
}
@@ -3260,9 +3256,6 @@ static int
qemuDomainObjPrivateXMLParseBlockjobDataCommit(qemuBlockJobDataPtr job,
xmlXPathContextPtr ctxt)
{
- g_autofree xmlNodePtr *nodes = NULL;
- ssize_t nnodes;
-
if (job->type == QEMU_BLOCKJOB_TYPE_COMMIT) {
qemuDomainObjPrivateXMLParseBlockjobNodename(job,
"string(./topparent/@node)",
@@ -3289,21 +3282,6 @@ qemuDomainObjPrivateXMLParseBlockjobDataCommit(qemuBlockJobDataPtr
job,
!job->data.commit.base)
return -1;
- if ((nnodes = virXPathNodeSet("./disabledBaseBitmaps/bitmap", ctxt,
&nodes)) > 0) {
- size_t i;
-
- job->data.commit.disabledBitmapsBase = g_new0(char *, nnodes + 1);
-
- for (i = 0; i < nnodes; i++) {
- char *tmp;
-
- if (!(tmp = virXMLPropString(nodes[i], "name")))
- return -1;
-
- job->data.commit.disabledBitmapsBase[i] = g_steal_pointer(&tmp);
- }
- }
-
return 0;
}
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 6256046762..8cf3629a80 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -18481,7 +18481,6 @@ qemuDomainBlockCommit(virDomainPtr dom,
const char *nodebase = NULL;
bool persistjob = false;
bool blockdev = false;
- VIR_AUTOSTRINGLIST bitmapDisableList = NULL;
virCheckFlags(VIR_DOMAIN_BLOCK_COMMIT_SHALLOW |
VIR_DOMAIN_BLOCK_COMMIT_ACTIVE |
@@ -18646,7 +18645,7 @@ qemuDomainBlockCommit(virDomainPtr dom,
}
if (!(job = qemuBlockJobDiskNewCommit(vm, disk, top_parent, topSource,
- baseSource, &bitmapDisableList,
+ baseSource,
flags & VIR_DOMAIN_BLOCK_COMMIT_DELETE,
flags)))
goto endjob;
diff --git a/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml
b/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml
index cc17a17ff4..ca6d110179 100644
--- a/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml
+++ b/tests/qemustatusxml2xmldata/blockjob-blockdev-in.xml
@@ -243,10 +243,6 @@
<base node='libvirt-19-format'/>
<top node='libvirt-17-format'/>
<deleteCommittedImages/>
- <disabledBaseBitmaps>
- <bitmap name='test'/>
- <bitmap name='test1'/>
- </disabledBaseBitmaps>
</blockjob>
<blockjob name='create-libvirt-1337-storage' type='create'
state='running'>
<create mode='storage'/>
--
2.26.2
9:46 a.m.
New subject: [PATCH 29/32] qemu: blockjob: Remove 'disabledBitmapsBase' field from commit job private data
On 6/15/20 12:10 PM, Peter Krempa wrote:
New semantics of the bitmap handling don't need this. Remove the
field
and all uses of it including the status XML.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_blockjob.c | 44 -------------------
src/qemu/qemu_blockjob.h | 3 --
src/qemu/qemu_domain.c | 22 ----------
src/qemu/qemu_driver.c | 3 +-
.../blockjob-blockdev-in.xml | 4 --
5 files changed, 1 insertion(+), 75 deletions(-)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
Another sign of how the new paradigm is simpler.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 30/32] qemu: Rewrite bitmap handling for block copy
Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc
implementatio of bitmap merging for block copy.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 113 ++-------------------------------------
src/qemu/qemu_blockjob.c | 40 ++++++++++++++
src/qemu/qemu_driver.c | 13 ++---
tests/qemublocktest.c | 6 ++-
4 files changed, 57 insertions(+), 115 deletions(-)
diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c
index e958fb71fa..0480f6b97d 100644
--- a/src/qemu/qemu_block.c
+++ b/src/qemu/qemu_block.c
@@ -3068,38 +3068,6 @@ qemuBlockBitmapChainIsValid(virStorageSourcePtr src,
}
-struct qemuBlockBitmapsHandleBlockcopyConcatData {
- virHashTablePtr bitmaps_merge;
- virJSONValuePtr actions;
- const char *mirrornodeformat;
- bool has_bitmaps;
-};
-
-
-static int
-qemuBlockBitmapsHandleBlockcopyConcatActions(void *payload,
- const void *name,
- void *opaque)
-{
- struct qemuBlockBitmapsHandleBlockcopyConcatData *data = opaque;
- virJSONValuePtr createactions = payload;
- const char *bitmapname = name;
- g_autoptr(virJSONValue) mergebitmaps = virHashSteal(data->bitmaps_merge,
bitmapname);
-
- data->has_bitmaps = true;
-
- virJSONValueArrayConcat(data->actions, createactions);
-
- if (qemuMonitorTransactionBitmapMerge(data->actions,
- data->mirrornodeformat,
- bitmapname,
- &mergebitmaps) < 0)
- return -1;
-
- return 0;
-}
-
-
/**
* qemuBlockBitmapsHandleBlockcopy:
* @src: disk source
@@ -3122,86 +3090,15 @@ qemuBlockBitmapsHandleBlockcopy(virStorageSourcePtr src,
bool shallow,
virJSONValuePtr *actions)
{
- g_autoptr(virHashTable) bitmaps = virHashNew(virJSONValueHashFree);
- g_autoptr(virHashTable) bitmaps_merge = virHashNew(virJSONValueHashFree);
- g_autoptr(virHashTable) bitmaps_skip = virHashNew(NULL);
- g_autoptr(virJSONValue) tmpactions = virJSONValueNewArray();
- qemuBlockNamedNodeDataPtr entry;
- virStorageSourcePtr n;
- size_t i;
- struct qemuBlockBitmapsHandleBlockcopyConcatData data = { .bitmaps_merge =
bitmaps_merge,
- .actions = tmpactions,
- .mirrornodeformat =
mirror->nodeformat,
- .has_bitmaps = false, };
-
- for (n = src; n; n = n->backingStore) {
- if (!(entry = virHashLookup(blockNamedNodeData, n->nodeformat)))
- continue;
-
- for (i = 0; i < entry->nbitmaps; i++) {
- qemuBlockNamedNodeDataBitmapPtr bitmap = entry->bitmaps[i];
- virJSONValuePtr bitmap_merge;
-
- if (virHashHasEntry(bitmaps_skip, bitmap->name))
- continue;
-
- if (!(bitmap_merge = virHashLookup(bitmaps_merge, bitmap->name))) {
- g_autoptr(virJSONValue) tmp = NULL;
- bool disabled = !bitmap->recording;
-
- /* disable any non top-layer bitmaps */
- if (n != src)
- disabled = true;
-
- if (!bitmap->persistent ||
- !(qemuBlockBitmapChainIsValid(n, bitmap->name,
- blockNamedNodeData))) {
- ignore_value(virHashAddEntry(bitmaps_skip, bitmap->name, NULL));
- continue;
- }
-
- /* prepare the data for adding the bitmap to the mirror */
- tmp = virJSONValueNewArray();
-
- if (qemuMonitorTransactionBitmapAdd(tmp,
- mirror->nodeformat,
- bitmap->name,
- true,
- disabled,
- bitmap->granularity) < 0)
- return -1;
+ virStorageSourcePtr base = NULL;
- if (virHashAddEntry(bitmaps, bitmap->name, tmp) < 0)
- return -1;
-
- tmp = NULL;
-
- /* prepare array for merging all the bitmaps from the original chain */
- tmp = virJSONValueNewArray();
-
- if (virHashAddEntry(bitmaps_merge, bitmap->name, tmp) < 0)
- return -1;
-
- bitmap_merge = g_steal_pointer(&tmp);
- }
+ if (shallow)
+ base = src->backingStore;
- if (qemuMonitorTransactionBitmapMergeSourceAddBitmap(bitmap_merge,
- n->nodeformat,
- bitmap->name) <
0)
- return -1;
- }
-
- if (shallow)
- break;
- }
-
- if (virHashForEach(bitmaps, qemuBlockBitmapsHandleBlockcopyConcatActions,
- &data) < 0)
+ if (qemuBlockGetBitmapMergeActions(src, base, mirror, NULL, NULL, mirror, actions,
+ blockNamedNodeData) < 0)
return -1;
- if (data.has_bitmaps)
- *actions = g_steal_pointer(&tmpactions);
-
return 0;
}
diff --git a/src/qemu/qemu_blockjob.c b/src/qemu/qemu_blockjob.c
index 58ba3e6e96..7e4530f48b 100644
--- a/src/qemu/qemu_blockjob.c
+++ b/src/qemu/qemu_blockjob.c
@@ -1282,6 +1282,43 @@ qemuBlockJobProcessEventCompletedActiveCommit(virQEMUDriverPtr
driver,
}
+static int
+qemuBlockJobProcessEventCompletedCopyBitmaps(virDomainObjPtr vm,
+ qemuBlockJobDataPtr job,
+ qemuDomainAsyncJob asyncJob)
+{
+ qemuDomainObjPrivatePtr priv = vm->privateData;
+ g_autoptr(virHashTable) blockNamedNodeData = NULL;
+ g_autoptr(virJSONValue) actions = NULL;
+ bool shallow = job->jobflags & VIR_DOMAIN_BLOCK_COPY_SHALLOW;
+
+ if (!virQEMUCapsGet(priv->qemuCaps, QEMU_CAPS_BLOCKDEV_REOPEN))
+ return 0;
+
+ if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm, asyncJob)))
+ return -1;
+
+ if (qemuBlockBitmapsHandleBlockcopy(job->disk->src,
+ job->disk->mirror,
+ blockNamedNodeData,
+ shallow,
+ &actions) < 0)
+ return 0;
+
+ if (!actions)
+ return 0;
+
+ if (qemuDomainObjEnterMonitorAsync(priv->driver, vm, asyncJob) < 0)
+ return -1;
+
+ qemuMonitorTransaction(priv->mon, &actions);
+
+ if (qemuDomainObjExitMonitor(priv->driver, vm) < 0)
+ return -1;
+
+ return 0;
+}
+
static void
qemuBlockJobProcessEventConcludedCopyPivot(virQEMUDriverPtr driver,
virDomainObjPtr vm,
@@ -1296,6 +1333,8 @@ qemuBlockJobProcessEventConcludedCopyPivot(virQEMUDriverPtr driver,
!job->disk->mirror)
return;
+ qemuBlockJobProcessEventCompletedCopyBitmaps(vm, job, asyncJob);
+
/* for shallow copy without reusing external image the user can either not
* specify the backing chain in which case libvirt will open and use the
* chain the user provided or not specify a chain in which case we'll
@@ -1329,6 +1368,7 @@ qemuBlockJobProcessEventConcludedCopyAbort(virQEMUDriverPtr driver,
!job->disk->mirror)
return;
+ /* activeWrite bitmap is removed automatically here */
qemuBlockJobEventProcessConcludedRemoveChain(driver, vm, asyncJob,
job->disk->mirror);
virObjectUnref(job->disk->mirror);
job->disk->mirror = NULL;
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index 8cf3629a80..c4d8a13004 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -17314,14 +17314,15 @@ qemuDomainBlockPivot(virQEMUDriverPtr driver,
if (blockdev && !job->jobflagsmissing) {
bool shallow = job->jobflags & VIR_DOMAIN_BLOCK_COPY_SHALLOW;
bool reuse = job->jobflags & VIR_DOMAIN_BLOCK_COPY_REUSE_EXT;
- g_autoptr(virHashTable) blockNamedNodeData = NULL;
- if (!(blockNamedNodeData = qemuBlockGetNamedNodeData(vm,
QEMU_ASYNC_JOB_NONE)))
- return -1;
+ actions = virJSONValueNewArray();
- if (qemuBlockBitmapsHandleBlockcopy(disk->src, disk->mirror,
- blockNamedNodeData,
- shallow, &actions) < 0)
+ if (qemuMonitorTransactionBitmapAdd(actions,
+ disk->mirror->nodeformat,
+ "libvirt-tmp-activewrite",
+ false,
+ false,
+ 0) < 0)
return -1;
/* Open and install the backing chain of 'mirror' late if we can use
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 4a737e19d3..1c6517646b 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -881,6 +881,7 @@ testQemuBlockBitmapBlockcopy(const void *opaque)
g_autoptr(virJSONValue) nodedatajson = NULL;
g_autoptr(virHashTable) nodedata = NULL;
g_autoptr(virStorageSource) fakemirror = virStorageSourceNew();
+ g_auto(virBuffer) buf = VIR_BUFFER_INITIALIZER;
if (!fakemirror)
return -1;
@@ -903,10 +904,13 @@ testQemuBlockBitmapBlockcopy(const void *opaque)
data->shallow, &actions) < 0)
return -1;
+
if (actions &&
- !(actual = virJSONValueToString(actions, true)))
+ virJSONValueToBuffer(actions, &buf, true) < 0)
return -1;
+ actual = virBufferContentAndReset(&buf);
+
return virTestCompareToFile(actual, expectpath);
}
--
2.26.2
9:50 a.m.
New subject: [PATCH 30/32] qemu: Rewrite bitmap handling for block copy
On 6/15/20 12:10 PM, Peter Krempa wrote:
Reuse qemuBlockGetBitmapMergeActions which allows to remove the
ad-hoc
allows the removal of
implementatio of bitmap merging for block copy.
implementation
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
src/qemu/qemu_block.c | 113 ++-------------------------------------
src/qemu/qemu_blockjob.c | 40 ++++++++++++++
src/qemu/qemu_driver.c | 13 ++---
tests/qemublocktest.c | 6 ++-
4 files changed, 57 insertions(+), 115 deletions(-)
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
6:11 a.m.
New subject: [PATCH 30/32] qemu: Rewrite bitmap handling for block copy
On Fri, Jun 19, 2020 at 09:50:16 -0500, Eric Blake wrote:
On 6/15/20 12:10 PM, Peter Krempa wrote:
> Reuse qemuBlockGetBitmapMergeActions which allows to remove the ad-hoc
allows the removal of
> implementatio of bitmap merging for block copy.
implementation
Too much copy and paste :)
12:10 p.m.
New subject: [PATCH 31/32] qemublocktest: Add test cases for handling bitmaps during block-copy
Test both 'basic' and 'snapshots' cases on shallow and deep copy modes.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 5 +
.../bitmapblockcopy/basic-deep-out.json | 144 ++++++++++++++
.../bitmapblockcopy/basic-shallow-out.json | 144 ++++++++++++++
.../bitmapblockcopy/snapshots-deep-out.json | 180 ++++++++++++++++++
.../snapshots-shallow-out.json | 144 ++++++++++++++
5 files changed, 617 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
diff --git a/tests/qemublocktest.c b/tests/qemublocktest.c
index 1c6517646b..0ab599dd67 100644
--- a/tests/qemublocktest.c
+++ b/tests/qemublocktest.c
@@ -1346,6 +1346,11 @@ mymain(void)
TEST_BITMAP_BLOCKCOPY("empty-shallow", true, "empty");
TEST_BITMAP_BLOCKCOPY("empty-deep", false, "empty");
+ TEST_BITMAP_BLOCKCOPY("basic-shallow", true, "basic");
+ TEST_BITMAP_BLOCKCOPY("basic-deep", false, "basic");
+
+ TEST_BITMAP_BLOCKCOPY("snapshots-shallow", true, "snapshots");
+ TEST_BITMAP_BLOCKCOPY("snapshots-deep", false, "snapshots");
#define TEST_BITMAP_BLOCKCOMMIT(testname, topimg, baseimg, ndf) \
do {\
diff --git a/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
b/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
new file mode 100644
index 0000000000..a8c2749879
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
@@ -0,0 +1,144 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
b/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
new file mode 100644
index 0000000000..a8c2749879
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
@@ -0,0 +1,144 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
b/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
new file mode 100644
index 0000000000..ac2ffb4ce2
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
@@ -0,0 +1,180 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "c"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "c"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "b"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "b"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-3-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-4-format",
+ "name": "a"
+ },
+ {
+ "node": "libvirt-5-format",
+ "name": "a"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "libvirt-2-format",
+ "name": "d"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
diff --git a/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
b/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
new file mode 100644
index 0000000000..3a9942569a
--- /dev/null
+++ b/tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
@@ -0,0 +1,144 @@
+[
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "current",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "current",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "current"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "c",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "c",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "c"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "b",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "b",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "b"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "a",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "a",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "a"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-add",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "d",
+ "persistent": true,
+ "disabled": false,
+ "granularity": 65536
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-merge",
+ "data": {
+ "node": "mirror-format-node",
+ "target": "d",
+ "bitmaps": [
+ {
+ "node": "libvirt-1-format",
+ "name": "d"
+ },
+ {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ ]
+ }
+ },
+ {
+ "type": "block-dirty-bitmap-remove",
+ "data": {
+ "node": "mirror-format-node",
+ "name": "libvirt-tmp-activewrite"
+ }
+ }
+]
--
2.26.2
9:52 a.m.
New subject: [PATCH 31/32] qemublocktest: Add test cases for handling bitmaps during block-copy
On 6/15/20 12:10 PM, Peter Krempa wrote:
Test both 'basic' and 'snapshots' cases on shallow
and deep copy modes.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
tests/qemublocktest.c | 5 +
.../bitmapblockcopy/basic-deep-out.json | 144 ++++++++++++++
.../bitmapblockcopy/basic-shallow-out.json | 144 ++++++++++++++
.../bitmapblockcopy/snapshots-deep-out.json | 180 ++++++++++++++++++
.../snapshots-shallow-out.json | 144 ++++++++++++++
5 files changed, 617 insertions(+)
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/basic-shallow-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-deep-out.json
create mode 100644 tests/qemublocktestdata/bitmapblockcopy/snapshots-shallow-out.json
Reviewed-by: Eric Blake <eblake(a)redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
12:10 p.m.
New subject: [PATCH 32/32] kbase: Add document outlining internals of incremental backup in qemu
Outline the basics and how to integrate with externally created
overlays. Other topics will continue later.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
docs/kbase.html.in | 3 +
docs/kbase/incrementalbackupinternals.rst | 210 ++++++++++++++++++++++
2 files changed, 213 insertions(+)
create mode 100644 docs/kbase/incrementalbackupinternals.rst
diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..4257e52b7e 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -36,6 +36,9 @@
<dt><a
href="kbase/virtiofs.html">Virtio-FS</a></dt>
<dd>Share a filesystem between the guest and the host</dd>
+
+ <dt><a
href="kbase/incrementalbackupinternals.html">Incremental backup
internals</a></dt>
+ <dd>Incremental backup implementation details relevant for
users</dd>
</dl>
</div>
diff --git a/docs/kbase/incrementalbackupinternals.rst
b/docs/kbase/incrementalbackupinternals.rst
new file mode 100644
index 0000000000..adf12002d2
--- /dev/null
+++ b/docs/kbase/incrementalbackupinternals.rst
@@ -0,0 +1,210 @@
+================================================
+Internals of incremental backup handling in qemu
+================================================
+
+.. contents::
+
+Libvirt's implementation of incremental backups in the ``qemu`` driver uses
+qemu's ``block-dirty-bitmaps`` under the hood to track the guest visible disk
+state changes correspoiding to the points in time described by a libvirt
+checkpoint.
+
+There are some semantical implications how libvirt creates and manages the
+bitmaps which de-facto become API as they are written into the disk images and
+this document will try to sumarize them.
+
+Glossary
+========
+
+Checkpoint
+
+ A libvirt object which represents a named point in time of the life of the
+ vm where libvirt tracks writes the VM has done and allows then a backup of
+ block which changed. Note that state of the VM memory is _not_ captured.
+
+ A checkpoint can be created either explicitly via the corresponding API
+ which isn't very useful or is created as part of creating an
+ incremental or full backup of the VM using the ``virDomainBackupBegin`` API
+ which allows a next backup to only copy the differences.
+
+Backup
+
+ A copy of either all blocks of selected disks (full backup) or blocks changed
+ since a checkpoint (incremental backup) at the time the backup job was
+ started. (Blocks modified while the backup job is running are not part of the
+ backup!)
+
+Snapshot
+
+ Similarly to a checkpoint it's a point in time in the lifecycle of the VM
+ but the state of the VM including memory is captured at that point allowing
+ returning to the state later.
+
+Blockjob
+
+ A long running job which modifies the shape and/or location of the disk
+ backing chain (images storing the disk contents). Libvirt supports
+ ``block pull`` where data is moved up the chain towards the active layer,
+ ``block commit`` where data is moved down the chain towards the base/oldest
+ image. These blockjobs always remove images from the backing chain. Lastly
+ ``block copy`` where image is moved to a different location (and possibly
+ collapsed moving all of the data into the new location into the one image).
+
+block-dirty-bitmap (bitmap)
+
+ A data structure in qemu tracking which blocks were written by the guest
+ OS since the bitmap was created.
+
+Relationships of bitmaps, checkpoints and VM disks
+==================================================
+
+When a checkpoint is created libvirt creates a block-dirty-bitmap for every
+configured VM disk named the same way as chcheckpoint. The bitmap is actively
+recording which blocks were changed by the guest OS from that point on. Other
+bitmaps are not impacted by any way as they are self-contained:
+
+::
+
+ +----------------+ +----------------+
+ | disk: vda | | disk: vdb |
+ +--------+-------+ +--------+-------+
+ | |
+ +--------v-------+ +--------v-------+
+ | vda-1.qcow2 | | vdb-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a | | bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Bitmaps are created at the same time to track changes to all disks in sync and
+are active and persisted in the QCOW2 image. Oter formats currently don't
+support this feature.
+
+Modification of bitmaps outside of libvirt is not recommended, but when adrering
+to the same semantics which the document will describe it should be safe to do
+so but obviously we can't guarantee that.
+
+
+Integration with external snapshots
+===================================
+
+Handling of bitmaps
+-------------------
+
+Creating an external snapshot involves adding a new layer to the backing chain
+on top of the previous chain. In this step there are no new bitmaps created by
+default, which would mean that backups become impossible after this step.
+
+To prevent this from happening we need to re-create the active bitmaps in the
+new top/active layer of the backing chain which allows us to continue tracking
+the changes with same granularity as before and also allows libvirt to stitch
+together all the corresponding bitmaps to do a backup acorss snapshots.
+
+After taking a snapshot of the ``vda`` disk from the example above placed into
+``vda-2.qcow2`` the following topology will be created:
+
+::
+
+ +----------------+
+ | disk: vda |
+ +-------+--------+
+ |
+ +-------v--------+ +----------------+
+ | vda-2.qcow2 | | vda-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a +----> bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Checking bitmap health
+----------------------
+
+QEMU optimizes disk writes by only updating the bitmaps in certain cases. This
+also can cause problems in cases when e.g. QEMU crashes.
+
+For a chain of bitmaps corresponding in a backing chain to be considered valid
+and eligible for use with ``virDomainBackupBegin`` it must conform to the
+following rules:
+
+1) Top image must contain the bitmap
+2) If any of the backing images in the chain contain the bitmap too all
+ contiguous images must have the bitmap (no gaps)
+3) all of the above bitmaps must be marked as active
+ (``auto`` flag in ``qemu-img`` output, ``recording`` in qemu)
+4) none of the above bitmaps can be inconsistent
+ (``in-use`` flag in ``qemu-img`` provided that it's not used on image which
+ is currently in use by a qemu instance, or ``inconsistent`` in qemu)
+
+::
+
+ # check that image has bitmaps
+ $ qemu-img info vda-1.qcow2
+ image: vda-1.qcow2
+ file format: qcow2
+ virtual size: 100 MiB (104857600 bytes)
+ disk size: 220 KiB
+ cluster_size: 65536
+ Format specific information:
+ compat: 1.1
+ compression type: zlib
+ lazy refcounts: false
+ bitmaps:
+ [0]:
+ flags:
+ [0]: in-use
+ [1]: auto
+ name: chk-a
+ granularity: 65536
+ [1]:
+ flags:
+ [0]: auto
+ name: chk-b
+ granularity: 65536
+ refcount bits: 16
+ corrupt: false
+
+(See also the ``qemuBlockBitmapChainIsValid`` helper method in
+``src/qemu/qemu_block.c``)
+
+Creating external checkpoints manually
+--------------------------------------
+
+To create the same topology outside of libvirt (e.g when doing snapshots offline)
+a new ``qemu-img`` which supports the ``bitmap`` subcomand is necessary. The
+following algorithm then ensures that the new image after snapshot will work
+with backups (note that ``jq`` is a JSON processor):
+
+::
+
+ # arguments
+ SNAP_IMG="vda-2.qcow2"
+ BACKING_IMG="vda-1.qcow2"
+
+ # constants - snapshots and bitmaps work only with qcow2
+ SNAP_FMT="qcow2"
+ BACKING_IMG_FMT="qcow2"
+
+ # create snapshot overlay
+ qemu-img create -f "$SNAP_FMT" -F "$BACKING_IMG_FMT" -b
"$BACKING_IMG" "$SNAP_IMG"
+
+ BACKING_IMG_INFO=$(qemu-img info --output=json -f "$BACKING_IMG_FMT"
"$BACKING_IMG")
+ BACKING_BITMAPS=$(jq '."format-specific".data.bitmaps' <<<
"$BACKING_IMG_INFO")
+
+ if [ "x$BACKING_BITMAPS" == "xnull" ]; then
+ exit 0
+ fi
+
+ for BACKING_BITMAP_ in $(jq -c '.[]' <<<
"$BACKING_BITMAPS"); do
+ BITMAP_FLAGS=$(jq -c -r '.flags[]' <<<
"$BACKING_BITMAP_")
+ BITMAP_NAME=$(jq -r '.name' <<< "$BACKING_BITMAP_")
+
+ if grep 'in-use' <<< "$BITMAP_FLAGS" ||
+ grep -v 'auto' <<< "$BITMAP_FLAGS"; then
+ continue
+ fi
+
+ qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add
"$BITMAP_NAME"
+
+ done
--
2.26.2
10:10 a.m.
New subject: [PATCH 32/32] kbase: Add document outlining internals of incremental backup in qemu
On 6/15/20 12:10 PM, Peter Krempa wrote:
Outline the basics and how to integrate with externally created
overlays. Other topics will continue later.
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
docs/kbase.html.in | 3 +
docs/kbase/incrementalbackupinternals.rst | 210 ++++++++++++++++++++++
2 files changed, 213 insertions(+)
create mode 100644 docs/kbase/incrementalbackupinternals.rst
+++ b/docs/kbase/incrementalbackupinternals.rst
@@ -0,0 +1,210 @@
+================================================
+Internals of incremental backup handling in qemu
+================================================
+
+.. contents::
+
+Libvirt's implementation of incremental backups in the ``qemu`` driver uses
+qemu's ``block-dirty-bitmaps`` under the hood to track the guest visible disk
+state changes correspoiding to the points in time described by a libvirt
corresponding
+checkpoint.
+
+There are some semantical implications how libvirt creates and manages the
semantic implications with how
+bitmaps which de-facto become API as they are written into the disk
images and
images,
+this document will try to sumarize them.
summarize
+
+Glossary
+========
+
+Checkpoint
+
+ A libvirt object which represents a named point in time of the life of the
+ vm where libvirt tracks writes the VM has done and allows then a backup of
has done, thereby allowing a backup of only the blocks which changed
+ block which changed. Note that state of the VM memory is _not_
captured.
+
+ A checkpoint can be created either explicitly via the corresponding API
+ which isn't very useful or is created as part of creating an
+ incremental or full backup of the VM using the ``virDomainBackupBegin`` API
+ which allows a next backup to only copy the differences.
Maybe:
A checkpoint can be created either explicitly via the corresponding API
(although this isn't very useful on its own), or simultaneously with an
incremental or full backup of the VM
+
+Backup
+
+ A copy of either all blocks of selected disks (full backup) or blocks changed
+ since a checkpoint (incremental backup) at the time the backup job was
+ started. (Blocks modified while the backup job is running are not part of the
+ backup!)
+
+Snapshot
+
+ Similarly to a checkpoint it's a point in time in the lifecycle of the VM
+ but the state of the VM including memory is captured at that point allowing
+ returning to the state later.
Hmm. We have disk-only snapshots which do not save the state of memory.
Does this paragraph need adjustment to mention the difference between a
disk-only snapshot and a full state capture? Are we redefining any of
the terms in domainstatecapture.rst, and/or should those two documents
have cross-references?
+
+Blockjob
+
+ A long running job which modifies the shape and/or location of the disk
+ backing chain (images storing the disk contents). Libvirt supports
If qemu adds block-dirty-bitmap-populate, blockjobs can also manipulate
just bitmaps.
+ ``block pull`` where data is moved up the chain towards the
active layer,
+ ``block commit`` where data is moved down the chain towards the base/oldest
+ image. These blockjobs always remove images from the backing chain. Lastly
+ ``block copy`` where image is moved to a different location (and possibly
+ collapsed moving all of the data into the new location into the one image).
+
+block-dirty-bitmap (bitmap)
+
+ A data structure in qemu tracking which blocks were written by the guest
+ OS since the bitmap was created.
+
+Relationships of bitmaps, checkpoints and VM disks
+==================================================
+
+When a checkpoint is created libvirt creates a block-dirty-bitmap for every
+configured VM disk named the same way as chcheckpoint. The bitmap is actively
s/chcheckpoint/the checkpoint/
+recording which blocks were changed by the guest OS from that point
on. Other
+bitmaps are not impacted by any way as they are self-contained:
+
+::
+
+ +----------------+ +----------------+
+ | disk: vda | | disk: vdb |
+ +--------+-------+ +--------+-------+
+ | |
+ +--------v-------+ +--------v-------+
+ | vda-1.qcow2 | | vdb-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a | | bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Bitmaps are created at the same time to track changes to all disks in sync and
+are active and persisted in the QCOW2 image. Oter formats currently don't
Other
+support this feature.
+
+Modification of bitmaps outside of libvirt is not recommended, but when adrering
adhering
+to the same semantics which the document will describe it should be
safe to do
+so but obviously we can't guarantee that.
do so, even if we obviously can't guarantee that
+
+
+Integration with external snapshots
+===================================
+
+Handling of bitmaps
+-------------------
+
+Creating an external snapshot involves adding a new layer to the backing chain
+on top of the previous chain. In this step there are no new bitmaps created by
+default, which would mean that backups become impossible after this step.
+
+To prevent this from happening we need to re-create the active bitmaps in the
+new top/active layer of the backing chain which allows us to continue tracking
+the changes with same granularity as before and also allows libvirt to stitch
+together all the corresponding bitmaps to do a backup acorss snapshots.
across
+
+After taking a snapshot of the ``vda`` disk from the example above placed into
+``vda-2.qcow2`` the following topology will be created:
+
+::
+
+ +----------------+
+ | disk: vda |
+ +-------+--------+
+ |
+ +-------v--------+ +----------------+
+ | vda-2.qcow2 | | vda-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a +----> bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Checking bitmap health
+----------------------
+
+QEMU optimizes disk writes by only updating the bitmaps in certain cases. This
+also can cause problems in cases when e.g. QEMU crashes.
+
+For a chain of bitmaps corresponding in a backing chain to be considered valid
corresponding bitmaps
+and eligible for use with ``virDomainBackupBegin`` it must conform
to the
+following rules:
+
+1) Top image must contain the bitmap
+2) If any of the backing images in the chain contain the bitmap too all
too,
+ contiguous images must have the bitmap (no gaps)
+3) all of the above bitmaps must be marked as active
+ (``auto`` flag in ``qemu-img`` output, ``recording`` in qemu)
+4) none of the above bitmaps can be inconsistent
+ (``in-use`` flag in ``qemu-img`` provided that it's not used on image which
+ is currently in use by a qemu instance, or ``inconsistent`` in qemu)
+
+::
+
+ # check that image has bitmaps
+ $ qemu-img info vda-1.qcow2
+ image: vda-1.qcow2
+ file format: qcow2
+ virtual size: 100 MiB (104857600 bytes)
+ disk size: 220 KiB
+ cluster_size: 65536
+ Format specific information:
+ compat: 1.1
+ compression type: zlib
+ lazy refcounts: false
+ bitmaps:
+ [0]:
+ flags:
+ [0]: in-use
+ [1]: auto
+ name: chk-a
+ granularity: 65536
+ [1]:
+ flags:
+ [0]: auto
+ name: chk-b
+ granularity: 65536
+ refcount bits: 16
+ corrupt: false
+
+(See also the ``qemuBlockBitmapChainIsValid`` helper method in
+``src/qemu/qemu_block.c``)
+
+Creating external checkpoints manually
s/checkpoints/snapshots/
+--------------------------------------
+
+To create the same topology outside of libvirt (e.g when doing snapshots offline)
+a new ``qemu-img`` which supports the ``bitmap`` subcomand is necessary. The
subcommand
s/necessary/recommended/ (as it is also possible to use 'qemu-kvm -S' to
do the same actions via QMP commands - although I'm not sure if it is
worth documenting that fallback)
+following algorithm then ensures that the new image after snapshot
will work
+with backups (note that ``jq`` is a JSON processor):
+
+::
+
+ # arguments
+ SNAP_IMG="vda-2.qcow2"
+ BACKING_IMG="vda-1.qcow2"
+
+ # constants - snapshots and bitmaps work only with qcow2
+ SNAP_FMT="qcow2"
+ BACKING_IMG_FMT="qcow2"
+
+ # create snapshot overlay
+ qemu-img create -f "$SNAP_FMT" -F "$BACKING_IMG_FMT" -b
"$BACKING_IMG" "$SNAP_IMG"
+
+ BACKING_IMG_INFO=$(qemu-img info --output=json -f "$BACKING_IMG_FMT"
"$BACKING_IMG")
+ BACKING_BITMAPS=$(jq '."format-specific".data.bitmaps' <<<
"$BACKING_IMG_INFO")
<<< is a bashism.
+
+ if [ "x$BACKING_BITMAPS" == "xnull" ]; then
So is == instead of =. Either we should tweak this to be portable to
dash, or you should add a #!/bin/bash line to the top of the example.
+ exit 0
+ fi
+
+ for BACKING_BITMAP_ in $(jq -c '.[]' <<<
"$BACKING_BITMAPS"); do
+ BITMAP_FLAGS=$(jq -c -r '.flags[]' <<<
"$BACKING_BITMAP_")
+ BITMAP_NAME=$(jq -r '.name' <<< "$BACKING_BITMAP_")
+
+ if grep 'in-use' <<< "$BITMAP_FLAGS" ||
+ grep -v 'auto' <<< "$BITMAP_FLAGS"; then
+ continue
+ fi
+
+ qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add
"$BITMAP_NAME"
Do you want to also copy the --granularity of the bitmaps being added?
+
+ done
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
7:48 a.m.
New subject: [PATCH 32/32] kbase: Add document outlining internals of incremental backup in qemu
On Fri, Jun 19, 2020 at 10:10:36 -0500, Eric Blake wrote:
On 6/15/20 12:10 PM, Peter Krempa wrote:
> Outline the basics and how to integrate with externally created
> overlays. Other topics will continue later.
>
> Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
> ---
> docs/kbase.html.in | 3 +
> docs/kbase/incrementalbackupinternals.rst | 210 ++++++++++++++++++++++
> 2 files changed, 213 insertions(+)
> create mode 100644 docs/kbase/incrementalbackupinternals.rst
[...]
> +Snapshot
> +
> + Similarly to a checkpoint it's a point in time in the lifecycle of the VM
> + but the state of the VM including memory is captured at that point allowing
> + returning to the state later.
Hmm. We have disk-only snapshots which do not save the state of memory. Does
this paragraph need adjustment to mention the difference between a disk-only
snapshot and a full state capture? Are we redefining any of the terms in
domainstatecapture.rst, and/or should those two documents have
cross-references?
Good idea! I'll keep this as is, but I'll link to the
domainstatecapture.rst document at the beginning of the glossary.
[...]
> + fi
> +
> + for BACKING_BITMAP_ in $(jq -c '.[]' <<<
"$BACKING_BITMAPS"); do
> + BITMAP_FLAGS=$(jq -c -r '.flags[]' <<<
"$BACKING_BITMAP_")
> + BITMAP_NAME=$(jq -r '.name' <<<
"$BACKING_BITMAP_")
> +
> + if grep 'in-use' <<< "$BITMAP_FLAGS" ||
> + grep -v 'auto' <<< "$BITMAP_FLAGS"; then
> + continue
> + fi
> +
> + qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add
"$BITMAP_NAME"
Do you want to also copy the --granularity of the bitmaps being
added?
Hmm, yeah, that's what libvirt does.
6:40 p.m.
New subject: [PATCH 32/32] kbase: Add document outlining internals of incremental backup in qemu
On Mon, Jun 15, 2020 at 8:13 PM Peter Krempa <pkrempa(a)redhat.com> wrote:
Outline the basics and how to integrate with externally created
overlays. Other topics will continue later.
Thanks, this is very helpful!
Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
---
docs/kbase.html.in | 3 +
docs/kbase/incrementalbackupinternals.rst | 210 ++++++++++++++++++++++
2 files changed, 213 insertions(+)
create mode 100644 docs/kbase/incrementalbackupinternals.rst
diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c586e0f676..4257e52b7e 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -36,6 +36,9 @@
<dt><a
href="kbase/virtiofs.html">Virtio-FS</a></dt>
<dd>Share a filesystem between the guest and the host</dd>
+
+ <dt><a
href="kbase/incrementalbackupinternals.html">Incremental backup
internals</a></dt>
+ <dd>Incremental backup implementation details relevant for
users</dd>
</dl>
</div>
diff --git a/docs/kbase/incrementalbackupinternals.rst
b/docs/kbase/incrementalbackupinternals.rst
new file mode 100644
index 0000000000..adf12002d2
--- /dev/null
+++ b/docs/kbase/incrementalbackupinternals.rst
@@ -0,0 +1,210 @@
+================================================
+Internals of incremental backup handling in qemu
+================================================
+
+.. contents::
+
+Libvirt's implementation of incremental backups in the ``qemu`` driver uses
+qemu's ``block-dirty-bitmaps`` under the hood to track the guest visible disk
+state changes correspoiding to the points in time described by a libvirt
+checkpoint.
+
+There are some semantical implications how libvirt creates and manages the
+bitmaps which de-facto become API as they are written into the disk images and
+this document will try to sumarize them.
+
+Glossary
+========
+
+Checkpoint
+
+ A libvirt object which represents a named point in time of the life of the
+ vm where libvirt tracks writes the VM has done and allows then a backup of
+ block which changed. Note that state of the VM memory is _not_ captured.
+
+ A checkpoint can be created either explicitly via the corresponding API
+ which isn't very useful or is created as part of creating an
+ incremental or full backup of the VM using the ``virDomainBackupBegin`` API
+ which allows a next backup to only copy the differences.
+
+Backup
+
+ A copy of either all blocks of selected disks (full backup) or blocks changed
+ since a checkpoint (incremental backup) at the time the backup job was
+ started. (Blocks modified while the backup job is running are not part of the
+ backup!)
+
+Snapshot
+
+ Similarly to a checkpoint it's a point in time in the lifecycle of the VM
+ but the state of the VM including memory is captured at that point allowing
+ returning to the state later.
+
+Blockjob
+
+ A long running job which modifies the shape and/or location of the disk
+ backing chain (images storing the disk contents). Libvirt supports
+ ``block pull`` where data is moved up the chain towards the active layer,
+ ``block commit`` where data is moved down the chain towards the base/oldest
+ image. These blockjobs always remove images from the backing chain. Lastly
+ ``block copy`` where image is moved to a different location (and possibly
+ collapsed moving all of the data into the new location into the one image).
+
+block-dirty-bitmap (bitmap)
+
+ A data structure in qemu tracking which blocks were written by the guest
+ OS since the bitmap was created.
+
+Relationships of bitmaps, checkpoints and VM disks
+==================================================
+
+When a checkpoint is created libvirt creates a block-dirty-bitmap for every
+configured VM disk named the same way as chcheckpoint. The bitmap is actively
+recording which blocks were changed by the guest OS from that point on. Other
+bitmaps are not impacted by any way as they are self-contained:
+
+::
+
+ +----------------+ +----------------+
+ | disk: vda | | disk: vdb |
+ +--------+-------+ +--------+-------+
+ | |
+ +--------v-------+ +--------v-------+
+ | vda-1.qcow2 | | vdb-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a | | bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Bitmaps are created at the same time to track changes to all disks in sync and
+are active and persisted in the QCOW2 image. Oter formats currently don't
+support this feature.
+
+Modification of bitmaps outside of libvirt is not recommended, but when adrering
+to the same semantics which the document will describe it should be safe to do
+so but obviously we can't guarantee that.
+
+
+Integration with external snapshots
+===================================
+
+Handling of bitmaps
+-------------------
+
+Creating an external snapshot involves adding a new layer to the backing chain
+on top of the previous chain. In this step there are no new bitmaps created by
+default, which would mean that backups become impossible after this step.
+
+To prevent this from happening we need to re-create the active bitmaps in the
+new top/active layer of the backing chain which allows us to continue tracking
+the changes with same granularity as before and also allows libvirt to stitch
+together all the corresponding bitmaps to do a backup acorss snapshots.
+
+After taking a snapshot of the ``vda`` disk from the example above placed into
+``vda-2.qcow2`` the following topology will be created:
+
+::
+
+ +----------------+
+ | disk: vda |
+ +-------+--------+
+ |
+ +-------v--------+ +----------------+
+ | vda-2.qcow2 | | vda-1.qcow2 |
+ | | | |
+ | bitmaps: chk-a +----> bitmaps: chk-a |
+ | chk-b | | chk-b |
+ | | | |
+ +----------------+ +----------------+
+
+Checking bitmap health
+----------------------
+
+QEMU optimizes disk writes by only updating the bitmaps in certain cases. This
+also can cause problems in cases when e.g. QEMU crashes.
+
+For a chain of bitmaps corresponding in a backing chain to be considered valid
+and eligible for use with ``virDomainBackupBegin`` it must conform to the
+following rules:
+
+1) Top image must contain the bitmap
+2) If any of the backing images in the chain contain the bitmap too all
+ contiguous images must have the bitmap (no gaps)
+3) all of the above bitmaps must be marked as active
+ (``auto`` flag in ``qemu-img`` output, ``recording`` in qemu)
+4) none of the above bitmaps can be inconsistent
+ (``in-use`` flag in ``qemu-img`` provided that it's not used on image which
+ is currently in use by a qemu instance, or ``inconsistent`` in qemu)
Can you add a chapter of about the old format and how it was different
from the new
format?
Looks like the differences are:
- all bitmaps are always active, so no need to enable or disable them
- based on next section, new snapshost contain all the bitmaps from
the previous snapshots
(since all of them active and we copy all of them to the new snapshot)
How qemu knows which bitmap should track changes if all bitmaps are active?
When libvirt starts a VM, it knows nothing about the checkpoints. We
define the checkpoints
right before the first backup after starting a VM. So both libvirt and
qemu know nothing about
the bitmaps at this point.
Do you expect to have the checkpoints defined before the guest is started?
If I understand this correctly, if we do:
- create base image
- do full backup (check-1)
- do incremental backup 1 (check-2)
- create snapshot-1
- do incremental backup 2 (check-3)
- do incremental backup 3 (check-4)
- create snapshot-2
- do incremental backup 4 (check-5)
This will be the image structure:
- base image
- check-1
- check-2
- snapshot-1
- check-1
- check-2
- check-3
- check-4
- snapshot-2
- check-1
- check-2
- check-3
- check-4
- check-5
So we are duplicating bitmaps that have no content in all snapshot?
Why not copy only the last (current) bitmap?
- base image
- check-1
- check-2
- snapshot-1
- check-2
- check-3
- check-4
- snapshot-2
- check-4
- check-5
+::
+
+ # check that image has bitmaps
+ $ qemu-img info vda-1.qcow2
+ image: vda-1.qcow2
+ file format: qcow2
+ virtual size: 100 MiB (104857600 bytes)
+ disk size: 220 KiB
+ cluster_size: 65536
+ Format specific information:
+ compat: 1.1
+ compression type: zlib
+ lazy refcounts: false
+ bitmaps:
+ [0]:
+ flags:
+ [0]: in-use
+ [1]: auto
+ name: chk-a
+ granularity: 65536
+ [1]:
+ flags:
+ [0]: auto
+ name: chk-b
+ granularity: 65536
+ refcount bits: 16
+ corrupt: false
+
+(See also the ``qemuBlockBitmapChainIsValid`` helper method in
+``src/qemu/qemu_block.c``)
Looks like oVirt needs to implement this a well, otherwise we will waste time
creating bitmaps on a snapshot when libvirt will fail the backup later since
there was an inconsistent or disabled bitmap.
+Creating external checkpoints manually
+--------------------------------------
+
+To create the same topology outside of libvirt (e.g when doing snapshots offline)
+a new ``qemu-img`` which supports the ``bitmap`` subcomand is necessary. The
+following algorithm then ensures that the new image after snapshot will work
+with backups (note that ``jq`` is a JSON processor):
+
+::
+
+ # arguments
+ SNAP_IMG="vda-2.qcow2"
+ BACKING_IMG="vda-1.qcow2"
+
+ # constants - snapshots and bitmaps work only with qcow2
+ SNAP_FMT="qcow2"
+ BACKING_IMG_FMT="qcow2"
+
+ # create snapshot overlay
+ qemu-img create -f "$SNAP_FMT" -F "$BACKING_IMG_FMT" -b
"$BACKING_IMG" "$SNAP_IMG"
+
+ BACKING_IMG_INFO=$(qemu-img info --output=json -f "$BACKING_IMG_FMT"
"$BACKING_IMG")
+ BACKING_BITMAPS=$(jq '."format-specific".data.bitmaps' <<<
"$BACKING_IMG_INFO")
+
+ if [ "x$BACKING_BITMAPS" == "xnull" ]; then
+ exit 0
+ fi
+
+ for BACKING_BITMAP_ in $(jq -c '.[]' <<<
"$BACKING_BITMAPS"); do
+ BITMAP_FLAGS=$(jq -c -r '.flags[]' <<<
"$BACKING_BITMAP_")
+ BITMAP_NAME=$(jq -r '.name' <<< "$BACKING_BITMAP_")
+
+ if grep 'in-use' <<< "$BITMAP_FLAGS" ||
+ grep -v 'auto' <<< "$BITMAP_FLAGS"; then
+ continue
+ fi
+
+ qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add
"$BITMAP_NAME"
So what we have to do is:
- get a list of bitmaps that should be in this disk from oVirt engine
- get list of bitmaps with "auto" flag and without the "in-use" flag
in the backing file
- if the lists do not match we can delete all bitmaps and relevant
checkpoints on engine
side, since the next backup will fail anyway.
- if the lists match, maybe verify that bitmaps are not missing in
lower layers (gaps)
- if the lists match, create empty bitmap with the same name and
granularity in the top image
What do we have to do for the old format? or we just not implement
this until we get the new
format?
Can you add a section explaining how bitmaps should be handled in blockCommit?
We may need to implement this for cold merge using qemu-img commit.
Looking at the current structure, it looks like we have to do:
1. commit top layer to base layer (we support only one layer commit)
2. merge all bitmaps from top layer to base layer
3. copy all bitmaps in top layer that are not in base layer to base layer
Nir
+
+ done
--
2.26.2
1:32 a.m.
New subject: [PATCH 32/32] kbase: Add document outlining internals of incremental backup in qemu
On Mon, Jun 22, 2020 at 02:40:18 +0300, Nir Soffer wrote:
On Mon, Jun 15, 2020 at 8:13 PM Peter Krempa
<pkrempa(a)redhat.com> wrote:
>
> Outline the basics and how to integrate with externally created
> overlays. Other topics will continue later.
Thanks, this is very helpful!
> Signed-off-by: Peter Krempa <pkrempa(a)redhat.com>
> ---
> docs/kbase.html.in | 3 +
> docs/kbase/incrementalbackupinternals.rst | 210 ++++++++++++++++++++++
> 2 files changed, 213 insertions(+)
> create mode 100644 docs/kbase/incrementalbackupinternals.rst
[...]
> +
> +Checking bitmap health
> +----------------------
> +
> +QEMU optimizes disk writes by only updating the bitmaps in certain cases. This
> +also can cause problems in cases when e.g. QEMU crashes.
> +
> +For a chain of bitmaps corresponding in a backing chain to be considered valid
> +and eligible for use with ``virDomainBackupBegin`` it must conform to the
> +following rules:
> +
> +1) Top image must contain the bitmap
> +2) If any of the backing images in the chain contain the bitmap too all
> + contiguous images must have the bitmap (no gaps)
> +3) all of the above bitmaps must be marked as active
> + (``auto`` flag in ``qemu-img`` output, ``recording`` in qemu)
> +4) none of the above bitmaps can be inconsistent
> + (``in-use`` flag in ``qemu-img`` provided that it's not used on image which
> + is currently in use by a qemu instance, or ``inconsistent`` in qemu)
Can you add a chapter of about the old format and how it was different
from the new
format?
No, I don't plan to do that. This feature was never enabled upstream
yet, so except for the users who hacked-in the support by adding the
correct capability, nobody was exposed to bitmaps managed by libvirt
yet.
As of such I don't feel it's necessary or even worth documenting the old
state.
Looks like the differences are:
- all bitmaps are always active, so no need to enable or disable them
- based on next section, new snapshost contain all the bitmaps from
the previous snapshots
(since all of them active and we copy all of them to the new snapshot)
Yes.
How qemu knows which bitmap should track changes if all bitmaps are
active?
When libvirt starts a VM, it knows nothing about the checkpoints. We
define the checkpoints
right before the first backup after starting a VM. So both libvirt and
qemu know nothing about
the bitmaps at this point.
QEMU tracks changes for all active bitmaps, that's why they are active.
Libvirt doesn't care at startup, but cares if you want to do a backup.
Do you expect to have the checkpoints defined before the guest is
started?
If I understand this correctly, if we do:
- create base image
- do full backup (check-1)
- do incremental backup 1 (check-2)
- create snapshot-1
- do incremental backup 2 (check-3)
- do incremental backup 3 (check-4)
- create snapshot-2
- do incremental backup 4 (check-5)
This will be the image structure:
- base image
- check-1
- check-2
- snapshot-1
- check-1
- check-2
- check-3
- check-4
- snapshot-2
- check-1
- check-2
- check-3
- check-4
- check-5
So we are duplicating bitmaps that have no content in all snapshot?
Yes. It's way easier to manage them that way.
Why not copy only the last (current) bitmap?
- base image
- check-1
- check-2
- snapshot-1
- check-2
- check-3
- check-4
- snapshot-2
- check-4
- check-5
This is too complicated to deal with. That's what we did for now and
it's a nightmare to compute what to do.
Ideally when 'block-dirty-bitmap-populate' materializes, I'll add a flag
for API creating snapshots which will stop pulling bitmaps up, so you'll
end up with:
- base image
- check-1
- check-2
- snapshot-1
- check-3
- check-4
- snapshot-2
- check-5
Since the rest can be re-calculated It actually doesn't make sense to
use any bitmap after a snapshot.
> +::
> +
> + # check that image has bitmaps
> + $ qemu-img info vda-1.qcow2
> + image: vda-1.qcow2
> + file format: qcow2
> + virtual size: 100 MiB (104857600 bytes)
> + disk size: 220 KiB
> + cluster_size: 65536
> + Format specific information:
> + compat: 1.1
> + compression type: zlib
> + lazy refcounts: false
> + bitmaps:
> + [0]:
> + flags:
> + [0]: in-use
> + [1]: auto
> + name: chk-a
> + granularity: 65536
> + [1]:
> + flags:
> + [0]: auto
> + name: chk-b
> + granularity: 65536
> + refcount bits: 16
> + corrupt: false
> +
> +(See also the ``qemuBlockBitmapChainIsValid`` helper method in
> +``src/qemu/qemu_block.c``)
Looks like oVirt needs to implement this a well, otherwise we will waste time
creating bitmaps on a snapshot when libvirt will fail the backup later since
there was an inconsistent or disabled bitmap.
> +Creating external checkpoints manually
> +--------------------------------------
> +
> +To create the same topology outside of libvirt (e.g when doing snapshots offline)
> +a new ``qemu-img`` which supports the ``bitmap`` subcomand is necessary. The
> +following algorithm then ensures that the new image after snapshot will work
> +with backups (note that ``jq`` is a JSON processor):
> +
> +::
> +
> + # arguments
> + SNAP_IMG="vda-2.qcow2"
> + BACKING_IMG="vda-1.qcow2"
> +
> + # constants - snapshots and bitmaps work only with qcow2
> + SNAP_FMT="qcow2"
> + BACKING_IMG_FMT="qcow2"
> +
> + # create snapshot overlay
> + qemu-img create -f "$SNAP_FMT" -F "$BACKING_IMG_FMT" -b
"$BACKING_IMG" "$SNAP_IMG"
> +
> + BACKING_IMG_INFO=$(qemu-img info --output=json -f "$BACKING_IMG_FMT"
"$BACKING_IMG")
> + BACKING_BITMAPS=$(jq '."format-specific".data.bitmaps'
<<< "$BACKING_IMG_INFO")
> +
> + if [ "x$BACKING_BITMAPS" == "xnull" ]; then
> + exit 0
> + fi
> +
> + for BACKING_BITMAP_ in $(jq -c '.[]' <<<
"$BACKING_BITMAPS"); do
> + BITMAP_FLAGS=$(jq -c -r '.flags[]' <<<
"$BACKING_BITMAP_")
> + BITMAP_NAME=$(jq -r '.name' <<<
"$BACKING_BITMAP_")
> +
> + if grep 'in-use' <<< "$BITMAP_FLAGS" ||
> + grep -v 'auto' <<< "$BITMAP_FLAGS"; then
> + continue
> + fi
> +
> + qemu-img bitmap -f "$SNAP_FMT" "$SNAP_IMG" --add
"$BITMAP_NAME"
So what we have to do is:
- get a list of bitmaps that should be in this disk from oVirt engine
- get list of bitmaps with "auto" flag and without the "in-use" flag
in the backing file
- if the lists do not match we can delete all bitmaps and relevant
checkpoints on engine
side, since the next backup will fail anyway.
- if the lists match, maybe verify that bitmaps are not missing in
lower layers (gaps)
- if the lists match, create empty bitmap with the same name and
granularity in the top image
What do we have to do for the old format? or we just not implement
this until we get the new
format?
The above algorithm will work even for the old format, by the way.
There's just one active bitmap at any point. The validation algorithm
will not work obviously.
The problem of the "old" format is not in snapshots, but rather in any
other operation.
Can you add a section explaining how bitmaps should be handled in
blockCommit?
We may need to implement this for cold merge using qemu-img commit.
Yes, right after this get's merged. By the way, block commit is one of
the things where the 'old' format was becoming insane.
Looking at the current structure, it looks like we have to do:
1. commit top layer to base layer (we support only one layer commit)
2. merge all bitmaps from top layer to base layer
3. copy all bitmaps in top layer that are not in base layer to base layer
Yes, for that special case it's true. Technically I'd write it as:
2) create bitmaps in base layer which are in top but not in base (they
are empty now
3) merge all bitmaps from top into corresponding bitmap in base
1614
days inactive
1621
days old
71 comments
3 participants
participants (3)
-
Eric Blake -
Nir Soffer -
Peter Krempa