[libvirt] [PATCH v10 00/19] Checkpoint APIs (incremental backup saga)
by Eric Blake
This is a subset of v10 of incremental backup, fixing Peter's review
comments from v9 (and even some from v8 that I had missed earlier).
There's still a lot more rebasing churn to resolve in the backup
portion of the series before I can post a full backup-v10 label.
There's still a decision to make: do we want this series in 5.6
(possibly with the addition of just patch 2/10 at [1] to introduce the
backup API to make it possible for downstream to backport features
without bumping .so)? This series has checkpoint support for test and
qemu drivers, and I think is probably clean enough to finally satisfy
everything Peter has been pointing out; but while the backup API
itself seems reasonable, the qemu implementation will likely miss 5.6
(as that half of the v9 posting was further behind, and still has a
lot of rebase churn to resolve). Or do we delay the checkpoint API to
5.7, to only go in with backup API? There's also still an outstanding
question of whether the backup API needs a tweak to use 'const char *'
instead of 'int' for the job identifier, given that Peter has a
proposal for overhauling the representation of libvirt jobs.
[1] https://www.redhat.com/archives/libvir-list/2019-July/msg00330.html
Here's a comparison of these patches to v9:
- rebase on top of master
- use more VIR_AUTOFREE and friends (but probably still missed some
spots where it would be worthwhile)
- split several patches
- add in a new patch to ensure the node names are learned at domain
startup (but I didn't play enough with hotplug to see if that also
has problems); necessary in order to...
- get rid of detecting node names in checkpoint code
- completely got rid of the notion of a current checkpoint in the
public API (the internal code still has one, but solely to track
which checkpoint becomes the parent of a new one to maintain a linear
chain; when loading checkpoints on libvirtd restart, there should be
at most one leaf node which becomes the internal current node)
- improved code for handling partial checkpoints (the code now correctly
handles: c1: disks A, B; c2: disks B, C; c3: disks A, B)
001/19:[down] 'snapshot: Don't leak moment obj list metaroot to callers'
002/19:[down] 'snapshot: Saner error message for duplicate create'
003/19:[down] 'snapshot: Documentation and comment improvements'
004/19:[down] 'backup: qemu: Detect node names at domain startup'
005/19:[0101] [FC] 'backup: Document new XML for checkpoints'
006/19:[0031] [FC] 'backup: Introduce virDomainCheckpoint APIs'
007/19:[0051] [FC] 'backup: Document nuances between different state capture APIs'
008/19:[0189] [FC] 'backup: Parse and output checkpoint XML'
009/19:[0066] [FC] 'backup: Allow for lists of checkpoint objects'
010/19:[----] [--] 'backup: Add new domain:checkpoint access control'
011/19:[----] [--] 'backup: Implement checkpoint APIs for remote driver'
012/19:[0161] [FC] 'backup: Implement virsh support for checkpoints'
013/19:[0047] [FC] 'backup: test: Implement metadata tracking for checkpoint APIs'
014/19:[0044] [FC] 'backup: Add virsh-checkpoints test'
015/19:[down] 'backup: qemu: Add directory for tracking checkpoints'
016/19:[0193] [FC] 'backup: qemu: Implement metadata tracking for checkpoint APIs'
017/19:[down] 'backup: qemu: Add helper API for looking up node name'
018/19:[0119] [FC] 'backup: Wire up qemu checkpoint commands over QMP'
019/19:[down] 'backup: Prevent snapshots and checkpoints at same time'
Eric Blake (19):
snapshot: Don't leak moment obj list metaroot to callers
snapshot: Saner error message for duplicate create
snapshot: Documentation and comment improvements
backup: qemu: Detect node names at domain startup
backup: Document new XML for checkpoints
backup: Introduce virDomainCheckpoint APIs
backup: Document nuances between different state capture APIs
backup: Parse and output checkpoint XML
backup: Allow for lists of checkpoint objects
backup: Add new domain:checkpoint access control
backup: Implement checkpoint APIs for remote driver
backup: Implement virsh support for checkpoints
backup: test: Implement metadata tracking for checkpoint APIs
backup: Add virsh-checkpoints test
backup: qemu: Add directory for tracking checkpoints
backup: qemu: Implement metadata tracking for checkpoint APIs
backup: qemu: Add helper API for looking up node name
backup: Wire up qemu checkpoint commands over QMP
backup: Prevent snapshots and checkpoints at same time
include/libvirt/libvirt-domain-checkpoint.h | 137 ++
include/libvirt/libvirt-domain.h | 6 +
include/libvirt/libvirt.h | 5 +-
src/access/viraccessperm.h | 6 +
src/conf/checkpoint_conf.h | 91 ++
src/conf/domain_conf.h | 2 +
src/conf/virconftypes.h | 9 +
src/conf/virdomaincheckpointobjlist.h | 72 +
src/conf/virdomainmomentobjlist.h | 10 +-
src/conf/virdomainobjlist.h | 7 +-
src/conf/virdomainsnapshotobjlist.h | 2 +
src/driver-hypervisor.h | 38 +
src/qemu/qemu_conf.h | 2 +
src/qemu/qemu_domain.h | 18 +
tools/virsh-checkpoint.h | 26 +
tools/virsh-completer.h | 4 +
tools/virsh-util.h | 3 +
tools/virsh.h | 1 +
docs/Makefile.am | 3 +
docs/apibuild.py | 2 +
docs/docs.html.in | 4 +-
docs/format.html.in | 1 +
docs/formatcheckpoint.html.in | 196 +++
docs/formatsnapshot.html.in | 14 +-
docs/index.html.in | 3 +-
docs/kbase.html.in | 5 +
docs/kbase/domainstatecapture.html.in | 303 +++++
docs/schemas/domaincheckpoint.rng | 94 ++
libvirt.spec.in | 2 +
mingw-libvirt.spec.in | 4 +
po/POTFILES | 3 +
src/Makefile.am | 2 +
src/access/viraccessperm.c | 3 +-
src/conf/Makefile.inc.am | 4 +
src/conf/checkpoint_conf.c | 597 +++++++++
src/conf/domain_conf.c | 6 +
src/conf/virdomaincheckpointobjlist.c | 243 ++++
src/conf/virdomainmomentobjlist.c | 55 +-
src/conf/virdomainobjlist.c | 11 +
src/conf/virdomainsnapshotobjlist.c | 9 +
src/libvirt-domain-checkpoint.c | 567 ++++++++
src/libvirt-domain.c | 19 +-
src/libvirt_private.syms | 29 +-
src/libvirt_public.syms | 16 +
src/qemu/qemu_conf.c | 5 +
src/qemu/qemu_domain.c | 190 +++
src/qemu/qemu_driver.c | 731 ++++++++++-
src/qemu/qemu_process.c | 5 +
src/remote/remote_daemon_dispatch.c | 20 +
src/remote/remote_driver.c | 24 +-
src/remote/remote_protocol.x | 123 +-
src/remote_protocol-structs | 69 +
src/rpc/gendispatch.pl | 32 +-
src/test/test_driver.c | 386 +++++-
tests/Makefile.am | 12 +-
.../disk-default.xml | 7 +
.../disk-invalid.xml | 8 +
tests/qemudomaincheckpointxml2xmlin/empty.xml | 1 +
.../name-invalid.xml | 4 +
.../qemudomaincheckpointxml2xmlin/sample.xml | 7 +
tests/qemudomaincheckpointxml2xmlin/size.xml | 4 +
.../disk-default.xml | 11 +
.../qemudomaincheckpointxml2xmlout/empty.xml | 7 +
.../internal-active-invalid.xml | 53 +
.../internal-inactive-invalid.xml | 53 +
.../redefine.xml | 63 +
.../qemudomaincheckpointxml2xmlout/sample.xml | 12 +
tests/qemudomaincheckpointxml2xmlout/size.xml | 11 +
tests/qemudomaincheckpointxml2xmltest.c | 213 +++
tests/virschematest.c | 2 +
tests/virsh-checkpoint | 174 +++
tests/virsh-snapshot | 15 +-
tools/Makefile.am | 1 +
tools/virsh-checkpoint.c | 1153 +++++++++++++++++
tools/virsh-completer.c | 51 +
tools/virsh-domain-monitor.c | 23 +
tools/virsh-domain.c | 7 +
tools/virsh-util.c | 11 +
tools/virsh.c | 2 +
tools/virsh.pod | 201 ++-
80 files changed, 6262 insertions(+), 63 deletions(-)
create mode 100644 include/libvirt/libvirt-domain-checkpoint.h
create mode 100644 src/conf/checkpoint_conf.h
create mode 100644 src/conf/virdomaincheckpointobjlist.h
create mode 100644 tools/virsh-checkpoint.h
create mode 100644 docs/formatcheckpoint.html.in
create mode 100644 docs/kbase/domainstatecapture.html.in
create mode 100644 docs/schemas/domaincheckpoint.rng
create mode 100644 src/conf/checkpoint_conf.c
create mode 100644 src/conf/virdomaincheckpointobjlist.c
create mode 100644 src/libvirt-domain-checkpoint.c
create mode 100644 tests/qemudomaincheckpointxml2xmlin/disk-default.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlin/disk-invalid.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlin/empty.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlin/name-invalid.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlin/sample.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlin/size.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/disk-default.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/empty.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/internal-active-invalid.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/internal-inactive-invalid.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/redefine.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/sample.xml
create mode 100644 tests/qemudomaincheckpointxml2xmlout/size.xml
create mode 100644 tests/qemudomaincheckpointxml2xmltest.c
create mode 100755 tests/virsh-checkpoint
create mode 100644 tools/virsh-checkpoint.c
--
2.20.1
5 years, 9 months
[libvirt] [PATCH] commandtest: Remove commandhelper.log in test27
by Michal Privoznik
The recently added test27 spawns commandhelper. This is fine,
except, one of the things that commandhelper does is it records
arguments it was spawn with into commandhelper.log. Other test
cases then use checkoutput() to compare the arguments against the
expected ones and also unlink() the log file. However, test27()
is not doing that and thus it leaves the file behind. This
breaks distcheck.
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
Another approach would be to explicitly just unlink() the file somewhere
in the cleanup label. But I figured, comparing the args and unlinking
the file afterwards is more robust check IMO.
tests/commanddata/test27.log | 20 ++++++++++++++++++++
tests/commandtest.c | 3 +++
2 files changed, 23 insertions(+)
create mode 100644 tests/commanddata/test27.log
diff --git a/tests/commanddata/test27.log b/tests/commanddata/test27.log
new file mode 100644
index 0000000000..40b2627c07
--- /dev/null
+++ b/tests/commanddata/test27.log
@@ -0,0 +1,20 @@
+ARG:--readfd
+ARG:5
+ARG:--readfd
+ARG:7
+ENV:DISPLAY=:0.0
+ENV:HOME=/home/test
+ENV:HOSTNAME=test
+ENV:LANG=C
+ENV:LOGNAME=test
+ENV:PATH=/usr/bin:/bin
+ENV:TMPDIR=/tmp
+ENV:USER=test
+FD:0
+FD:1
+FD:2
+FD:5
+FD:7
+DAEMON:no
+CWD:/tmp
+UMASK:0022
diff --git a/tests/commandtest.c b/tests/commandtest.c
index dfd15a2079..d7ab588969 100644
--- a/tests/commandtest.c
+++ b/tests/commandtest.c
@@ -1231,6 +1231,9 @@ static int test27(const void *unused ATTRIBUTE_UNUSED)
goto cleanup;
}
+ if (checkoutput("test27", NULL) < 0)
+ goto cleanup;
+
ret = 0;
cleanup:
--
2.21.0
5 years, 9 months
[libvirt] [PATCH v2 0/2] test_driver: implement virDomainGetCPUStats
by Ilias Stamatis
Changes since v1:
* Removed failing test cases
* Total CPUTIME now is the sum of the individual CPUs
Ilias Stamatis (2):
tests: virsh-optparse: remove no longer valid cpu-stats test cases
test_driver: implement virDomainGetCPUStats
src/test/test_driver.c | 132 +++++++++++++++++++++++++++++++++++++++++
tests/virsh-optparse | 20 -------
2 files changed, 132 insertions(+), 20 deletions(-)
--
2.22.0
5 years, 9 months
[libvirt] [PATCH 0/5] Fix hyperv and kvm features with QEMU 4.1
by Jiri Denemark
Originally the names of the hyperv and kvm CPU features were only used
internally for looking up their CPUID bits. But with QEMU 4.1 we check
which features were enabled or disabled by a freshly started QEMU
process using their names rather than their CPUID bits (mostly because
of MSR features). Thus we need to change our made up internal names into
the actual names used by QEMU.
Otherwise libvirt would mistakenly report the features as unavailable
and refuse to start any domain using them with QEMU 4.1.
Reported-by: Vitaly Kuznetsov <vkuznets(a)redhat.com>
Jiri Denemark (5):
qemu: Fix hyperv features with QEMU 4.1
qemu: Prefer dashes for hyperv features
cpu: Drop KVM_ from hyperv feature macros
cpu: Drop unused KVM features
qemu: Fix KVM features with QEMU 4.1
src/cpu/cpu_x86.c | 77 +++++++------------
src/cpu/cpu_x86_data.h | 38 ++++-----
src/qemu/qemu_command.c | 19 +++--
src/qemu/qemu_process.c | 2 +-
.../clock-timer-hyperv-rtc.args | 2 +-
tests/qemuxml2argvdata/hyperv-panic.args | 2 +-
tests/qemuxml2argvdata/hyperv.args | 4 +-
tests/qemuxml2argvdata/panic-double.args | 2 +-
8 files changed, 61 insertions(+), 85 deletions(-)
--
2.22.0
5 years, 9 months
[libvirt] [PATCH 0/6] Add support for Direct Mode for Hyper-V Synthetic timers
by Vitaly Kuznetsov
QEMU-4.1 will bring us Direct Mode for Hyper-V Synthetic timers support,
we need to support it in libvirt too. As this is not a new enlightenment
but rather an enhancement of an existing one ('stimer'), support it in
<stimer state='on'>
<direct state='on'/>
</stimer>
form. Backwards compatibility is (hopefully) preserved.
Because of an existing imcompatibility between libvirt-5.5+ and
QEMU-4.1-git (see https://www.redhat.com/archives/libvir-list/2019-July/msg01435.html)
I had to backport hv-stimer-direct to QEMU-4.0 to test this. If someone wants to
do it too here is a list of QEMU patches needed:
128531d9e1 i386/kvm: add support for Direct Mode for Hyper-V synthetic timers
8caba36db5 i386/kvm: hv-evmcs requires hv-vapic
bd59fbdf4f i386/kvm: hv-tlbflush/ipi require hv-vpindex
c686193072 i386/kvm: hv-stimer requires hv-time and hv-synic
e48ddcc6ce i386/kvm: implement 'hv-passthrough' mode
fb19f72b77 i386/kvm: document existing Hyper-V enlightenments
2344d22e50 i386/kvm: move Hyper-V CPUID filling to hyperv_handle_properties()
6760bd2002 i386/kvm: add support for KVM_GET_SUPPORTED_HV_CPUID
2d384d7c83 i386/kvm: convert hyperv enlightenments properties from bools to bits
Vitaly Kuznetsov (6):
docs: formatdomain: add synic flag to the example xml
docs: formatdomain: fix 'SynIC' spelling
docs: formatdomain: move 'msrs' out of Hyper-V Enlightenments
conf: add support for Direct Mode for Hyper-V Synthetic timers
qemu: add support for Direct Mode for Hyper-V Synthetic timers
news: mention Direct Mode for Hyper-V Synthetic timers support
docs/formatdomain.html.in | 13 ++-
docs/news.xml | 9 ++
docs/schemas/domaincommon.rng | 16 +++-
src/conf/domain_conf.c | 138 +++++++++++++++++++++++++---
src/conf/domain_conf.h | 8 ++
src/cpu/cpu_x86.c | 51 +++++-----
src/cpu/cpu_x86_data.h | 2 +
src/libvirt_private.syms | 2 +
src/qemu/qemu_command.c | 22 ++++-
src/qemu/qemu_process.c | 39 +++++++-
tests/qemuxml2argvdata/hyperv.args | 4 +-
tests/qemuxml2argvdata/hyperv.xml | 4 +-
tests/qemuxml2xmloutdata/hyperv.xml | 4 +-
13 files changed, 263 insertions(+), 49 deletions(-)
--
2.20.1
5 years, 9 months
[libvirt] [PATCH] qemumonitorjsontest: Don't leak 'query-jobs' info
by Michal Privoznik
The returned array of qemuMonitorJobInfo structs must be freed.
164 (16 direct, 148 indirect) bytes in 1 blocks are definitely lost in loss record 64 of 84
at 0x4A3568B: realloc (vg_replace_malloc.c:826)
by 0x4D888BD: virReallocN (viralloc.c:244)
by 0x4D889B3: virExpandN (viralloc.c:293)
by 0x4D88C87: virInsertElementsN (viralloc.c:435)
by 0x214004: qemuMonitorJSONGetJobInfo (qemu_monitor_json.c:9185)
by 0x148B3F: testQueryJobs (qemumonitorjsontest.c:2979)
by 0x14C192: virTestRun (testutils.c:174)
by 0x14BF36: mymain (qemumonitorjsontest.c:3286)
by 0x14E256: virTestMain (testutils.c:1096)
by 0x14BFD9: main (qemumonitorjsontest.c:3298)
Signed-off-by: Michal Privoznik <mprivozn(a)redhat.com>
---
tests/qemumonitorjsontest.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tests/qemumonitorjsontest.c b/tests/qemumonitorjsontest.c
index bf89f49aca..522ff5a3d5 100644
--- a/tests/qemumonitorjsontest.c
+++ b/tests/qemumonitorjsontest.c
@@ -2996,6 +2996,9 @@ testQueryJobs(const void *opaque)
ret = 0;
cleanup:
+ for (i = 0; i < njobs; i++)
+ qemuMonitorJobInfoFree(jobs[i]);
+ VIR_FREE(jobs);
qemuMonitorTestFree(test);
return ret;
}
--
2.21.0
5 years, 9 months
[libvirt] UID/GID during kvm/qemu migrate
by Stephan von Krawczynski
Hello,
is there some immanent code in libvirt that forces UID/GID of the libvirt
standard user to be the same on two boxes migrating qemu vms against each
other?
The migration itself uses root obviously (password is requested). But if a vm
xml does not contain any definition regarding UID/GID what else could
prevent this from working?
I believe I ran into such a problem trying to migrate and ending up in an
error, a vm still working on original host but its fs (netfs pool (nfs/raw))
being switched to read-only...
--
Regards,
Stephan
5 years, 9 months
[libvirt] [PATCH 0/9] qemu: Support block pull/commit jobs with blockdev (blockdev-add saga)
by Peter Krempa
One of the final chapters of the saga. At the end we finally support
first blockjobs with blockdev.
This series depends on some of the previous postings so for convenience
it can be fetched at:
git fetch https://gitlab.com/pipo.sk/libvirt.git blockjobs-send-partial
The above branch also contains patches which enable blockdev support,
which is ideal for testing.
Peter Krempa (9):
qemu: blockjob: Use proper value when setting disk's READY state
tests: qemustatusxml: Add few disks with backing chains for blockdev
job testing
qemu: blockjob: Document qemuBlockJobRegister
qemu: blockjob: Register disk->mirror with a job only when required
qemu: blockjob: Split out update of persistent XML disk's source on
mirror jobs
qemu: blockjob: Esure that persistent definition source does not copy
irrelevant data
qemu: blockjob: Ensure that config disk source is identical when
modifying it
qemu: Add -blockdev support for block pull job
qemu: Add -blockdev support for block commit job
src/qemu/qemu_blockjob.c | 525 +++++++++++++++++-
src/qemu/qemu_blockjob.h | 40 ++
src/qemu/qemu_domain.c | 123 +++-
src/qemu/qemu_driver.c | 83 ++-
.../blockjob-blockdev-in.xml | 285 +++++++++-
5 files changed, 1001 insertions(+), 55 deletions(-)
--
2.21.0
5 years, 9 months
