[libvirt] [PATCH v2 RFC 00/12] introduce push backups
by Nikolay Shirokovskiy
* Diff from v1 [3]
- add man/html docs
- add backup xml schema
Push backup is a backup when hypervisor itself copy backup data to destination
in contrast to pull backup when hypervisor exports backup data thru some
interface and mgmt itself make a copy.
This patch series basically adds creating backup to API/remote/qemu/virsh and
initial backup XML definition.
Just like other blockjobs backup creation is asynchronous. That is creation is
merely a backup start and client should track backup error/completion thru
blockjob events. As backup is done transactionally all individual disk backup
jobs will be aborted by qemu itself in case of error, client need not to do it
manually. Client can cancel the backup by aborting blockjob on any disk being
backed up.
Backup xml desription is similar to snapshot one with some exceptions and is
described in more details in definition patch [p1] or in html docs.
I guess good client will track progress for every disk in backup to report
progress and detect hangs so it don't need extra backup complete/error event
that aggregate the overall backup result. However it looks like aborting backup
can be implemented in libvirt as code will be common for all clients. We need
to abort some of not yet completed per disk backups and retry if job to be
aborted is completed meanwhile.
Of coures this series is far from being complete. Incremental backups and
backup persistent metadata is to be implemented. Let's just start work in this
direction.
Links:
In [1] we came to agreement to create distinct API to support backup operations
and there is a discussion of pull backup series in [2]. The latter is blocked
as some necessary operations are still experimental in qemu. [3] is the
first version of series.
[1] https://www.redhat.com/archives/libvir-list/2016-March/msg00937.html
[2] https://www.redhat.com/archives/libvir-list/2016-September/msg00192.html
[3] https://www.redhat.com/archives/libvir-list/2017-May/msg00130.html
Nikolay Shirokovskiy (12):
api: backup: add api to create backup
api: backup: add driver based implementation
remote: backup: add create backup implementation
backup: qemu: monitor: add drive-backup command
backup: misc: add backup block job type
conf: backup: add backup xml definition [p1]
qemu: backup: add qemuDomainBackupCreateXML implementation
qemu: backup: check backup destination before start
qemu: backup: prepare backup destination
virsh: backup: add backup-create command
schema: backup: add schema and its tests
docs: add backup html docs
daemon/remote.c | 8 +
docs/Makefile.am | 3 +
docs/apibuild.py | 2 +
docs/docs.html.in | 4 +-
docs/format.html.in | 1 +
docs/formatbackup.html.in | 58 +++++
docs/index.html.in | 3 +-
docs/schemas/domainbackup.rng | 79 +++++++
examples/object-events/event-test.c | 3 +
include/libvirt/libvirt-domain-backup.h | 59 +++++
include/libvirt/libvirt-domain.h | 3 +
include/libvirt/libvirt.h | 1 +
include/libvirt/virterror.h | 2 +
po/POTFILES.in | 2 +
src/Makefile.am | 3 +
src/access/viraccessperm.c | 3 +-
src/access/viraccessperm.h | 6 +
src/conf/backup_conf.c | 299 +++++++++++++++++++++++++
src/conf/backup_conf.h | 70 ++++++
src/conf/domain_conf.c | 2 +-
src/datatypes.c | 60 +++++
src/datatypes.h | 29 +++
src/driver-hypervisor.h | 5 +
src/libvirt-domain-backup.c | 209 +++++++++++++++++
src/libvirt_private.syms | 9 +
src/libvirt_public.syms | 10 +
src/qemu/qemu_conf.h | 1 +
src/qemu/qemu_driver.c | 239 +++++++++++++++++++-
src/qemu/qemu_monitor.c | 14 ++
src/qemu/qemu_monitor.h | 5 +
src/qemu/qemu_monitor_json.c | 36 +++
src/qemu/qemu_monitor_json.h | 5 +
src/remote/remote_driver.c | 7 +
src/remote/remote_protocol.x | 23 +-
src/rpc/gendispatch.pl | 29 ++-
src/util/virerror.c | 6 +
tests/domainbackupxml/block_target.xml | 5 +
tests/domainbackupxml/explicit_description.xml | 6 +
tests/domainbackupxml/explicit_file_type.xml | 5 +
tests/domainbackupxml/explicit_format.xml | 5 +
tests/domainbackupxml/explicit_name.xml | 6 +
tests/domainbackupxml/multi_disk.xml | 8 +
tests/domainbackupxml/ref_by_path.xml | 5 +
tests/virschematest.c | 1 +
tools/Makefile.am | 1 +
tools/virsh-backup.c | 100 +++++++++
tools/virsh-backup.h | 29 +++
tools/virsh-domain.c | 3 +-
tools/virsh-util.c | 11 +
tools/virsh-util.h | 3 +
tools/virsh.c | 2 +
tools/virsh.h | 1 +
tools/virsh.pod | 20 ++
tools/virt-xml-validate.in | 5 +-
54 files changed, 1499 insertions(+), 15 deletions(-)
create mode 100644 docs/formatbackup.html.in
create mode 100644 docs/schemas/domainbackup.rng
create mode 100644 include/libvirt/libvirt-domain-backup.h
create mode 100644 src/conf/backup_conf.c
create mode 100644 src/conf/backup_conf.h
create mode 100644 src/libvirt-domain-backup.c
create mode 100644 tests/domainbackupxml/block_target.xml
create mode 100644 tests/domainbackupxml/explicit_description.xml
create mode 100644 tests/domainbackupxml/explicit_file_type.xml
create mode 100644 tests/domainbackupxml/explicit_format.xml
create mode 100644 tests/domainbackupxml/explicit_name.xml
create mode 100644 tests/domainbackupxml/multi_disk.xml
create mode 100644 tests/domainbackupxml/ref_by_path.xml
create mode 100644 tools/virsh-backup.c
create mode 100644 tools/virsh-backup.h
--
1.8.3.1
7 years, 6 months
[libvirt] [PATCH 0/2] improve detection of UNIX path generated by libvirt
by Pavel Hrdina
Pavel Hrdina (2):
util: introduce virStringMatch
qemu: improve detection of UNIX path generated by libvirt
src/libvirt_private.syms | 1 +
src/qemu/qemu_domain.c | 78 +++++++++++++++++++---
src/util/virstring.c | 34 ++++++++++
src/util/virstring.h | 3 +
.../qemuxml2argv-channel-unix-gen-path1.xml | 17 +++++
.../qemuxml2argv-channel-unix-gen-path2.xml | 17 +++++
.../qemuxml2argv-channel-unix-gen-path3.xml | 17 +++++
.../qemuxml2argv-channel-unix-user-path.xml | 17 +++++
.../qemuxml2xmlout-channel-unix-gen-path1.xml | 32 +++++++++
.../qemuxml2xmlout-channel-unix-gen-path2.xml | 32 +++++++++
.../qemuxml2xmlout-channel-unix-gen-path3.xml | 32 +++++++++
.../qemuxml2xmlout-channel-unix-user-path.xml | 33 +++++++++
tests/qemuxml2xmltest.c | 5 ++
tests/virstringtest.c | 47 +++++++++++++
14 files changed, 356 insertions(+), 9 deletions(-)
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path1.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path2.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path3.xml
create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-user-path.xml
create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path1.xml
create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path2.xml
create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path3.xml
create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-user-path.xml
--
2.12.2
7 years, 6 months
[libvirt] [PATCH] conf: Fix resource leak in virCapabilitiesInitCaches
by John Ferlan
The @type from virFileReadValueString needs to be VIR_FREE each time
through the loop since it's not saved and since cleanup can be reached
prior to decoding it for @kernel_type amd bank->type, the cleanup code
needs to also have a VIR_FREE
Found by Coverity
Signed-off-by: John Ferlan <jferlan(a)redhat.com>
---
src/conf/capabilities.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c
index c36ca40..d699b08 100644
--- a/src/conf/capabilities.c
+++ b/src/conf/capabilities.c
@@ -1611,10 +1611,10 @@ virCapabilitiesInitCaches(virCapsPtr caps)
if (kernel_type < 0) {
virReportError(VIR_ERR_INTERNAL_ERROR,
_("Unknown cache type '%s'"), type);
- VIR_FREE(type);
goto cleanup;
}
bank->type = kernel_type;
+ VIR_FREE(type);
for (i = 0; i < caps->host.ncaches; i++) {
if (virCapsHostCacheBankEquals(bank, caps->host.caches[i]))
@@ -1637,6 +1637,7 @@ virCapabilitiesInitCaches(virCapsPtr caps)
ret = 0;
cleanup:
+ VIR_FREE(type);
VIR_FREE(path);
virDirClose(&dirp);
virCapsHostCacheBankFree(bank);
--
2.9.3
7 years, 6 months
[libvirt] [PATCH v2] qemu: reduce packet loss rate for vm with macvtap passthrough mode in migration
by ZhiPeng Lu
Before libvirt that calls virNetDevMacVLanCreateWithVPortProfile sets mac address
or vlan of a Virtual Function(VF) linked to a macvtap passthrough device of migration
destination host in migration start step. If we ping the migrating vm,
we get the network does not pass. Because VFs of migration source and destination
have the same MAC address. The patch later calling qemuMigrationVPAssociatePortProfiles
sets mac address of VF in migration finish step instead of start step.
The patch aims to reduce packet loss rate.
Signed-off-by: ZhiPeng Lu <lu.zhipeng(a)zte.com.cn>
---
src/qemu/qemu_migration.c | 18 ++++++++++++++++--
src/util/virnetdevmacvlan.c | 17 +++++++++++------
2 files changed, 27 insertions(+), 8 deletions(-)
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
index 09adb04..795ed71 100644
--- a/src/qemu/qemu_migration.c
+++ b/src/qemu/qemu_migration.c
@@ -5004,7 +5004,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver,
}
static int
-qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def)
+qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def, const char *stateDir)
{
size_t i;
int last_good_net = -1;
@@ -5013,6 +5013,20 @@ qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def)
for (i = 0; i < def->nnets; i++) {
net = def->nets[i];
if (virDomainNetGetActualType(net) == VIR_DOMAIN_NET_TYPE_DIRECT) {
+ if ((!virDomainNetGetActualVirtPortProfile(net) || (virDomainNetGetActualVirtPortProfile(net) &&
+ virDomainNetGetActualVirtPortProfile(net)->virtPortType != VIR_NETDEV_VPORT_PROFILE_8021QBG &&
+ virDomainNetGetActualVirtPortProfile(net)->virtPortType != VIR_NETDEV_VPORT_PROFILE_8021QBH)) &&
+ virDomainNetGetActualDirectMode(net) ==
+ VIR_NETDEV_MACVLAN_MODE_PASSTHRU) {
+ if (virNetDevSaveNetConfig(virDomainNetGetActualDirectDev(net),
+ -1, stateDir, false) < 0) {
+ goto err_exit;
+ }
+ if (virNetDevSetNetConfig(virDomainNetGetActualDirectDev(net),
+ -1, NULL, virDomainNetGetActualVlan(net), &net->mac, false) < 0) {
+ goto err_exit;
+ }
+ }
if (virNetDevVPortProfileAssociate(net->ifname,
virDomainNetGetActualVirtPortProfile(net),
&net->mac,
@@ -5187,7 +5201,7 @@ qemuMigrationFinish(virQEMUDriverPtr driver,
goto endjob;
}
- if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0)
+ if (qemuMigrationVPAssociatePortProfiles(vm->def, cfg->stateDir) < 0)
goto endjob;
if (mig->network && qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0)
diff --git a/src/util/virnetdevmacvlan.c b/src/util/virnetdevmacvlan.c
index 7222b0f..682dcd1 100644
--- a/src/util/virnetdevmacvlan.c
+++ b/src/util/virnetdevmacvlan.c
@@ -1020,12 +1020,17 @@ virNetDevMacVLanCreateWithVPortProfile(const char *ifnameRequested,
*/
setVlan = false;
}
-
- if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0)
- return -1;
-
- if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, setVlan) < 0)
- return -1;
+ if (vmOp != VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START &&
+ virtPortProfile && (virtPortProfile->virtPortType == VIR_NETDEV_VPORT_PROFILE_8021QBH ||
+ virtPortProfile->virtPortType == VIR_NETDEV_VPORT_PROFILE_8021QBG)) {
+ if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) {
+ return -1;
+ }
+ if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress,
+ setVlan) < 0) {
+ return -1;
+ }
+ }
}
if (ifnameRequested) {
--
1.8.3.1
7 years, 6 months
[libvirt] [PATCH] qemu: reduce packet loss rate for vm with macvtap passthrough mode in migration
by ZhiPeng Lu
Before libvirt that calls virNetDevMacVLanCreateWithVPortProfile sets mac address
or vlan of a Virtual Function(VF) linked to a macvtap passthrough device of migration
destination host in migration start step. If we ping the migrating vm,
we get the network does not pass. Because VFs of migration source and destination
have the same MAC address. The patch later calling qemuMigrationVPAssociatePortProfiles
sets mac address of VF in migration finish step instead of start step.
The patch aims to reduce packet loss rate.
Signed-off-by: ZhiPeng Lu <lu.zhipeng(a)zte.com.cn>
---
src/qemu/qemu_migration.c | 17 +++++++++++++++--
src/util/virnetdevmacvlan.c | 17 +++++++++++------
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
index 09adb04..0308043 100644
--- a/src/qemu/qemu_migration.c
+++ b/src/qemu/qemu_migration.c
@@ -5004,7 +5004,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver,
}
static int
-qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def)
+qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def, const char *stateDir)
{
size_t i;
int last_good_net = -1;
@@ -5013,6 +5013,19 @@ qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def)
for (i = 0; i < def->nnets; i++) {
net = def->nets[i];
if (virDomainNetGetActualType(net) == VIR_DOMAIN_NET_TYPE_DIRECT) {
+ if (virDomainNetGetActualVirtPortProfile(net) != VIR_NETDEV_VPORT_PROFILE_8021QBG &&
+ virDomainNetGetActualVirtPortProfile(net) != VIR_NETDEV_VPORT_PROFILE_8021QBH &&
+ (virDomainNetGetActualDirectMode(net) ==
+ VIR_NETDEV_MACVLAN_MODE_PASSTHRU)) {
+ if (virNetDevSaveNetConfig(virDomainNetGetActualDirectDev(net),
+ -1, stateDir, false) < 0) {
+ goto err_exit;
+ }
+ if (virNetDevSetNetConfig(virDomainNetGetActualDirectDev(net),
+ -1, NULL, virDomainNetGetActualVlan(net), &net->mac, false) < 0) {
+ goto err_exit;
+ }
+ }
if (virNetDevVPortProfileAssociate(net->ifname,
virDomainNetGetActualVirtPortProfile(net),
&net->mac,
@@ -5187,7 +5200,7 @@ qemuMigrationFinish(virQEMUDriverPtr driver,
goto endjob;
}
- if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0)
+ if (qemuMigrationVPAssociatePortProfiles(vm->def, cfg->stateDir) < 0)
goto endjob;
if (mig->network && qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0)
diff --git a/src/util/virnetdevmacvlan.c b/src/util/virnetdevmacvlan.c
index 7222b0f..fc08c43 100644
--- a/src/util/virnetdevmacvlan.c
+++ b/src/util/virnetdevmacvlan.c
@@ -1020,12 +1020,17 @@ virNetDevMacVLanCreateWithVPortProfile(const char *ifnameRequested,
*/
setVlan = false;
}
-
- if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0)
- return -1;
-
- if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, setVlan) < 0)
- return -1;
+ if (vmOp != VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START &&
+ (virtPortProfile == VIR_NETDEV_VPORT_PROFILE_8021QBH ||
+ virtPortProfile == VIR_NETDEV_VPORT_PROFILE_8021QBG)) {
+ if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) {
+ return -1;
+ }
+ if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress,
+ setVlan) < 0) {
+ return -1;
+ }
+ }
}
if (ifnameRequested) {
--
1.8.3.1
7 years, 6 months
[libvirt] [PATCH v2] Detect VMDK version 3 files
by Daniel P. Berrange
The metadata libvirt cares about is identical for version 3
as for previous versions, so we merely need list the new
version number.
Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com>
---
src/util/virstoragefile.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c
index 94a77ce..b43acf6 100644
--- a/src/util/virstoragefile.c
+++ b/src/util/virstoragefile.c
@@ -113,7 +113,7 @@ enum {
BACKING_STORE_ERROR,
};
-#define FILE_TYPE_VERSIONS_LAST 2
+#define FILE_TYPE_VERSIONS_LAST 3
struct FileEncryptionInfo {
int format; /* Encryption format to assign */
@@ -374,7 +374,7 @@ static struct FileTypeInfo const fileTypeInfo[] = {
},
[VIR_STORAGE_FILE_VMDK] = {
0, "KDMV", NULL,
- LV_LITTLE_ENDIAN, 4, 4, {1, 2},
+ LV_LITTLE_ENDIAN, 4, 4, {1, 2, 3},
4+4+4, 8, 512, NULL, vmdk4GetBackingStore, NULL
},
};
--
2.9.3
7 years, 6 months
[libvirt] [PATCH] Add "io" option to virsh attach-disk sub-command.
by Gordon Messmer
---
tools/virsh-domain.c | 14 +++++++++++---
tools/virsh.pod | 2 ++
2 files changed, 13 insertions(+), 3 deletions(-)
diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c
index 0d19d0e..d2a2a05 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -267,6 +267,10 @@ static const vshCmdOptDef opts_attach_disk[] = {
.type = VSH_OT_STRING,
.help = N_("cache mode of disk device")
},
+ {.name = "io",
+ .type = VSH_OT_STRING,
+ .help = N_("io mode of disk device")
+ },
{.name = "type",
.type = VSH_OT_STRING,
.help = N_("target device type")
@@ -504,8 +508,9 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd)
virDomainPtr dom = NULL;
const char *source = NULL, *target = NULL, *driver = NULL,
*subdriver = NULL, *type = NULL, *mode = NULL,
- *iothread = NULL, *cache = NULL, *serial = NULL,
- *straddr = NULL, *wwn = NULL, *targetbus = NULL;
+ *iothread = NULL, *cache = NULL, *io = NULL,
+ *serial = NULL, *straddr = NULL, *wwn = NULL,
+ *targetbus = NULL;
struct DiskAddress diskAddr;
bool isFile = false, functionReturn = false;
int ret;
@@ -537,6 +542,7 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd)
vshCommandOptStringReq(ctl, cmd, "mode", &mode) < 0 ||
vshCommandOptStringReq(ctl, cmd, "iothread", &iothread) < 0 ||
vshCommandOptStringReq(ctl, cmd, "cache", &cache) < 0 ||
+ vshCommandOptStringReq(ctl, cmd, "io", &io) < 0 ||
vshCommandOptStringReq(ctl, cmd, "serial", &serial) < 0 ||
vshCommandOptStringReq(ctl, cmd, "wwn", &wwn) < 0 ||
vshCommandOptStringReq(ctl, cmd, "address", &straddr) < 0 ||
@@ -579,7 +585,7 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd)
virBufferAddLit(&buf, ">\n");
virBufferAdjustIndent(&buf, 2);
- if (driver || subdriver || iothread || cache) {
+ if (driver || subdriver || iothread || cache || io) {
virBufferAddLit(&buf, "<driver");
if (driver)
@@ -590,6 +596,8 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd)
virBufferAsprintf(&buf, " iothread='%s'", iothread);
if (cache)
virBufferAsprintf(&buf, " cache='%s'", cache);
+ if (io)
+ virBufferAsprintf(&buf, " io='%s'", io);
virBufferAddLit(&buf, "/>\n");
}
diff --git a/tools/virsh.pod b/tools/virsh.pod
index cd1f25f..9656411 100644
--- a/tools/virsh.pod
+++ b/tools/virsh.pod
@@ -2860,6 +2860,8 @@ I<mode> can specify the two specific mode
I<readonly> or I<shareable>.
I<sourcetype> can indicate the type of source (block|file)
I<cache> can be one of "default", "none", "writethrough", "writeback",
"directsync" or "unsafe".
+I<io> io is "threads", or "native" and selects between pthread based disk
+I/O and native Linux AIO.
I<iothread> is the number within the range of domain IOThreads to which
this disk may be attached (QEMU only).
I<serial> is the serial of disk device. I<wwn> is the wwn of disk device.
--
2.9.3
7 years, 6 months
[libvirt] [PATCH 0/2] spec: Minor improvements
by Andrea Bolognani
*** BLURBY MCBLURBFACE ***
Andrea Bolognani (2):
spec: Support maintenance releases on mingw
spec: Use HTTPS instead of HTTP
libvirt.spec.in | 4 ++--
mingw-libvirt.spec.in | 8 ++++++--
2 files changed, 8 insertions(+), 4 deletions(-)
--
2.7.4
7 years, 6 months
