- Devel - libvirt List Archives

[libvirt] changing compat when create volume from volume via libvirt api
by Vasiliy Tolstov 12 May '17

12 May '17

I'm use via php binding https://libvirt.org/html/libvirt-libvirt-storage.html#virStorageVolCreateXM… and see that image grows: image: new.qcow2 file format: qcow2 virtual size: 4.9G (5242880000 bytes) disk size: 854M cluster_size: 65536 Format specific information: compat: 0.10 refcount bits: 16 image: old.qcow2 file format: qcow2 virtual size: 4.9G (5242880000 bytes) disk size: 277M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false Why libvirt create new image with different compat? -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

1 2

[libvirt] [PATCH v2 RFC 00/12] introduce push backups
by Nikolay Shirokovskiy 12 May '17

12 May '17

* Diff from v1 [3] - add man/html docs - add backup xml schema Push backup is a backup when hypervisor itself copy backup data to destination in contrast to pull backup when hypervisor exports backup data thru some interface and mgmt itself make a copy. This patch series basically adds creating backup to API/remote/qemu/virsh and initial backup XML definition. Just like other blockjobs backup creation is asynchronous. That is creation is merely a backup start and client should track backup error/completion thru blockjob events. As backup is done transactionally all individual disk backup jobs will be aborted by qemu itself in case of error, client need not to do it manually. Client can cancel the backup by aborting blockjob on any disk being backed up. Backup xml desription is similar to snapshot one with some exceptions and is described in more details in definition patch [p1] or in html docs. I guess good client will track progress for every disk in backup to report progress and detect hangs so it don't need extra backup complete/error event that aggregate the overall backup result. However it looks like aborting backup can be implemented in libvirt as code will be common for all clients. We need to abort some of not yet completed per disk backups and retry if job to be aborted is completed meanwhile. Of coures this series is far from being complete. Incremental backups and backup persistent metadata is to be implemented. Let's just start work in this direction. Links: In [1] we came to agreement to create distinct API to support backup operations and there is a discussion of pull backup series in [2]. The latter is blocked as some necessary operations are still experimental in qemu. [3] is the first version of series. [1] https://www.redhat.com/archives/libvir-list/2016-March/msg00937.html [2] https://www.redhat.com/archives/libvir-list/2016-September/msg00192.html [3] https://www.redhat.com/archives/libvir-list/2017-May/msg00130.html Nikolay Shirokovskiy (12): api: backup: add api to create backup api: backup: add driver based implementation remote: backup: add create backup implementation backup: qemu: monitor: add drive-backup command backup: misc: add backup block job type conf: backup: add backup xml definition [p1] qemu: backup: add qemuDomainBackupCreateXML implementation qemu: backup: check backup destination before start qemu: backup: prepare backup destination virsh: backup: add backup-create command schema: backup: add schema and its tests docs: add backup html docs daemon/remote.c | 8 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 4 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 58 +++++ docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 79 +++++++ examples/object-events/event-test.c | 3 + include/libvirt/libvirt-domain-backup.h | 59 +++++ include/libvirt/libvirt-domain.h | 3 + include/libvirt/libvirt.h | 1 + include/libvirt/virterror.h | 2 + po/POTFILES.in | 2 + src/Makefile.am | 3 + src/access/viraccessperm.c | 3 +- src/access/viraccessperm.h | 6 + src/conf/backup_conf.c | 299 +++++++++++++++++++++++++ src/conf/backup_conf.h | 70 ++++++ src/conf/domain_conf.c | 2 +- src/datatypes.c | 60 +++++ src/datatypes.h | 29 +++ src/driver-hypervisor.h | 5 + src/libvirt-domain-backup.c | 209 +++++++++++++++++ src/libvirt_private.syms | 9 + src/libvirt_public.syms | 10 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_driver.c | 239 +++++++++++++++++++- src/qemu/qemu_monitor.c | 14 ++ src/qemu/qemu_monitor.h | 5 + src/qemu/qemu_monitor_json.c | 36 +++ src/qemu/qemu_monitor_json.h | 5 + src/remote/remote_driver.c | 7 + src/remote/remote_protocol.x | 23 +- src/rpc/gendispatch.pl | 29 ++- src/util/virerror.c | 6 + tests/domainbackupxml/block_target.xml | 5 + tests/domainbackupxml/explicit_description.xml | 6 + tests/domainbackupxml/explicit_file_type.xml | 5 + tests/domainbackupxml/explicit_format.xml | 5 + tests/domainbackupxml/explicit_name.xml | 6 + tests/domainbackupxml/multi_disk.xml | 8 + tests/domainbackupxml/ref_by_path.xml | 5 + tests/virschematest.c | 1 + tools/Makefile.am | 1 + tools/virsh-backup.c | 100 +++++++++ tools/virsh-backup.h | 29 +++ tools/virsh-domain.c | 3 +- tools/virsh-util.c | 11 + tools/virsh-util.h | 3 + tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 20 ++ tools/virt-xml-validate.in | 5 +- 54 files changed, 1499 insertions(+), 15 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 include/libvirt/libvirt-domain-backup.h create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/libvirt-domain-backup.c create mode 100644 tests/domainbackupxml/block_target.xml create mode 100644 tests/domainbackupxml/explicit_description.xml create mode 100644 tests/domainbackupxml/explicit_file_type.xml create mode 100644 tests/domainbackupxml/explicit_format.xml create mode 100644 tests/domainbackupxml/explicit_name.xml create mode 100644 tests/domainbackupxml/multi_disk.xml create mode 100644 tests/domainbackupxml/ref_by_path.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 1.8.3.1

1 12

[libvirt] error in docs https://libvirt.org/html/libvirt-libvirt-domain.html
by Vasiliy Tolstov 12 May '17

12 May '17

virDomainMigrateFlags have errors VIR_MIGRATE_LIVE have invalid comment and as i see all comments are shifted down by one point. -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

3 4

[libvirt] [PATCH 0/2] improve detection of UNIX path generated by libvirt
by Pavel Hrdina 12 May '17

12 May '17

Pavel Hrdina (2): util: introduce virStringMatch qemu: improve detection of UNIX path generated by libvirt src/libvirt_private.syms | 1 + src/qemu/qemu_domain.c | 78 +++++++++++++++++++--- src/util/virstring.c | 34 ++++++++++ src/util/virstring.h | 3 + .../qemuxml2argv-channel-unix-gen-path1.xml | 17 +++++ .../qemuxml2argv-channel-unix-gen-path2.xml | 17 +++++ .../qemuxml2argv-channel-unix-gen-path3.xml | 17 +++++ .../qemuxml2argv-channel-unix-user-path.xml | 17 +++++ .../qemuxml2xmlout-channel-unix-gen-path1.xml | 32 +++++++++ .../qemuxml2xmlout-channel-unix-gen-path2.xml | 32 +++++++++ .../qemuxml2xmlout-channel-unix-gen-path3.xml | 32 +++++++++ .../qemuxml2xmlout-channel-unix-user-path.xml | 33 +++++++++ tests/qemuxml2xmltest.c | 5 ++ tests/virstringtest.c | 47 +++++++++++++ 14 files changed, 356 insertions(+), 9 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path1.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path2.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path3.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-user-path.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path1.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path2.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path3.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-user-path.xml -- 2.12.2

3 9

[libvirt] [PATCH] conf: Fix resource leak in virCapabilitiesInitCaches
by John Ferlan 12 May '17

12 May '17

The @type from virFileReadValueString needs to be VIR_FREE each time through the loop since it's not saved and since cleanup can be reached prior to decoding it for @kernel_type amd bank->type, the cleanup code needs to also have a VIR_FREE Found by Coverity Signed-off-by: John Ferlan <jferlan(a)redhat.com> --- src/conf/capabilities.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/conf/capabilities.c b/src/conf/capabilities.c index c36ca40..d699b08 100644 --- a/src/conf/capabilities.c +++ b/src/conf/capabilities.c @@ -1611,10 +1611,10 @@ virCapabilitiesInitCaches(virCapsPtr caps) if (kernel_type < 0) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Unknown cache type '%s'"), type); - VIR_FREE(type); goto cleanup; } bank->type = kernel_type; + VIR_FREE(type); for (i = 0; i < caps->host.ncaches; i++) { if (virCapsHostCacheBankEquals(bank, caps->host.caches[i])) @@ -1637,6 +1637,7 @@ virCapabilitiesInitCaches(virCapsPtr caps) ret = 0; cleanup: + VIR_FREE(type); VIR_FREE(path); virDirClose(&dirp); virCapsHostCacheBankFree(bank); -- 2.9.3

2 1

[libvirt] [PATCH v2] qemu: reduce packet loss rate for vm with macvtap passthrough mode in migration
by ZhiPeng Lu 12 May '17

12 May '17

Before libvirt that calls virNetDevMacVLanCreateWithVPortProfile sets mac address or vlan of a Virtual Function(VF) linked to a macvtap passthrough device of migration destination host in migration start step. If we ping the migrating vm, we get the network does not pass. Because VFs of migration source and destination have the same MAC address. The patch later calling qemuMigrationVPAssociatePortProfiles sets mac address of VF in migration finish step instead of start step. The patch aims to reduce packet loss rate. Signed-off-by: ZhiPeng Lu <lu.zhipeng(a)zte.com.cn> --- src/qemu/qemu_migration.c | 18 ++++++++++++++++-- src/util/virnetdevmacvlan.c | 17 +++++++++++------ 2 files changed, 27 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 09adb04..795ed71 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5004,7 +5004,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver, } static int -qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def) +qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def, const char *stateDir) { size_t i; int last_good_net = -1; @@ -5013,6 +5013,20 @@ qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def) for (i = 0; i < def->nnets; i++) { net = def->nets[i]; if (virDomainNetGetActualType(net) == VIR_DOMAIN_NET_TYPE_DIRECT) { + if ((!virDomainNetGetActualVirtPortProfile(net) || (virDomainNetGetActualVirtPortProfile(net) && + virDomainNetGetActualVirtPortProfile(net)->virtPortType != VIR_NETDEV_VPORT_PROFILE_8021QBG && + virDomainNetGetActualVirtPortProfile(net)->virtPortType != VIR_NETDEV_VPORT_PROFILE_8021QBH)) && + virDomainNetGetActualDirectMode(net) == + VIR_NETDEV_MACVLAN_MODE_PASSTHRU) { + if (virNetDevSaveNetConfig(virDomainNetGetActualDirectDev(net), + -1, stateDir, false) < 0) { + goto err_exit; + } + if (virNetDevSetNetConfig(virDomainNetGetActualDirectDev(net), + -1, NULL, virDomainNetGetActualVlan(net), &net->mac, false) < 0) { + goto err_exit; + } + } if (virNetDevVPortProfileAssociate(net->ifname, virDomainNetGetActualVirtPortProfile(net), &net->mac, @@ -5187,7 +5201,7 @@ qemuMigrationFinish(virQEMUDriverPtr driver, goto endjob; } - if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0) + if (qemuMigrationVPAssociatePortProfiles(vm->def, cfg->stateDir) < 0) goto endjob; if (mig->network && qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0) diff --git a/src/util/virnetdevmacvlan.c b/src/util/virnetdevmacvlan.c index 7222b0f..682dcd1 100644 --- a/src/util/virnetdevmacvlan.c +++ b/src/util/virnetdevmacvlan.c @@ -1020,12 +1020,17 @@ virNetDevMacVLanCreateWithVPortProfile(const char *ifnameRequested, */ setVlan = false; } - - if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) - return -1; - - if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, setVlan) < 0) - return -1; + if (vmOp != VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START && + virtPortProfile && (virtPortProfile->virtPortType == VIR_NETDEV_VPORT_PROFILE_8021QBH || + virtPortProfile->virtPortType == VIR_NETDEV_VPORT_PROFILE_8021QBG)) { + if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) { + return -1; + } + if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, + setVlan) < 0) { + return -1; + } + } } if (ifnameRequested) { -- 1.8.3.1

1 0

[libvirt] [PATCH] qemu: reduce packet loss rate for vm with macvtap passthrough mode in migration
by ZhiPeng Lu 12 May '17

12 May '17

Before libvirt that calls virNetDevMacVLanCreateWithVPortProfile sets mac address or vlan of a Virtual Function(VF) linked to a macvtap passthrough device of migration destination host in migration start step. If we ping the migrating vm, we get the network does not pass. Because VFs of migration source and destination have the same MAC address. The patch later calling qemuMigrationVPAssociatePortProfiles sets mac address of VF in migration finish step instead of start step. The patch aims to reduce packet loss rate. Signed-off-by: ZhiPeng Lu <lu.zhipeng(a)zte.com.cn> --- src/qemu/qemu_migration.c | 17 +++++++++++++++-- src/util/virnetdevmacvlan.c | 17 +++++++++++------ 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index 09adb04..0308043 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -5004,7 +5004,7 @@ qemuMigrationPerform(virQEMUDriverPtr driver, } static int -qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def) +qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def, const char *stateDir) { size_t i; int last_good_net = -1; @@ -5013,6 +5013,19 @@ qemuMigrationVPAssociatePortProfiles(virDomainDefPtr def) for (i = 0; i < def->nnets; i++) { net = def->nets[i]; if (virDomainNetGetActualType(net) == VIR_DOMAIN_NET_TYPE_DIRECT) { + if (virDomainNetGetActualVirtPortProfile(net) != VIR_NETDEV_VPORT_PROFILE_8021QBG && + virDomainNetGetActualVirtPortProfile(net) != VIR_NETDEV_VPORT_PROFILE_8021QBH && + (virDomainNetGetActualDirectMode(net) == + VIR_NETDEV_MACVLAN_MODE_PASSTHRU)) { + if (virNetDevSaveNetConfig(virDomainNetGetActualDirectDev(net), + -1, stateDir, false) < 0) { + goto err_exit; + } + if (virNetDevSetNetConfig(virDomainNetGetActualDirectDev(net), + -1, NULL, virDomainNetGetActualVlan(net), &net->mac, false) < 0) { + goto err_exit; + } + } if (virNetDevVPortProfileAssociate(net->ifname, virDomainNetGetActualVirtPortProfile(net), &net->mac, @@ -5187,7 +5200,7 @@ qemuMigrationFinish(virQEMUDriverPtr driver, goto endjob; } - if (qemuMigrationVPAssociatePortProfiles(vm->def) < 0) + if (qemuMigrationVPAssociatePortProfiles(vm->def, cfg->stateDir) < 0) goto endjob; if (mig->network && qemuDomainMigrateOPDRelocate(driver, vm, mig) < 0) diff --git a/src/util/virnetdevmacvlan.c b/src/util/virnetdevmacvlan.c index 7222b0f..fc08c43 100644 --- a/src/util/virnetdevmacvlan.c +++ b/src/util/virnetdevmacvlan.c @@ -1020,12 +1020,17 @@ virNetDevMacVLanCreateWithVPortProfile(const char *ifnameRequested, */ setVlan = false; } - - if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) - return -1; - - if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, setVlan) < 0) - return -1; + if (vmOp != VIR_NETDEV_VPORT_PROFILE_OP_MIGRATE_IN_START && + (virtPortProfile == VIR_NETDEV_VPORT_PROFILE_8021QBH || + virtPortProfile == VIR_NETDEV_VPORT_PROFILE_8021QBG)) { + if (virNetDevSaveNetConfig(linkdev, -1, stateDir, setVlan) < 0) { + return -1; + } + if (virNetDevSetNetConfig(linkdev, -1, NULL, vlan, macaddress, + setVlan) < 0) { + return -1; + } + } } if (ifnameRequested) { -- 1.8.3.1

1 0

[libvirt] [PATCH v2] Detect VMDK version 3 files
by Daniel P. Berrange 12 May '17

12 May '17

The metadata libvirt cares about is identical for version 3 as for previous versions, so we merely need list the new version number. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/util/virstoragefile.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/util/virstoragefile.c b/src/util/virstoragefile.c index 94a77ce..b43acf6 100644 --- a/src/util/virstoragefile.c +++ b/src/util/virstoragefile.c @@ -113,7 +113,7 @@ enum { BACKING_STORE_ERROR, }; -#define FILE_TYPE_VERSIONS_LAST 2 +#define FILE_TYPE_VERSIONS_LAST 3 struct FileEncryptionInfo { int format; /* Encryption format to assign */ @@ -374,7 +374,7 @@ static struct FileTypeInfo const fileTypeInfo[] = { }, [VIR_STORAGE_FILE_VMDK] = { 0, "KDMV", NULL, - LV_LITTLE_ENDIAN, 4, 4, {1, 2}, + LV_LITTLE_ENDIAN, 4, 4, {1, 2, 3}, 4+4+4, 8, 512, NULL, vmdk4GetBackingStore, NULL }, }; -- 2.9.3

2 1

[libvirt] [PATCH] Add "io" option to virsh attach-disk sub-command.
by Gordon Messmer 12 May '17

12 May '17

--- tools/virsh-domain.c | 14 +++++++++++--- tools/virsh.pod | 2 ++ 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index 0d19d0e..d2a2a05 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -267,6 +267,10 @@ static const vshCmdOptDef opts_attach_disk[] = { .type = VSH_OT_STRING, .help = N_("cache mode of disk device") }, + {.name = "io", + .type = VSH_OT_STRING, + .help = N_("io mode of disk device") + }, {.name = "type", .type = VSH_OT_STRING, .help = N_("target device type") @@ -504,8 +508,9 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) virDomainPtr dom = NULL; const char *source = NULL, *target = NULL, *driver = NULL, *subdriver = NULL, *type = NULL, *mode = NULL, - *iothread = NULL, *cache = NULL, *serial = NULL, - *straddr = NULL, *wwn = NULL, *targetbus = NULL; + *iothread = NULL, *cache = NULL, *io = NULL, + *serial = NULL, *straddr = NULL, *wwn = NULL, + *targetbus = NULL; struct DiskAddress diskAddr; bool isFile = false, functionReturn = false; int ret; @@ -537,6 +542,7 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) vshCommandOptStringReq(ctl, cmd, "mode", &mode) < 0 || vshCommandOptStringReq(ctl, cmd, "iothread", &iothread) < 0 || vshCommandOptStringReq(ctl, cmd, "cache", &cache) < 0 || + vshCommandOptStringReq(ctl, cmd, "io", &io) < 0 || vshCommandOptStringReq(ctl, cmd, "serial", &serial) < 0 || vshCommandOptStringReq(ctl, cmd, "wwn", &wwn) < 0 || vshCommandOptStringReq(ctl, cmd, "address", &straddr) < 0 || @@ -579,7 +585,7 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) virBufferAddLit(&buf, ">\n"); virBufferAdjustIndent(&buf, 2); - if (driver || subdriver || iothread || cache) { + if (driver || subdriver || iothread || cache || io) { virBufferAddLit(&buf, "<driver"); if (driver) @@ -590,6 +596,8 @@ cmdAttachDisk(vshControl *ctl, const vshCmd *cmd) virBufferAsprintf(&buf, " iothread='%s'", iothread); if (cache) virBufferAsprintf(&buf, " cache='%s'", cache); + if (io) + virBufferAsprintf(&buf, " io='%s'", io); virBufferAddLit(&buf, "/>\n"); } diff --git a/tools/virsh.pod b/tools/virsh.pod index cd1f25f..9656411 100644 --- a/tools/virsh.pod +++ b/tools/virsh.pod @@ -2860,6 +2860,8 @@ I<mode> can specify the two specific mode I<readonly> or I<shareable>. I<sourcetype> can indicate the type of source (block|file) I<cache> can be one of "default", "none", "writethrough", "writeback", "directsync" or "unsafe". +I<io> io is "threads", or "native" and selects between pthread based disk +I/O and native Linux AIO. I<iothread> is the number within the range of domain IOThreads to which this disk may be attached (QEMU only). I<serial> is the serial of disk device. I<wwn> is the wwn of disk device. -- 2.9.3

3 8

[libvirt] [PATCH 0/2] spec: Minor improvements
by Andrea Bolognani 12 May '17

12 May '17

*** BLURBY MCBLURBFACE *** Andrea Bolognani (2): spec: Support maintenance releases on mingw spec: Use HTTPS instead of HTTP libvirt.spec.in | 4 ++-- mingw-libvirt.spec.in | 8 ++++++-- 2 files changed, 8 insertions(+), 4 deletions(-) -- 2.7.4

2 3