- Devel - libvirt List Archives

[libvirt] [PATCH] safer default storage dir permissions
by Christian Ehrhardt 15 May '17

15 May '17

Hi, while cleaning out patchs that we held for a while on top of libvirt I found this patch of Serge (thanks!) which I think would make just as much sense in the upstream project itself. Or in case the discussion might unveil why it might not make sense, that would also be a win for us to adapt. I modified the older patch to match latest master (files owning the code changed) and added the matching doc update to be complete. Serge Hallyn (1): storage: use 0711 as the default perms for dirs docs/formatstorage.html.in | 2 +- src/storage/storage_util.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) -- 2.7.4

4 9

[libvirt] [PATCH v2 0/3] improve detection of UNIX path generated by libvirt
by Pavel Hrdina 15 May '17

15 May '17

Pavel Hrdina (3): util: introduce virStringMatch util: introduce virBufferEscapeRegex qemu: improve detection of UNIX path generated by libvirt src/libvirt_private.syms | 2 + src/qemu/qemu_domain.c | 51 ++++++++++++++++++---- src/util/virbuffer.c | 19 ++++++++ src/util/virbuffer.h | 3 ++ src/util/virstring.c | 33 ++++++++++++++ src/util/virstring.h | 3 ++ .../qemuxml2argv-channel-unix-gen-path1.xml | 17 ++++++++ .../qemuxml2argv-channel-unix-gen-path2.xml | 17 ++++++++ .../qemuxml2argv-channel-unix-gen-path3.xml | 17 ++++++++ .../qemuxml2argv-channel-unix-user-path.xml | 17 ++++++++ .../qemuxml2xmlout-channel-unix-gen-path1.xml | 32 ++++++++++++++ .../qemuxml2xmlout-channel-unix-gen-path2.xml | 32 ++++++++++++++ .../qemuxml2xmlout-channel-unix-gen-path3.xml | 32 ++++++++++++++ .../qemuxml2xmlout-channel-unix-user-path.xml | 33 ++++++++++++++ tests/qemuxml2xmltest.c | 5 +++ tests/virbuftest.c | 40 +++++++++++++++++ tests/virstringtest.c | 47 ++++++++++++++++++++ 17 files changed, 391 insertions(+), 9 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path1.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path2.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-gen-path3.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-channel-unix-user-path.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path1.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path2.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-gen-path3.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-channel-unix-user-path.xml -- 2.13.0

2 9

[libvirt] [PATCH v2 00/38] Implement sparse streams for libvirt
by Michal Privoznik 15 May '17

15 May '17

v2 of: https://www.redhat.com/archives/libvir-list/2017-April/msg00671.html All the patches can be found on my github: https://github.com/zippy2/libvirt/tree/sparse_iohelper5 diff to v1: - Do not drop the pipe in 06/38. Use it for signaling to the event loop that there's a message in the queue waiting to be processed. - Dropped the error report in virNetClientStreamRecvPacket in 31/38. It was useless. Michal Privoznik (38): fdstreamtest: Rename tempdir fdstreamtest: Print more info on read failure fdstream: s/struct virFDStreamData */virFDStreamDataPtr/ virFDStreamData: Turn into virObjectLockable virfdstream: Drop iohelper in favour of a thread virfdstream: Use messages instead of pipe iohelper: Remove unused mode util: Introduce virFileInData Introduce virStreamRecvFlags Implement virStreamRecvFlags to some drivers Introduce virStreamSkip Introduce virStreamHoleSize Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag Introduce virStreamSparseRecvAll Introduce virStreamSparseSendAll Introduce virStreamInData virNetClientStreamNew: Track origin stream Track if stream is skippable RPC: Introduce virNetStreamSkip Introduce VIR_NET_STREAM_SKIP message type Teach wireshark plugin about VIR_NET_STREAM_SKIP daemon: Introduce virNetServerProgramSendStreamSkip virnetclientstream: Introduce virNetClientStreamSendSkip daemon: Implement VIR_NET_STREAM_SKIP handling virnetclientstream: Introduce virNetClientStreamHandleSkip remote_driver: Implement virStreamSkip virNetClientStreamRecvPacket: Introduce @flags argument Introduce virNetClientStreamHoleSize remote: Implement virStreamHoleSize virNetClientStream: Wire up VIR_NET_STREAM_SKIP remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE daemonStreamHandleRead: Wire up seekable stream daemon: Don't call virStreamInData so often daemon: Don't call virStreamInData so often gendispatch: Introduce @sparseflag for our calls Introduce virStorageVol{Download,Upload}Flags virsh: Implement sparse stream to vol-download virsh: Implement sparse stream to vol-upload daemon/remote.c | 2 +- daemon/stream.c | 147 +++++- daemon/stream.h | 3 +- include/libvirt/libvirt-storage.h | 9 + include/libvirt/libvirt-stream.h | 102 +++- src/driver-stream.h | 23 + src/esx/esx_stream.c | 16 +- src/libvirt-storage.c | 4 +- src/libvirt-stream.c | 456 +++++++++++++++++ src/libvirt_internal.h | 3 + src/libvirt_private.syms | 2 + src/libvirt_public.syms | 9 + src/libvirt_remote.syms | 3 + src/remote/remote_driver.c | 89 +++- src/remote/remote_protocol.x | 2 + src/rpc/gendispatch.pl | 21 +- src/rpc/virnetclient.c | 1 + src/rpc/virnetclientstream.c | 201 +++++++- src/rpc/virnetclientstream.h | 17 +- src/rpc/virnetprotocol.x | 16 +- src/rpc/virnetserverprogram.c | 33 ++ src/rpc/virnetserverprogram.h | 7 + src/storage/storage_driver.c | 4 +- src/storage/storage_util.c | 10 +- src/util/iohelper.c | 72 +-- src/util/virfdstream.c | 926 ++++++++++++++++++++++++++++------- src/util/virfdstream.h | 2 +- src/util/virfile.c | 81 +++ src/util/virfile.h | 3 + src/virnetprotocol-structs | 4 + tests/fdstreamtest.c | 12 +- tests/virfiletest.c | 203 ++++++++ tools/virsh-util.c | 65 +++ tools/virsh-util.h | 29 ++ tools/virsh-volume.c | 50 +- tools/virsh.pod | 6 +- tools/wireshark/src/packet-libvirt.c | 48 ++ tools/wireshark/src/packet-libvirt.h | 2 + 38 files changed, 2369 insertions(+), 314 deletions(-) -- 2.10.2

3 93

[libvirt] [PATCH] RFE: virsh: add domxml-to-native <fmt> [--domain DOMAIN] option
by Dan 15 May '17

15 May '17

Bug 835476 RFE: virsh: add domxml-to-native --domain option (for existing VM) [1] virsh DOMAIN COMMAND domxml-to-native did not support domain (id|uuid|name) as input for generating hypervisor agent native command, instead only supported XML input from STDIN. Here in this patch, it supports the following syntax: domxml-to-native <format> { [--domain DOMAIN] | [XML] }, i.e., it supports either designating domain (domain id, uuid, or name), or path to XML domain configuration file; NOTE that it deprecated existing STDIN input passing XML functionality. It was tested on the test mock driver and QEMU. [1]. https://bugzilla.redhat.com/show_bug.cgi?id=835476 Dan --- tools/virsh-domain.c | 68 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 55 insertions(+), 13 deletions(-) diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c index db8accfe4..9ac855b19 100644 --- a/tools/virsh-domain.c +++ b/tools/virsh-domain.c @@ -60,6 +60,7 @@ #include "virsh-nodedev.h" #include "viruri.h" + /* Gnulib doesn't guarantee SA_SIGINFO support. */ #ifndef SA_SIGINFO # define SA_SIGINFO 0 @@ -9811,9 +9812,13 @@ static const vshCmdOptDef opts_domxmltonative[] = { .flags = VSH_OFLAG_REQ, .help = N_("target config data type format") }, + {.name = "domain", + .type = VSH_OT_DATA, + .flags = VSH_OFLAG_REQ_OPT, + .help = N_("domain name, id or uuid") + }, {.name = "xml", .type = VSH_OT_DATA, - .flags = VSH_OFLAG_REQ, .help = N_("xml data file to export from") }, {.name = NULL} @@ -9822,31 +9827,68 @@ static const vshCmdOptDef opts_domxmltonative[] = { static bool cmdDomXMLToNative(vshControl *ctl, const vshCmd *cmd) { - bool ret = true; const char *format = NULL; const char *xmlFile = NULL; - char *configData; - char *xmlData; + const char *domain = NULL; + char *configData = NULL; + char *xmlData = NULL; unsigned int flags = 0; + unsigned int domflags = 0; virshControlPtr priv = ctl->privData; + virDomainPtr dom = NULL; - if (vshCommandOptStringReq(ctl, cmd, "format", &format) < 0 || - vshCommandOptStringReq(ctl, cmd, "xml", &xmlFile) < 0) + if (vshCommandOptStringReq(ctl, cmd, "format", &format) < 0) return false; + + if (vshCommandOptStringReq(ctl, cmd, "domain", &domain) < 0) + return false; - if (virFileReadAll(xmlFile, VSH_MAX_XML_FILE, &xmlData) < 0) - return false; + if (vshCommandOptStringReq(ctl, cmd, "xml", &xmlFile) < 0) + return false; + + VSH_EXCLUSIVE_OPTIONS_VAR(domain, xmlFile); + + if (domain) { + domflags = VIRSH_BYID | VIRSH_BYUUID | VIRSH_BYNAME; + dom = virshLookupDomainBy(ctl, domain, domflags); + } + + if (!dom && !xmlFile) { + vshError(ctl, _("need either xml file input or domain")); + return false; + } + + if(dom){ + xmlData = virDomainGetXMLDesc(dom, flags); + if (xmlData == NULL) { + virshDomainFree(dom); + return false; + } + } + + if (xmlFile) { + if (virFileReadAll(xmlFile, VSH_MAX_XML_FILE, &xmlData) < 0) { + vshReportError(ctl); + VIR_FREE(xmlData); + return false; + } + } configData = virConnectDomainXMLToNative(priv->conn, format, xmlData, flags); + if (configData != NULL) { - vshPrint(ctl, "%s", configData); - VIR_FREE(configData); + vshPrint(ctl, "%s", configData); + VIR_FREE(xmlData); + VIR_FREE(configData); + virDomainFree(dom); + return true; } else { - ret = false; + VIR_FREE(xmlData); + virDomainFree(dom); + return false; } - VIR_FREE(xmlData); - return ret; + return false; } /* -- 2.12.2

2 2

[libvirt] [PATCH] util: conf: Don't log when adding commented out lines
by Peter Krempa 15 May '17

15 May '17

virConfAddEntry spams debug logs even for fully commented out lines. Skip such messages to avoid: 2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil) 2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil) 2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil) 2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil) 2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil) ... This also fixes NULL passed to printf. --- src/util/virconf.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/util/virconf.c b/src/util/virconf.c index 9840ca6c7..4498d253a 100644 --- a/src/util/virconf.c +++ b/src/util/virconf.c @@ -238,7 +238,10 @@ virConfAddEntry(virConfPtr conf, char *name, virConfValuePtr value, char *comm) if ((comm == NULL) && (name == NULL)) return NULL; - VIR_DEBUG("Add entry %s %p", name, value); + /* don't log fully commented out lines */ + if (name) + VIR_DEBUG("Add entry %s %p", name, value); + if (VIR_ALLOC(ret) < 0) return NULL; -- 2.12.2

2 2

[libvirt] [PATCH] test-wrap-argv.pl: Accept short parameter -i for --in-place
by Martin Kletzander 12 May '17

12 May '17

I like to use it that way and every time I try running it I just instinctively use '-i' (like with sed, etc.) and it makes sense, IMHO. Signed-off-by: Martin Kletzander <mkletzan(a)redhat.com> --- tests/test-wrap-argv.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test-wrap-argv.pl b/tests/test-wrap-argv.pl index f0d3c0ba2d10..7867e9d7194e 100755 --- a/tests/test-wrap-argv.pl +++ b/tests/test-wrap-argv.pl @@ -31,7 +31,7 @@ $in_place = 0; $check = 0; -if (@ARGV[0] eq "--in-place") { +if (@ARGV[0] eq "--in-place" or @ARGV[0] eq "-i") { $in_place = 1; shift @ARGV; } elsif (@ARGV[0] eq "--check") { -- 2.12.2

3 8

[libvirt] [PATCH] docs: install html fonts and related
by Nikolay Shirokovskiy 12 May '17

12 May '17

--- docs/Makefile.am | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/Makefile.am b/docs/Makefile.am index 105fe68..7a10a50 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -54,6 +54,20 @@ css = \ libvirt.css \ main.css +fonts = \ + fonts/LICENSE.md \ + fonts/stylesheet.css \ + fonts/overpass-bold-italic.woff \ + fonts/overpass-bold.woff \ + fonts/overpass-italic.woff \ + fonts/overpass-light-italic.woff \ + fonts/overpass-light.woff \ + fonts/overpass-mono-bold.woff \ + fonts/overpass-mono-light.woff \ + fonts/overpass-mono-regular.woff \ + fonts/overpass-mono-semibold.woff \ + fonts/overpass-regular.woff + devhelppng = \ devhelp/home.png \ devhelp/left.png \ @@ -163,7 +177,7 @@ EXTRA_DIST= \ $(devhelphtml) $(devhelppng) $(devhelpcss) $(devhelpxsl) \ $(xml) $(qemu_xml) $(lxc_xml) $(admin_xml) $(fig) $(png) $(css) \ $(logofiles) $(patches) $(dot_php_in) $(dot_php_code_in) $(dot_php)\ - $(internals_html_in) $(internals_html) \ + $(internals_html_in) $(internals_html) $(fonts) \ aclperms.htmlinc \ hvsupport.pl \ $(schema_DATA) @@ -383,9 +397,13 @@ install-data-local: for file in $(devhelphtml) $(devhelppng) $(devhelpcss); do \ $(INSTALL) -m 0644 $(srcdir)/$${file} $(DESTDIR)$(DEVHELP_DIR) ; \ done + $(mkinstalldirs) $(DESTDIR)$(HTML_DIR)/fonts + for f in $(fonts); do \ + $(INSTALL) -m 0644 $(srcdir)/$$f $(DESTDIR)$(HTML_DIR)/fonts; \ + done uninstall-local: - for f in $(css) $(dot_html) $(gif) $(png); do \ + for f in $(css) $(dot_html) $(gif) $(png) $(fonts); do \ rm -f $(DESTDIR)$(HTML_DIR)/$$f; \ done for f in $(logofiles); do \ -- 1.8.3.1

3 2

[libvirt] changing compat when create volume from volume via libvirt api
by Vasiliy Tolstov 12 May '17

12 May '17

I'm use via php binding https://libvirt.org/html/libvirt-libvirt-storage.html#virStorageVolCreateXM… and see that image grows: image: new.qcow2 file format: qcow2 virtual size: 4.9G (5242880000 bytes) disk size: 854M cluster_size: 65536 Format specific information: compat: 0.10 refcount bits: 16 image: old.qcow2 file format: qcow2 virtual size: 4.9G (5242880000 bytes) disk size: 277M cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false Why libvirt create new image with different compat? -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

1 2

[libvirt] [PATCH v2 RFC 00/12] introduce push backups
by Nikolay Shirokovskiy 12 May '17

12 May '17

* Diff from v1 [3] - add man/html docs - add backup xml schema Push backup is a backup when hypervisor itself copy backup data to destination in contrast to pull backup when hypervisor exports backup data thru some interface and mgmt itself make a copy. This patch series basically adds creating backup to API/remote/qemu/virsh and initial backup XML definition. Just like other blockjobs backup creation is asynchronous. That is creation is merely a backup start and client should track backup error/completion thru blockjob events. As backup is done transactionally all individual disk backup jobs will be aborted by qemu itself in case of error, client need not to do it manually. Client can cancel the backup by aborting blockjob on any disk being backed up. Backup xml desription is similar to snapshot one with some exceptions and is described in more details in definition patch [p1] or in html docs. I guess good client will track progress for every disk in backup to report progress and detect hangs so it don't need extra backup complete/error event that aggregate the overall backup result. However it looks like aborting backup can be implemented in libvirt as code will be common for all clients. We need to abort some of not yet completed per disk backups and retry if job to be aborted is completed meanwhile. Of coures this series is far from being complete. Incremental backups and backup persistent metadata is to be implemented. Let's just start work in this direction. Links: In [1] we came to agreement to create distinct API to support backup operations and there is a discussion of pull backup series in [2]. The latter is blocked as some necessary operations are still experimental in qemu. [3] is the first version of series. [1] https://www.redhat.com/archives/libvir-list/2016-March/msg00937.html [2] https://www.redhat.com/archives/libvir-list/2016-September/msg00192.html [3] https://www.redhat.com/archives/libvir-list/2017-May/msg00130.html Nikolay Shirokovskiy (12): api: backup: add api to create backup api: backup: add driver based implementation remote: backup: add create backup implementation backup: qemu: monitor: add drive-backup command backup: misc: add backup block job type conf: backup: add backup xml definition [p1] qemu: backup: add qemuDomainBackupCreateXML implementation qemu: backup: check backup destination before start qemu: backup: prepare backup destination virsh: backup: add backup-create command schema: backup: add schema and its tests docs: add backup html docs daemon/remote.c | 8 + docs/Makefile.am | 3 + docs/apibuild.py | 2 + docs/docs.html.in | 4 +- docs/format.html.in | 1 + docs/formatbackup.html.in | 58 +++++ docs/index.html.in | 3 +- docs/schemas/domainbackup.rng | 79 +++++++ examples/object-events/event-test.c | 3 + include/libvirt/libvirt-domain-backup.h | 59 +++++ include/libvirt/libvirt-domain.h | 3 + include/libvirt/libvirt.h | 1 + include/libvirt/virterror.h | 2 + po/POTFILES.in | 2 + src/Makefile.am | 3 + src/access/viraccessperm.c | 3 +- src/access/viraccessperm.h | 6 + src/conf/backup_conf.c | 299 +++++++++++++++++++++++++ src/conf/backup_conf.h | 70 ++++++ src/conf/domain_conf.c | 2 +- src/datatypes.c | 60 +++++ src/datatypes.h | 29 +++ src/driver-hypervisor.h | 5 + src/libvirt-domain-backup.c | 209 +++++++++++++++++ src/libvirt_private.syms | 9 + src/libvirt_public.syms | 10 + src/qemu/qemu_conf.h | 1 + src/qemu/qemu_driver.c | 239 +++++++++++++++++++- src/qemu/qemu_monitor.c | 14 ++ src/qemu/qemu_monitor.h | 5 + src/qemu/qemu_monitor_json.c | 36 +++ src/qemu/qemu_monitor_json.h | 5 + src/remote/remote_driver.c | 7 + src/remote/remote_protocol.x | 23 +- src/rpc/gendispatch.pl | 29 ++- src/util/virerror.c | 6 + tests/domainbackupxml/block_target.xml | 5 + tests/domainbackupxml/explicit_description.xml | 6 + tests/domainbackupxml/explicit_file_type.xml | 5 + tests/domainbackupxml/explicit_format.xml | 5 + tests/domainbackupxml/explicit_name.xml | 6 + tests/domainbackupxml/multi_disk.xml | 8 + tests/domainbackupxml/ref_by_path.xml | 5 + tests/virschematest.c | 1 + tools/Makefile.am | 1 + tools/virsh-backup.c | 100 +++++++++ tools/virsh-backup.h | 29 +++ tools/virsh-domain.c | 3 +- tools/virsh-util.c | 11 + tools/virsh-util.h | 3 + tools/virsh.c | 2 + tools/virsh.h | 1 + tools/virsh.pod | 20 ++ tools/virt-xml-validate.in | 5 +- 54 files changed, 1499 insertions(+), 15 deletions(-) create mode 100644 docs/formatbackup.html.in create mode 100644 docs/schemas/domainbackup.rng create mode 100644 include/libvirt/libvirt-domain-backup.h create mode 100644 src/conf/backup_conf.c create mode 100644 src/conf/backup_conf.h create mode 100644 src/libvirt-domain-backup.c create mode 100644 tests/domainbackupxml/block_target.xml create mode 100644 tests/domainbackupxml/explicit_description.xml create mode 100644 tests/domainbackupxml/explicit_file_type.xml create mode 100644 tests/domainbackupxml/explicit_format.xml create mode 100644 tests/domainbackupxml/explicit_name.xml create mode 100644 tests/domainbackupxml/multi_disk.xml create mode 100644 tests/domainbackupxml/ref_by_path.xml create mode 100644 tools/virsh-backup.c create mode 100644 tools/virsh-backup.h -- 1.8.3.1

1 12

[libvirt] error in docs https://libvirt.org/html/libvirt-libvirt-domain.html
by Vasiliy Tolstov 12 May '17

12 May '17

virDomainMigrateFlags have errors VIR_MIGRATE_LIVE have invalid comment and as i see all comments are shifted down by one point. -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

3 4