- Devel - libvirt List Archives

[libvirt] [PATCH 0/2] docs: news: add per-release links
by Ján Tomko 17 Jul '19

17 Jul '19

Ján Tomko (2): docs: news: remove sed sorcery docs: news: add per-release links docs/Makefile.am | 8 +------- docs/news-html.xsl | 21 ++++++++++++++------- 2 files changed, 15 insertions(+), 14 deletions(-) -- 2.20.1

4 8

[libvirt] [PATCH 0/7] docs: assorted cleanups (virtio-fs epopee)
by Ján Tomko 17 Jul '19

17 Jul '19

Got a bit lost while navigating around the <filesystem> element documentation. Ján Tomko (7): docs: link to networkportformat.html in format.html docs: drvqemu: fix a typo docs: drvqemu: remove relative time reference docs: formatdomain: remove stray nbsp docs: formatdomain: tsc is supported by QEMU docs: formatdomain: remove "Since" references older than 1.1.1 docs: remove "Since" references older than 1.1.1 docs/drvnodedev.html.in | 8 +- docs/drvqemu.html.in | 15 +- docs/format.html.in | 1 + docs/formatcaps.html.in | 22 ++- docs/formatdomain.html.in | 340 +++++++++++++------------------------- docs/storage.html.in | 5 - 6 files changed, 138 insertions(+), 253 deletions(-) -- 2.20.1

2 10

[libvirt] [PATCH 0/2] qemu: blockjob: Tweak config saving in legacy block job handling (blockdev-add saga)
by Peter Krempa 17 Jul '19

17 Jul '19

This is based on quite a lot of patches that were ACK'd but I didn't push them yet, so feel free to fetch this at: git fetch https://gitlab.com/pipo.sk/libvirt.git legacy-job-refactor Peter Krempa (2): qemu: blockjob: Save config only in qemuBlockJobEventProcessLegacyCompleted qemu: blockjob: Update new job state earlier in qemuBlockJobEventProcessLegacy src/qemu/qemu_blockjob.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) -- 2.21.0

2 4

[libvirt] [jenkins-ci PATCH 00/20] quayadmin: Introduce Quay API client
by Andrea Bolognani 17 Jul '19

17 Jul '19

Dan started building this a while ago, and I finally managed to find some time to take it from a PoC to something that's actually useful for our day to day operations. There's still plenty of work to be done for sure, but it's already at the point where I can at least partially script the process of refreshing container images, which is something that up until now had to be done slowly and painfully through the Web interface. I will probably spend some more time on it in a couple of weeks, after I've used it to refresh images right before the 5.6.0 release and inevitably spotted a bunch of issues while doing so O:-) Andrea Bolognani (20): quayadmin: Import initial implementation quayadmin: Fix argument parsing for delete-repo quayadmin: Fix quiet mode for show-repo quayadmin: Tweak show-repo output format quayadmin: Tweak list-repos output format quayadmin: Build endpoint, params and payload separately quayadmin: Implement debug mode quayadmin: Remove clientid and clientsecret quayamdin: Introduce configuration object quayadmin: Read configuration from file quayadmin: Add support for the PUT method quayadmin: Add list-triggers command quayadmin: Add show-trigger command quayadmin: Add activate-trigger command quayadmin: Add list-builds command quayadmin: Add show-build command quayadmin: Add list-tags command quayadmin: Add show-tag command quayadmin: Add create-tag command quayadmin: Add delete-tag command guests/quayadmin | 563 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 563 insertions(+) create mode 100755 guests/quayadmin -- 2.21.0

3 29

[libvirt] [PATCH 00/29] Split the libvirtd daemon into per-driver daemons
by Daniel P. Berrangé 17 Jul '19

17 Jul '19

This is what all the driver refactoring I've done has been about enabling. We gain new daemons for each driver, for the primary virt drivers: virtlibxld virtlxcd virtqemud virtvboxd virtvzd And again for the secondary drivers virtinterfaced virtnetworkd virtnodedevd virtnwfilterd virtsecretd virtstoraged Finally to support IP connectivity, and also the legacy lbivirtd UNIX domain socket (for the old libvirt remote driver SSH tunnelling): virtproxyd The the sake of facilitating upgrades, the existing libvirtd still exists and works the same way it always has. You either run libvirtd, or you run the per-driver daemons, never both. The remote driver will look to see whether libvirtd is running to figure out whether to connect to libvirtd or the new per-driver daemons. When auto-spawning daemons for nonroot users, we default to spawning the per-driver daemons. This can be controlled with a UR parameter "?mode=direct|legacy|auto", where 'direct' means per-driver and 'legacy' means libvirtd (or indirect via virtproxyd if that's running). Still todo - Add systemd unit files for the new daemons - Make it possible to disable build of libvirtd, or of the per-driver daemons so downstream vendors can decide which to ship - Tuning of the daemon defaults for worker threads to better suit the fact that we have per-driver daemons - More work on RPM packaging to allow install of per-driver daemosn without pulling in libvirtd too - A bunch of stuff that doesn't occurr to me right now - Identify & fix more bugs I've created here Daniel P. Berrangé (29): rpc: add API for checking whether an auth scheme is in use on a server remote: simplify libvirtd code for deciding if SASL auth is needed logging: pass binary name not logfile name when enabling logging remote: conditionalize socket names in libvirtd daemon remote: conditionalize daemon name in libvirtd daemon remote: conditionalize driver loading in libvirtd daemon remote: conditionalize IP socket usage in libvirtd daemon remote: conditionalize IP socket config in libvirtd.conf remote: conditionalize IP socket config in augeas definitions remote: refactor & rename variables for building libvirtd secret: introduce virtsecretd daemon network: introduce virtnetworkd daemon interface: introduce virtinterfaced daemon storage: introduce virtstoraged daemon nodedev: introduce virtnodedevd daemon nwfilter: introduce virtnwfilterd daemon libxl: introduce virtlibxld daemon qemu: introduce virtqemud daemon lxc: introduce virtlxcd daemon vbox: introduce virtvboxd daemon bhyve: introduce virtbhyved daemon vz: introduce virtvzd daemon remote: introduce virtproxyd daemon to handle IP connectivity remote: open secondary drivers via remote driver if needed remote: use enum helpers for parsing remote driver transport remote: refactor the code for choosing the UNIX socket path remote: switch to connect to per-driver daemons by default all: don't wait for driver lock during startup interface: fix driver name in state directory path .gitignore | 12 + build-aux/augeas-gentest.pl | 2 +- libvirt.spec.in | 10 + src/bhyve/Makefile.inc.am | 14 + src/bhyve/bhyve_driver.c | 2 +- src/driver.h | 2 + src/interface/Makefile.inc.am | 14 + src/interface/interface_backend_netcf.c | 6 +- src/interface/interface_backend_udev.c | 6 +- src/libvirt.c | 24 ++ src/libvirt_remote.syms | 1 + src/libxl/Makefile.inc.am | 14 + src/libxl/libxl_driver.c | 2 +- src/locking/lock_daemon.c | 2 +- src/logging/log_daemon.c | 2 +- src/lxc/Makefile.inc.am | 15 + src/lxc/lxc_driver.c | 2 +- src/network/Makefile.inc.am | 14 + src/network/leaseshelper.c | 2 +- src/node_device/Makefile.inc.am | 14 + src/node_device/node_device_hal.c | 2 +- src/node_device/node_device_udev.c | 2 +- src/nwfilter/Makefile.inc.am | 14 + src/nwfilter/nwfilter_driver.c | 2 +- src/qemu/Makefile.inc.am | 14 + src/qemu/qemu_driver.c | 2 +- src/remote/Makefile.inc.am | 153 ++++--- src/remote/{libvirtd.aug => libvirtd.aug.in} | 24 +- .../{libvirtd.conf => libvirtd.conf.in} | 42 +- src/remote/remote_daemon.c | 221 +++++++--- src/remote/remote_daemon_config.c | 41 +- src/remote/remote_daemon_config.h | 9 +- src/remote/remote_daemon_dispatch.c | 82 +++- src/remote/remote_driver.c | 391 ++++++++++++------ src/remote/remote_driver.h | 4 - src/remote/test_libvirtd.aug.in | 16 +- src/rpc/virnetserver.c | 17 + src/rpc/virnetserver.h | 3 + src/secret/Makefile.inc.am | 14 + src/secret/secret_driver.c | 2 +- src/storage/Makefile.inc.am | 14 + src/util/virlog.c | 20 +- src/vbox/Makefile.inc.am | 14 + src/vz/Makefile.inc.am | 14 + src/vz/vz_driver.c | 2 +- 45 files changed, 938 insertions(+), 341 deletions(-) rename src/remote/{libvirtd.aug => libvirtd.aug.in} (88%) rename src/remote/{libvirtd.conf => libvirtd.conf.in} (95%) -- 2.21.0

6 43

[libvirt] [PATCH 0/9] qemu: refactor fsdev formatting (virtio-fs epopee)
by Ján Tomko 16 Jul '19

16 Jul '19

Ján Tomko (9): tests: qemu: minimize fs9p.xml qemu: add CAPS_LATEST tests for 9pfs qemu: address: remove useless comment qemu: command: rename qemuBuildFSDevCommandLine qemu: command: re-introduce qemuBuildFSDevCommandLine qemu: command: use VIR_AUTOFREE in qemuBuildFSDevCommandLine qemu: introduce qemuDomainDeviceDefValidateFS qemu: command: use VIR_AUTOCLEAN in qemuBuildFS* qemu: command: remove qemuDomainFSDriver src/qemu/qemu_command.c | 111 ++++++------------ src/qemu/qemu_domain.c | 46 +++++++- src/qemu/qemu_domain_address.c | 3 +- .../fs9p-ccw.s390x-latest.args | 44 +++++++ tests/qemuxml2argvdata/fs9p.args | 6 +- .../qemuxml2argvdata/fs9p.x86_64-latest.args | 41 +++++++ tests/qemuxml2argvdata/fs9p.xml | 9 +- tests/qemuxml2argvtest.c | 2 + 8 files changed, 175 insertions(+), 87 deletions(-) create mode 100644 tests/qemuxml2argvdata/fs9p-ccw.s390x-latest.args create mode 100644 tests/qemuxml2argvdata/fs9p.x86_64-latest.args -- 2.19.2

2 11

[libvirt] [PATCH v2 0/4] Optimize mass closing of FDs on child spwaning
by Michal Privoznik 16 Jul '19

16 Jul '19

v2 of: https://www.redhat.com/archives/libvir-list/2019-July/msg00089.html diff to v1: - Added a test for stats parsing, - Added big fat warning as requested by Eric that malloc() in between fork() and exec() is okay with glibc (I guess we don't care about other libc-s like musl or uclibc). Michal Prívozník (4): virNetDevOpenvswitchInterfaceStats: Optimize for speed test: Introduce virnetdevopenvswitchtest vircommand: Separate mass FD closing into a function virCommand: use procfs to learn opened FDs src/libvirt_private.syms | 1 + src/util/vircommand.c | 122 +++++++++++++++--- src/util/virnetdevopenvswitch.c | 136 +++++++++++++++------ src/util/virnetdevopenvswitch.h | 4 + tests/Makefile.am | 13 +- tests/virnetdevopenvswitchdata/stats1.json | 1 + tests/virnetdevopenvswitchdata/stats2.json | 1 + tests/virnetdevopenvswitchtest.c | 101 +++++++++++++++ 8 files changed, 321 insertions(+), 58 deletions(-) create mode 100644 tests/virnetdevopenvswitchdata/stats1.json create mode 100644 tests/virnetdevopenvswitchdata/stats2.json create mode 100644 tests/virnetdevopenvswitchtest.c -- 2.21.0

2 7

[libvirt] [PATCH v2] qapi: add dirty-bitmaps to query-named-block-nodes result
by John Snow 15 Jul '19

15 Jul '19

From: Vladimir Sementsov-Ogievskiy <vsementsov(a)virtuozzo.com> Let's add a possibility to query dirty-bitmaps not only on root nodes. It is useful when dealing both with snapshots and incremental backups. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov(a)virtuozzo.com> [Added deprecation and feature flag information. --js] Signed-off-by: John Snow <jsnow(a)redhat.com> --- block/qapi.c | 5 +++++ qapi/block-core.json | 14 +++++++++++++- qemu-deprecated.texi | 12 ++++++++++++ 3 files changed, 30 insertions(+), 1 deletion(-) diff --git a/block/qapi.c b/block/qapi.c index 917435f022..15f1030264 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk, info->backing_file = g_strdup(bs->backing_file); } + if (!QLIST_EMPTY(&bs->dirty_bitmaps)) { + info->has_dirty_bitmaps = true; + info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs); + } + info->detect_zeroes = bs->detect_zeroes; if (blk && blk_get_public(blk)->throttle_group_member.throttle_state) { diff --git a/qapi/block-core.json b/qapi/block-core.json index 0d43d4f37c..0d67dd245c 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -360,6 +360,16 @@ # @write_threshold: configured write threshold for the device. # 0 if disabled. (Since 2.3) # +# @dirty-bitmaps: dirty bitmaps information (only present if node +# has one or more dirty bitmaps) (Since 4.2) +# +# Features: +# @node-dirty-bitmaps: Signals the capability to return dirty bitmap information +# per-node instead of per-drive. If this flag is present, +# dirty-bitmaps should not be read from the BlockInfo +# structure, the top-level data for query-block. +# (Since 4.2) +# # Since: 0.14.0 # ## @@ -378,7 +388,8 @@ '*bps_wr_max_length': 'int', '*iops_max_length': 'int', '*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int', '*iops_size': 'int', '*group': 'str', 'cache': 'BlockdevCacheInfo', - 'write_threshold': 'int' } } + 'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo'] }, + 'features': [ { 'name': 'node-dirty-bitmaps' } ] } ## # @BlockDeviceIoStatus: @@ -656,6 +667,7 @@ # # @dirty-bitmaps: dirty bitmaps information (only present if the # driver has one or more dirty bitmaps) (Since 2.0) +# Deprecated in 4.2; see BlockDirtyInfo instead. # # @io-status: @BlockDeviceIoStatus. Only present if the device # supports it and the VM is configured to stop on errors diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi index c90b08d553..bc4e5ac1d7 100644 --- a/qemu-deprecated.texi +++ b/qemu-deprecated.texi @@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo'' structure, returned by the query-block command is deprecated. Two new boolean fields, ``recording'' and ``busy'' effectively replace it. +@subsection query-block result field dirty-bitmaps (Since 4.2) + +The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by +the query-block command is itself now deprecated. The ``dirty-bitmaps`` +field of the ``BlockDeviceInfo`` struct should be used instead, which is the +type of the ``inserted`` field in query-block replies, as well as the +type of array items in query-named-block-nodes. + +In the absence of bitmaps on either structure, management APIs may use the +presence of the ``node-dirty-bitmaps`` feature flag on the ``BlockDeviceInfo`` +structure to know where to anticipate bitmap data when present. + @subsection query-cpus (since 2.12.0) The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command. -- 2.21.0

1 0

[libvirt] [PATCH 0/3] Optimize mass closing of FDs on child spwaning
by Michal Privoznik 15 Jul '19

15 Jul '19

If the limit for max opened files is way bigger than the default (1024), say 3 orders bigger then spawning a child through virCommand can be expensive because we iterate over ALL FDs within the limit and close them. There's no need to that since we can learn the list of opened FDs from /proc/self/fd/. Michal Prívozník (3): virNetDevOpenvswitchInterfaceStats: Optimize for speed vircommand: Separate mass FD closing into a function virCommand: use procfs to learn opened FDs src/util/vircommand.c | 114 +++++++++++++++++++++++++++----- src/util/virnetdevopenvswitch.c | 111 ++++++++++++++++++++----------- 2 files changed, 170 insertions(+), 55 deletions(-) -- 2.21.0

7 20

[libvirt] [PATCH] tests: don't assume "localhost" only resolves to 1/2 IPs
by Daniel P. Berrangé 15 Jul '19

15 Jul '19

On Debian derived distros "localhost" can resolve to the normal "127.0.0.1" and "::1", but it can also resolve to "127.0.1.1" Rewrite the code so that it doesn't assume a fixed number of IPs. Signed-off-by: Daniel P. Berrangé <berrange(a)redhat.com> --- tests/virsystemdtest.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/tests/virsystemdtest.c b/tests/virsystemdtest.c index cd031914ab..dd87caeebf 100644 --- a/tests/virsystemdtest.c +++ b/tests/virsystemdtest.c @@ -548,22 +548,21 @@ testActivation(bool useNames) size_t nsockIP; int ret = -1; size_t i; - const char *names2 = "demo-unix.socket:demo-ip.socket"; - const char *names3 = "demo-unix.socket:demo-ip.socket:demo-ip.socket"; char nfdstr[INT_BUFSIZE_BOUND(size_t)]; char pidstr[INT_BUFSIZE_BOUND(pid_t)]; virSystemdActivationMap map[2]; int *fds = NULL; size_t nfds = 0; VIR_AUTOPTR(virSystemdActivation) act = NULL; + virBuffer names = VIR_BUFFER_INITIALIZER; + + virBufferAddLit(&names, "demo-unix.socket"); if (testActivationCreateFDs(&sockUNIX, &sockIP, &nsockIP) < 0) return -1; - if (nsockIP != 1 && nsockIP != 2) { - fprintf(stderr, "Got %zu IP sockets but expected only 1 or 2\n", nsockIP); - goto cleanup; - } + for (i = 0; i < nsockIP; i++) + virBufferAddLit(&names, ":demo-ip.socket"); snprintf(nfdstr, sizeof(nfdstr), "%zu", 1 + nsockIP); snprintf(pidstr, sizeof(pidstr), "%lld", (long long)getpid()); @@ -571,8 +570,11 @@ testActivation(bool useNames) setenv("LISTEN_FDS", nfdstr, 1); setenv("LISTEN_PID", pidstr, 1); + if (virBufferError(&names)) + goto cleanup; + if (useNames) - setenv("LISTEN_FDNAMES", nsockIP == 1 ? names2 : names3, 1); + setenv("LISTEN_FDNAMES", virBufferCurrentContent(&names), 1); else unsetenv("LISTEN_FDNAMES"); @@ -627,6 +629,7 @@ testActivation(bool useNames) ret = 0; cleanup: + virBufferFreeAndReset(&names); virObjectUnref(sockUNIX); for (i = 0; i < nsockIP; i++) virObjectUnref(sockIP[i]); -- 2.21.0

2 1