November 2017 - Devel - libvirt List Archives

[libvirt] [PATCH 00/21] Initial support for resctrl (CAT)
by Martin Kletzander 21 Nov '17

21 Nov '17

This is an initial support for resctrl, so that domains can make use of Cache Allocation Technology (🐈). After bunch of initial discussions I followed most of those things except one detail. In order to distinguish different allocations while minimizing the number of them at the same time, I changed the XML design, but only slightly. See docs in patch 17 for detail. There is still much missing, but all is designed in a way that should not collied with the future additions. Missing features include: - After allocation is created the file should be re-read and parsed into the al the allocation so that the masks reflect what kernel actually uses (just in case there was a change) - When formatting live XML, used masks for allocations should be formatted as well - Each <cache/> should support shared allocations, for now every single allocation is exclusive only - Suuport for different backing, particularly if allocation has type='both' (code as well as data) and the host supports CDP (Code Data Partitioning), meaning it can allocate code and data portions separately the code currently fails even though it is possible to convert such allocation. - APIs for manipulation with various resctrl groups. What's needed is reading and setting values of the default group as well as any cachetune of a domain. Martin Kletzander (21): Only output initialized capabilities with VIR_TEST_DEBUG > 1 util: Introduce virPrettySize util: Make prefix optional in virBitampString util: Rename virBitmapString to virBitmapToString util: Rename virBitmapDataToString to virBitmapDataFormat util: Don't output too many zeros from virBitmapToString util: Introduce virBitmapNewString util: Reintroduce virBitmapSubtract util: Introduce virBitmapShrink conf: Sort cache banks in capabilities XML conf: Format cache banks in capabilities with virPrettySize resctrl: Instantiate all resctrl information at once tests: Remove executable bits on plain data files tests: Change some schemata for the default group caps2xml resctrl-skx-twocaches resctrl: Add functions to work with resctrl allocations conf: cachetune tests: Minor adjustments for test data tests: Add virresctrltest qemu: Add support for resctrl docs: Add CAT (resctrl) support into news.xml docs/formatdomain.html.in | 24 + docs/news.xml | 11 + docs/schemas/domaincommon.rng | 32 + po/POTFILES.in | 1 + src/Makefile.am | 2 +- src/conf/capabilities.c | 113 +- src/conf/capabilities.h | 4 +- src/conf/domain_conf.c | 249 ++++ src/conf/domain_conf.h | 21 + src/libvirt_private.syms | 24 +- src/qemu/qemu_capabilities.c | 4 +- src/qemu/qemu_process.c | 61 +- src/util/virbitmap.c | 118 +- src/util/virbitmap.h | 15 +- src/util/virresctrl.c | 1317 ++++++++++++++++++-- src/util/virresctrl.h | 81 +- src/util/virresctrlpriv.h | 32 + src/util/virutil.c | 50 + src/util/virutil.h | 3 + tests/Makefile.am | 8 +- .../genericxml2xmlindata/generic-cachetune-cdp.xml | 36 + .../generic-cachetune-colliding-allocs.xml | 30 + .../generic-cachetune-colliding-tunes.xml | 32 + .../generic-cachetune-colliding-types.xml | 30 + .../generic-cachetune-small.xml | 29 + tests/genericxml2xmlindata/generic-cachetune.xml | 33 + tests/genericxml2xmltest.c | 10 + tests/testutils.c | 2 +- tests/virbitmaptest.c | 84 +- tests/vircaps2xmldata/linux-caches/system | 1 + .../linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask | 0 .../resctrl/info/L3CODE/min_cbm_bits | 0 .../resctrl/info/L3CODE/num_closids | 0 .../linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask | 0 .../resctrl/info/L3DATA/min_cbm_bits | 0 .../resctrl/info/L3DATA/num_closids | 0 .../linux-resctrl-cdp/resctrl/schemata | 4 +- .../resctrl/info/L3/cbm_mask | 1 + .../resctrl/info/L3}/min_cbm_bits | 0 .../resctrl/info/L3/num_closids | 1 + .../linux-resctrl-skx-twocaches/resctrl/schemata | 1 + .../resctrl/some_reservation/schemata | 1 + .../system/cpu/cpu0/cache/index0/id | 1 + .../system/cpu/cpu0/cache/index0/level | 1 + .../system/cpu/cpu0/cache/index0/shared_cpu_list | 1 + .../system/cpu/cpu0/cache/index0/shared_cpu_map | 1 + .../system/cpu/cpu0/cache/index0/size | 1 + .../system/cpu/cpu0/cache/index0/type | 1 + .../system/cpu/cpu0/cache/index1/id} | 0 .../system/cpu/cpu0/cache/index1/level | 1 + .../system/cpu/cpu0/cache/index1/shared_cpu_list | 1 + .../system/cpu/cpu0/cache/index1/shared_cpu_map | 1 + .../system/cpu/cpu0/cache/index1/size | 1 + .../system/cpu/cpu0/cache/index1/type | 1 + .../system/cpu/cpu0/online} | 0 .../system/cpu/cpu0/topology/core_id | 1 + .../system/cpu/cpu0/topology/core_siblings | 1 + .../system/cpu/cpu0/topology/core_siblings_list | 1 + .../system/cpu/cpu0/topology/physical_package_id | 1 + .../system/cpu/cpu0/topology/thread_siblings | 1 + .../system/cpu/cpu0/topology/thread_siblings_list | 1 + .../linux-resctrl-skx-twocaches/system/cpu/online | 1 + .../linux-resctrl-skx-twocaches/system/cpu/present | 1 + .../system/node/node0/cpu0 | 1 + .../system/node/node0/cpulist | 1 + .../system/node/node0/cpumap | 1 + .../system/node/node0/distance | 1 + .../linux-resctrl-skx-twocaches/system/node/online | 1 + .../linux-resctrl-skx/resctrl/empty/schemata | 0 .../linux-resctrl-skx/resctrl/schemata | 2 +- .../vircaps2xmldata/linux-resctrl/resctrl/schemata | 2 +- tests/vircaps2xmldata/vircaps-x86_64-caches.xml | 2 +- .../vircaps2xmldata/vircaps-x86_64-resctrl-cdp.xml | 4 +- ...ml => vircaps-x86_64-resctrl-skx-twocaches.xml} | 7 +- .../vircaps2xmldata/vircaps-x86_64-resctrl-skx.xml | 4 +- tests/vircaps2xmldata/vircaps-x86_64-resctrl.xml | 4 +- tests/vircaps2xmltest.c | 1 + .../resctrl--cachetune/vcpus-0-1.alloc | 1 + .../resctrl--cachetune/vcpus-3.alloc | 1 + .../resctrl-cdp--cachetune-cdp/vcpus-0-1.alloc | 2 + .../resctrl-cdp--cachetune-cdp/vcpus-2.alloc | 1 + .../resctrl-cdp--cachetune-cdp/vcpus-3.alloc | 1 + tests/virresctrldata/resctrl-cdp.schemata | 2 + .../virresctrldata/resctrl-skx-twocaches.schemata | 1 + tests/virresctrldata/resctrl-skx.schemata | 1 + tests/virresctrldata/resctrl.schemata | 1 + tests/virresctrltest.c | 277 ++++ tools/virsh-domain.c | 4 +- tools/virsh-host.c | 2 +- 89 files changed, 2626 insertions(+), 183 deletions(-) create mode 100644 src/util/virresctrlpriv.h create mode 100644 tests/genericxml2xmlindata/generic-cachetune-cdp.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-allocs.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-tunes.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-colliding-types.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune-small.xml create mode 100644 tests/genericxml2xmlindata/generic-cachetune.xml create mode 120000 tests/vircaps2xmldata/linux-caches/system mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/cbm_mask mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3CODE/num_closids mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/cbm_mask mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/min_cbm_bits mode change 100755 => 100644 tests/vircaps2xmldata/linux-resctrl-cdp/resctrl/info/L3DATA/num_closids create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/info/L3/cbm_mask copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE => linux-resctrl-skx-twocaches/resctrl/info/L3}/min_cbm_bits (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/info/L3/num_closids create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/schemata create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/resctrl/some_reservation/schemata create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/level create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/shared_cpu_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/shared_cpu_map create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/size create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index0/type copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits => linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/id} (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/level create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/shared_cpu_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/shared_cpu_map create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/size create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/cache/index1/type copy tests/vircaps2xmldata/{linux-resctrl-cdp/resctrl/info/L3CODE/min_cbm_bits => linux-resctrl-skx-twocaches/system/cpu/cpu0/online} (100%) mode change 100755 => 100644 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_siblings create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/core_siblings_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/physical_package_id create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/thread_siblings create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/cpu0/topology/thread_siblings_list create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/online create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/cpu/present create mode 120000 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpu0 create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpulist create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/cpumap create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/node0/distance create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx-twocaches/system/node/online create mode 100644 tests/vircaps2xmldata/linux-resctrl-skx/resctrl/empty/schemata copy tests/vircaps2xmldata/{vircaps-x86_64-resctrl-skx.xml => vircaps-x86_64-resctrl-skx-twocaches.xml} (68%) create mode 100644 tests/virresctrldata/resctrl--cachetune/vcpus-0-1.alloc create mode 100644 tests/virresctrldata/resctrl--cachetune/vcpus-3.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-0-1.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-2.alloc create mode 100644 tests/virresctrldata/resctrl-cdp--cachetune-cdp/vcpus-3.alloc create mode 100644 tests/virresctrldata/resctrl-cdp.schemata create mode 100644 tests/virresctrldata/resctrl-skx-twocaches.schemata create mode 100644 tests/virresctrldata/resctrl-skx.schemata create mode 100644 tests/virresctrldata/resctrl.schemata create mode 100644 tests/virresctrltest.c -- 2.15.0

3 57

[libvirt] [PATCH v4 1/2] remote: Extract common clearing of event callbacks of client private data
by xinhua.Cao 20 Nov '17

20 Nov '17

Extract common clearing of event callbacks as remoteClientFreePrivateCallbacks. the common function also separation including the sysident handling. --- daemon/remote.c | 73 ++++++++++++++++++++++++++++++--------------------------- 1 file changed, 39 insertions(+), 34 deletions(-) diff --git a/daemon/remote.c b/daemon/remote.c index cbcb6e8..2dcec1e 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -1689,6 +1689,44 @@ void remoteRelayConnectionClosedEvent(virConnectPtr conn ATTRIBUTE_UNUSED, int r neventCallbacks = 0; \ } while (0); + +static void +remoteClientFreePrivateCallbacks(struct daemonClientPrivate *priv) +{ + virIdentityPtr sysident = virIdentityGetSystem(); + virIdentitySetCurrent(sysident); + + DEREG_CB(priv->conn, priv->domainEventCallbacks, + priv->ndomainEventCallbacks, + virConnectDomainEventDeregisterAny, "domain"); + DEREG_CB(priv->conn, priv->networkEventCallbacks, + priv->nnetworkEventCallbacks, + virConnectNetworkEventDeregisterAny, "network"); + DEREG_CB(priv->conn, priv->storageEventCallbacks, + priv->nstorageEventCallbacks, + virConnectStoragePoolEventDeregisterAny, "storage"); + DEREG_CB(priv->conn, priv->nodeDeviceEventCallbacks, + priv->nnodeDeviceEventCallbacks, + virConnectNodeDeviceEventDeregisterAny, "node device"); + DEREG_CB(priv->conn, priv->secretEventCallbacks, + priv->nsecretEventCallbacks, + virConnectSecretEventDeregisterAny, "secret"); + DEREG_CB(priv->conn, priv->qemuEventCallbacks, + priv->nqemuEventCallbacks, + virConnectDomainQemuMonitorEventDeregister, "qemu monitor"); + + if (priv->closeRegistered) { + if (virConnectUnregisterCloseCallback(priv->conn, + remoteRelayConnectionClosedEvent) < 0) + VIR_WARN("unexpected close callback event deregister failure"); + } + + virIdentitySetCurrent(NULL); + virObjectUnref(sysident); +} +#undef DEREG_CB + + /* * You must hold lock for at least the client * We don't free stuff here, merely disconnect the client's @@ -1702,44 +1740,11 @@ void remoteClientFreeFunc(void *data) /* Deregister event delivery callback */ if (priv->conn) { - virIdentityPtr sysident = virIdentityGetSystem(); - - virIdentitySetCurrent(sysident); - - DEREG_CB(priv->conn, priv->domainEventCallbacks, - priv->ndomainEventCallbacks, - virConnectDomainEventDeregisterAny, "domain"); - DEREG_CB(priv->conn, priv->networkEventCallbacks, - priv->nnetworkEventCallbacks, - virConnectNetworkEventDeregisterAny, "network"); - DEREG_CB(priv->conn, priv->storageEventCallbacks, - priv->nstorageEventCallbacks, - virConnectStoragePoolEventDeregisterAny, "storage"); - DEREG_CB(priv->conn, priv->nodeDeviceEventCallbacks, - priv->nnodeDeviceEventCallbacks, - virConnectNodeDeviceEventDeregisterAny, "node device"); - DEREG_CB(priv->conn, priv->secretEventCallbacks, - priv->nsecretEventCallbacks, - virConnectSecretEventDeregisterAny, "secret"); - DEREG_CB(priv->conn, priv->qemuEventCallbacks, - priv->nqemuEventCallbacks, - virConnectDomainQemuMonitorEventDeregister, "qemu monitor"); - - if (priv->closeRegistered) { - if (virConnectUnregisterCloseCallback(priv->conn, - remoteRelayConnectionClosedEvent) < 0) - VIR_WARN("unexpected close callback event deregister failure"); - } - + remoteClientFreePrivateCallbacks(priv); virConnectClose(priv->conn); - - virIdentitySetCurrent(NULL); - virObjectUnref(sysident); } - VIR_FREE(priv); } -#undef DEREG_CB static void remoteClientCloseFunc(virNetServerClientPtr client) -- 2.8.3

3 5

[libvirt] [PATCH v2 00/10] Implement query-dump command
by John Ferlan 20 Nov '17

20 Nov '17

v1: https://www.redhat.com/archives/libvir-list/2017-November/msg00731.html Differences from v1 (besides more patches) Rather than use a timing loop in order to fetch the dump stats and the presence of the 'query-dump' command, let's use the DUMP_COMPLETED event and wire that up in order to determine when the dump has been completed. In the mean time, the previous timing loop is converted into a qemuDomainGetJobInfoDumpStats to be called if the "right conditions" exist. Also found/fixed a couple of minor issues from v1 regarding usage "dumpformat" instead of just "format" from my patch 2 adjustment and a type for qemuMonitorDumpStatus (was qemuMontiorDumpStatus in v1). NB: (from v1 cover): Details in the patches. Essentially though QEMU 2.6 added the ability to perform the 'dump-guest-memory' via a thread by adding a 'detach' boolean to the command. In order to watch the progress of the thread the 'query-dump' command was added. The query-dump will return just the current status, the total size to be dumped, and the current progress. So using the migrate stats data in the DomainJobInfo we can then save our current progress allowing tools such as virsh to watch that progress. As an added benefit, the dump-guest-memory is now truly asynchronous. John Ferlan (10): qemu: Clean up style for the qemuDumpToFd definition qemu: Alter dump-guest-memory command generation qemu: Add support for DUMP_COMPLETED event qemu: Introduce qemuProcessHandleDumpCompleted qemu: Introduce qemuMonitor[JSON]QueryDump qemu: Add new parameter to qemuMonitorDumpToFd qemu: Introduce qemuDomainGetJobInfoMigrationStats qemu: Introduce qemuDomainGetJobInfoDumpStats qemu: Add dump completed event to the capabilities qemu: Allow showing the dump progress for memory only dump src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_domain.c | 2 + src/qemu/qemu_domain.h | 2 + src/qemu/qemu_driver.c | 157 ++++++++++++++++++--- src/qemu/qemu_monitor.c | 39 ++++- src/qemu/qemu_monitor.h | 33 ++++- src/qemu/qemu_monitor_json.c | 122 ++++++++++++++-- src/qemu/qemu_monitor_json.h | 7 +- src/qemu/qemu_process.c | 23 +++ .../caps_2.10.0-gicv2.aarch64.xml | 1 + .../caps_2.10.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + .../caps_2.6.0-gicv2.aarch64.xml | 1 + .../caps_2.6.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 3 +- 27 files changed, 365 insertions(+), 42 deletions(-) -- 2.13.6

1 10

[libvirt] [jenkins-ci PATCH 0/5] Fix environment handling
by Andrea Bolognani 20 Nov '17

20 Nov '17

My own attempt at fixing libosinfo-master-rpm's failing status, and then some. This is an extension of Pavel's attempt[1] in a way; except of course I think my approach is superior, otherwise I wouldn't have bothered ;) It also sets the groundwork for finally moving $VIRT_PREFIX and friends out of the Jenkins server configuration, where they are invisible to anyone but the CI admins, and into the git repository so that they can be tracked alongside the jobs using them. [1] https://www.redhat.com/archives/libvir-list/2017-November/msg00045.html Andrea Bolognani (5): jobs: Rename {make_env} to {global_env} jobs: Include {global_env} everywhere jobs: Rename {check_env} to {env} jobs: Include {env} everywhere jobs: Include $PATH definition in {global_env} jobs/autotools.yaml | 17 ++++++++++++----- jobs/defaults.yaml | 5 +++-- jobs/generic.yaml | 20 ++++++++++++++++---- jobs/go.yaml | 4 ++++ jobs/perl-makemaker.yaml | 9 ++++++--- jobs/perl-modulebuild.yaml | 6 ++++++ jobs/python-distutils.yaml | 6 ++++++ projects/libosinfo.yaml | 2 +- projects/libvirt.yaml | 2 +- projects/osinfo-db.yaml | 4 ---- 10 files changed, 55 insertions(+), 20 deletions(-) -- 2.14.3

2 8

[libvirt] [PATCH 00/14] Fix serial console behavior on non-x86 architectures
by Andrea Bolognani 20 Nov '17

20 Nov '17

See the last commit for a short explanation of what this series is all about. Andrea Bolognani (10): qemu: Add QEMU_CAPS_DEVICE_SPAPR_VTY conf,qemu: Use type-aware switches where possible qemu: Introduce qemuDomainChrDefPostParse() conf: Run devicePostParse() again for the first serial device conf: Introduce VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_NONE conf: Drop virDomainChrDeviceType.targetTypeAttr conf: Add VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_SPAPR qemu: Support usb-serial and pci-serial on pSeries conf: Add VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_PL011 news: Update for serial console fixes Pino Toscano (4): conf: add VIR_DOMAIN_CHR_SERIAL_TARGET_TYPE_SCLP/SCLPLM conf: pass parseFlags down to virDomainDefAddConsoleCompat conf: convert sclp/sclplm <console> as <serial> qemu: switch s390/s390x default console back to serial docs/formatdomain.html.in | 13 +- docs/news.xml | 12 ++ docs/schemas/domaincommon.rng | 4 + src/conf/domain_conf.c | 132 +++++++++++++++----- src/conf/domain_conf.h | 11 +- src/qemu/qemu_capabilities.c | 2 + src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 136 +++++++++++++-------- src/qemu/qemu_domain.c | 95 ++++++++++++-- src/qemu/qemu_domain_address.c | 8 +- src/vz/vz_sdk.c | 5 +- .../qemuargv2xml-console-compat.xml | 2 +- tests/qemuargv2xmldata/qemuargv2xml-serial-dev.xml | 2 +- .../qemuargv2xmldata/qemuargv2xml-serial-file.xml | 2 +- .../qemuargv2xmldata/qemuargv2xml-serial-many.xml | 4 +- tests/qemuargv2xmldata/qemuargv2xml-serial-pty.xml | 2 +- .../qemuargv2xml-serial-tcp-telnet.xml | 2 +- tests/qemuargv2xmldata/qemuargv2xml-serial-tcp.xml | 2 +- tests/qemuargv2xmldata/qemuargv2xml-serial-udp.xml | 4 +- .../qemuargv2xmldata/qemuargv2xml-serial-unix.xml | 2 +- tests/qemuargv2xmldata/qemuargv2xml-serial-vc.xml | 2 +- tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + ...otplug-console-compat-2-live+console-virtio.xml | 4 +- .../qemuhotplug-console-compat-2-live.xml | 4 +- .../qemuxml2argv-mach-virt-console-native.args | 1 + .../qemuxml2argv-mach-virt-console-native.xml | 17 +++ ... => qemuxml2argv-mach-virt-console-virtio.args} | 15 +-- .../qemuxml2argv-mach-virt-console-virtio.xml | 19 +++ ...muxml2argv-mach-virt-serial+console-native.args | 1 + ...emuxml2argv-mach-virt-serial+console-native.xml | 18 +++ .../qemuxml2argv-mach-virt-serial-compat.args | 1 + .../qemuxml2argv-mach-virt-serial-compat.xml | 19 +++ ...muxml2argv-mach-virt-serial-invalid-machine.xml | 19 +++ ...s => qemuxml2argv-mach-virt-serial-native.args} | 12 +- .../qemuxml2argv-mach-virt-serial-native.xml | 16 +++ .../qemuxml2argv-mach-virt-serial-pci.args | 26 ++++ .../qemuxml2argv-mach-virt-serial-pci.xml | 18 +++ .../qemuxml2argv-mach-virt-serial-usb.args | 27 ++++ .../qemuxml2argv-mach-virt-serial-usb.xml | 21 ++++ .../qemuxml2argv-pseries-basic.args | 2 +- .../qemuxml2argv-pseries-console-native.args | 1 + .../qemuxml2argv-pseries-console-native.xml | 17 +++ ...gs => qemuxml2argv-pseries-console-virtio.args} | 10 +- .../qemuxml2argv-pseries-console-virtio.xml | 19 +++ .../qemuxml2argv-pseries-cpu-compat-power9.args | 2 +- .../qemuxml2argv-pseries-cpu-compat.args | 2 +- .../qemuxml2argv-pseries-cpu-exact.args | 2 +- .../qemuxml2argv-pseries-cpu-le.args | 2 +- .../qemuxml2argv-pseries-panic-missing.args | 2 +- .../qemuxml2argv-pseries-panic-no-address.args | 2 +- ...qemuxml2argv-pseries-serial+console-native.args | 1 + .../qemuxml2argv-pseries-serial+console-native.xml | 18 +++ .../qemuxml2argv-pseries-serial-compat.args | 1 + .../qemuxml2argv-pseries-serial-compat.xml | 19 +++ ...qemuxml2argv-pseries-serial-invalid-machine.xml | 19 +++ ...rgs => qemuxml2argv-pseries-serial-native.args} | 7 +- .../qemuxml2argv-pseries-serial-native.xml | 16 +++ ...c.args => qemuxml2argv-pseries-serial-pci.args} | 7 +- .../qemuxml2argv-pseries-serial-pci.xml | 18 +++ ...c.args => qemuxml2argv-pseries-serial-usb.args} | 8 +- .../qemuxml2argv-pseries-serial-usb.xml | 21 ++++ .../qemuxml2argv-pseries-usb-default.args | 2 +- .../qemuxml2argv-pseries-usb-kbd.args | 2 +- .../qemuxml2argv-pseries-usb-multi.args | 2 +- .../qemuxml2argv-pseries-vio-user-assigned.args | 4 +- .../qemuxml2argvdata/qemuxml2argv-pseries-vio.args | 4 +- ....args => qemuxml2argv-s390-console2serial.args} | 11 +- .../qemuxml2argv-s390-console2serial.xml | 19 +++ ...power9.args => qemuxml2argv-s390-serial-2.args} | 14 +-- .../qemuxml2argv-s390-serial-2.xml | 17 +++ ....args => qemuxml2argv-s390-serial-console.args} | 11 +- .../qemuxml2argv-s390-serial-console.xml | 15 +++ ...es-basic.args => qemuxml2argv-s390-serial.args} | 11 +- .../qemuxml2argvdata/qemuxml2argv-s390-serial.xml | 14 +++ ...muxml2argv-serial-tcp-tlsx509-chardev-notls.xml | 4 +- .../qemuxml2argvdata/qemuxml2argv-user-aliases.xml | 4 +- tests/qemuxml2argvtest.c | 83 +++++++++++++ .../qemuxml2xmlout-aarch64-virtio-pci-default.xml | 2 +- .../qemuxml2xmlout-bios-nvram-os-interleave.xml | 2 +- .../qemuxml2xmlout-chardev-label.xml | 4 +- .../qemuxml2xmlout-console-compat-auto.xml | 2 +- .../qemuxml2xmlout-console-compat.xml | 2 +- .../qemuxml2xmlout-console-compat2.xml | 2 +- .../qemuxml2xmlout-console-virtio-many.xml | 2 +- .../qemuxml2xmlout-interface-driver.xml | 2 +- .../qemuxml2xmlout-interface-server.xml | 4 +- .../qemuxml2xmlout-mach-virt-console-native.xml | 1 + ...=> qemuxml2xmlout-mach-virt-console-virtio.xml} | 19 +-- ...uxml2xmlout-mach-virt-serial+console-native.xml | 1 + .../qemuxml2xmlout-mach-virt-serial-compat.xml | 29 +++++ .../qemuxml2xmlout-mach-virt-serial-native.xml | 1 + .../qemuxml2xmlout-mach-virt-serial-pci.xml | 42 +++++++ .../qemuxml2xmlout-mach-virt-serial-usb.xml | 39 ++++++ .../qemuxml2xmlout-net-bandwidth.xml | 2 +- .../qemuxml2xmlout-net-bandwidth2.xml | 2 +- .../qemuxml2xmlout-net-coalesce.xml | 2 +- .../qemuxml2xmloutdata/qemuxml2xmlout-net-mtu.xml | 2 +- .../qemuxml2xmlout-panic-pseries.xml | 2 +- .../qemuxml2xmlout-pseries-console-native.xml | 1 + ...l => qemuxml2xmlout-pseries-console-virtio.xml} | 16 +-- .../qemuxml2xmlout-pseries-cpu-compat-power9.xml | 2 +- .../qemuxml2xmlout-pseries-cpu-compat.xml | 2 +- .../qemuxml2xmlout-pseries-cpu-exact.xml | 2 +- .../qemuxml2xmlout-pseries-panic-missing.xml | 2 +- .../qemuxml2xmlout-pseries-panic-no-address.xml | 2 +- ...emuxml2xmlout-pseries-serial+console-native.xml | 1 + .../qemuxml2xmlout-pseries-serial-compat.xml | 1 + ...ml => qemuxml2xmlout-pseries-serial-native.xml} | 8 +- ...g.xml => qemuxml2xmlout-pseries-serial-pci.xml} | 14 +-- ...g.xml => qemuxml2xmlout-pseries-serial-usb.xml} | 11 +- .../qemuxml2xmlout-q35-virt-manager-basic.xml | 2 +- .../qemuxml2xmlout-s390-defaultconsole.xml | 6 +- .../qemuxml2xmlout-s390-serial-2.xml | 29 +++++ ....xml => qemuxml2xmlout-s390-serial-console.xml} | 18 +-- ...tconsole.xml => qemuxml2xmlout-s390-serial.xml} | 18 +-- .../qemuxml2xmlout-serial-spiceport-nospice.xml | 2 +- .../qemuxml2xmlout-serial-spiceport.xml | 2 +- .../qemuxml2xmlout-serial-target-port-auto.xml | 6 +- .../qemuxml2xmlout-serial-tcp-tlsx509-chardev.xml | 4 +- .../qemuxml2xmlout-tap-vhost-incorrect.xml | 2 +- .../qemuxml2xmlout-tap-vhost.xml | 2 +- .../qemuxml2xmlout-vhost_queues.xml | 2 +- tests/qemuxml2xmltest.c | 54 ++++++++ 125 files changed, 1186 insertions(+), 284 deletions(-) create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-console-native.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-console-native.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-mach-virt-console-virtio.args} (53%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-console-virtio.xml create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial+console-native.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial+console-native.xml create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-compat.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-compat.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-invalid-machine.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-mach-virt-serial-native.args} (62%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-native.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-pci.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-pci.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-usb.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-mach-virt-serial-usb.xml create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-pseries-console-native.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-console-native.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-pseries-console-virtio.args} (59%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-console-virtio.xml create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial+console-native.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial+console-native.xml create mode 120000 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-compat.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-compat.xml create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-invalid-machine.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-pseries-serial-native.args} (70%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-native.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-pseries-serial-pci.args} (70%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-pci.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-pseries-serial-usb.args} (65%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-pseries-serial-usb.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-s390-console2serial.args} (71%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-s390-console2serial.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-cpu-compat-power9.args => qemuxml2argv-s390-serial-2.args} (62%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-s390-serial-2.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-s390-serial-console.args} (71%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-s390-serial-console.xml copy tests/qemuxml2argvdata/{qemuxml2argv-pseries-basic.args => qemuxml2argv-s390-serial.args} (71%) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-s390-serial.xml create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-console-native.xml copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-s390-defaultconsole.xml => qemuxml2xmlout-mach-virt-console-virtio.xml} (50%) create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-serial+console-native.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-serial-compat.xml create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-serial-native.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-serial-pci.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-mach-virt-serial-usb.xml create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-pseries-console-native.xml copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-panic-pseries.xml => qemuxml2xmlout-pseries-console-virtio.xml} (75%) create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-pseries-serial+console-native.xml create mode 120000 tests/qemuxml2xmloutdata/qemuxml2xmlout-pseries-serial-compat.xml copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-pseries-panic-missing.xml => qemuxml2xmlout-pseries-serial-native.xml} (82%) copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-pseries-panic-missing.xml => qemuxml2xmlout-pseries-serial-pci.xml} (74%) copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-pseries-panic-missing.xml => qemuxml2xmlout-pseries-serial-usb.xml} (75%) create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-s390-serial-2.xml copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-s390-defaultconsole.xml => qemuxml2xmlout-s390-serial-console.xml} (50%) copy tests/qemuxml2xmloutdata/{qemuxml2xmlout-s390-defaultconsole.xml => qemuxml2xmlout-s390-serial.xml} (50%) -- 2.13.6

5 46

[libvirt] [jenkins-ci PATCH 0/3] Add Fedora 27, remove Fedora 25
by Andrea Bolognani 20 Nov '17

20 Nov '17

'tis the season. Andrea Bolognani (3): Add Fedora 27 builder Remove Fedora 25 builder guests: Adjust Fedora 26 install URL guests/host_vars/libvirt-fedora-26/install.yml | 2 +- .../install.yml | 2 +- .../main.yml | 0 guests/inventory | 2 +- guests/vars/vault.yml | 90 +++++++++++----------- projects/libosinfo.yaml | 4 +- projects/libvirt-cim.yaml | 2 +- projects/libvirt-glib.yaml | 4 +- projects/libvirt-go-xml.yaml | 2 +- projects/libvirt-go.yaml | 2 +- projects/libvirt-perl.yaml | 4 +- projects/libvirt-python.yaml | 4 +- projects/libvirt-sandbox.yaml | 4 +- projects/libvirt-tck.yaml | 4 +- projects/libvirt.yaml | 6 +- projects/osinfo-db-tools.yaml | 4 +- projects/osinfo-db.yaml | 4 +- projects/virt-manager.yaml | 4 +- projects/virt-viewer.yaml | 4 +- 19 files changed, 74 insertions(+), 74 deletions(-) rename guests/host_vars/{libvirt-fedora-25 => libvirt-fedora-27}/install.yml (74%) rename guests/host_vars/{libvirt-fedora-25 => libvirt-fedora-27}/main.yml (100%) -- 2.14.3

2 6

[libvirt] [PATCH 0/2] tests: commandtest: handle tcmalloc hacking environment
by Nikolay Shirokovskiy 20 Nov '17

20 Nov '17

3 11

[libvirt] [PATCH 0/6] Implement query-dump command
by John Ferlan 20 Nov '17

20 Nov '17

https://bugzilla.redhat.com/show_bug.cgi?id=916061 Details in the patches. Essentially though QEMU 2.6 added the ability to perform the 'dump-guest-memory' via a thread by adding a 'detach' boolean to the command. In order to watch the progress of the thread the 'query-dump' command was added. The query-dump will return just the current status, the total size to be dumped, and the current progress. So using the migrate stats data in the DomainJobInfo we can then save our current progress allowing tools such as virsh to watch that progress. As an added benefit, the dump-guest-memory is now truly asynchronous. John Ferlan (6): qemu: Fix the qemuDumpToFd definition qemu: Alter dump-guest-memory command generation qemu: Add query-dump to the capabilities qemu: Introduce qemuMonitor[JSON]QueryDump qemu: Add new @detach parameter to dump-guest-memory qemu: Allow showing the dump progress for memory only dump src/qemu/qemu_capabilities.c | 4 +- src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_driver.c | 95 ++++++++++++++++++++-- src/qemu/qemu_monitor.c | 25 +++++- src/qemu/qemu_monitor.h | 25 +++++- src/qemu/qemu_monitor_json.c | 91 +++++++++++++++++---- src/qemu/qemu_monitor_json.h | 7 +- .../caps_2.10.0-gicv2.aarch64.xml | 1 + .../caps_2.10.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.10.0.x86_64.xml | 1 + .../caps_2.6.0-gicv2.aarch64.xml | 1 + .../caps_2.6.0-gicv3.aarch64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.6.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.7.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.8.0.x86_64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.ppc64.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.s390x.xml | 1 + tests/qemucapabilitiesdata/caps_2.9.0.x86_64.xml | 1 + tests/qemumonitorjsontest.c | 3 +- 24 files changed, 240 insertions(+), 27 deletions(-) -- 2.13.6

2 8

[libvirt] [PATCH] docs: don't use https in XML namespace URIs
by Daniel P. Berrange 20 Nov '17

20 Nov '17

The XML namespace URI for the QEMU/LXC drivers must use http as the protocol otherwise it won't match the parser's expectations. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- docs/drvlxc.html.in | 2 +- docs/drvqemu.html.in | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/drvlxc.html.in b/docs/drvlxc.html.in index 94c6b82cfd..58a336ed30 100644 --- a/docs/drvlxc.html.in +++ b/docs/drvlxc.html.in @@ -601,7 +601,7 @@ rather than creating new network namespace for the container. In this case privn ignored. <pre> -<domain type='lxc' xmlns:lxc='https://libvirt.org/schemas/domain/lxc/1.0'> +<domain type='lxc' xmlns:lxc='http://libvirt.org/schemas/domain/lxc/1.0'> ... <lxc:namespace> <lxc:sharenet type='netns' value='red'/> diff --git a/docs/drvqemu.html.in b/docs/drvqemu.html.in index e2a0797cfb..cb41b505d7 100644 --- a/docs/drvqemu.html.in +++ b/docs/drvqemu.html.in @@ -546,7 +546,7 @@ $ virsh domxml-to-native qemu-argv demo.xml (Since 0.8.3). In order to use the XML additions, it is necessary to issue an XML namespace request (the special <code>xmlns:name</code> attribute) that - pulls in <code>https://libvirt.org/schemas/domain/qemu/1.0</code>; + pulls in <code>http://libvirt.org/schemas/domain/qemu/1.0</code>; typically, the namespace is given the name of <code>qemu</code>. With the namespace in place, it is then possible to add an element <code><qemu:commandline></code> @@ -566,7 +566,7 @@ $ virsh domxml-to-native qemu-argv demo.xml and optional <code>value</code>.</dd> </dl> Example:<pre> -<domain type='qemu' xmlns:qemu='https://libvirt.org/schemas/domain/qemu/1.0'> +<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>QEmu-fedora-i686</name> <memory>219200</memory> <os> -- 2.14.3

2 1

[libvirt] [PATCH] AppArmor: add rules needed with additional mediation features brought by Linux 4.14.
by intrigeri 19 Nov '17

19 Nov '17

--- examples/apparmor/libvirt-qemu | 2 ++ examples/apparmor/usr.sbin.libvirtd | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/examples/apparmor/libvirt-qemu b/examples/apparmor/libvirt-qemu index b341e31f42..5994a35042 100644 --- a/examples/apparmor/libvirt-qemu +++ b/examples/apparmor/libvirt-qemu @@ -16,6 +16,8 @@ network inet stream, network inet6 stream, + signal (receive) set=("term") peer=/usr/sbin/libvirtd, + /dev/net/tun rw, /dev/kvm rw, /dev/ptmx rw, diff --git a/examples/apparmor/usr.sbin.libvirtd b/examples/apparmor/usr.sbin.libvirtd index 819068ffc3..17b5ee38ff 100644 --- a/examples/apparmor/usr.sbin.libvirtd +++ b/examples/apparmor/usr.sbin.libvirtd @@ -30,6 +30,8 @@ # Needed for vfio capability sys_resource, + mount, + network inet stream, network inet dgram, network inet6 stream, @@ -37,11 +39,18 @@ network packet dgram, network packet raw, + network netlink raw, + network unix dgram, + network unix stream, + ptrace (trace) peer=unconfined, ptrace (trace) peer=/usr/sbin/libvirtd, ptrace (trace) peer=/usr/sbin/dnsmasq, ptrace (trace) peer=libvirt-*, + signal (send) set=("hup") peer=/usr/sbin/dnsmasq, + signal (send) set=("term") peer=libvirt-*, + # Very lenient profile for libvirtd since we want to first focus on confining # the guests. Guests will have a very restricted profile. / r, -- 2.15.0.rc2

5 16