[libvirt] [PATCH v1 0/3] Adding the 'ccf-assist' pSeries capability
Hi, This short series exposes, implements and documents a pSeries guest capability called Count Cache Flush Assist (ccf-assist), which was added in the Linux kernel since 5.1 and in QEMU since 4.0.0. The reason why this capability needs to exposed via Libvirt is because it is defaulted to 'off' in QEMU to not break migration compatibility in the Power 9 processor class. However, there is a performance gain in activating it, thus the user might want to choose less migration options for more power. More details can be found in the commit message of patch 1. Daniel Henrique Barboza (3): qemu: Add capability for the ccf-assist pSeries feature qemu: Implement the ccf-assist pSeries feature news: Update for the ccf-assist pSeries feature docs/formatdomain.html.in | 9 +++++++++ docs/news.xml | 9 +++++++++ docs/schemas/domaincommon.rng | 5 +++++ src/conf/domain_conf.c | 4 ++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 20 +++++++++++++++++++ src/qemu/qemu_domain.c | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qemuxml2argvdata/pseries-features.args | 2 +- tests/qemuxml2argvdata/pseries-features.xml | 1 + tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmloutdata/pseries-features.xml | 1 + tests/qemuxml2xmltest.c | 1 + 15 files changed, 58 insertions(+), 1 deletion(-) -- 2.21.0
Linux kernel 5.1 added a new PPC KVM capability named KVM_PPC_CPU_CHAR_BCCTR_FLUSH_ASSIST, which is exposed to the QEMU guest since QEMU commit 8ff43ee404d under a new sPAPR capability called SPAPR_CAP_CCF_ASSIST. This cap indicates whether the processor supports hardware acceleration for the count cache flush workaround, which is a software workaround that flushes the count cache on context switch. If the processor has this hardware acceleration, the software flush can be shortened, resulting in performance gain. This hardware acceleration is defaulted to 'off' in QEMU. The reason is that earlier versions of the Power 9 processor didn't support it (it is available on Power 9 DD2.3 and newer), and defaulting this option to 'on' would break migration compatibility between the Power 9 processor class. However, the user running a P9 DD2.3+ hypervisor might want to create guests with ccf-assist=on, accepting the downside of only being able to migrate them only between other P9 DD2.3+ hosts running upstream kernel 5.1+, to get a performance boost. This patch adds this new capability to Libvirt, with the name of QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST. Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + 3 files changed, 4 insertions(+) diff --git a/src/qemu/qemu_capabilities.c b/src/qemu/qemu_capabilities.c index 9b19930964..17d9edee8f 100644 --- a/src/qemu/qemu_capabilities.c +++ b/src/qemu/qemu_capabilities.c @@ -539,6 +539,7 @@ VIR_ENUM_IMPL(virQEMUCaps, "migration-file-drop-cache", "net-socket-dgram", "dbus-vmstate", + "machine.pseries.cap-ccf-assist", ); @@ -1430,6 +1431,7 @@ static struct virQEMUCapsStringFlags virQEMUCapsMachinePropsPSeries[] = { { "cap-hpt-max-page-size", QEMU_CAPS_MACHINE_PSERIES_CAP_HPT_MAX_PAGE_SIZE }, { "cap-htm", QEMU_CAPS_MACHINE_PSERIES_CAP_HTM }, { "cap-nested-hv", QEMU_CAPS_MACHINE_PSERIES_CAP_NESTED_HV }, + { "cap-ccf-assist", QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST }, }; static struct virQEMUCapsStringFlags virQEMUCapsMachinePropsVirt[] = { diff --git a/src/qemu/qemu_capabilities.h b/src/qemu/qemu_capabilities.h index 54f91151c6..2747a8b3d8 100644 --- a/src/qemu/qemu_capabilities.h +++ b/src/qemu/qemu_capabilities.h @@ -520,6 +520,7 @@ typedef enum { /* virQEMUCapsFlags grouping marker for syntax-check */ QEMU_CAPS_MIGRATION_FILE_DROP_CACHE, /* migration with disk cache on is safe for type='file' disks */ QEMU_CAPS_NET_SOCKET_DGRAM, /* -net socket,fd= with dgram socket */ QEMU_CAPS_DBUS_VMSTATE, /* -object dbus-vmstate */ + QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST, /* -machine pseries.cap-ccf-assist */ QEMU_CAPS_LAST /* this must always be the last item */ } virQEMUCapsFlags; diff --git a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml b/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml index 73859becab..a88c8f327f 100644 --- a/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml +++ b/tests/qemucapabilitiesdata/caps_4.0.0.ppc64.xml @@ -171,6 +171,7 @@ <flag name='bochs-display'/> <flag name='migration-file-drop-cache'/> <flag name='net-socket-dgram'/> + <flag name='machine.pseries.cap-ccf-assist'/> <version>4000000</version> <kvmVersion>0</kvmVersion> <microcodeVersion>42900758</microcodeVersion> -- 2.21.0
This patch adds the implementation of the ccf-assist pSeries feature, based on the QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST capability that was added in the previous patch. Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> --- docs/formatdomain.html.in | 9 +++++++++ docs/schemas/domaincommon.rng | 5 +++++ src/conf/domain_conf.c | 4 ++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_command.c | 20 +++++++++++++++++++ src/qemu/qemu_domain.c | 1 + tests/qemuxml2argvdata/pseries-features.args | 2 +- tests/qemuxml2argvdata/pseries-features.xml | 1 + tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmloutdata/pseries-features.xml | 1 + tests/qemuxml2xmltest.c | 1 + 11 files changed, 45 insertions(+), 1 deletion(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 86a5261e47..109db25889 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -2059,6 +2059,7 @@ <tseg unit='MiB'>48</tseg> </smm> <htm state='on'/> + <ccf-assist state='on'/> <msrs unknown='ignore'/> </features> ...</pre> @@ -2357,6 +2358,14 @@ will not be ignored. <span class="since">Since 5.1.0</span> (bhyve only) </dd> + <dt><code>ccf-assist</code></dt> + <dd>Configure ccf-assist (Count Cache Flush Assist) availability for + pSeries guests. + Possible values for the <code>state</code> attribute + are <code>on</code> and <code>off</code>. If the attribute is not + defined, the hypervisor default will be used. + <span class="since">Since 5.8.0</span> (QEMU/KVM only) + </dd> </dl> <h3><a id="elementsTime">Time keeping</a></h3> diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index cae3be639e..7f55ce1fd2 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -5107,6 +5107,11 @@ <optional> <ref name="msrs"/> </optional> + <optional> + <element name="ccf-assist"> + <ref name="featurestate"/> + </element> + </optional> </interleave> </element> </optional> diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 96e9223e21..1cd18903bd 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -172,6 +172,7 @@ VIR_ENUM_IMPL(virDomainFeature, "htm", "nested-hv", "msrs", + "ccf-assist", ); VIR_ENUM_IMPL(virDomainCapabilitiesPolicy, @@ -20309,6 +20310,7 @@ virDomainDefParseXML(xmlDocPtr xml, case VIR_DOMAIN_FEATURE_HTM: case VIR_DOMAIN_FEATURE_NESTED_HV: + case VIR_DOMAIN_FEATURE_CCF_ASSIST: if (!(tmp = virXMLPropString(nodes[i], "state"))) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("missing state attribute '%s' of feature '%s'"), @@ -22550,6 +22552,7 @@ virDomainDefFeaturesCheckABIStability(virDomainDefPtr src, case VIR_DOMAIN_FEATURE_VMCOREINFO: case VIR_DOMAIN_FEATURE_HTM: case VIR_DOMAIN_FEATURE_NESTED_HV: + case VIR_DOMAIN_FEATURE_CCF_ASSIST: if (src->features[i] != dst->features[i]) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("State of feature '%s' differs: " @@ -28111,6 +28114,7 @@ virDomainDefFormatFeatures(virBufferPtr buf, case VIR_DOMAIN_FEATURE_VMPORT: case VIR_DOMAIN_FEATURE_HTM: case VIR_DOMAIN_FEATURE_NESTED_HV: + case VIR_DOMAIN_FEATURE_CCF_ASSIST: switch ((virTristateSwitch) def->features[i]) { case VIR_TRISTATE_SWITCH_LAST: case VIR_TRISTATE_SWITCH_ABSENT: diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index 82631ecb07..6a9121f73b 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -1740,6 +1740,7 @@ typedef enum { VIR_DOMAIN_FEATURE_HTM, VIR_DOMAIN_FEATURE_NESTED_HV, VIR_DOMAIN_FEATURE_MSRS, + VIR_DOMAIN_FEATURE_CCF_ASSIST, VIR_DOMAIN_FEATURE_LAST } virDomainFeature; diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c index f795f2e987..a52d9d2198 100644 --- a/src/qemu/qemu_command.c +++ b/src/qemu/qemu_command.c @@ -7377,6 +7377,26 @@ qemuBuildMachineCommandLine(virCommandPtr cmd, virBufferAsprintf(&buf, ",cap-nested-hv=%s", str); } + if (def->features[VIR_DOMAIN_FEATURE_CCF_ASSIST] != VIR_TRISTATE_SWITCH_ABSENT) { + const char *str; + + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("ccf-assist configuration is not supported by this " + "QEMU binary")); + return -1; + } + + str = virTristateSwitchTypeToString(def->features[VIR_DOMAIN_FEATURE_CCF_ASSIST]); + if (!str) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", + _("Invalid setting for ccf-assist state")); + return -1; + } + + virBufferAsprintf(&buf, ",cap-ccf-assist=%s", str); + } + if (cpu && cpu->model && cpu->mode == VIR_CPU_MODE_HOST_MODEL && qemuDomainIsPSeries(def) && diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c index bd247628cb..b7b9c4561a 100644 --- a/src/qemu/qemu_domain.c +++ b/src/qemu/qemu_domain.c @@ -4655,6 +4655,7 @@ qemuDomainDefValidateFeatures(const virDomainDef *def, case VIR_DOMAIN_FEATURE_HPT: case VIR_DOMAIN_FEATURE_HTM: case VIR_DOMAIN_FEATURE_NESTED_HV: + case VIR_DOMAIN_FEATURE_CCF_ASSIST: if (def->features[i] != VIR_TRISTATE_SWITCH_ABSENT && !qemuDomainIsPSeries(def)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, diff --git a/tests/qemuxml2argvdata/pseries-features.args b/tests/qemuxml2argvdata/pseries-features.args index 9fde54b37a..7aa357a54e 100644 --- a/tests/qemuxml2argvdata/pseries-features.args +++ b/tests/qemuxml2argvdata/pseries-features.args @@ -11,7 +11,7 @@ QEMU_AUDIO_DRV=none \ -name guest \ -S \ -machine pseries,accel=tcg,usb=off,dump-guest-core=off,resize-hpt=required,\ -cap-hpt-max-page-size=1048576k,cap-htm=on,cap-nested-hv=off \ +cap-hpt-max-page-size=1048576k,cap-htm=on,cap-nested-hv=off,cap-ccf-assist=on \ -m 512 \ -realtime mlock=off \ -smp 1,sockets=1,cores=1,threads=1 \ diff --git a/tests/qemuxml2argvdata/pseries-features.xml b/tests/qemuxml2argvdata/pseries-features.xml index 6f7d32b065..8ccc1b73d8 100644 --- a/tests/qemuxml2argvdata/pseries-features.xml +++ b/tests/qemuxml2argvdata/pseries-features.xml @@ -12,6 +12,7 @@ </hpt> <htm state='on'/> <nested-hv state='off'/> + <ccf-assist state='on'/> </features> <devices> <emulator>/usr/bin/qemu-system-ppc64</emulator> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index abff3d1c61..4667b81d72 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1937,6 +1937,7 @@ mymain(void) QEMU_CAPS_MACHINE_PSERIES_CAP_HPT_MAX_PAGE_SIZE, QEMU_CAPS_MACHINE_PSERIES_CAP_HTM, QEMU_CAPS_MACHINE_PSERIES_CAP_NESTED_HV, + QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST, QEMU_CAPS_MACHINE_PSERIES_RESIZE_HPT); DO_TEST_FAILURE("pseries-features", QEMU_CAPS_DEVICE_SPAPR_PCI_HOST_BRIDGE); diff --git a/tests/qemuxml2xmloutdata/pseries-features.xml b/tests/qemuxml2xmloutdata/pseries-features.xml index 7e12bc9c03..a5df840394 100644 --- a/tests/qemuxml2xmloutdata/pseries-features.xml +++ b/tests/qemuxml2xmloutdata/pseries-features.xml @@ -14,6 +14,7 @@ </hpt> <htm state='on'/> <nested-hv state='off'/> + <ccf-assist state='on'/> </features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index d5c66d8791..b9364f942f 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -608,6 +608,7 @@ mymain(void) QEMU_CAPS_MACHINE_PSERIES_CAP_HPT_MAX_PAGE_SIZE, QEMU_CAPS_MACHINE_PSERIES_CAP_HTM, QEMU_CAPS_MACHINE_PSERIES_CAP_NESTED_HV, + QEMU_CAPS_MACHINE_PSERIES_CAP_CCF_ASSIST, QEMU_CAPS_MACHINE_PSERIES_RESIZE_HPT); DO_TEST("pseries-serial-native", -- 2.21.0
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> --- docs/news.xml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/news.xml b/docs/news.xml index f6aee74884..c2e95227c6 100644 --- a/docs/news.xml +++ b/docs/news.xml @@ -41,6 +41,15 @@ <libvirt> <release version="v5.8.0" date="unreleased"> <section title="New features"> + <change> + <summary> + qemu: Implement the ccf-assist pSeries feature + </summary> + <description> + Users can now decide whether ccf-assist (Count Cache Flush Assist) + support should be available to pSeries guests. + </description> + </change> </section> <section title="Removed features"> <change> -- 2.21.0
Ping On 9/16/19 3:37 PM, Daniel Henrique Barboza wrote:
Hi,
This short series exposes, implements and documents a pSeries guest capability called Count Cache Flush Assist (ccf-assist), which was added in the Linux kernel since 5.1 and in QEMU since 4.0.0.
The reason why this capability needs to exposed via Libvirt is because it is defaulted to 'off' in QEMU to not break migration compatibility in the Power 9 processor class. However, there is a performance gain in activating it, thus the user might want to choose less migration options for more power. More details can be found in the commit message of patch 1.
Daniel Henrique Barboza (3): qemu: Add capability for the ccf-assist pSeries feature qemu: Implement the ccf-assist pSeries feature news: Update for the ccf-assist pSeries feature
docs/formatdomain.html.in | 9 +++++++++ docs/news.xml | 9 +++++++++ docs/schemas/domaincommon.rng | 5 +++++ src/conf/domain_conf.c | 4 ++++ src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 + src/qemu/qemu_command.c | 20 +++++++++++++++++++ src/qemu/qemu_domain.c | 1 + .../qemucapabilitiesdata/caps_4.0.0.ppc64.xml | 1 + tests/qemuxml2argvdata/pseries-features.args | 2 +- tests/qemuxml2argvdata/pseries-features.xml | 1 + tests/qemuxml2argvtest.c | 1 + tests/qemuxml2xmloutdata/pseries-features.xml | 1 + tests/qemuxml2xmltest.c | 1 + 15 files changed, 58 insertions(+), 1 deletion(-)
participants (1)
-
Daniel Henrique Barboza