[libvirt] [PATCH 0/4] Add HAP to domain features
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP (hardware assisted paging) to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. I've seen reports [1] of users asking for this capability in the past. Patch 3 is an attempt to advertise the feature in hvm guest capabilities for the Xen hypervisor. I have not found a way to discover the presence of HAP support through the xen tools. Instead, the feature is only advertised on Xen >= 3.3, which sadly won't accommodate distro backports to older Xen versions. Thanks for your comments, Jim [1] http://www.mail-archive.com/fedora-xen@redhat.com/msg02680.html Jim Fehlig (4): Add HAP to virDomainFeature enum Add support for HAP feature to xen drivers Add HAP to xen hypervisor capabilities Document HAP domain feature docs/formatdomain.html.in | 5 +++++ docs/schemas/domain.rng | 7 ++++++- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + src/xen/xen_hypervisor.c | 10 ++++++++++ src/xen/xend_internal.c | 4 ++++ src/xen/xm_internal.c | 8 ++++++++ src/xenapi/xenapi_driver.c | 2 ++ src/xenapi/xenapi_utils.c | 2 ++ 9 files changed, 40 insertions(+), 2 deletions(-) -- 1.7.3.1
Extend the virDomainFeature enumeration to include HAP (hardware assisted paging) feature. Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+), 1 deletions(-) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index c857a89..8392c8e 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -75,7 +75,8 @@ VIR_ENUM_IMPL(virDomainBoot, VIR_DOMAIN_BOOT_LAST, VIR_ENUM_IMPL(virDomainFeature, VIR_DOMAIN_FEATURE_LAST, "acpi", "apic", - "pae") + "pae", + "hap") VIR_ENUM_IMPL(virDomainLifecycle, VIR_DOMAIN_LIFECYCLE_LAST, "destroy", diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h index a459a22..6a8ec64 100644 --- a/src/conf/domain_conf.h +++ b/src/conf/domain_conf.h @@ -730,6 +730,7 @@ enum virDomainFeature { VIR_DOMAIN_FEATURE_ACPI, VIR_DOMAIN_FEATURE_APIC, VIR_DOMAIN_FEATURE_PAE, + VIR_DOMAIN_FEATURE_HAP, VIR_DOMAIN_FEATURE_LAST }; -- 1.7.3.1
On Wed, Jan 05, 2011 at 04:32:39PM -0700, Jim Fehlig wrote:
Extend the virDomainFeature enumeration to include HAP (hardware assisted paging) feature.
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+), 1 deletions(-)
ACK Daniel
Daniel P. Berrange wrote:
On Wed, Jan 05, 2011 at 04:32:39PM -0700, Jim Fehlig wrote:
Extend the virDomainFeature enumeration to include HAP (hardware assisted paging) feature.
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+), 1 deletions(-)
ACK
Thanks Daniel. I've pushed this series now. Regards, Jim
On 01/13/2011 10:20 AM, Jim Fehlig wrote:
Daniel P. Berrange wrote:
On Wed, Jan 05, 2011 at 04:32:39PM -0700, Jim Fehlig wrote:
Extend the virDomainFeature enumeration to include HAP (hardware assisted paging) feature.
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+), 1 deletions(-)
ACK
Thanks Daniel. I've pushed this series now.
Unfortunately, this series is making 'make check' fail, like this: 57) Xen XM-2-XML Parse pci-devs ... Expect [l] Actual [hap = 0 l] ... FAILED 58) Xen XM-2-XML Format pci-devs ... OK FAIL: xmconfigtest Can you work on a followup patch to adjust all the test .xm files to account for the new "hap = 0" line? -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
Eric Blake wrote:
On 01/13/2011 10:20 AM, Jim Fehlig wrote:
Daniel P. Berrange wrote:
On Wed, Jan 05, 2011 at 04:32:39PM -0700, Jim Fehlig wrote:
Extend the virDomainFeature enumeration to include HAP (hardware assisted paging) feature.
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. --- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + 2 files changed, 3 insertions(+), 1 deletions(-)
ACK
Thanks Daniel. I've pushed this series now.
Unfortunately, this series is making 'make check' fail, like this:
Hmm, a bug in my test scripts caused me to miss this. Looking at logs make check actually failed for me here: CC secaatest.o cc1: warnings being treated as errors secaatest.c: In function 'main': secaatest.c:18:5: error: implicit declaration of function 'virSecurityDriverStartup' [-Wimplicit-function-declaration] secaatest.c:18:5: error: nested extern declaration of 'virSecurityDriverStartup' [-Wnested-externs] secaatest.c:28:13: error: expected expression before 'virSecurityDriverGetModel' secaatest.c:36:11: error: expected expression before 'virSecurityDriverGetDOI'
57) Xen XM-2-XML Parse pci-devs ... Expect [l] Actual [hap = 0 l]
... FAILED 58) Xen XM-2-XML Format pci-devs ... OK FAIL: xmconfigtest
Can you work on a followup patch to adjust all the test .xm files to account for the new "hap = 0" line?
Yes. Regards, Jim
On 01/14/2011 09:05 AM, Jim Fehlig wrote:
Hmm, a bug in my test scripts caused me to miss this. Looking at logs make check actually failed for me here:
CC secaatest.o cc1: warnings being treated as errors secaatest.c: In function 'main':
Yep, secaatest is an independent failure, brought up in this thread: https://www.redhat.com/archives/libvir-list/2011-January/msg00615.html
57) Xen XM-2-XML Parse pci-devs ... Expect [l] Actual [hap = 0 l]
... FAILED 58) Xen XM-2-XML Format pci-devs ... OK FAIL: xmconfigtest
Can you work on a followup patch to adjust all the test .xm files to account for the new "hap = 0" line?
Yes.
Thanks. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
xen-unstable c/s 16931 introduced a per-domain setting for hvm guests to enable/disable hardware assisted paging. If disabled, software techniques such as shadow page tables are used. If enabled, and the feature exists in underlying hardware, hardware support for paging is used. This provides implementation for mapping HAP setting to/from domxml/native formats in xen drivers. --- src/xen/xend_internal.c | 4 ++++ src/xen/xm_internal.c | 8 ++++++++ src/xenapi/xenapi_driver.c | 2 ++ src/xenapi/xenapi_utils.c | 2 ++ 4 files changed, 16 insertions(+), 0 deletions(-) diff --git a/src/xen/xend_internal.c b/src/xen/xend_internal.c index 6ce0c3f..d3633ee 100644 --- a/src/xen/xend_internal.c +++ b/src/xen/xend_internal.c @@ -2210,6 +2210,8 @@ xenDaemonParseSxpr(virConnectPtr conn, def->features |= (1 << VIR_DOMAIN_FEATURE_APIC); if (sexpr_int(root, "domain/image/hvm/pae")) def->features |= (1 << VIR_DOMAIN_FEATURE_PAE); + if (sexpr_int(root, "domain/image/hvm/hap")) + def->features |= (1 << VIR_DOMAIN_FEATURE_HAP); /* Old XenD only allows localtime here for HVM */ if (sexpr_int(root, "domain/image/hvm/localtime")) @@ -5923,6 +5925,8 @@ xenDaemonFormatSxpr(virConnectPtr conn, virBufferAddLit(&buf, "(apic 1)"); if (def->features & (1 << VIR_DOMAIN_FEATURE_PAE)) virBufferAddLit(&buf, "(pae 1)"); + if (def->features & (1 << VIR_DOMAIN_FEATURE_HAP)) + virBufferAddLit(&buf, "(hap 1)"); virBufferAddLit(&buf, "(usb 1)"); diff --git a/src/xen/xm_internal.c b/src/xen/xm_internal.c index 4d6b41b..ec618aa 100644 --- a/src/xen/xm_internal.c +++ b/src/xen/xm_internal.c @@ -830,6 +830,10 @@ xenXMDomainConfigParse(virConnectPtr conn, virConfPtr conf) { goto cleanup; else if (val) def->features |= (1 << VIR_DOMAIN_FEATURE_APIC); + if (xenXMConfigGetBool(conf, "hap", &val, 0) < 0) + goto cleanup; + else if (val) + def->features |= (1 << VIR_DOMAIN_FEATURE_HAP); } if (xenXMConfigGetBool(conf, "localtime", &vmlocaltime, 0) < 0) goto cleanup; @@ -2409,6 +2413,10 @@ virConfPtr xenXMDomainConfigFormat(virConnectPtr conn, (1 << VIR_DOMAIN_FEATURE_APIC)) ? 1 : 0) < 0) goto no_memory; + if (xenXMConfigSetInt(conf, "hap", + (def->features & + (1 << VIR_DOMAIN_FEATURE_HAP)) ? 1 : 0) < 0) + goto no_memory; if (def->clock.offset == VIR_DOMAIN_CLOCK_OFFSET_LOCALTIME) { if (def->clock.data.timezone) { diff --git a/src/xenapi/xenapi_driver.c b/src/xenapi/xenapi_driver.c index 6fff276..fffa617 100644 --- a/src/xenapi/xenapi_driver.c +++ b/src/xenapi/xenapi_driver.c @@ -1358,6 +1358,8 @@ xenapiDomainDumpXML (virDomainPtr dom, int flags ATTRIBUTE_UNUSED) defPtr->features = defPtr->features | (1<<VIR_DOMAIN_FEATURE_APIC); else if (STREQ(result->contents[i].key, "pae")) defPtr->features = defPtr->features | (1<<VIR_DOMAIN_FEATURE_PAE); + else if (STREQ(result->contents[i].key, "hap")) + defPtr->features = defPtr->features | (1<<VIR_DOMAIN_FEATURE_HAP); } } xen_string_string_map_free(result); diff --git a/src/xenapi/xenapi_utils.c b/src/xenapi/xenapi_utils.c index a7e2a4b..2ec5f9e 100644 --- a/src/xenapi/xenapi_utils.c +++ b/src/xenapi/xenapi_utils.c @@ -529,6 +529,8 @@ createVMRecordFromXml (virConnectPtr conn, virDomainDefPtr def, allocStringMap(&strings, (char *)"apic", (char *)"true"); if (def->features & (1 << VIR_DOMAIN_FEATURE_PAE)) allocStringMap(&strings, (char *)"pae", (char *)"true"); + if (def->features & (1 << VIR_DOMAIN_FEATURE_HAP)) + allocStringMap(&strings, (char *)"hap", (char *)"true"); } if (strings != NULL) (*record)->platform = strings; -- 1.7.3.1
On Wed, Jan 05, 2011 at 04:32:40PM -0700, Jim Fehlig wrote:
xen-unstable c/s 16931 introduced a per-domain setting for hvm guests to enable/disable hardware assisted paging. If disabled, software techniques such as shadow page tables are used. If enabled, and the feature exists in underlying hardware, hardware support for paging is used.
This provides implementation for mapping HAP setting to/from domxml/native formats in xen drivers. --- src/xen/xend_internal.c | 4 ++++ src/xen/xm_internal.c | 8 ++++++++ src/xenapi/xenapi_driver.c | 2 ++ src/xenapi/xenapi_utils.c | 2 ++ 4 files changed, 16 insertions(+), 0 deletions(-)
ACK Daniel
xen-unstable c/s 16931 introduced a per-domain setting for hvm guests to enable/disable hardware assisted paging. If disabled, software techniques such as shadow page tables are used. If enabled, and the feature exists in underlying hardware, hardware support for paging is used. Xen does not provide a mechanism to discover the HAP capability, so we advertise its availability for hvm guests on Xen >= 3.3. --- src/xen/xen_hypervisor.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diff --git a/src/xen/xen_hypervisor.c b/src/xen/xen_hypervisor.c index ead8ee9..229ef3d 100644 --- a/src/xen/xen_hypervisor.c +++ b/src/xen/xen_hypervisor.c @@ -2321,6 +2321,16 @@ xenHypervisorBuildCapabilities(virConnectPtr conn, hv_minor > 0 ? 0 : 1)) == NULL) goto no_memory; + + /* Xen 3.3.x and beyond supports enabling/disabling + * hardware assisted paging. Default is off. + */ + if ((hv_major == 3 && hv_minor >= 3) || (hv_major > 3)) + if (virCapabilitiesAddGuestFeature(guest, + "hap", + 0, + 1) == NULL) + goto no_memory; } } -- 1.7.3.1
On Wed, Jan 05, 2011 at 04:32:41PM -0700, Jim Fehlig wrote:
xen-unstable c/s 16931 introduced a per-domain setting for hvm guests to enable/disable hardware assisted paging. If disabled, software techniques such as shadow page tables are used. If enabled, and the feature exists in underlying hardware, hardware support for paging is used.
Xen does not provide a mechanism to discover the HAP capability, so we advertise its availability for hvm guests on Xen >= 3.3. --- src/xen/xen_hypervisor.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
diff --git a/src/xen/xen_hypervisor.c b/src/xen/xen_hypervisor.c index ead8ee9..229ef3d 100644 --- a/src/xen/xen_hypervisor.c +++ b/src/xen/xen_hypervisor.c @@ -2321,6 +2321,16 @@ xenHypervisorBuildCapabilities(virConnectPtr conn, hv_minor > 0 ? 0 : 1)) == NULL) goto no_memory; + + /* Xen 3.3.x and beyond supports enabling/disabling + * hardware assisted paging. Default is off. + */ + if ((hv_major == 3 && hv_minor >= 3) || (hv_major > 3)) + if (virCapabilitiesAddGuestFeature(guest, + "hap", + 0, + 1) == NULL) + goto no_memory; } }
ACK Daniel
Add HAP feature to schema and documentation. --- docs/formatdomain.html.in | 5 +++++ docs/schemas/domain.rng | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index e9fcea1..0b1a851 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -437,6 +437,7 @@ <pae/> <acpi/> <apic/> + <hap/> </features> ...</pre> @@ -456,6 +457,10 @@ <dd>ACPI is useful for power management, for example, with KVM guests it is required for graceful shutdown to work. </dd> + <dt><code>hap</code></dt> + <dd>Enable use of Hardware Assisted Paging if available in + the hardware. + </dd> </dl> <h3><a name="elementsTime">Time keeping</a></h3> diff --git a/docs/schemas/domain.rng b/docs/schemas/domain.rng index eee9864..f043234 100644 --- a/docs/schemas/domain.rng +++ b/docs/schemas/domain.rng @@ -1741,7 +1741,7 @@ </element> </define> <!-- - A set of optional features: PAE, APIC and ACPI support + A set of optional features: PAE, APIC, ACPI, and HAP support --> <define name="features"> <optional> @@ -1762,6 +1762,11 @@ <empty/> </element> </optional> + <optional> + <element name="hap"> + <empty/> + </element> + </optional> </interleave> </element> </optional> -- 1.7.3.1
On Wed, Jan 05, 2011 at 04:32:42PM -0700, Jim Fehlig wrote:
Add HAP feature to schema and documentation. --- docs/formatdomain.html.in | 5 +++++ docs/schemas/domain.rng | 7 ++++++- 2 files changed, 11 insertions(+), 1 deletions(-)
ACK Daniel
Any comments on this patch set? In hindsight, I probably should have sent this as a single patch but I wanted to break out #3 since I wasn't sure about the capabilities advertisement. In doing so I went a little overboard ... Thanks, Jim Jim Fehlig wrote:
Hardware features such as Extended Page Table and Nested Page Table augment hypervisor software techniques such as shadow page table. Adding HAP (hardware assisted paging) to the virDomainFeature enumeration allows users to select between hardware and software memory management mechanisms for their guests. I've seen reports [1] of users asking for this capability in the past.
Patch 3 is an attempt to advertise the feature in hvm guest capabilities for the Xen hypervisor. I have not found a way to discover the presence of HAP support through the xen tools. Instead, the feature is only advertised on Xen >= 3.3, which sadly won't accommodate distro backports to older Xen versions.
Thanks for your comments, Jim
[1] http://www.mail-archive.com/fedora-xen@redhat.com/msg02680.html
Jim Fehlig (4): Add HAP to virDomainFeature enum Add support for HAP feature to xen drivers Add HAP to xen hypervisor capabilities Document HAP domain feature
docs/formatdomain.html.in | 5 +++++ docs/schemas/domain.rng | 7 ++++++- src/conf/domain_conf.c | 3 ++- src/conf/domain_conf.h | 1 + src/xen/xen_hypervisor.c | 10 ++++++++++ src/xen/xend_internal.c | 4 ++++ src/xen/xm_internal.c | 8 ++++++++ src/xenapi/xenapi_driver.c | 2 ++ src/xenapi/xenapi_utils.c | 2 ++ 9 files changed, 40 insertions(+), 2 deletions(-)
participants (3)
-
Daniel P. Berrange -
Eric Blake -
Jim Fehlig