8:07 a.m.
From: Wim ten Have <wim.ten.have(a)oracle.com>
This patch enhances host-passthrough capability to advertise the
required vendor CPU virtualization feature which will be used to
enable 'nestedhvm' in the libxl driver.
For Intel virtualization (VT-x)
..
<cpu mode='host-passthrough'>
<feature policy='require' name='vmx'/>
</cpu>
For AMD virtualization (AMD-V)
..
<cpu mode='host-passthrough'>
<feature policy='require' name='svm'/>
</cpu>
Joao Martins (2):
libxl: set nestedhvm when vmx|svm is required
xenconfig: add conversion of domxml nestedhvm
Wim ten Have (1):
xlconfigtest: add test and data for 'nestedhvm' support
src/libxl/libxl_conf.c | 9 ++++
src/xenconfig/xen_xl.c | 41 +++++++++++++++++
tests/xlconfigdata/test-fullvirt-nestedhvm.cfg | 26 +++++++++++
tests/xlconfigdata/test-fullvirt-nestedhvm.xml | 61 ++++++++++++++++++++++++++
tests/xlconfigtest.c | 1 +
5 files changed, 138 insertions(+)
create mode 100644 tests/xlconfigdata/test-fullvirt-nestedhvm.cfg
create mode 100644 tests/xlconfigdata/test-fullvirt-nestedhvm.xml
--
2.9.3
8:07 a.m.
New subject: [libvirt] [PATCH v1 1/3] libxl: set nestedhvm when vmx|svm is required
From: Joao Martins <joao.m.martins(a)oracle.com>
nestedhvm is the option on Xen 4.4+ which enables nested
virtualization on Xen. We set this field when one of the
following features is included in a host-passthrough CPU
element:
<feature policy='require' name='vmx'/>
or:
<feature policy='require' name='svm'/>
Signed-off-by: Joao Martins <joao.m.martins(a)oracle.com>
Signed-off-by: Wim ten Have <wim.ten.have(a)oracle.com>
---
src/libxl/libxl_conf.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/src/libxl/libxl_conf.c b/src/libxl/libxl_conf.c
index f5b788b..8c4c511 100644
--- a/src/libxl/libxl_conf.c
+++ b/src/libxl/libxl_conf.c
@@ -374,6 +374,15 @@ libxlMakeDomBuildInfo(virDomainDefPtr def,
def->features[VIR_DOMAIN_FEATURE_ACPI] ==
VIR_TRISTATE_SWITCH_ON);
+ if (def->cpu && def->cpu->nfeatures) {
+ for (i = 0; i < def->cpu->nfeatures; i++) {
+ if (def->cpu->features[i].policy == VIR_CPU_FEATURE_REQUIRE
&&
+ (STREQ(def->cpu->features[i].name, "vmx") ||
+ STREQ(def->cpu->features[i].name, "svm")))
+ libxl_defbool_set(&b_info->u.hvm.nested_hvm, true);
+ }
+ }
+
if (def->nsounds > 0) {
/*
* Use first sound device. man xl.cfg(5) describes soundhw as
--
2.9.3
8:07 a.m.
New subject: [libvirt] [PATCH v1 2/3] xenconfig: add conversion of domxml nestedhvm
From: Joao Martins <joao.m.martins(a)oracle.com>
In other words, on configurations containing (within its
CPU element) one of the following for a host-passthrough
mode:
<feature policy='require' name='vmx'/>
<feature policy='require' name='svm'/>
It will then generate (or parse) for nestedhvm=1 in/from xl format.
Signed-off-by: Joao Martins <joao.m.martins(a)oracle.com>
Signed-off-by: Wim ten Have <wim.ten.have(a)oracle.com>
---
src/xenconfig/xen_xl.c | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/src/xenconfig/xen_xl.c b/src/xenconfig/xen_xl.c
index 74f68b3..738dcd0 100644
--- a/src/xenconfig/xen_xl.c
+++ b/src/xenconfig/xen_xl.c
@@ -106,6 +106,7 @@ xenParseXLOS(virConfPtr conf, virDomainDefPtr def, virCapsPtr caps)
if (def->os.type == VIR_DOMAIN_OSTYPE_HVM) {
const char *bios;
const char *boot;
+ int val;
if (xenConfigGetString(conf, "bios", &bios, NULL) < 0)
return -1;
@@ -164,6 +165,35 @@ xenParseXLOS(virConfPtr conf, virDomainDefPtr def, virCapsPtr caps)
}
def->os.nBootDevs++;
}
+
+ if (xenConfigGetBool(conf, "nestedhvm", &val, 0) < 0) {
+ return -1;
+ } else if (val) {
+ virCPUDefPtr cpu = NULL;
+
+ if (VIR_ALLOC(cpu) < 0)
+ return -1;
+
+ if (VIR_ALLOC_N(cpu->features, 1) < 0)
+ goto cleanup;
+
+ cpu->features[0].policy = VIR_CPU_FEATURE_REQUIRE;
+ if (VIR_STRDUP(cpu->features[0].name, "vmx") < 0)
+ goto cleanup;
+
+ cpu->nfeatures = cpu->nfeatures_max = 1;
+ cpu->mode = VIR_CPU_MODE_HOST_PASSTHROUGH;
+ cpu->type = VIR_CPU_TYPE_GUEST;
+ def->cpu = cpu;
+ cpu = NULL;
+
+ cleanup:
+ if (cpu) {
+ VIR_FREE(cpu->features);
+ VIR_FREE(cpu);
+ return -1;
+ }
+ }
} else {
if (xenConfigCopyStringOpt(conf, "bootloader",
&def->os.bootloader) < 0)
return -1;
@@ -897,6 +927,17 @@ xenFormatXLOS(virConfPtr conf, virDomainDefPtr def)
if (xenConfigSetString(conf, "boot", boot) < 0)
return -1;
+ if (def->cpu && def->cpu->nfeatures) {
+ for (i = 0; i < def->cpu->nfeatures; i++) {
+ if (def->cpu->features[i].policy == VIR_CPU_FEATURE_REQUIRE
&&
+ (STREQ(def->cpu->features[i].name, "vmx") ||
+ STREQ(def->cpu->features[i].name, "svm")))
+ if (xenConfigSetInt(conf, "nestedhvm", 1) < 0)
+ return -1;
+ }
+ }
+
+
/* XXX floppy disks */
} else {
if (def->os.bootloader &&
--
2.9.3
8:07 a.m.
New subject: [libvirt] [PATCH v1 3/3] xlconfigtest: add test and data for 'nestedhvm' support
From: Wim ten Have <wim.ten.have(a)oracle.com>
Configurations containing (within its CPU element) following:
Intel virtualization (VT-x):
<feature policy='require' name='vmx'/>
AMD virtualization (AMD-V):
<feature policy='require' name='svm'/>
generate nestedhvm=1 in/from xl format.
Signed-off-by: Wim ten Have <wim.ten.have(a)oracle.com>
---
tests/xlconfigdata/test-fullvirt-nestedhvm.cfg | 26 +++++++++++
tests/xlconfigdata/test-fullvirt-nestedhvm.xml | 61 ++++++++++++++++++++++++++
tests/xlconfigtest.c | 1 +
3 files changed, 88 insertions(+)
create mode 100644 tests/xlconfigdata/test-fullvirt-nestedhvm.cfg
create mode 100644 tests/xlconfigdata/test-fullvirt-nestedhvm.xml
diff --git a/tests/xlconfigdata/test-fullvirt-nestedhvm.cfg
b/tests/xlconfigdata/test-fullvirt-nestedhvm.cfg
new file mode 100644
index 0000000..281f126
--- /dev/null
+++ b/tests/xlconfigdata/test-fullvirt-nestedhvm.cfg
@@ -0,0 +1,26 @@
+name = "XenGuest2"
+uuid = "c7a5fdb2-cdaf-9455-926a-d65c16db1809"
+maxmem = 579
+memory = 394
+vcpus = 1
+pae = 1
+acpi = 1
+apic = 1
+viridian = 0
+rtc_timeoffset = 0
+localtime = 0
+on_poweroff = "destroy"
+on_reboot = "restart"
+on_crash = "restart"
+device_model = "/usr/lib/xen/bin/qemu-system-i386"
+sdl = 0
+vnc = 1
+vncunused = 1
+vnclisten = "127.0.0.1"
+vif = [ "mac=00:16:3e:66:92:9c,bridge=xenbr1,script=vif-bridge,model=e1000" ]
+parallel = "none"
+serial = "none"
+builder = "hvm"
+boot = "d"
+nestedhvm = 1
+disk = [
"format=raw,vdev=hda,access=rw,backendtype=phy,target=/dev/HostVG/XenGuest2",
"format=qcow2,vdev=hdb,access=rw,backendtype=qdisk,target=/var/lib/libvirt/images/XenGuest2-home",
"format=raw,vdev=hdc,access=ro,backendtype=qdisk,devtype=cdrom,target=/root/boot.iso"
]
diff --git a/tests/xlconfigdata/test-fullvirt-nestedhvm.xml
b/tests/xlconfigdata/test-fullvirt-nestedhvm.xml
new file mode 100644
index 0000000..a1f637a
--- /dev/null
+++ b/tests/xlconfigdata/test-fullvirt-nestedhvm.xml
@@ -0,0 +1,61 @@
+<domain type='xen'>
+ <name>XenGuest2</name>
+ <uuid>c7a5fdb2-cdaf-9455-926a-d65c16db1809</uuid>
+ <memory unit='KiB'>592896</memory>
+ <currentMemory unit='KiB'>403456</currentMemory>
+ <vcpu placement='static'>1</vcpu>
+ <os>
+ <type arch='x86_64' machine='xenfv'>hvm</type>
+ <loader type='rom'>/usr/lib/xen/boot/hvmloader</loader>
+ <boot dev='cdrom'/>
+ </os>
+ <features>
+ <acpi/>
+ <apic/>
+ <pae/>
+ </features>
+ <cpu mode='host-passthrough'>
+ <feature policy='require' name='vmx'/>
+ </cpu>
+ <clock offset='variable' adjustment='0' basis='utc'/>
+ <on_poweroff>destroy</on_poweroff>
+ <on_reboot>restart</on_reboot>
+ <on_crash>restart</on_crash>
+ <devices>
+ <emulator>/usr/lib/xen/bin/qemu-system-i386</emulator>
+ <disk type='block' device='disk'>
+ <driver name='phy' type='raw'/>
+ <source dev='/dev/HostVG/XenGuest2'/>
+ <target dev='hda' bus='ide'/>
+ <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
+ </disk>
+ <disk type='file' device='disk'>
+ <driver name='qemu' type='qcow2'/>
+ <source file='/var/lib/libvirt/images/XenGuest2-home'/>
+ <target dev='hdb' bus='ide'/>
+ <address type='drive' controller='0' bus='0'
target='0' unit='1'/>
+ </disk>
+ <disk type='file' device='cdrom'>
+ <driver name='qemu' type='raw'/>
+ <source file='/root/boot.iso'/>
+ <target dev='hdc' bus='ide'/>
+ <readonly/>
+ <address type='drive' controller='0' bus='1'
target='0' unit='0'/>
+ </disk>
+ <controller type='ide' index='0'/>
+ <interface type='bridge'>
+ <mac address='00:16:3e:66:92:9c'/>
+ <source bridge='xenbr1'/>
+ <script path='vif-bridge'/>
+ <model type='e1000'/>
+ </interface>
+ <input type='mouse' bus='ps2'/>
+ <input type='keyboard' bus='ps2'/>
+ <graphics type='vnc' port='-1' autoport='yes'
listen='127.0.0.1'>
+ <listen type='address' address='127.0.0.1'/>
+ </graphics>
+ <video>
+ <model type='cirrus' vram='8192' heads='1'
primary='yes'/>
+ </video>
+ </devices>
+</domain>
diff --git a/tests/xlconfigtest.c b/tests/xlconfigtest.c
index e74e4d6..d2cc950 100644
--- a/tests/xlconfigtest.c
+++ b/tests/xlconfigtest.c
@@ -268,6 +268,7 @@ mymain(void)
DO_TEST("fullvirt-hpet-timer");
DO_TEST("fullvirt-tsc-timer");
DO_TEST("fullvirt-multi-timer");
+ DO_TEST("fullvirt-nestedhvm");
DO_TEST("paravirt-cmdline");
DO_TEST_FORMAT("paravirt-cmdline-extra-root", false);
--
2.9.3
8:13 a.m.
On Wed, Mar 08, 2017 at 03:07:12PM +0100, Wim Ten Have wrote:
From: Wim ten Have <wim.ten.have(a)oracle.com>
This patch enhances host-passthrough capability to advertise the
required vendor CPU virtualization feature which will be used to
enable 'nestedhvm' in the libxl driver.
For Intel virtualization (VT-x)
..
<cpu mode='host-passthrough'>
<feature policy='require' name='vmx'/>
</cpu>
For AMD virtualization (AMD-V)
..
<cpu mode='host-passthrough'>
<feature policy='require' name='svm'/>
</cpu>
If using host-passthrough or host-model, and the host's CPU includes
vmx or svm, then I would expecte nested-HVM to be enabled, without
needing any extra <feature> flag in the XML. That would match the
semantics used with the QEMU driver.
The only time we would need to use <feature> is if using mode=custom
along with a named CPU model which lacks vmx/svm flags.
BTW, I wonder how hard it would be to wire up the libxl driver to use
the CPU model infrastructure in src/cpu ? It feels a little odd to use
XML <cpu mode='host-passthrough'/> if we're not then making sure it
actually uses host-passthrough when configuring the guest.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
9:09 a.m.
On 03/08/2017 02:13 PM, Daniel P. Berrange wrote:
On Wed, Mar 08, 2017 at 03:07:12PM +0100, Wim Ten Have wrote:
> From: Wim ten Have <wim.ten.have(a)oracle.com>
>
> This patch enhances host-passthrough capability to advertise the
> required vendor CPU virtualization feature which will be used to
> enable 'nestedhvm' in the libxl driver.
>
> For Intel virtualization (VT-x)
> ..
> <cpu mode='host-passthrough'>
> <feature policy='require' name='vmx'/>
> </cpu>
>
> For AMD virtualization (AMD-V)
> ..
> <cpu mode='host-passthrough'>
> <feature policy='require' name='svm'/>
> </cpu>
If using host-passthrough or host-model, and the host's CPU includes
vmx or svm, then I would expecte nested-HVM to be enabled, without
needing any extra <feature> flag in the XML. That would match the
semantics used with the QEMU driver.
The only time we would need to use <feature> is if using mode=custom
along with a named CPU model which lacks vmx/svm flags.
Ah OK - I was kinda off
unclear on that. So using host-passthrough should be
enough then. (while making sure libxl checks if host->cpu does have vmx or svm
in its features)
BTW, I wonder how hard it would be to wire up the libxl driver to
use
the CPU model infrastructure in src/cpu ? It feels a little odd to use
XML <cpu mode='host-passthrough'/> if we're not then making sure it
actually uses host-passthrough when configuring the guest.
While xen libxl do allow
to mangle the cpuid, it is meant for disabling features
at this point. libxl follows a "host-model" first, meaning the default is to
expose as much as features as possible to the guest (depending on whether it's
PV or HVM). However, nested virt is a slightly special case since the toolstack
will do more than simply unmasking vmx/svm bits (actually within libxl, a sysctl
to Xen is performed to enable nested virt to the domain, in which case libxc
will handle any vendor specific bits). IOW, even when we improve libxl cpuid
policy management to the point we can wire up to libvirt cpu model
infrastruture, we would still need to handle the nestedhvm special case (which
btw this field would work even for ARM whenever supported).
Joao
9:07 a.m.
On Wed, Mar 08, 2017 at 03:09:11PM +0000, Joao Martins wrote:
On 03/08/2017 02:13 PM, Daniel P. Berrange wrote:
> On Wed, Mar 08, 2017 at 03:07:12PM +0100, Wim Ten Have wrote:
>> From: Wim ten Have <wim.ten.have(a)oracle.com>
>>
>> This patch enhances host-passthrough capability to advertise the
>> required vendor CPU virtualization feature which will be used to
>> enable 'nestedhvm' in the libxl driver.
>>
>> For Intel virtualization (VT-x)
>> ..
>> <cpu mode='host-passthrough'>
>> <feature policy='require' name='vmx'/>
>> </cpu>
>>
>> For AMD virtualization (AMD-V)
>> ..
>> <cpu mode='host-passthrough'>
>> <feature policy='require' name='svm'/>
>> </cpu>
>
> If using host-passthrough or host-model, and the host's CPU includes
> vmx or svm, then I would expecte nested-HVM to be enabled, without
> needing any extra <feature> flag in the XML. That would match the
> semantics used with the QEMU driver.
>
> The only time we would need to use <feature> is if using mode=custom
> along with a named CPU model which lacks vmx/svm flags.
Ah OK - I was kinda off unclear on that. So using host-passthrough should be
enough then. (while making sure libxl checks if host->cpu does have vmx or svm
in its features)
> BTW, I wonder how hard it would be to wire up the libxl driver to use
> the CPU model infrastructure in src/cpu ? It feels a little odd to use
> XML <cpu mode='host-passthrough'/> if we're not then making sure
it
> actually uses host-passthrough when configuring the guest.
While xen libxl do allow to mangle the cpuid, it is meant for disabling features
at this point. libxl follows a "host-model" first, meaning the default is to
expose as much as features as possible to the guest (depending on whether it's
PV or HVM). However, nested virt is a slightly special case since the toolstack
will do more than simply unmasking vmx/svm bits (actually within libxl, a sysctl
to Xen is performed to enable nested virt to the domain, in which case libxc
will handle any vendor specific bits). IOW, even when we improve libxl cpuid
policy management to the point we can wire up to libvirt cpu model
infrastruture, we would still need to handle the nestedhvm special case (which
btw this field would work even for ARM whenever supported).
Yep, I see. So with libxl using host-model by default, then
- If the host CPU includes svm/vmx, turn on nested virt in the guest
- Allow use of <feature name="vmx" policy="disable"/> to block
nested
virt in the guest, even if available in host CPU
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
9:32 a.m.
On 03/08/2017 03:07 PM, Daniel P. Berrange wrote:
On Wed, Mar 08, 2017 at 03:09:11PM +0000, Joao Martins wrote:
> On 03/08/2017 02:13 PM, Daniel P. Berrange wrote:
>> On Wed, Mar 08, 2017 at 03:07:12PM +0100, Wim Ten Have wrote:
>>> From: Wim ten Have <wim.ten.have(a)oracle.com>
>>>
>>> This patch enhances host-passthrough capability to advertise the
>>> required vendor CPU virtualization feature which will be used to
>>> enable 'nestedhvm' in the libxl driver.
>>>
>>> For Intel virtualization (VT-x)
>>> ..
>>> <cpu mode='host-passthrough'>
>>> <feature policy='require' name='vmx'/>
>>> </cpu>
>>>
>>> For AMD virtualization (AMD-V)
>>> ..
>>> <cpu mode='host-passthrough'>
>>> <feature policy='require' name='svm'/>
>>> </cpu>
>>
>> If using host-passthrough or host-model, and the host's CPU includes
>> vmx or svm, then I would expecte nested-HVM to be enabled, without
>> needing any extra <feature> flag in the XML. That would match the
>> semantics used with the QEMU driver.
>>
>> The only time we would need to use <feature> is if using mode=custom
>> along with a named CPU model which lacks vmx/svm flags.
> Ah OK - I was kinda off unclear on that. So using host-passthrough should be
> enough then. (while making sure libxl checks if host->cpu does have vmx or svm
> in its features)
>
>> BTW, I wonder how hard it would be to wire up the libxl driver to use
>> the CPU model infrastructure in src/cpu ? It feels a little odd to use
>> XML <cpu mode='host-passthrough'/> if we're not then making
sure it
>> actually uses host-passthrough when configuring the guest.
> While xen libxl do allow to mangle the cpuid, it is meant for disabling features
> at this point. libxl follows a "host-model" first, meaning the default is
to
> expose as much as features as possible to the guest (depending on whether it's
> PV or HVM). However, nested virt is a slightly special case since the toolstack
> will do more than simply unmasking vmx/svm bits (actually within libxl, a sysctl
> to Xen is performed to enable nested virt to the domain, in which case libxc
> will handle any vendor specific bits). IOW, even when we improve libxl cpuid
> policy management to the point we can wire up to libvirt cpu model
> infrastruture, we would still need to handle the nestedhvm special case (which
> btw this field would work even for ARM whenever supported).
Yep, I see. So with libxl using host-model by default, then
- If the host CPU includes svm/vmx, turn on nested virt in the guest
- Allow use of <feature name="vmx" policy="disable"/> to block
nested
virt in the guest, even if available in host CPU
Cool, thanks!
Joao
2816
days inactive
2816
days old
7 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Joao Martins -
Wim Ten Have