[libvirt] Question about hypervisor <features> that are not tristate
Happy new year! Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg. The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before. To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use. Regards, Jim [1] https://www.redhat.com/archives/libvir-list/2016-February/msg00197.html [2] https://www.redhat.com/archives/libvir-list/2016-March/msg00001.html
On 01/06/2017 05:31 PM, Jim Fehlig wrote: Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it? Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users? Thanks for your comments! Regards, Jim
[1] https://www.redhat.com/archives/libvir-list/2016-February/msg00197.html [2] https://www.redhat.com/archives/libvir-list/2016-March/msg00001.html
On Wed, Jan 11, 2017 at 10:49:48AM -0700, Jim Fehlig wrote:
On 01/06/2017 05:31 PM, Jim Fehlig wrote:
Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it?
Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users?
If the guest arch is declared as x86_64, then you should unconditionally force 'pae' to be present, otherwise you'd be giving the guest an i686 environment. The app should request arch == i686 explicitly if they want an i686 environment. Allowing pae to be toggled for i686 guests is ok, since that indicates whethre the i686 CPU can address > 2GB of RAM. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
On 01/11/2017 10:55 AM, Daniel P. Berrange wrote:
On Wed, Jan 11, 2017 at 10:49:48AM -0700, Jim Fehlig wrote:
On 01/06/2017 05:31 PM, Jim Fehlig wrote:
Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it?
Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users?
If the guest arch is declared as x86_64, then you should unconditionally force 'pae' to be present, otherwise you'd be giving the guest an i686 environment.
Right.
The app should request arch == i686 explicitly if they want an i686 environment. Allowing pae to be toggled for i686 guests is ok, since that indicates whethre the i686 CPU can address > 2GB of RAM.
Good point. I'll look into a patch implementing this logic. Regards, Jim
On 11/01/17 17:49, Jim Fehlig wrote:
On 01/06/2017 05:31 PM, Jim Fehlig wrote:
Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it?
Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users?
ISTR the main use of this knob being to cause 32bit versions of windows to avoid using PAE paging, making them more efficient to shadow. Now that 64bit and EPT/NPT are fairly ubiquitous, there should be no reason to turn it off. Can people still play with the CPUID policy if they really need to disable it? It is unfortunate that this option was ever exposed via a non-CPUID mechanism, but I am trying to clean this up at the hypervisor interface to ensure that the *only* way to alter details like this is via the appropriate interface. ~Andrew
On 01/11/2017 11:00 AM, Andrew Cooper wrote:
On 11/01/17 17:49, Jim Fehlig wrote:
On 01/06/2017 05:31 PM, Jim Fehlig wrote:
Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it?
Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users?
ISTR the main use of this knob being to cause 32bit versions of windows to avoid using PAE paging, making them more efficient to shadow.
Now that 64bit and EPT/NPT are fairly ubiquitous, there should be no reason to turn it off.
Okay, thanks.
Can people still play with the CPUID policy if they really need to disable it?
ATM, not through libvirt. We still have some work to do in this area.
It is unfortunate that this option was ever exposed via a non-CPUID mechanism, but I am trying to clean this up at the hypervisor interface to ensure that the *only* way to alter details like this is via the appropriate interface.
And we'll need to make use of this interface in libvirt. Regards, Jim
On Wed, Jan 11, 2017 at 10:49:48AM -0700, Jim Fehlig wrote:
On 01/06/2017 05:31 PM, Jim Fehlig wrote:
Adding xen-devel for a question below...
Happy new year!
Nearly a year ago I reported an issue with the <hap> hypervisor feature on Xen [1] and am now seeing a similar issue with the <pae> feature. Setting the default value of pae changed between xend and libxl. When not specified, xend would enable pae for HVM domains. Clients such as xm and the old libvirt driver did not have to explicitly enable it. In libxl, the pae field within libxl_domain_build_info is initialized to 0. Clients must enable pae, and indeed xl will do so if pae=1 is not specified in the xl.cfg.
The xend behavior prevents libvirt from disabling pae, whereas the libxl behvior causes a guest ABI change (config that worked with libvirt+xend doesn't with libvirt+libxl). The libxl behavior also forces management software (e.g. OpenStack nova) to add <pae> where it wasn't needed before.
To solve this problem for <hap>, it was changed it to a tristate [2], allowing it to be turned off with explicit <hap state='off'/>, and on if not specified or <hap/> or <hap state='on'/>. Should <pae> (and the remaining hypervisor features that are not tristate) be converted to tristate similar to <hap>? Alternatively, I could simply set pae=1 for all HVM domains in the libxl driver. Like the old libvirt+xend behavior it couldn't be turned off, but I don't think there is a practical use-case to do so. At least no one has complained over all the years of libvirt+xend use.
Xen folks, what is your opinion of always enabling pae for HVM domains in the libvirt libxl driver? Is there a need these days to disable it?
No. I think it should be enabled all the time.
Jan had mentioned that some old, buggy guest OS's (Win 9x) might need it disabled, and perhaps some cases where it may be desirable to suppress a guest OS entering 64-bit mode. But in practice do you think it is necessary to expose this knob to users?
Um, Win 98? People use that? Microsoft doesn't even support that. I would ignore such ancient OSes.
Thanks for your comments!
Regards, Jim
[1] https://www.redhat.com/archives/libvir-list/2016-February/msg00197.html [2] https://www.redhat.com/archives/libvir-list/2016-March/msg00001.html
_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
participants (4)
-
Andrew Cooper -
Daniel P. Berrange -
Jim Fehlig -
Konrad Rzeszutek Wilk