[libvirt] [RFC] Host CPU passthrough
Hi, AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced. I think we should add a possibility to passthrough host CPU into a domain no matter what the CPU looks like. Domains configured in such a way will not be allowed to be migrated since we cannot guarantee the same CPU on destination host but they would be able to see the real CPU. Another benefit is that a single XML configuration would work on any CPU without modifications. Since possible issues with such domains will be harder to debug we can mark them as tainted just like we do if someone calls qemu monitor commands directly. I'm not sure what XML representation makes the most sense so I'll mention several options which I'm thinking about: - "host" ("native", "passthrough", whatever) special model; same name space as real CPU models, I don't like this too much: <cpu match='exact'> <model>host</model> </cpu> - "host" match policy; has nice first-look semantics that the guest CPU matches host's but doesn't play very nice with the defined semantics of "match" attribute: <cpu match='host'/> - new "mode" attribute; IMHO the best one, similar to what we have for SMBIOS: <cpu mode='host'/> vs. <cpu mode='custom' match='...'> <model>...</model. ... </cpu> What do you think? Jirka
On Mon, Aug 08, 2011 at 13:48:59 +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
There have been no comments on this so far. Perhaps the topic is not so controversial as I thought it was. But more likely it's just that people are busy with other things. IIRC, Daniel used to have a strong opinion on this matter, is that right? Jirka
On Tue, Aug 16, 2011 at 10:55:32PM +0200, Jiri Denemark wrote:
On Mon, Aug 08, 2011 at 13:48:59 +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
There have been no comments on this so far. Perhaps the topic is not so controversial as I thought it was. But more likely it's just that people are busy with other things. IIRC, Daniel used to have a strong opinion on this matter, is that right?
I'm thinking that this boils down essentially to syntactic sugar. Would it not be possible to create a <cpu>host</cpu> that simply automates the process of copying the host capabilities into the running guest XML? That would allow libvirt to do pre-migration validation that the destination host was suitable, but also permit users to specify one value of <cpu> that should in theory run with the maximum capabilities of the particular host where the domain was started and not have to go through the work of copying the host capabilities every time before booting the guest. Dave
Jirka
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Tue, Aug 16, 2011 at 17:46:43 -0400, Dave Allan wrote:
On Tue, Aug 16, 2011 at 10:55:32PM +0200, Jiri Denemark wrote:
On Mon, Aug 08, 2011 at 13:48:59 +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
There have been no comments on this so far. Perhaps the topic is not so controversial as I thought it was. But more likely it's just that people are busy with other things. IIRC, Daniel used to have a strong opinion on this matter, is that right?
I'm thinking that this boils down essentially to syntactic sugar.
Would it not be possible to create a <cpu>host</cpu> that simply automates the process of copying the host capabilities into the running guest XML? That would allow libvirt to do pre-migration validation that the destination host was suitable, but also permit users to specify one value of <cpu> that should in theory run with the maximum capabilities of the particular host where the domain was started and not have to go through the work of copying the host capabilities every time before booting the guest.
Dave and I agreed privately on IRC that in fact both use cases are valid and it would be nice if the XML configuration could allow both of them. Jirka
On Tue, Aug 16, 2011 at 05:46:43PM -0400, Dave Allan wrote:
On Tue, Aug 16, 2011 at 10:55:32PM +0200, Jiri Denemark wrote:
On Mon, Aug 08, 2011 at 13:48:59 +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
There have been no comments on this so far. Perhaps the topic is not so controversial as I thought it was. But more likely it's just that people are busy with other things. IIRC, Daniel used to have a strong opinion on this matter, is that right?
I'm thinking that this boils down essentially to syntactic sugar.
Would it not be possible to create a <cpu>host</cpu> that simply automates the process of copying the host capabilities into the running guest XML? That would allow libvirt to do pre-migration validation that the destination host was suitable, but also permit users to specify one value of <cpu> that should in theory run with the maximum capabilities of the particular host where the domain was started and not have to go through the work of copying the host capabilities every time before booting the guest.
The goal of the XML is that it should always reflect the canonical configuration of the guest. So when the guest is running it should not show 'host' as the CPU model, but rather the real CPU model that the guest is using. Although libvirt has the info internally, we want that to be equally visible via the XML description to the app. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Tue, Aug 16, 2011 at 03:46:23PM -0700, Daniel P. Berrange wrote:
On Tue, Aug 16, 2011 at 05:46:43PM -0400, Dave Allan wrote:
On Tue, Aug 16, 2011 at 10:55:32PM +0200, Jiri Denemark wrote:
On Mon, Aug 08, 2011 at 13:48:59 +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
There have been no comments on this so far. Perhaps the topic is not so controversial as I thought it was. But more likely it's just that people are busy with other things. IIRC, Daniel used to have a strong opinion on this matter, is that right?
I'm thinking that this boils down essentially to syntactic sugar.
Would it not be possible to create a <cpu>host</cpu> that simply automates the process of copying the host capabilities into the running guest XML? That would allow libvirt to do pre-migration validation that the destination host was suitable, but also permit users to specify one value of <cpu> that should in theory run with the maximum capabilities of the particular host where the domain was started and not have to go through the work of copying the host capabilities every time before booting the guest.
The goal of the XML is that it should always reflect the canonical configuration of the guest. So when the guest is running it should not show 'host' as the CPU model, but rather the real CPU model that the guest is using. Although libvirt has the info internally, we want that to be equally visible via the XML description to the app.
Agreed. Dave
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Aug 08, 2011 at 01:48:59PM +0200, Jiri Denemark wrote:
Hi,
AFAIK this topic is not new but I think we still do not have a good solution for it. Libvirt already supports specifying what CPU and its features a guest should see but imagine one wants to run a guest on the best possible CPU. The current way is to copy the <cpu> element from capabilities XML into domain XML. This approach is fine since it provides reproducible environment and such domain can even be migrated to a different host. But the CPU shown provided to a guest is not the same as the real host CPU. It's based on a model which doesn't reflect all aspects of real CPUs. Ideally, we would model everything but that's quite complicated and may need updating anytime a new CPU is introduced.
I think we should add a possibility to passthrough host CPU into a domain no matter what the CPU looks like. Domains configured in such a way will not be allowed to be migrated since we cannot guarantee the same CPU on destination host but they would be able to see the real CPU. Another benefit is that a single XML configuration would work on any CPU without modifications. Since possible issues with such domains will be harder to debug we can mark them as tainted just like we do if someone calls qemu monitor commands directly.
I'm not sure what XML representation makes the most sense so I'll mention several options which I'm thinking about:
- "host" ("native", "passthrough", whatever) special model; same name space as real CPU models, I don't like this too much:
<cpu match='exact'> <model>host</model> </cpu>
This is not nice because the end mgmt app doesn't see what the guest is actually running with. So you get the result that the app has to have two sets of logic, one looking at the guest XML CPU model, and other case where the app has to look at the host capabilities CPU model.
- "host" match policy; has nice first-look semantics that the guest CPU matches host's but doesn't play very nice with the defined semantics of "match" attribute:
<cpu match='host'/>
Yeah I don't much like that.
- new "mode" attribute; IMHO the best one, similar to what we have for SMBIOS:
<cpu mode='host'/>
vs.
<cpu mode='custom' match='...'> <model>...</model. ... </cpu>
This option is almost right with one change. When the guest is inactive: <cpu mode='host'/> But when the guest is actually launched, it should get filled in with the real data. ie <cpu mode='host' match='...'> <model>...</model> ... </cpu> and then cleared out again when the guest stops. This is what we do with the SELinux dynamic security labels. During save/restore/ migration, of course the contents should remain present. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange -
Dave Allan -
Jiri Denemark