[libvirt] [RFC] Specific vcpu hot-(un)plug API proposal
Hypervisors are starting to support hot-(un)plugging of specific vcpus. This adds more flexibility for the management tools to decide which CPU should be added or removed. Libvirt's API in current state does not allow to choose arbitrary vCPU id's for the new vCPU's and does not support removing arbitrary vCPUs either. I propose a following API to enable working with specific vCPUs: /** * virDomainAddVcpu: * @domain: pointer to domain object * @vcpu: ID of the vcpu socket to plug the virtual CPU to * @flags: bitwise-OR of virDomainModificationImpact * * Dynamicaly add a CPU to the domain. Attach the cpu to the ID specified * by @vcpu. Note that this call may fail if the underlying virtualization * hypervisor does not support adding cpu's with specific ID or if maximum * number of CPUs is arbitrary limited. * * The @vcpu parameter identifies the vcpu ID the new vcpu should be attached * to. If -1 is specified, the new cpu is added to the first available ID. * * @flags may include VIR_DOMAIN_AFFECT_LIVE to affect a running * domain (which may fail if domain is not active), or * VIR_DOMAIN_AFFECT_CONFIG to affect the next boot via the XML * description of the domain. Both flags may be set. * If neither flag is specified (that is, @flags is VIR_DOMAIN_AFFECT_CURRENT), * then an inactive domain modifies persistent setup, while an active domain * is hypervisor-dependent on whether just live or both live and persistent * state is changed. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainAddVcpu(virDomainPtr domain, int vcpu, unsigned int flags) /** * virDomainRemoveVcpu: * @domain: pointer to domain object * @vcpu: ID of the vcpu socket to plug the virtual CPU to * @flags: bitwise-OR of virDomainModificationImpact * * Dynamicaly remove a CPU from the domain. Remove the the vcpu with ID * specified by @vcpu. Note that this call may fail if the underlying * virtualization hypervisor does not support hot-unplugging vcpus with * specific ID or vcpus in general. * * The @vcpu parameter identifies the vcpu ID that should be removed. * If -1 is specified, the cpu with the highest ID is removed. * * @flags may include VIR_DOMAIN_AFFECT_LIVE to affect a running * domain (which may fail if domain is not active), or * VIR_DOMAIN_AFFECT_CONFIG to affect the next boot via the XML * description of the domain. Both flags may be set. * If neither flag is specified (that is, @flags is VIR_DOMAIN_AFFECT_CURRENT), * then an inactive domain modifies persistent setup, while an active domain * is hypervisor-dependent on whether just live or both live and persistent * state is changed. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainRemoveVcpu(virDomainPtr domain, int vcpu, unsigned int flags) What are your thoughts on this? Thanks Peter
On 02/06/2012 10:10 AM, Peter Krempa wrote:
Hypervisors are starting to support hot-(un)plugging of specific vcpus. This adds more flexibility for the management tools to decide which CPU should be added or removed.
Libvirt's API in current state does not allow to choose arbitrary vCPU id's for the new vCPU's and does not support removing arbitrary vCPUs either.
I propose a following API to enable working with specific vCPUs:
/** * virDomainAddVcpu: * @domain: pointer to domain object * @vcpu: ID of the vcpu socket to plug the virtual CPU to * @flags: bitwise-OR of virDomainModificationImpact * * Dynamicaly add a CPU to the domain. Attach the cpu to the ID specified
s/Dynamicaly/Dynamically/
* by @vcpu. Note that this call may fail if the underlying virtualization * hypervisor does not support adding cpu's with specific ID or if maximum * number of CPUs is arbitrary limited. * * The @vcpu parameter identifies the vcpu ID the new vcpu should be attached * to. If -1 is specified, the new cpu is added to the first available ID.
This can only add one vcpu at a time, along with the counterpart that only removes one at a time; while the current virDomainSetVcpus can both add and subtract an arbitrary number (well, up to the hypervisor limits) of vcpus at once. I'm wondering if a better interface would be a single function that takes a cpu map as input, and which both hot-plugs and hot-unplugs vcpus until the guest matches the passed-in bitmap. That is, something like: /** * virDomainSetVcpuMap: * @domain: pointer to domain object * @vcpumap: pointer to bit map of vcpus to enable. Each bit set to * 1 means the vcpu is plugged in. Bytes are stored in * little-endian order: VCPU0-7, 8-15..., with lowest vcpu * number in least significant bit. * @maplen: number of bytes in vcpumap * @flags: bitwise-OR of virDomainModificationImpact * * Dynamically alter the set of vcpus enabled in a domain. Some * hypervisors only support hot-plug, rather than hot-unplug. * * @vcpumap must contain at least 1 set bit. If @maplen is * shorter than the hypervisor maximum, unspecified bytes are * treated as clear. Attempts to enable a vcpu beyond the * current maximum vcpu of the domain will fail; raise the * maximum first, by using virDomainSetVcpusFlags(). * * @flags controls whether the change is to a running domain, * configuration for next boot of a persistent domain, or both. */ int virDomainSetVcpuMap(virDomainPtr domain, unsigned char *vcpumap, int maplen, unsigned int flags)
What are your thoughts on this?
You also need to propose proper XML to represent a domain with particular vcpus unplugged. Right now, <vcpu current='2'>4</vcpu> is hard-coded to stating that vcpu 0 and 1 are enabled, and vcpu 2 and 3 are disabled. But if you let me call char map = 0x5; virDomainSetVcpuMap(dom, &map, 1, 0); in order to have vcpu 0 and 2 enabled, and vcpu 1 and 3 unplugged, then you've got to represent that in the XML somehow. And you may want a convenience counterpart method that queries the vcpu map directly, rather than making the user have to parse XML to see the current set of enabled vcpus. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Mon, Feb 06, 2012 at 10:33:59AM -0700, Eric Blake wrote:
On 02/06/2012 10:10 AM, Peter Krempa wrote:
Hypervisors are starting to support hot-(un)plugging of specific vcpus. This adds more flexibility for the management tools to decide which CPU should be added or removed.
Libvirt's API in current state does not allow to choose arbitrary vCPU id's for the new vCPU's and does not support removing arbitrary vCPUs either.
I propose a following API to enable working with specific vCPUs:
/** * virDomainAddVcpu: * @domain: pointer to domain object * @vcpu: ID of the vcpu socket to plug the virtual CPU to * @flags: bitwise-OR of virDomainModificationImpact * * Dynamicaly add a CPU to the domain. Attach the cpu to the ID specified
s/Dynamicaly/Dynamically/
* by @vcpu. Note that this call may fail if the underlying virtualization * hypervisor does not support adding cpu's with specific ID or if maximum * number of CPUs is arbitrary limited. * * The @vcpu parameter identifies the vcpu ID the new vcpu should be attached * to. If -1 is specified, the new cpu is added to the first available ID.
This can only add one vcpu at a time, along with the counterpart that only removes one at a time; while the current virDomainSetVcpus can both add and subtract an arbitrary number (well, up to the hypervisor limits) of vcpus at once.
I'm wondering if a better interface would be a single function that takes a cpu map as input, and which both hot-plugs and hot-unplugs vcpus until the guest matches the passed-in bitmap. That is, something like:
The downside with such a command is in the failure scenarios. eg, if we get 1/2 way through doing an operation and hotplug/unplug fails, what should we do. Do we try to revert to the original config (which might also fail), do we ignore the failure & carry on trying other vCPUs, or do we just stop leaving the 1/2 way completed state. If we have an API for just changing the online/offline state of a single vCPU at a time, then the mgmt application can make its own decision about how to handle failure of individual vCPUs. So personally I'd go for an API virDomainSetVCPUState(virDomainPtr dom, unsigned int vcpunum, int online)
What are your thoughts on this?
You also need to propose proper XML to represent a domain with particular vcpus unplugged. Right now, <vcpu current='2'>4</vcpu> is hard-coded to stating that vcpu 0 and 1 are enabled, and vcpu 2 and 3 are disabled. But if you let me call
char map = 0x5; virDomainSetVcpuMap(dom, &map, 1, 0);
in order to have vcpu 0 and 2 enabled, and vcpu 1 and 3 unplugged, then you've got to represent that in the XML somehow. And you may want a convenience counterpart method that queries the vcpu map directly, rather than making the user have to parse XML to see the current set of enabled vcpus.
Agreed, we need to cope with XML mapping, at the same time as doing this in the XML, so that we can deal with migration / XML state / etc Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Daniel P. Berrange -
Eric Blake -
Peter Krempa