[Libvir] Request for additional entry points

As discussed on the xen-cim call on 3/31, the Xen CIM provider needs some additional entry points in libvirt. I would like to open a discussion about adding the following entry points to libvirt. virDomainSetConfig(virConnectPtr conn, const char *xmlDesc) Stores the domU config data in xenstore. The domU is not running yet but enumerating domains would return the config for the domain as well as any running domains. This would support the notion of a defined but inactive virtual machine. Daniel noted that the config could be cached in libvirt, preventing (to some degree) modifying the config out-of-band prior to activating the domain. Note that the current virConnectListDomains() implies enumerating only running domains since a list of domain IDs is returned. Perhaps virConnectListDomains() will have to be expanded to include defined domains or another entry point to enumerate defined domains. Related would be activating a defined domain. Would clients get the domain config (via virDomainGetXMLDesc()) and subsequently pass it to virDomainCreateLinux() or another entry point to create a defined domain? virDomainDeleteConfig(virConnectPtr conn, const char *name) Removed domU config from xenstore (or cache). DomU no longer "exists". What if domain is active upon invocation? virDomainSetCurrentMemory(virDomainPtr domain, unsigned long memory) Adjust the current memory usage for a domain. virDomainReboot(virDomainPtr domain) Reboot the domain. virDomainMigrate(virDomainPtr domain, const char *host) ?? Not sure what can of worms might be opened by this one :-). Thoughts? BTW, I can certainly help with the effort if an agreement is reached on adding this functionality. Regards, Jim

On Fri, Mar 31, 2006 at 10:56:06AM -0700, Jim Fehlig wrote:
As discussed on the xen-cim call on 3/31, the Xen CIM provider needs some additional entry points in libvirt. I would like to open a discussion about adding the following entry points to libvirt.
virDomainSetConfig(virConnectPtr conn, const char *xmlDesc) Stores the domU config data in xenstore. The domU is not running yet but enumerating domains would return the config for the domain as well as any running domains. This would support the notion of a defined but inactive virtual machine. Daniel noted that the config could be cached in libvirt, preventing (to some degree) modifying the config out-of-band prior to activating the domain. Note that the current virConnectListDomains() implies enumerating only running domains since a list of domain IDs is returned. Perhaps virConnectListDomains() will have to be expanded to include defined domains or another entry point to enumerate defined domains.
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains(). In general libvirt doesn't yet create a unique proxy object per running domain, the unicity need to be garanteed, it's a TODO and add other constraints like reference counting and mutex when modifying a domain.
Related would be activating a defined domain. Would clients get the domain config (via virDomainGetXMLDesc()) and subsequently pass it to virDomainCreateLinux() or another entry point to create a defined domain?
I would rather add an xmlDomainCreate(domain) taking as the argument the returned domain from virDomainSetConfig().
virDomainDeleteConfig(virConnectPtr conn, const char *name) Removed domU config from xenstore (or cache). DomU no longer "exists". What if domain is active upon invocation?
Semantic would have to be defined. I would think of a destroy if the reference count for the domain goes down to 0 then.
virDomainSetCurrentMemory(virDomainPtr domain, unsigned long memory) Adjust the current memory usage for a domain.
We have SetMaxMemory already, I understand there is a difference in Xen between the max defined memory and the current target, but we need to nail down precisely the meaning of the 2 apis if introducing a new one.
virDomainReboot(virDomainPtr domain) Reboot the domain.
Trivial kind of cut and paste from the Shutdown routines.
virDomainMigrate(virDomainPtr domain, const char *host) ?? Not sure what can of worms might be opened by this one :-).
That one is premature IMHO, and the second parameter would be a virConnectPtr itselt open with a target specifying the host. But it's too deep in the future to really say yes. All other APIs looks fine and should be doable relatively easilly and quickly. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

On Fri, Mar 31, 2006 at 04:47:38PM -0500, Daniel Veillard wrote:
virDomainReboot(virDomainPtr domain) Reboot the domain.
Trivial kind of cut and paste from the Shutdown routines.
I just added the following: /** * virDomainReboot: * @domain: a domain object * @flags: extra flags for the reboot operation, not used yet * * Reboot a domain, the domain object is still usable there after but * the domain OS is being stopped for a restart. * Note that the guest OS may ignore the request. * * Returns 0 in case of success and -1 in case of failure. */ int virDomainReboot(virDomainPtr domain, unsigned int flags) there is also a new "virsh reboot" command to test it. However there when trying it I saw that the guest OS shutdown in reboot mode: ------------------------ Turning off quotas: Unmounting pipe file systems: Please stand by while rebooting the system... md: stopping all md devices. md: md0 switched to read-only mode. Restarting system. . ------------------------ but the domain was not relaunched by default. I guess this also requires to set up specific xenstore attributes so that the domain is actually restarted. So not completely done yet. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Daniel Veillard wrote:
On Fri, Mar 31, 2006 at 04:47:38PM -0500, Daniel Veillard wrote:
I just added the following:
/** * virDomainReboot: * @domain: a domain object * @flags: extra flags for the reboot operation, not used yet * * Reboot a domain, the domain object is still usable there after but * the domain OS is being stopped for a restart. * Note that the guest OS may ignore the request. * * Returns 0 in case of success and -1 in case of failure. */ int virDomainReboot(virDomainPtr domain, unsigned int flags)
there is also a new "virsh reboot" command to test it. However there when trying it I saw that the guest OS shutdown in reboot mode:
------------------------ Turning off quotas: Unmounting pipe file systems: Please stand by while rebooting the system... md: stopping all md devices. md: md0 switched to read-only mode. Restarting system. . ------------------------
but the domain was not relaunched by default. I guess this also requires to set up specific xenstore attributes so that the domain is actually restarted. So not completely done yet.
You are probably aware of this but FYI... I used virDomainReboot() with success when the domain was created with "xm create" and "on_reboot = 'restart'" in domain config file. Looking at virDomainParseXMLDesc() in xml.c I see we are not accounting for a <on_reboot> element (or on_crash, on_poweroff) in the XML. Are you working on this Daniel or should I take a stab at it? I'm guessing you just want elements like <on_reboot>restart</on_reboot> <on_poweroff>destroy</on_poweroff> <on_crash>restart</on_crash> Same goes turning sxp to XML in xend_parse_sexp_desc() in xend_internal.c. Regards, Jim

On Tue, Apr 04, 2006 at 05:45:56PM -0600, Jim Fehlig wrote:
but the domain was not relaunched by default. I guess this also requires to set up specific xenstore attributes so that the domain is actually restarted. So not completely done yet.
You are probably aware of this but FYI...
I used virDomainReboot() with success when the domain was created with "xm create" and "on_reboot = 'restart'" in domain config file.
Yup, that's what's missing, it's one of the XenStore attributes
Looking at virDomainParseXMLDesc() in xml.c I see we are not accounting for a <on_reboot> element (or on_crash, on_poweroff) in the XML. Are
I wasn't sure how much this was needed, clearly it is now.
you working on this Daniel or should I take a stab at it? I'm guessing you just want elements like
<on_reboot>restart</on_reboot> <on_poweroff>destroy</on_poweroff> <on_crash>restart</on_crash>
Yes. Please go ahead, I didn't yet applied your previous patch but it looks sensible, just some bogus logic was left from the old code.
Same goes turning sxp to XML in xend_parse_sexp_desc() in xend_internal.c.
Yup. just provide information about the new elements added and their values so I can extend the documentation page about the XML format (at some point I will also provide a Relax-NG or XSD schemas to validate them). Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

Daniel Veillard wrote:
Looking at virDomainParseXMLDesc() in xml.c I see we are not accounting for a <on_reboot> element (or on_crash, on_poweroff) in the XML. Are
I wasn't sure how much this was needed, clearly it is now.
you working on this Daniel or should I take a stab at it? I'm guessing you just want elements like
<on_reboot>restart</on_reboot> <on_poweroff>destroy</on_poweroff> <on_crash>restart</on_crash>
Yes. Please go ahead, I didn't yet applied your previous patch but it looks sensible, just some bogus logic was left from the old code.
Same goes turning sxp to XML in xend_parse_sexp_desc() in xend_internal.c.
Yup. just provide information about the new elements added and their values so I can extend the documentation page about the XML format (at some point I will also provide a Relax-NG or XSD schemas to validate them).
Ok, here is a patch that accounts for various lifecycle events, e.g. on_reboot, on_poweroff, on_crash. The new elements (children of domain element) are <on_reboot>action</on_reboot> <on_poweroff>action</on_poweroff> <on_crash>action</on_crash> on_reboot: Action to be taken when domain is rebooted. on_poweroff: Action to be taken when a power off or shutdown operation is performed inside the domain. on_crash: Action to be taken when domain crashes. action values can be one of destroy: The domain is cleaned up as normal. restart: A new domain is started in place of the old one. preserve: The domain will remain in memory until it is destroyed manually. rename-restart: The domain will remain in memory until it is destroyed manually, and a new, renamed domain is started. Regards, Jim

On Fri, Apr 07, 2006 at 03:40:10PM -0600, Jim Fehlig wrote:
Ok, here is a patch that accounts for various lifecycle events, e.g. on_reboot, on_poweroff, on_crash. The new elements (children of domain element) are <on_reboot>action</on_reboot> <on_poweroff>action</on_poweroff> <on_crash>action</on_crash>
Looks fine, applied and commited, thanks ! I also added the description of the new elements to the documentation. Thanks ! Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

On Fri, 31 Mar 2006 16:47:38 -0500, Daniel Veillard wrote:
On Fri, Mar 31, 2006 at 10:56:06AM -0700, Jim Fehlig wrote:
As discussed on the xen-cim call on 3/31, the Xen CIM provider needs some additional entry points in libvirt. I would like to open a discussion about adding the following entry points to libvirt.
virDomainSetConfig(virConnectPtr conn, const char *xmlDesc) Stores the domU config data in xenstore. The domU is not running yet but enumerating domains would return the config for the domain as well as any running domains. This would support the notion of a defined but inactive virtual machine. Daniel noted that the config could be cached in libvirt, preventing (to some degree) modifying the config out-of-band prior to activating the domain. Note that the current virConnectListDomains() implies enumerating only running domains since a list of domain IDs is returned. Perhaps virConnectListDomains() will have to be expanded to include defined domains or another entry point to enumerate defined domains.
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains(). In general libvirt doesn't yet create a unique proxy object per running domain, the unicity need to be garanteed, it's a TODO and add other constraints like reference counting and mutex when modifying a domain.
Related would be activating a defined domain. Would clients get the domain config (via virDomainGetXMLDesc()) and subsequently pass it to virDomainCreateLinux() or another entry point to create a defined domain?
I would rather add an xmlDomainCreate(domain) taking as the argument the returned domain from virDomainSetConfig().
virDomainDeleteConfig(virConnectPtr conn, const char *name) Removed domU config from xenstore (or cache). DomU no longer "exists". What if domain is active upon invocation?
Semantic would have to be defined. I would think of a destroy if the reference count for the domain goes down to 0 then.
virDomainSetCurrentMemory(virDomainPtr domain, unsigned long memory) Adjust the current memory usage for a domain.
We have SetMaxMemory already, I understand there is a difference in Xen between the max defined memory and the current target, but we need to nail down precisely the meaning of the 2 apis if introducing a new one.
MaxMemory is the cap allocation for a domain. It can never allocate more than MaxMemory. Note, that a domain can "balloon" up to or down from MaxMemory. SetCurrentMemory "requests" that a domain "balloon" itself to a particular target. Note that setting MaxMemory at runtime has been broken since the 2.0.x series. I don't know why we bother even exposing it anymore in Xend as I don't think there are any plans to fix it. Regards, Anthony Liguori
virDomainReboot(virDomainPtr domain) Reboot the domain.
Trivial kind of cut and paste from the Shutdown routines.
virDomainMigrate(virDomainPtr domain, const char *host) ?? Not sure what can of worms might be opened by this one :-).
That one is premature IMHO, and the second parameter would be a virConnectPtr itselt open with a target specifying the host. But it's too deep in the future to really say yes. All other APIs looks fine and should be doable relatively easilly and quickly.
Daniel

On Fri, 2006-03-31 at 16:47 -0500, Daniel Veillard wrote:
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains().
How would that jive with other ways to create inactive domains ? Right now I can create a file in /etc/xen and immediately have an incative domain; I always thought that the xm scripts would eventually be replaced by libvirt-based scripts and conventions for where to put files, but that libvirt would remain focused on active domains. I agree that it would be nice to have a standard way to enumerate 'defined' (as opposed to 'active') domains, but it seems a little odd to require a library to do what amounts to putting a file into a well-defined place. David

On Tue, Apr 04, 2006 at 02:35:15PM -0700, David Lutterkort wrote:
On Fri, 2006-03-31 at 16:47 -0500, Daniel Veillard wrote:
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains().
How would that jive with other ways to create inactive domains ? Right now I can create a file in /etc/xen and immediately have an incative domain; I always thought that the xm scripts would eventually be replaced by libvirt-based scripts and conventions for where to put files, but that libvirt would remain focused on active domains.
I agree that it would be nice to have a standard way to enumerate 'defined' (as opposed to 'active') domains, but it seems a little odd to require a library to do what amounts to putting a file into a well-defined place.
Yeah. To be honest I don't understand what is advantage of this solution (configs in xenstore). I think we already have in better and wide supported solutions how share data between more nodes (hosts). I don't think that we should replace OS filesystem with something like xenstore. I think we can implement "a standard way to enumerate inactive domain" without xenstore (for example by some search $PATH and standard FS). Karel -- Karel Zak <kzak@redhat.com>

On Fri, Mar 31, 2006 at 04:47:38PM -0500, Daniel Veillard wrote:
On Fri, Mar 31, 2006 at 10:56:06AM -0700, Jim Fehlig wrote:
As discussed on the xen-cim call on 3/31, the Xen CIM provider needs some additional entry points in libvirt. I would like to open a discussion about adding the following entry points to libvirt.
virDomainSetConfig(virConnectPtr conn, const char *xmlDesc) Stores the domU config data in xenstore. The domU is not running yet but enumerating domains would return the config for the domain as well as any running domains. This would support the notion of a defined but inactive virtual machine. Daniel noted that the config could be cached in libvirt, preventing (to some degree) modifying the config out-of-band prior to activating the domain. Note that the current virConnectListDomains() implies enumerating only running domains since a list of domain IDs is returned. Perhaps virConnectListDomains() will have to be expanded to include defined domains or another entry point to enumerate defined domains.
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains(). In general libvirt doesn't yet create a unique proxy object per running domain, the unicity need to be garanteed, it's a TODO and add other constraints like reference counting and mutex when modifying a domain.
I'd actually question whether libvirt should have any concept of 'passive' or 'inactive' domains at all. Just to play out a few scenarios: * Definitions of inactive domains are stored on a remote server, libvirt has no visibilty into this system. virDomainSetConfig() would not help since there is no concept of a domain being inactive on a single host, rather inactive domains can be deployed to any host in the server farm at will. * Definitions are visible on the local filesystem, but this FS is shared NFS / SAN storage. Again while their config files may be technically visible via the shared filesystem, there is no meaningful association between an inactive config & a host. * Domains are created from config 'templates'. The template defines a generic resource configuration, turned into a fully fledged config on-demand - perhaps with the variables plugged into the template coming from an alternate datastore (SQL databsae perhaps). In this case there would be no sense in storing pre-made XML configs, since the master info is a combo of the template & SQL database entries. So, really the only case where there is any meaningful concept of 'passive' domain configs associated with a static single host model - for example that provided by XenD which happen to have a simple store of inactive domains in /etc/xen. There are so many other interesting management scenarios where inactive domains don't really have a persistent association with a single host, or indeed any passive configuration at all, that I'm not sure there would be much value in libvirt to representing passive domains at all. Thinking about what tools might need information on inactive domains: - Desktop management tool - cf VMWare Workstation. Possibly, but even here there is no reason such a tool represents inactive domains in terms of a libvirt XML config file. It may well have generic templates eg 'Linux Web Server and instances 'web0.google.com', 'web1.google.com', 'web2.google.com', where the only instance specific data is the hostname and master disk image. - Data center management tool - it may well have a concept of inactive domain configs, but it almost certainly won't be associated with a single host, so unlikely to be asking libvirt to list inactive domains. I struggle to think of cases where libvirt would need to provide info on inactive domains, which are not distorted to serve the needs of a specific application's representational model. If a local app wants to maintain a local list of config files for inactive domains, let it do so itself. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

On Wed, Apr 12, 2006 at 04:08:53AM +0100, Daniel P. Berrange wrote:
I'd actually question whether libvirt should have any concept of 'passive' or 'inactive' domains at all. Just to play out a few scenarios:
The 2 first scenarios assume the passive domains are stored on the disk. This doesn't has to be the case, libvirt could just cache those informations in the management tool memory, also xenstore is acessed via RPC, those might be extended for remote machines.
* Definitions of inactive domains are stored on a remote server, libvirt has no visibilty into this system. virDomainSetConfig() would not help since there is no concept of a domain being inactive on a single host, rather inactive domains can be deployed to any host in the server farm at will.
* Definitions are visible on the local filesystem, but this FS is shared NFS / SAN storage. Again while their config files may be technically visible via the shared filesystem, there is no meaningful association between an inactive config & a host.
* Domains are created from config 'templates'. The template defines a generic resource configuration, turned into a fully fledged config on-demand - perhaps with the variables plugged into the template coming from an alternate datastore (SQL databsae perhaps). In this case there would be no sense in storing pre-made XML configs, since the master info is a combo of the template & SQL database entries.
In general I agree that reliance on some filesystem storage for them is a serious problem.
So, really the only case where there is any meaningful concept of 'passive' domain configs associated with a static single host model - for example that provided by XenD which happen to have a simple store of inactive domains in /etc/xen.
No that could work too if libvirt used to manage remote domains from a centralized location and keep the passive domain definitions on that centralized server.
Thinking about what tools might need information on inactive domains:
- Desktop management tool - cf VMWare Workstation. Possibly, but even here there is no reason such a tool represents inactive domains in terms of a libvirt XML config file. It may well have generic templates eg 'Linux Web Server and instances 'web0.google.com', 'web1.google.com', 'web2.google.com', where the only instance specific data is the hostname and master disk image.
- Data center management tool - it may well have a concept of inactive domain configs, but it almost certainly won't be associated with a single host, so unlikely to be asking libvirt to list inactive domains.
The need comes from the second one, this is part of the CIM model. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

On Wed, Apr 12, 2006 at 04:25:49AM -0400, Daniel Veillard wrote:
On Wed, Apr 12, 2006 at 04:08:53AM +0100, Daniel P. Berrange wrote:
I'd actually question whether libvirt should have any concept of 'passive' or 'inactive' domains at all. Just to play out a few scenarios:
The 2 first scenarios assume the passive domains are stored on the disk. This doesn't has to be the case, libvirt could just cache those informations in the management tool memory, also xenstore is acessed via RPC, those might be extended for remote machines.
* Definitions of inactive domains are stored on a remote server, libvirt has no visibilty into this system. virDomainSetConfig() would not help since there is no concept of a domain being inactive on a single host, rather inactive domains can be deployed to any host in the server farm at will.
* Definitions are visible on the local filesystem, but this FS is shared NFS / SAN storage. Again while their config files may be technically visible via the shared filesystem, there is no meaningful association between an inactive config & a host.
* Domains are created from config 'templates'. The template defines a generic resource configuration, turned into a fully fledged config on-demand - perhaps with the variables plugged into the template coming from an alternate datastore (SQL databsae perhaps). In this case there would be no sense in storing pre-made XML configs, since the master info is a combo of the template & SQL database entries.
In general I agree that reliance on some filesystem storage for them is a serious problem.
I'm not refering to any particular storage mechanism for libvirt - that's a backend implementation detail. I'm questioning whether it is relevant at all & what the semantics are for various use cases & what the value of such an API is for applications which might need such capabilities. If the inactive domains are stored in some higher level descriptive format rather than the raw domain XML defintion, any list of inactive domains configs would always be empty. So what is the value to an application, if an API for 'list inactive domains' returned an empty list in 50% of common uses ?
So, really the only case where there is any meaningful concept of 'passive' domain configs associated with a static single host model - for example that provided by XenD which happen to have a simple store of inactive domains in /etc/xen.
No that could work too if libvirt used to manage remote domains from a centralized location and keep the passive domain definitions on that centralized server.
This central management server could have an arbitrary backend for storing its master list of inactive domains, in a potentially application specific model. What would the value be in having the application take all these definitions from its master storage & convert their format & load them into libvirt - its just introduced a large burden on the application to ensure that its master store is always in sync with what it has loaded into libvirt - with no clear benefit for the application, since its still going to want to operate on its master store, not the denormalization. It has not obviously helped application inter-operability either, because the domains loaded into libvirt on the management server, would not be visible to an application using libvirt on the remote host itself.
Thinking about what tools might need information on inactive domains:
- Desktop management tool - cf VMWare Workstation. Possibly, but even here there is no reason such a tool represents inactive domains in terms of a libvirt XML config file. It may well have generic templates eg 'Linux Web Server and instances 'web0.google.com', 'web1.google.com', 'web2.google.com', where the only instance specific data is the hostname and master disk image.
- Data center management tool - it may well have a concept of inactive domain configs, but it almost certainly won't be associated with a single host, so unlikely to be asking libvirt to list inactive domains.
The need comes from the second one, this is part of the CIM model.
Its hard to reconcile the needs without knowing the conceptual requirements of the CIM model, rather than just an API implementation requirements. While I can see the value in CIM using libvirt to getting information on active domains since there is a very clear semantic & representational model which is not application specific. This is not true of inactive domains, since if you get ito any more detail than just 'there is an inactive domain with a UUID XYZ', you'd really be imposing an arbitrary representational model on applications using the UI that may make no sense for them. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

Daniel P. Berrange wrote:
On Fri, Mar 31, 2006 at 04:47:38PM -0500, Daniel Veillard wrote:
On Fri, Mar 31, 2006 at 10:56:06AM -0700, Jim Fehlig wrote:
As discussed on the xen-cim call on 3/31, the Xen CIM provider needs some additional entry points in libvirt. I would like to open a discussion about adding the following entry points to libvirt.
virDomainSetConfig(virConnectPtr conn, const char *xmlDesc) Stores the domU config data in xenstore. The domU is not running yet but enumerating domains would return the config for the domain as well as any running domains. This would support the notion of a defined but inactive virtual machine. Daniel noted that the config could be cached in libvirt, preventing (to some degree) modifying the config out-of-band prior to activating the domain. Note that the current virConnectListDomains() implies enumerating only running domains since a list of domain IDs is returned. Perhaps virConnectListDomains() will have to be expanded to include defined domains or another entry point to enumerate defined domains.
Well libvirt doesn't have the notion of 'passive' domain yet, i.e. domain we know exists but are not running or activated at a given point in time. If we define virDomainSetConfig() then we have to be able to extract at least the name (and uuid) from the xmlDesc. And that routine could then returm a virDomainPtr associated to this unactive domain (or the associated active domain if it exists). Those unactive domains could then be listed in virConnectListDomains(). In general libvirt doesn't yet create a unique proxy object per running domain, the unicity need to be garanteed, it's a TODO and add other constraints like reference counting and mutex when modifying a domain.
I'd actually question whether libvirt should have any concept of 'passive' or 'inactive' domains at all. Just to play out a few scenarios:
We've discussed this before and I think the most compelling argument for passive domain support in libvirt (and even at a lower level such as Xend) is the following scenario: Let's say a user creates a virtual FC6 virtual machine that she is going to use to test her apps under Gnome (she's a KDE user). The machine isn't always there b/c she rarely needs it. She created it with some nifty KDE tool that has yet to be written. Of course, she finds out about this neat applet that you can use to managing running domains. If there isn't a common way of accessing passive domains then she won't see the FC6 VM. Now, I don't think that *all* domains should have to have passive equivalents (which is a requirement in the CIM provider today) because of some of the scenarios you outlined. However, I think not attempting to have a common API for passive domains is going to creative a mess as we get a larger number of management tools. Regards, Anthony Liguori
* Definitions of inactive domains are stored on a remote server, libvirt has no visibilty into this system. virDomainSetConfig() would not help since there is no concept of a domain being inactive on a single host, rather inactive domains can be deployed to any host in the server farm at will.
* Definitions are visible on the local filesystem, but this FS is shared NFS / SAN storage. Again while their config files may be technically visible via the shared filesystem, there is no meaningful association between an inactive config & a host.
* Domains are created from config 'templates'. The template defines a generic resource configuration, turned into a fully fledged config on-demand - perhaps with the variables plugged into the template coming from an alternate datastore (SQL databsae perhaps). In this case there would be no sense in storing pre-made XML configs, since the master info is a combo of the template & SQL database entries.
So, really the only case where there is any meaningful concept of 'passive' domain configs associated with a static single host model - for example that provided by XenD which happen to have a simple store of inactive domains in /etc/xen. There are so many other interesting management scenarios where inactive domains don't really have a persistent association with a single host, or indeed any passive configuration at all, that I'm not sure there would be much value in libvirt to representing passive domains at all.
Thinking about what tools might need information on inactive domains:
- Desktop management tool - cf VMWare Workstation. Possibly, but even here there is no reason such a tool represents inactive domains in terms of a libvirt XML config file. It may well have generic templates eg 'Linux Web Server and instances 'web0.google.com', 'web1.google.com', 'web2.google.com', where the only instance specific data is the hostname and master disk image.
- Data center management tool - it may well have a concept of inactive domain configs, but it almost certainly won't be associated with a single host, so unlikely to be asking libvirt to list inactive domains.
I struggle to think of cases where libvirt would need to provide info on inactive domains, which are not distorted to serve the needs of a specific application's representational model. If a local app wants to maintain a local list of config files for inactive domains, let it do so itself.
Dan.

On Wed, Apr 12, 2006 at 11:51:53AM -0500, Anthony Liguori wrote:
Daniel P. Berrange wrote:
I'd actually question whether libvirt should have any concept of 'passive' or 'inactive' domains at all. Just to play out a few scenarios:
We've discussed this before and I think the most compelling argument for passive domain support in libvirt (and even at a lower level such as Xend) is the following scenario:
Let's say a user creates a virtual FC6 virtual machine that she is going to use to test her apps under Gnome (she's a KDE user). The machine isn't always there b/c she rarely needs it. She created it with some nifty KDE tool that has yet to be written.
Of course, she finds out about this neat applet that you can use to managing running domains. If there isn't a common way of accessing passive domains then she won't see the FC6 VM.
Indeed, that interoperability is the core problem, however, I think that solving it higher up the stack could result in a more flexible & effective solution. Even if the solution is not part of libvirt, there is still plenty of scope for defining a standard way to enumerate & interact with passive domains. Storing the passive domains in libvirt in the low-level XML format will mean that the KDE virt tool will need to either manage all its passive domains in the libvirt XML format, or maintain a denormalization of its real persistent format & keep that synced with libvirt, neither of which are particularly nice. So say, the Virt Applet finds out about the passive domain 'Gnome test' that was creted with the KDE virt tool, and wants to present a drop down menu allowing the user to run it. Now it could get the XML def for the passive domain from a registry in libvirt, and then pass it backk to libvirt's create domain method - bypassing the original KDE virt tool which defined the domain. This would all work fine, however, it does constrain what you can do when starting the domain. For example, consider that the original KDE virt tool also has some idea of policy to mediate between different VMs running on the host. WIth the applet getting the config & launching the domain directly via libvirt, the KDE tool is now cut out of the loop - it can merely *react* to the newly started domain. An alternate model for dealing with passive domains could revolve around the provision of a standardized DBus service API, org.example.virt.Manager. At its simplest, this could contain two APIs, 'list inactive domains' and 'start domain'. So the applet can obtain a list of inactive domains and present them in a menu allowing a user to choose one to start. When the user asks the applet to start one, the applet can call the 'start domain' method in the service. At the other end of the picture, the KDE virt tool would provide an implementation of the org.example.virt.Manager DBus service. Thus the applet's request to start the 'Gnome test' domain gets routed to the KDE tool, which in turn talks to libvirt passing down the raw XML description of the domain. The virt tool now has much more flexibility in how it deals with passive domains. For example, it can maintain its master list of passive domains in any format it likes - it might like to separate out its definition of the domain's hardware (# of CPUs, # of NICS, MAC addresses), from the definition of policy (CPU priority, schedular settings, amount of memory). If starting the new domain would over-commit on the host's resources, the tool can now pro-actively suspend, pause or shutdown other domains, or give the user the option to reduce the memory for the new domain before starting. The virt tool could now even be fetching its list of inactive domain on demand from an external source. Or if one has an external management server, one could drop in an alternate implementation of the org.example.virt.Manager DBus service, which talks to this remote server. Thus the applet would transparently be listing & activating domains which are defined & managed remotely from the host itself. Going back to the original problem, the CIM provider would also be able to trivially talk to the local DBus service to get the listing of passive domains.
Now, I don't think that *all* domains should have to have passive equivalents (which is a requirement in the CIM provider today) because of some of the scenarios you outlined. However, I think not attempting to have a common API for passive domains is going to creative a mess as we get a larger number of management tools.
I defintely agree with the need for formalizing an API for dealing with passive domains, just not neccessarily from within libvirt, since it puts applications into a low level straight jacket of 'list domains' & 'start this domain config', preventing the more interesting interactions between both UI applications & other components such as virt policy service all of which run higher up the stack where libvirt has pretty limited visibility. Regards, Dan.
* Definitions of inactive domains are stored on a remote server, libvirt has no visibilty into this system. virDomainSetConfig() would not help since there is no concept of a domain being inactive on a single host, rather inactive domains can be deployed to any host in the server farm at will.
* Definitions are visible on the local filesystem, but this FS is shared NFS / SAN storage. Again while their config files may be technically visible via the shared filesystem, there is no meaningful association between an inactive config & a host.
* Domains are created from config 'templates'. The template defines a generic resource configuration, turned into a fully fledged config on-demand - perhaps with the variables plugged into the template coming from an alternate datastore (SQL databsae perhaps). In this case there would be no sense in storing pre-made XML configs, since the master info is a combo of the template & SQL database entries.
So, really the only case where there is any meaningful concept of 'passive' domain configs associated with a static single host model - for example that provided by XenD which happen to have a simple store of inactive domains in /etc/xen. There are so many other interesting management scenarios where inactive domains don't really have a persistent association with a single host, or indeed any passive configuration at all, that I'm not sure there would be much value in libvirt to representing passive domains at all.
Thinking about what tools might need information on inactive domains:
- Desktop management tool - cf VMWare Workstation. Possibly, but even here there is no reason such a tool represents inactive domains in terms of a libvirt XML config file. It may well have generic templates eg 'Linux Web Server and instances 'web0.google.com', 'web1.google.com', 'web2.google.com', where the only instance specific data is the hostname and master disk image.
- Data center management tool - it may well have a concept of inactive domain configs, but it almost certainly won't be associated with a single host, so unlikely to be asking libvirt to list inactive domains.
I struggle to think of cases where libvirt would need to provide info on inactive domains, which are not distorted to serve the needs of a specific application's representational model. If a local app wants to maintain a local list of config files for inactive domains, let it do so itself.
Dan.
-- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
participants (7)
-
Anthony Liguori
-
Anthony Liguori
-
Daniel P. Berrange
-
Daniel Veillard
-
David Lutterkort
-
Jim Fehlig
-
Karel Zak