Re: [Libvir] Request for additional entry points
On Thu, 2006-04-13 at 12:00 -0400, libvir-list-request@redhat.com wrote:
We've discussed this before and I think the most compelling argument for passive domain support in libvirt (and even at a lower level such as Xend) is the following scenario:
Let's say a user creates a virtual FC6 virtual machine that she is going to use to test her apps under Gnome (she's a KDE user). The machine isn't always there b/c she rarely needs it. She created it with some nifty KDE tool that has yet to be written.
Of course, she finds out about this neat applet that you can use to managing running domains. If there isn't a common way of accessing passive domains then she won't see the FC6 VM.
Now, I don't think that *all* domains should have to have passive equivalents (which is a requirement in the CIM provider today) because of some of the scenarios you outlined. However, I think not attempting to have a common API for passive domains is going to creative a mess as we get a larger number of management tools.
I think we're confusing the notion of what a passive domain is with what config files happen to be sitting on / exposed to the dom0 machine. I could very easily look at having an rdbms store the info about the passive domain, hand that down to the dom0 via rpc, and directly call the createLinux call. To me, that's still a passive domain, even though it's configs haven't touched disk yet. I guess I'm also struggling to understand why you'd toss this into xenstore... it just seems this is a higher level concept that needs to be tracked in too specific a way by management systems. --Bret
On Fri, Apr 14, 2006 at 10:38:23AM -0400, Bret McMillan wrote:
I think we're confusing the notion of what a passive domain is with what config files happen to be sitting on / exposed to the dom0 machine. I could very easily look at having an rdbms store the info about the passive domain, hand that down to the dom0 via rpc, and directly call the createLinux call. To me, that's still a passive domain, even though it's configs haven't touched disk yet.
Yup, to me they are different level.
I guess I'm also struggling to understand why you'd toss this into xenstore... it just seems this is a higher level concept that needs to be tracked in too specific a way by management systems.
The reason it would be useful to save this in xenstore is to garantee the same vision between different application managing that node (for example a remote supervision tool and a local launcher used by the user). Otherwise just keeping this information in libvirt own memory would be just fine, I'm still unsure the need to synchronize is really there. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, Apr 18, 2006 at 12:40:01PM -0400, Daniel Veillard wrote:
On Fri, Apr 14, 2006 at 10:38:23AM -0400, Bret McMillan wrote:
I think we're confusing the notion of what a passive domain is with what config files happen to be sitting on / exposed to the dom0 machine. I could very easily look at having an rdbms store the info about the passive domain, hand that down to the dom0 via rpc, and directly call the createLinux call. To me, that's still a passive domain, even though it's configs haven't touched disk yet.
Yup, to me they are different level.
I guess I'm also struggling to understand why you'd toss this into xenstore... it just seems this is a higher level concept that needs to be tracked in too specific a way by management systems.
The reason it would be useful to save this in xenstore is to garantee the same vision between different application managing that node (for example a remote supervision tool and a local launcher used by the user). Otherwise
The trouble is this imoses a representational model on the application, which does not neccessarily mesh with the requirements of the application. So by having this in libvirt/xenstore, either the application's model will be compromised, or the app will have its master model how it likes & have to worry about keeping a denormalization of the master model synced to libvirt / xenstore. In the latter case, you're now loosing the guareentee of shared app vision this was trying to achieve in the first place.
just keeping this information in libvirt own memory would be just fine, I'm still unsure the need to synchronize is really there.
What would be the application gain in this scenario - why wouldn't the app just store manage it directly itself & pass the XML to libvirt at time of domain creation? Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Tue, Apr 18, 2006 at 08:11:46PM +0100, Daniel P. Berrange wrote:
On Tue, Apr 18, 2006 at 12:40:01PM -0400, Daniel Veillard wrote:
The reason it would be useful to save this in xenstore is to garantee the same vision between different application managing that node (for example a remote supervision tool and a local launcher used by the user). Otherwise
The trouble is this imoses a representational model on the application, which does not neccessarily mesh with the requirements of the application. So by having this in libvirt/xenstore, either the application's model will be compromised, or the app will have its master model how it likes & have to worry about keeping a denormalization of the master model synced to libvirt / xenstore. In the latter case, you're now loosing the guareentee of shared app vision this was trying to achieve in the first place.
Can you explain your problems in more functional terms. Even with a PhD in CS I have troubles mapping your explanations to concrete case. So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API. As far as I know that would not change any other API except, except maybe listing domains (though I would rather advocate a new API for listing those non-running domains). So where is your problem exactly ? What master model does this break ? As far as I can tell the only change to the current set of API is that you can reserve a domain name without having it running and then a lookup for such a name will bring back this defined but not running domain. Existing code behaviour would remain exactly unchanged, unless you pick a an already defined name.
What would be the application gain in this scenario - why wouldn't the app just store manage it directly itself & pass the XML to libvirt at time of domain creation?
Reserve a domain name for example. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote:
So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API.
Since I have troubles understanding why you have such an issue with this, let's try to be as clear as possible. What I would expect is the following APIs to be added: /* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain); extensions to the current behaviour: - new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains. that's it. Now what is fundamentally wrong with that ? You don't have to use it if you don't need it I assume the problem is harder than this. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, 2006-04-18 at 17:40 -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote:
So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API.
Since I have troubles understanding why you have such an issue with this, let's try to be as clear as possible. What I would expect is the following APIs to be added:
/* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain);
extensions to the current behaviour:
- new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains.
that's it. Now what is fundamentally wrong with that ? You don't have to use it if you don't need it I assume the problem is harder than this.
What I don't understand is why these API's are needed at all - with the xm-based tools you can do all these things very easily from the command line without any need to pull in extra libraries. While it fits well into the CIM model of managing configurations, it seems very non-Unixy to have API's for something that amounts to simple file management tasks. What is the interplay between these API calls and storing config files in the local file system ? Where am I supposed to look to find all inactive domains on a system ? libvirt ? /etc/xen ? Somewhere else in /etc ? If I want to define an inactive domain on a system, where should I put the XML description ? I think there is a much harder question concerning the interplay of libvirt and the xm tools that this API discussion is somewhat sidestepping. Currently, the xm tools set a defacto standard for how you define inactive domains; libvirt will add a second mechanism with the proposed API. And since the libvirt XML descriptions are a much nicer way to describe a domain than the python scripts in /etc/xen, there's a big temptation to write libvirt based tools that use the XML description and replicate (some) of the xm functionality. That would give us three separate ways to define an inactive domain on a local system - madness ensues. I would be very curious to hear how people see how the libvirt XML descriptions and xm or libvirt-based xm-like tools would interact. David
On Tue, 2006-04-18 at 17:43 -0700, David Lutterkort wrote:
I think there is a much harder question concerning the interplay of libvirt and the xm tools that this API discussion is somewhat sidestepping. Currently, the xm tools set a defacto standard for how you define inactive domains; libvirt will add a second mechanism with the proposed API. And since the libvirt XML descriptions are a much nicer way to describe a domain than the python scripts in /etc/xen, there's a big temptation to write libvirt based tools that use the XML description and replicate (some) of the xm functionality. That would give us three separate ways to define an inactive domain on a local system - madness ensues.
I would be very curious to hear how people see how the libvirt XML descriptions and xm or libvirt-based xm-like tools would interact.
David, I think you're capturing my main concern here... deviation from the norm of other tools we'll end up possibly shipping in the distro. I can understand how, from an RHN point of view, these calls can be pretty easily used, but in ways that would be very RHN-specific. If that's different from how the commandline tools we ship w/ the distro deal with things, that could really cause some customer confusion. --Bret
On Tue, Apr 18, 2006 at 05:43:35PM -0700, David Lutterkort wrote:
On Tue, 2006-04-18 at 17:40 -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote: that's it. Now what is fundamentally wrong with that ? You don't have to use it if you don't need it I assume the problem is harder than this.
What I don't understand is why these API's are needed at all - with the xm-based tools you can do all these things very easily from the command line without any need to pull in extra libraries. While it fits well
The solution with xm-based tools and Python based config files is pretty closed for future extensions and GUI applications. I hope it will be replaced with tools based on libvirt which is based on C (=possible bindings to others langs) and XML (=common standard).
into the CIM model of managing configurations, it seems very non-Unixy
gconf <-- ?
to have API's for something that amounts to simple file management tasks.
The API is a way how inform all system tools about your inactive domains. There still will be possible use files and start up new domains by "virsh create /path/dom.conf".
What is the interplay between these API calls and storing config files in the local file system ? Where am I supposed to look to find all inactive domains on a system ? libvirt ? /etc/xen ? Somewhere else in /etc ? If I want to define an inactive domain on a system, where should I put the XML description ?
Everywhere. The API is a way how register your configuration to the system. I think the question is if we need this libvirt low-level solution or we can alive only with some non-libvirt high-level (dbus) solution or we should support both solutions in our tools.
I think there is a much harder question concerning the interplay of libvirt and the xm tools that this API discussion is somewhat sidestepping. Currently, the xm tools set a defacto standard for how you define inactive domains; libvirt will add a second mechanism with the
No second. It's replacement. We don't want to coexist with xm Python config files and libvirt XML definitions in one system.
proposed API. And since the libvirt XML descriptions are a much nicer way to describe a domain than the python scripts in /etc/xen, there's a big temptation to write libvirt based tools that use the XML description and replicate (some) of the xm functionality. That would give us three separate ways to define an inactive domain on a local system - madness ensues.
I would be very curious to hear how people see how the libvirt XML descriptions and xm or libvirt-based xm-like tools would interact.
Karel -- Karel Zak <kzak@redhat.com>
On Tue, 2006-04-18 at 17:40 -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote:
So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API.
Since I have troubles understanding why you have such an issue with this, let's try to be as clear as possible. What I would expect is the following APIs to be added:
/* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain);
extensions to the current behaviour:
- new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains.
that's it.
Daniel, Thanks for the summary, I think it makes it a good bit clearer about what folks are shooting for. Just a couple of questions for the folks thinking about using this state: 1) What state, local to the physical hardware, will be altered by calls to virDomainDefineXML? Just xenstore? Or would this stuff bleed over into configfiles getting dropped onto disk? Or is that simply up to the management app calling into libvirt? 2) Do we expect other local tools to adopt this notion of state? Or will this only live in libvirt-based code? --Bret
On Tue, Apr 18, 2006 at 09:16:19PM -0400, Bret McMillan wrote:
On Tue, 2006-04-18 at 17:40 -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote: /* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain);
extensions to the current behaviour:
- new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains.
that's it.
Daniel,
Thanks for the summary, I think it makes it a good bit clearer about what folks are shooting for. Just a couple of questions for the folks thinking about using this state:
1) What state, local to the physical hardware, will be altered by calls to virDomainDefineXML? Just xenstore? Or would this stuff bleed over into configfiles getting dropped onto disk? Or is that simply up to the management app calling into libvirt?
IMHO, this could mean absolutely no state change on the managed machine, libvirt could very well talk only to the remote node though just xend, libvirt would have those informations in memory, and even the remote xenstore could not be affected at all. Another situation could be a local tool run by the user where the configuration is pulled by the application from say $HOME/xen and configs are written to xenstore. For the CIM model feedback from an expert would be welcome :-)
2) Do we expect other local tools to adopt this notion of state? Or will this only live in libvirt-based code?
If you mean do we expect xm to be extended for this, this seems unlikely as xm is really based on the /etc/xen/ directory and libvirt precisely try to avoid dependancy on any such local database, I don't expect both set of use to be joined in general, but I may be wrong. If you start creating your domains with libvirt, well you won't have an entry in /etc/xen unless you really like to store your data in python scripts which I guess is not very common. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, Apr 18, 2006 at 05:40:07PM -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote:
So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API.
Since I have troubles understanding why you have such an issue with this, let's try to be as clear as possible. What I would expect is the following APIs to be added:
/* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain);
What would you anticipate the scope of the domains defined with these APIs to be. Would they be visible to only to the app defining them (ie just kept in process memory), shared between any locally running app using libvirt on the host, or shared between arbitrary apps connecting to the HV, even if connecting remotely ?
extensions to the current behaviour:
- new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains.
that's it. Now what is fundamentally wrong with that ? You don't have to use it if you don't need it I assume the problem is harder than this.
There is nothing fundamentally wrong - *if* you are only aiming to support the needs of a simple local management tool. In the broader case though it does not look to be effective because it: - Pre-supposes that there is a relation between a passive VM and a particular host. This may be true in the simple local case, of VMWare Workstation type tool, but in a distributed environment it is just doesn't make any sense - you have a group of passives VMs and a group of potential hosts to run them on - there is no 1-to-1 mapping between passive domains & hosts. - Pre-supposes that the application creating/editing the passive VM configurations wants to store them in the libvirt XML format. Again while this may be true in the basic case it doesn't bear up to more interesting scenarios, such as the idea of VM templates. In such a case a generic template would define # of CPUs, # disk adapters, and other general VM capabilities. A passive VM would only maintain perhaps its name, path to disk image & a template name. - Assumes that passive domains are concept which neccessarily even exist a head of time. A web server farm may merely comprise a stateless OS image, and a set of potential hosts - domains for new OS images are defined on the fly as demand requires. Now I know there is nothing that requires an application to make use of these APIs - one could simply call virDomainCreate(const char *xml) at the time the new domain is needed, but in doing so there are now a broad set of scenarios where 'virConnectListDefinedDomains' will return an empty list. So I think my core question is - what are the client application uses cases & scenarios which these APIs are intended to serve. If the app use cases / scenario were clearly described, then it could well be the case that the points I raise above are completely irrelevant. But without this info on uses cases I can't say whether the APIs described are sufficiently flexible or not. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Fri, Apr 21, 2006 at 10:33:29PM +0100, Daniel P. Berrange wrote: [...]
What would you anticipate the scope of the domains defined with these APIs to be. Would they be visible to only to the app defining them (ie just kept in process memory), shared between any locally running app using libvirt on the host, or shared between arbitrary apps connecting to the HV, even if connecting remotely ?
initially the first at least for the implementation test period, then probably the second because resiliency to restart and common view will be needed. The third one I don't think I can technically implement it without forcing xend to switch to a common format, and there is no way I can suggest this will be reachable. [...]
There is nothing fundamentally wrong - *if* you are only aiming to support the needs of a simple local management tool. In the broader case though it does not look to be effective because it: [...] So I think my core question is - what are the client application uses cases & scenarios which these APIs are intended to serve.
CIM providers primarily and similar local management tools or agents. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel P. Berrange wrote:
On Tue, Apr 18, 2006 at 05:40:07PM -0400, Daniel Veillard wrote:
On Tue, Apr 18, 2006 at 04:40:13PM -0400, Daniel Veillard wrote:
So we have 2 more APIs which allows to define the XML for a domain and name it. That then allow to reserve that name, and the domain may be started later with a simpler API.
Since I have troubles understanding why you have such an issue with this, let's try to be as clear as possible. What I would expect is the following APIs to be added:
/* define a domain, but does not start it */ virDomainPtr virDomainDefineXML(virConnectPtr conn, const char *xml); /* undefine a domain but does not stop it if running */ int virDomainUndefine(virDomainPtr domain); /* list the defined domains */ int virConnectListDefinedDomains(virConnectPtr conn, const char **names, int maxnames); /* launch a defined domain */ int virDomainCreate(virDomainPtr domain);
What would you anticipate the scope of the domains defined with these APIs to be. Would they be visible to only to the app defining them (ie just kept in process memory), shared between any locally running app using libvirt on the host, or shared between arbitrary apps connecting to the HV, even if connecting remotely ?
extensions to the current behaviour:
- new state for defined non-running domains showing in virNodeGetInfo - virDomainLookupByName() could return a defined non-running domain - virDomainCreateLinux() would fail if a domain with the same name is already defined - a number of existing APIs would fail on defined but non-running domains.
that's it. Now what is fundamentally wrong with that ? You don't have to use it if you don't need it I assume the problem is harder than this.
There is nothing fundamentally wrong - *if* you are only aiming to support the needs of a simple local management tool. In the broader case though it does not look to be effective because it:
- Pre-supposes that there is a relation between a passive VM and a particular host. This may be true in the simple local case, of VMWare Workstation type tool, but in a distributed environment it is just doesn't make any sense - you have a group of passives VMs and a group of potential hosts to run them on - there is no 1-to-1 mapping between passive domains & hosts.
- Pre-supposes that the application creating/editing the passive VM configurations wants to store them in the libvirt XML format. Again while this may be true in the basic case it doesn't bear up to more interesting scenarios, such as the idea of VM templates. In such a case a generic template would define # of CPUs, # disk adapters, and other general VM capabilities. A passive VM would only maintain perhaps its name, path to disk image & a template name.
- Assumes that passive domains are concept which neccessarily even exist a head of time. A web server farm may merely comprise a stateless OS image, and a set of potential hosts - domains for new OS images are defined on the fly as demand requires.
Now I know there is nothing that requires an application to make use of these APIs - one could simply call virDomainCreate(const char *xml) at the time the new domain is needed, but in doing so there are now a broad set of scenarios where 'virConnectListDefinedDomains' will return an empty list.
So I think my core question is - what are the client application uses cases & scenarios which these APIs are intended to serve. If the app use cases / scenario were clearly described, then it could well be the case that the points I raise above are completely irrelevant. But without this info on uses cases I can't say whether the APIs described are sufficiently flexible or not.
The DMTF System Virtualization Profile describes the idea of a "defined" computer system, which is a computer system residing at the hosting node in the disabled, stopped, or off state but which can be immediately activatable. The computer system's image, disks, etc are available and ready to be consumed at the hosting node. The requested APIs were an attempt to represent the defined state (a computer system that is ready to go, just needs to be activated) further down the stack as I would think this state would be useful for other management apps and models. Certainly in more interesting management scenarios, templates, config, images, etc. could be stored in other places and "deployed" to a selected host. However, it seems valid that the management app may also want to query hosts about previously deployed but inactive domains. E.g. a possible use case: - Management app user selects a VM for deployment and an appropriate target host. - Management app queries target host to see if it knows anything about the VM - finds a "defined" VM already at the host - ensure the config at host matches the centrally maintained config description - activate the VM - find no knowledge of VM at the host - deploy config to host - activate VM From a data center perspective, I don't see the defined state at a particular host as a long-term state but more transitional. A domain will be in the defined state from the time a user reserves the domain on the target host, while the host is being configured to run the domain, and until it is finally activated. Perhaps a similar scenario when the domain is "retired" from a host, i.e. the domain is stopped (transitions to defined state), config torn down, and finally the config deleted from the host. As has been pointed out in this thread, simple on-host management apps could use this as well to manage both defined and active domains. I don't see how the proposed APIs restrict the evolution of more interesting management solutions discussed in this thread. Regards, Jim
participants (6)
-
Bret McMillan -
Daniel P. Berrange -
Daniel Veillard -
David Lutterkort -
Jim Fehlig -
Karel Zak