[Libvir] Semantics for ListDomains/ ListDefinedDomains
So I was thinking a little about lifecycle support for passive domains in libvirt and wanted to clarify the intended semantics of the two methods: virConnectListDomains virConnectListDefinedDomains Am I correct in thinking that virConnectListDefinedDomains will list a domain if-and-only-if its state == 'shutoff', and that virConnectListDomains will list a domain if-and-only-if its state != 'shutoff' ie, when I start a defined domain with 'virDomainCreate', it will be removed from the 'ListDefinedDomains' set and added to the 'ListDomains' set. Likewise when shutting down it'll move back the other way. Or will the virConnectListDefinedDomains call always list a domain with a config file, even if it is running ? I realize this is a little hypothetical since XenD doesn't have lifecycle management yet, but it matters to the test backend, and any potential QEMU / UML backend, and the future XenD XML-RPC backend Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Wed, Aug 23, 2006 at 07:16:45PM +0100, Daniel P. Berrange wrote:
So I was thinking a little about lifecycle support for passive domains in libvirt and wanted to clarify the intended semantics of the two methods:
virConnectListDomains virConnectListDefinedDomains
Am I correct in thinking that virConnectListDefinedDomains will list a domain if-and-only-if its state == 'shutoff', and that virConnectListDomains will list a domain if-and-only-if its state != 'shutoff'
yes
ie, when I start a defined domain with 'virDomainCreate', it will be removed from the 'ListDefinedDomains' set and added to the 'ListDomains' set. Likewise when shutting down it'll move back the other way. Or will the virConnectListDefinedDomains call always list a domain with a config file, even if it is running ?
I expected to have 2 disjoint sets, a partition.
I realize this is a little hypothetical since XenD doesn't have lifecycle management yet, but it matters to the test backend, and any potential QEMU / UML backend, and the future XenD XML-RPC backend
I was tempted to do an implementation just local to the library instance in the case there is no support by the virtualization engine. If you think you will use it then I should really implement it :-) Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Wed, Aug 23, 2006 at 02:21:03PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 07:16:45PM +0100, Daniel P. Berrange wrote:
So I was thinking a little about lifecycle support for passive domains in libvirt and wanted to clarify the intended semantics of the two methods:
virConnectListDomains virConnectListDefinedDomains
Am I correct in thinking that virConnectListDefinedDomains will list a domain if-and-only-if its state == 'shutoff', and that virConnectListDomains will list a domain if-and-only-if its state != 'shutoff'
yes
Ok, that makes sense - although means there's a bug in test backend :-)
I realize this is a little hypothetical since XenD doesn't have lifecycle management yet, but it matters to the test backend, and any potential QEMU / UML backend, and the future XenD XML-RPC backend
I was tempted to do an implementation just local to the library instance in the case there is no support by the virtualization engine. If you think you will use it then I should really implement it :-)
Trouble is I think we really badly need an implementation that is persistent. Tools like the 'xeninst[1]' package are using libvirt for creating domains, but at the same time manually writing out config files into /etc/xen - this means they are loosing an important benefit of libvirt - namely isolation from Xen instability/changes. Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that hard part sooner rather than later so I've been trying to thing of ways we could attack it. Now the key problem is that the xm config files can contain/are in fact python code. * Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability * Fork an unprivileged helper python program to exec the config file and re-write it as XML which can be read back in by libvirt The former is more work, but makes me feel better from a security point of view. The latter is more genreal purpose but a small slip up could have big consequences. My personal preference would be the first option & say 'if...else' is unsupported for now Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure. Regards, Dan. [1] see '_get_xen_config' method, line 140 http://hg.et.redhat.com/virt/applications/xeninst--devel?f=d21d927fd7c4;file... -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Wed, Aug 23, 2006 at 09:22:08PM +0100, Daniel P. Berrange wrote:
On Wed, Aug 23, 2006 at 02:21:03PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 07:16:45PM +0100, Daniel P. Berrange wrote:
So I was thinking a little about lifecycle support for passive domains in libvirt and wanted to clarify the intended semantics of the two methods:
virConnectListDomains virConnectListDefinedDomains
Am I correct in thinking that virConnectListDefinedDomains will list a domain if-and-only-if its state == 'shutoff', and that virConnectListDomains will list a domain if-and-only-if its state != 'shutoff'
yes
Ok, that makes sense - although means there's a bug in test backend :-)
I realize this is a little hypothetical since XenD doesn't have lifecycle management yet, but it matters to the test backend, and any potential QEMU / UML backend, and the future XenD XML-RPC backend
I was tempted to do an implementation just local to the library instance in the case there is no support by the virtualization engine. If you think you will use it then I should really implement it :-)
Trouble is I think we really badly need an implementation that is persistent. Tools like the 'xeninst[1]' package are using libvirt for creating domains, but at the same time manually writing out config files into /etc/xen - this means they are loosing an important benefit of libvirt - namely isolation from Xen instability/changes.
Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that hard part sooner rather than later so I've been trying to thing of ways we could attack it. Now the key problem is that the xm config files can contain/are in fact python code.
I could see a problem with random apps writing to /etc/ , even if run as root that won't fly in general. But well if the goal is compatibility with existing xen tools, that may be a sufficient reason.
* Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability
I'm not too concerned by handling only a subset, this should be data and processed as such IMHO.
* Fork an unprivileged helper python program to exec the config file and re-write it as XML which can be read back in by libvirt
The former is more work, but makes me feel better from a security point of view.
Writing a parser doesn't frighten me too much :-)
The latter is more genreal purpose but a small slip up could have big consequences. My personal preference would be the first option & say 'if...else' is unsupported for now
and import, and ....
Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure.
We are basically in agreement :-) So I write that parser ? Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Wed, Aug 23, 2006 at 05:15:26PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 09:22:08PM +0100, Daniel P. Berrange wrote:
Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that hard part sooner rather than later so I've been trying to thing of ways we could attack it. Now the key problem is that the xm config files can contain/are in fact python code.
I could see a problem with random apps writing to /etc/ , even if run as root that won't fly in general. But well if the goal is compatibility with existing xen tools, that may be a sufficient reason.
Well there's unlikely to be random apps writing into /etc/xen unless they're related to Xen config. We can ust blacklisted the 'xend-config.sxp' file (& perhaps the xmexample* files)
* Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability
I'm not too concerned by handling only a subset, this should be data and processed as such IMHO.
Sounds good.
Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure.
We are basically in agreement :-) So I write that parser ?
Sounds like we should, unless anyone (CIM guys ?) listening in has better suggestions ? Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Pragmatically, I think a simple parser that only handles simple xm config files would probably be fine 98% of the time, especially if you expect in most situations the VMs will be created via libvirt anyway. The issue for us with our old libxm interface was for for our particular product offering we just didnt have the option to say "also supports most existing xm configurations" [in terms of official marketing product support statements it was 'better' to say we dont support any...]. If you are OK with recommending to users to use libvirt-based tools to create DomUs, but that most other typical xm config files are supported too (if you put them in such and such place) then I dont see a problem. At the *very* worst, is there a way you can pipe an arbitrary xm config file thru xm and have it spit out what it would have done, without actually doing it? That could be a reliable fallback migration path perhaps. - Gareth Dr. Gareth S. Bestor IBM Linux Technology Center M/S DES2-01 15300 SW Koll Parkway, Beaverton, OR 97006 503-578-3186, T/L 775-3186, Fax 503-578-3186 "Daniel P. Berrange" <berrange@redhat. To com> Daniel Veillard Sent by: <veillard@redhat.com> libvir-list-bounc cc es@redhat.com libvir-list@redhat.com Subject Re: [Libvir] Semantics for 08/23/06 05:23 PM ListDomains/ ListDefinedDomains Please respond to "Daniel P. Berrange" <berrange@redhat. com> On Wed, Aug 23, 2006 at 05:15:26PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 09:22:08PM +0100, Daniel P. Berrange wrote:
Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that hard part sooner rather than later so I've been trying to thing of ways we could attack it. Now the key problem is that the xm config files can contain/are in fact python code.
I could see a problem with random apps writing to /etc/ , even if run as root that won't fly in general. But well if the goal is compatibility with existing xen tools, that may be a sufficient reason.
Well there's unlikely to be random apps writing into /etc/xen unless they're related to Xen config. We can ust blacklisted the 'xend-config.sxp' file (& perhaps the xmexample* files)
* Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability
I'm not too concerned by handling only a subset, this should be data and processed as such IMHO.
Sounds good.
Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure.
We are basically in agreement :-) So I write that parser ?
Sounds like we should, unless anyone (CIM guys ?) listening in has better suggestions ? Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
On Wed, Aug 23, 2006 at 07:04:17PM -0700, Gareth S Bestor wrote:
Pragmatically, I think a simple parser that only handles simple xm config files would probably be fine 98% of the time, especially if you expect in most situations the VMs will be created via libvirt anyway. The issue for us with our old libxm interface was for for our particular product offering we just didnt have the option to say "also supports most existing xm configurations" [in terms of official marketing product support statements it was 'better' to say we dont support any...]. If you are OK with recommending to users to use libvirt-based tools to create DomUs, but that most other typical xm config files are supported too (if you put them in such and such place) then I dont see a problem.
Even if people are creating xm config files with non-libvirt tools, or even manually by hand, the likelihood of using the full python expressions is pretty small. For that small set, it would be trivial for an admin to change the config to remove the unsafe python expressions. IMHO, this is a very worthwhile tradeoff between security & features. If the libvirt parsr did detect unsupported python expression, we could trivially propagate the error condition 'UNSAFE_CONFIG_EXPRESSION' to the application to bring it to the user's attention. So in practical terms support would be very good.
At the *very* worst, is there a way you can pipe an arbitrary xm config file thru xm and have it spit out what it would have done, without actually doing it? That could be a reliable fallback migration path perhaps.
This brings us back to the issue that we don't want to execute arbitrary code to read a config file's data - its just asking for trouble IMHO. It is pretty easy for an admin to convert the config into the sanitised format. Regards, Dan.
On Wed, Aug 23, 2006 at 05:15:26PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 09:22:08PM +0100, Daniel P. Berrange wrote:
Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that hard part sooner rather than later so I've been trying to thing of ways we could attack it. Now the key problem is that the xm config files can contain/are in fact python code.
I could see a problem with random apps writing to /etc/ , even if run as root that won't fly in general. But well if the goal is compatibility with existing xen tools, that may be a sufficient reason.
Well there's unlikely to be random apps writing into /etc/xen unless they're related to Xen config. We can ust blacklisted the 'xend-config.sxp' file (& perhaps the xmexample* files)
* Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability
I'm not too concerned by handling only a subset, this should be data and processed as such IMHO.
Sounds good.
Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure.
We are basically in agreement :-) So I write that parser ?
Sounds like we should, unless anyone (CIM guys ?) listening in has better suggestions ?
Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
-- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
-- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Wed, Aug 23, 2006 at 05:15:26PM -0400, Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 09:22:08PM +0100, Daniel P. Berrange wrote:
On Wed, Aug 23, 2006 at 02:21:03PM -0400, Daniel Veillard wrote:
I was tempted to do an implementation just local to the library instance in the case there is no support by the virtualization engine. If you think you will use it then I should really implement it :-)
Trouble is I think we really badly need an implementation that is persistent. Tools like the 'xeninst[1]' package are using libvirt for creating domains, but at the same time manually writing out config files into /etc/xen - this means they are loosing an important benefit of libvirt - namely isolation from Xen instability/changes.
Now it would be pretty easy for libvirt to convert the XML file passed into virDefineDomain into a config file for xend & stuf it in /etc/xen The hard part is the reverse - enumerating the config files in the dir & turning them back into XML. I fear we may have to tackle that [...] * Write a tiny parser for a trivial subset - basically enough to handle the (key, string) pairs & (key, list of string) pairs. Certainly doable - depending on how general purpose we want to get - do we care about the 'if..else' conditional used in the sample /etc/xen/xmexample.vti config file ? We can certainly make a valid case saying we don't care about this because the libvirt XML -> xm config conversion would not generate config using that capability
I'm not too concerned by handling only a subset, this should be data and processed as such IMHO.
* Fork an unprivileged helper python program to exec the config file and re-write it as XML which can be read back in by libvirt
The former is more work, but makes me feel better from a security point of view.
Writing a parser doesn't frighten me too much :-) [...]
Not a perfect solution, but would satisfy a great deal of common use cases pretty easily without being intrusive into existing code base & pretty secure.
We are basically in agreement :-) So I write that parser ?
Okay, there is a a first version in CVS under libvirt/src/conf.[ch] It allows only variable assignments and comments, the types supported are the [] lists, integers and strings. Still that should be sufficient to handled most data oriented existing Python Xen configurations. It supports load/lookup/writes as defined in conf.h: ------------------------------------- typedef struct _virConf virConf; typedef virConf *virConfPtr; virConfPtr virConfReadFile (const char *filename); virConfPtr virConfReadMem (const char *memory, int len); int virConfFree (virConfPtr conf); virConfValuePtr virConfGetValue (virConfPtr conf, const char *setting); int virConfWriteFile (const char *filename, virConfPtr conf); int virConfWriteMem (char *memory, int *len, virConfPtr conf); ------------------------------------- Values are open structures: ------------------------------------- /** * virConfType: * one of the possible type for a value from the configuration file * * TODO: we probably need a float too. */ typedef enum { VIR_CONF_NONE = 0, /* undefined */ VIR_CONF_LONG = 1, /* a long int */ VIR_CONF_STRING = 2, /* a string */ VIR_CONF_LIST = 3 /* a list */ } virConfType; /** * virConfValue: * a value from the configuration file */ typedef struct _virConfValue virConfValue; typedef virConfValue *virConfValuePtr; struct _virConfValue { virConfType type; /* the virConfType */ virConfValuePtr next; /* next element if in a list */ long l; /* long integer */ char *str; /* pointer to 0 terminated string */ virConfValuePtr list; /* list of a list */ }; ------------------------------------- If needed it's possible to lookup a value, change the object and save back, I could extend the supported subset to handle expressions or reuse of variables in assignments, but I'm not sure it is really worth it. The parser is strict, if it meets a construct it doesn't recognize it fails and return NULL with an error. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Wed, Aug 30, 2006 at 10:06:29AM -0400, Daniel Veillard wrote:
Okay, there is a a first version in CVS under libvirt/src/conf.[ch] It allows only variable assignments and comments, the types supported are the [] lists, integers and strings. Still that should be sufficient to handled most data oriented existing Python Xen configurations.
I tested it against 30 or so configuration files I've accumulated on my Xen test systems and there were only two tiny adjustments I needed to make. First I extended it to allow '_' in names, eg to handle on_reboot = 'restart' Then I relaxed the list parser to allow a trailing ',' at the end of a list, eg bar = [ "foo", ] Since it looks like xenguest-install had auto-generated a huge number of my config files with trailing ',' in the list of disks / NICs. With those two changes it parses all my example config files perfectly :-) I'm half way through adding support for passsive domains using this config file parser & its looking very promising! Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Daniel Veillard wrote:
On Wed, Aug 23, 2006 at 07:16:45PM +0100, Daniel P. Berrange wrote:
So I was thinking a little about lifecycle support for passive domains in libvirt and wanted to clarify the intended semantics of the two methods:
virConnectListDomains virConnectListDefinedDomains
Am I correct in thinking that virConnectListDefinedDomains will list a domain if-and-only-if its state == 'shutoff', and that virConnectListDomains will list a domain if-and-only-if its state != 'shutoff'
yes
ie, when I start a defined domain with 'virDomainCreate', it will be removed from the 'ListDefinedDomains' set and added to the 'ListDomains' set. Likewise when shutting down it'll move back the other way. Or will the virConnectListDefinedDomains call always list a domain with a config file, even if it is running ?
I expected to have 2 disjoint sets, a partition.
FWIW, I used these exact semantics when implementing this functionality in the provider 'shim'. Jim
I realize this is a little hypothetical since XenD doesn't have lifecycle management yet, but it matters to the test backend, and any potential QEMU / UML backend, and the future XenD XML-RPC backend
I was tempted to do an implementation just local to the library instance in the case there is no support by the virtualization engine. If you think you will use it then I should really implement it :-)
Daniel
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Gareth S Bestor -
Jim Fehlig