[Libvir] [RFC] Life-cycle Management of the domain take2
Hi, I'm making a patch to provide a changing allocation dynamically and statically without keeping state at the library. And this is still uncompleted works, but I would like to hear your comments. Attached patch shows below: # virsh --help setmem NAME setmem - change memory allocation SYNOPSIS virsh setmem <domain> <kilobytes> [ --static | --dynamic ] DESCRIPTION Change the current memory allocation in the guest domain. OPTIONS <domain> domain name, id or uuid <kilobytes> number of kilobytes of memory --static static change --dynamic dynamic change First, I made upper interface. --- virsh.c ( attached file "add_option2virsh.patch ) - add 2 options as follows: --dynamic, --static - call new APIs: virDomainSetMaxMemory -> virDomainSetMaxMemoryScope virDomainSetMemory -> virDomainSetMemoryScope virDomainSetVcpus -> virDomainSetVcpusScope --- libvirt.c ( attached file "add_option2libvirt.patch ) - add 3 APIs: virDomainSetMaxMemoryScope virDomainSetMemoryScope virDomainSetVcpusScope and I will add lower driver as follows in the case of setmem. --- drivers ( attached file "add_option2driver.patch ) - xen_unified.c : add xenUnifiedDomainSetMemoryScope - xend_internal.c : add xenDaemonDomainSetMemoryScope - other : set NULL to domainSetMemoryScope I plan to achieve allocation dynamically and statically by such a way at xenDaemonDomainSetMemoryScope. 1. get configuration from the Xen Daemon using sexpr_get. 2. change the value when static option is specified, change the node of configuration using new API that I will make. when dynamic option is specified, change the allocation using xend_op 3. set configuration to the Xen Daemon using xend_op Why I did not use virDomainGetXMLDesc and virDomainDefineXML was some elements ware lost in current XML. For example, there is no "vncpasswd", "cpus", "localtime" in XML configuration with virDomainGetXMLDesc. So when I set the "vncpasswd" the "vncpasswd" will disappear after I define the XML configuration. How does this sound ? Thanks, Saori Fukuta.
On Fri, Apr 27, 2007 at 07:57:09PM +0900, Saori Fukuta wrote:
Hi,
I'm making a patch to provide a changing allocation dynamically and statically without keeping state at the library. And this is still uncompleted works, but I would like to hear your comments.
I dislike the fact that we are making API confusing. At the beginning, it was simple, a domain is a running instance, and API works on domain. Then we started adding support for defined domains, with the premice that a domain was either active or inactive, so the change in API semantic was minimal. Now you are proposing to override this with another layer of APIs which allow a domain to be both the definition file somewhere, or the running instance, or both or something next ... I'm saying, stop ! To me we are trying to push patches without keeping the coehrency of the definitions http://libvirt.org/intro.html "a domain is an instance of an operating system running on a virtualized machine provided by the hypervisor" Maybe we need a different definition for what is a domain description, and then we can add APIs for those, but the confusion generated by the drift of what a Domain may represent, starts now to impact the API and I don't want that ! The goal is to keep the API simple, when you start having APIs where an extra argument is added to change the primary signification of what one of the argument means: - that means you have lost track of the objects exposed by the API - the API is becoming confusing Now I understand you want a way to modify the Domain descriptions. This should *NOT* use the virDomainPtr as the argument in the API. You need to define a different type of object, not overload the semantic of similar API. Define APIs with a different object virDomainDescription which could be loaded from a file, a descriptor or memory, or grabbed from a running virDomain then defines similar API to modify them if needed. But the fact that you can't get some informations like the VNC port number from a running domain, while you could get it from a description file is no excuse to start multiplying APIs. You're just trying to use the wrong object for the job. I don't think the current patches you provided are in the right direction, I'm sorry I didn't manage to express my concern before more clearly but now I think I know why I was resistant to your earlier suggestions. You are trying to add functionalities in libvirt for object we don't have a good representation for (descriptions of domains). But even if we want to do similar kind of operation on those objects they are fundamentally of a different type. And to address this cleanly we need to add support for those type instead. This certainly deserve some discussions about how to define domain description types what kind of methods they would take, but the more I think about it the cleaner and easier it will be if we take that path instead. Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hi Daniel I'm sorry I might be disrespectful a little... I understood descriptions of domains, and I will think over again with members of my team. I appreciate your explaining. thanks a lot ! Saori On Fri, 27 Apr 2007 07:47:51 -0400 Daniel Veillard wrote:
I dislike the fact that we are making API confusing. At the beginning, it was simple, a domain is a running instance, and API works on domain. Then we started adding support for defined domains, with the premice that a domain was either active or inactive, so the change in API semantic was minimal. Now you are proposing to override this with another layer of APIs which allow a domain to be both the definition file somewhere, or the running instance, or both or something next ...
I'm saying, stop ! To me we are trying to push patches without keeping the coehrency of the definitions http://libvirt.org/intro.html
"a domain is an instance of an operating system running on a virtualized machine provided by the hypervisor"
Maybe we need a different definition for what is a domain description, and then we can add APIs for those, but the confusion generated by the drift of what a Domain may represent, starts now to impact the API and I don't want that ! The goal is to keep the API simple, when you start having APIs where an extra argument is added to change the primary signification of what one of the argument means: - that means you have lost track of the objects exposed by the API - the API is becoming confusing
Now I understand you want a way to modify the Domain descriptions. This should *NOT* use the virDomainPtr as the argument in the API. You need to define a different type of object, not overload the semantic of similar API.
Define APIs with a different object virDomainDescription which could be loaded from a file, a descriptor or memory, or grabbed from a running virDomain then defines similar API to modify them if needed. But the fact that you can't get some informations like the VNC port number from a running domain, while you could get it from a description file is no excuse to start multiplying APIs. You're just trying to use the wrong object for the job.
I don't think the current patches you provided are in the right direction, I'm sorry I didn't manage to express my concern before more clearly but now I think I know why I was resistant to your earlier suggestions. You are trying to add functionalities in libvirt for object we don't have a good representation for (descriptions of domains). But even if we want to do similar kind of operation on those objects they are fundamentally of a different type. And to address this cleanly we need to add support for those type instead.
This certainly deserve some discussions about how to define domain description types what kind of methods they would take, but the more I think about it the cleaner and easier it will be if we take that path instead.
Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Fri, Apr 27, 2007 at 10:32:33PM +0900, Saori Fukuta wrote:
Hi Daniel
Hi Saori,
I'm sorry I might be disrespectful a little...
No, you were not. I'm just stating I'm thinking of the problem in different terms, which I think are open to discussion.
I understood descriptions of domains, and I will think over again with members of my team.
I appreciate your explaining.
thanks a lot !
To me changing the memory allocation of a live domain and the value in configuration file are not the same kind of operation, they are similar but trying to keep identical APIs based on virDomainPtr becomes confusing. Sometime you want to change a running domain, somethimes you want to change the config file, sometimes both. By mixing the whole in a single entry point assume you can actually identify both kind of objects (i.e. the domain and the config file it came from). I do not think it is possible: - the domain may not be running - the config file may not be available - the domain may already have diverged from the config file Keeping a coherent view and clean handling of errors looks nearly impossible. So I have a problem with this, and I suggest to isolate configuration file APIs, which in practice would mean client code a bit more verbose but keeping the API simpler. We have a long week-end ahead, I will try to get back with more complete proposal, but I could see a set of API like: proposal 1: - create a config from a string (or file) we could autodetect the kind of configs here and automatically support XML, sexpr, old Xen config files, and also VMWare-like ones - create a config from a running domain (like virDomainGetXMLDesc does) - modify values from a config - get a value from a config - save a config (as libvirt XML) in a string or file proposal 2: Or we could keep with the idea that a virDomain could represent a config file as for Xen unactive guests, but then a virDomain could only represent one or the other but not both. Which mean if you wanted to modify both the config file and the running domain you would need 2 different virDomain objects (and then you would still be using virDomainSetMemory() but calling it twice). The main drawback of proposal 1 is that it increase the API, but would make clear what kind of objects are been managed, i would also allow more specific configuration only option in the future. The main drawback of proposal 2 is that it mixes 2 kind of objects with different capabilities (configuration and running domains), but would keep the API smaller. I certainly need to think more about this :-) Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Hi Daniel, Thank you for your proposal ! I considered the way with it. On Fri, 27 Apr 2007 10:35:15 -0400 Daniel Veillard wrote:
So I have a problem with this, and I suggest to isolate configuration file APIs, which in practice would mean client code a bit more verbose but keeping the API simpler. We have a long week-end ahead, I will try to get back with more complete proposal, but I could see a set of API like:
proposal 1:
- create a config from a string (or file) we could autodetect the kind of configs here and automatically support XML, sexpr, old Xen config files, and also VMWare-like ones - create a config from a running domain (like virDomainGetXMLDesc does) - modify values from a config - get a value from a config - save a config (as libvirt XML) in a string or file
The main drawback of proposal 1 is that it increase the API, but would make clear what kind of objects are been managed, i would also allow more specific configuration only option in the future.
I think that the proposal is good to keep the API simple, though, the number of API increases. Here is an example of the interface. We could provide new API (like virDomainSetMemoryConfig), and use only it for not running domain. running domain --> change the allocation using current API(virDomainSetMemory) not running domain --> change the configuration using new API(virDomainSetMemoryConfig) # virsh setmem foo 200 --- at virsh command 1. if the domain is running -> virDomainSetMemory (current API) 2. if the domain is not running -> virDomainSetMemoryConfig (new API) --- at new API for Xen 1. get a string using expr_get and create a config form a string 2. modify the value, in this case, modify "memory" to 200 3. save a config using define Or to keep the increase to the minimum, we could provide one new command. # virsh mod-config foo -memory 200 --- at virsh command 1. if the domain is running -> return error 2. call new API virDomainModifyConfig --- at new API for Xen 2. get a string using expr_get and create a config form a string 3. modify the value, in this case, modify "memory" to 200 4. save a config using define and other changing allocation commands are alike described above. I think this is the better way than I suggested before. How does this look ? Thanks, Saori Fukuta
On Thu, May 10, 2007 at 12:06:51PM +0900, Saori Fukuta wrote:
Hi Daniel,
Thank you for your proposal ! I considered the way with it.
On Fri, 27 Apr 2007 10:35:15 -0400 Daniel Veillard wrote:
So I have a problem with this, and I suggest to isolate configuration file APIs, which in practice would mean client code a bit more verbose but keeping the API simpler. We have a long week-end ahead, I will try to get back with more complete proposal, but I could see a set of API like:
proposal 1:
- create a config from a string (or file) we could autodetect the kind of configs here and automatically support XML, sexpr, old Xen config files, and also VMWare-like ones - create a config from a running domain (like virDomainGetXMLDesc does) - modify values from a config - get a value from a config - save a config (as libvirt XML) in a string or file
The main drawback of proposal 1 is that it increase the API, but would make clear what kind of objects are been managed, i would also allow more specific configuration only option in the future.
I think that the proposal is good to keep the API simple, though, the number of API increases.
Well I have though about it more, and I'm afraid that proposal 1 makes things more complex due to the duplication of APIs. I think proposal 2 makes more sense, i.e. be able to create a Domain object from a config file. We just need to add new API to generate an xmlDomainPtr from those data, then store the path or data in the domain structure, and also a new API to be able to differentiate whether a domain is about a running domain or just the config.
Here is an example of the interface.
We could provide new API (like virDomainSetMemoryConfig), and use only it for not running domain.
The problem is that one need to add one such API for every aspect of the domain configuration, if it was only memory that would not be that bad but we ultimately want to be able to modify everything.
# virsh setmem foo 200
We need to be cautious the Xen model will not match other virtualization engines, for example if you run a KVM the system won't try to keep a database of settings. [...]
I think this is the better way than I suggested before. How does this look ?
Better but I'm not sure it's the right way. I am sorry but I am slow to get there I don't want to make the wrong choise. I think it's better to allow the duality of virDomainPtr, accepting it to be either a representation for a running domain or a representation of the associated configuration file. The fact that with new versions of Xen one can change into the other doesn't sound something that will be common to other engines, and as you noted it actually makes things harder because if you call the API you don't know a-priori without checking the current state which aspect will be modified. So we need to separate the two states, but if we can keep the same APIs for most of the modification entry points and just add a few call to create the configuration domains and being able to ask if a domain is about a config or about a running one. I welcome feedback from others, I think it is a rather complex issue and would feel better commiting to an API here if I could feel some consensus, Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Thu, 10 May 2007 12:36:30 -0400 Daniel Veillard wrote:
Well I have though about it more, and I'm afraid that proposal 1 makes things more complex due to the duplication of APIs. I think proposal 2 makes more sense, i.e. be able to create a Domain object from a config file. We just need to add new API to generate an xmlDomainPtr from those data, then store the path or data in the domain structure, and also a new API to be able to differentiate whether a domain is about a running domain or just the config.
Can I ask something ? That means adding xmlDomainPtr to virDomainPtr, like this ? struct _virDomain { unsigned int magic; /* specific value to check */ int uses; /* reference count */ virConnectPtr conn; /* pointer back to the connection */ xmlDomainPtr conf; <----this one char *name; /* the domain external name */ int id; /* the domain ID */ unsigned char uuid[VIR_UUID_BUFLEN]; /* the domain unique identifier */ };
The problem is that one need to add one such API for every aspect of the domain configuration, if it was only memory that would not be that bad but we ultimately want to be able to modify everything.
Yes, I would like to modify another values, too. There are vcpus, disk, network interface, vncport...etc.
I think it's better to allow the duality of virDomainPtr, accepting it to be either a representation for a running domain or a representation of the associated configuration file. The fact that with new versions of Xen one can change into the other doesn't sound something that will be common to other engines, and as you noted it actually makes things harder because if you call the API you don't know a-priori without checking the current state which aspect will be modified. So we need to separate the two states, but if we can keep the same APIs for most of the modification entry points and just add a few call to create the configuration domains and being able to ask if a domain is about a config or about a running one.
Now, the result of command seems not to be consistent when the domain is not running. So I think if we can separate the API between running domain and not running domain, we get the consistent result. Thanks, Saori Fukuta
On Fri, May 11, 2007 at 10:57:10PM +0900, Saori Fukuta wrote:
On Thu, 10 May 2007 12:36:30 -0400 Daniel Veillard wrote:
Well I have though about it more, and I'm afraid that proposal 1 makes things more complex due to the duplication of APIs. I think proposal 2 makes more sense, i.e. be able to create a Domain object from a config file. We just need to add new API to generate an xmlDomainPtr from those data, then store the path or data in the domain structure, and also a new API to be able to differentiate whether a domain is about a running domain or just the config.
Can I ask something ? That means adding xmlDomainPtr to virDomainPtr, like this ?
struct _virDomain { unsigned int magic; /* specific value to check */ int uses; /* reference count */ virConnectPtr conn; /* pointer back to the connection */ xmlDomainPtr conf; <----this one char *name; /* the domain external name */ int id; /* the domain ID */ unsigned char uuid[VIR_UUID_BUFLEN]; /* the domain unique identifier */ };
I am not sure we can bind the two like this in general. That's something which would be possible if you use a configuration and then ask to start the domain (in a way similar to virDomainCreate() but from a configuration). But we can't expect to have a configuration everytime we get a Domain. If you start virsh and list the domains you will get the virDomainPtr of running domains, but you won't be able to guess in general where the config file for them would be. Does that make sense ?
The problem is that one need to add one such API for every aspect of the domain configuration, if it was only memory that would not be that bad but we ultimately want to be able to modify everything.
Yes, I would like to modify another values, too. There are vcpus, disk, network interface, vncport...etc.
Sounds normal.
I think it's better to allow the duality of virDomainPtr, accepting it to be either a representation for a running domain or a representation of the associated configuration file. The fact that with new versions of Xen one can change into the other doesn't sound something that will be common to other engines, and as you noted it actually makes things harder because if you call the API you don't know a-priori without checking the current state which aspect will be modified. So we need to separate the two states, but if we can keep the same APIs for most of the modification entry points and just add a few call to create the configuration domains and being able to ask if a domain is about a config or about a running one.
Now, the result of command seems not to be consistent when the domain is not running. So I think if we can separate the API between running domain and not running domain, we get the consistent result.
I'm not sure I understand. Say you use virDomainSetMemory() with a very large value. I agree that if you target a running domain you may get an error due to lack of memory on the host, while running the same call on a virDomainPtr addressing a configuration file you are not likely to see such error raised. Are you concerned by this kind of difference in behaviour or something else ? Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Fri, 11 May 2007 14:06:07 -0400 Daniel Veillard wrote:
Can I ask something ? That means adding xmlDomainPtr to virDomainPtr, like this ?
struct _virDomain { unsigned int magic; /* specific value to check */ int uses; /* reference count */ virConnectPtr conn; /* pointer back to the connection */ xmlDomainPtr conf; <----this one char *name; /* the domain external name */ int id; /* the domain ID */ unsigned char uuid[VIR_UUID_BUFLEN]; /* the domain unique identifier */ };
I am not sure we can bind the two like this in general. That's something which would be possible if you use a configuration and then ask to start the domain (in a way similar to virDomainCreate() but from a configuration). But we can't expect to have a configuration everytime we get a Domain. If you start virsh and list the domains you will get the virDomainPtr of running domains, but you won't be able to guess in general where the config file for them would be. Does that make sense ?
Hmm, sorry but I do not really understand why the xmlDomainPtr is needed because I do not want to use "config file". To give an example, I would like to achieve in a manner similar to the way the virt-manager remove the device. --- virtManager/domain.py:remove_device 1. dumpXML 2. modify the defined XML 3. define But, there are the necessary information that is Xen-specific or should not be shown to all-user in sxp-format, and we should not have such information in XML. So I selected using sxp-format instead of XML, i.e. modifying the config at lower layer (xend_internal.c) that I proposed before. --- xend_internal.c 1. root = sexpr_get(domain->conn, "/xend/domain/%s?detail=1",domain->name); 2. snprintf(buf, sizeof(buf), "%lu", memory >> 10); 3. sexpr_chg_node(root, "domain/memory", buf, sizeof(buf)); /* new API */ 4. ret = xend_op(domain->conn, "", "op", "new", "config", buf, NULL);
Now, the result of command seems not to be consistent when the domain is not running. So I think if we can separate the API between running domain and not running domain, we get the consistent result.
I'm not sure I understand. Say you use virDomainSetMemory() with a very large value. I agree that if you target a running domain you may get an error due to lack of memory on the host, while running the same call on a virDomainPtr addressing a configuration file you are not likely to see such error raised. Are you concerned by this kind of difference in behaviour or something else ?
And my concern is that there are 2 kinds of return (some command returns success but some command returns failure) when executing the virsh command while the domain is not running, because of depending on Xen. For example, there are following result for current Xen: success : setmem, setmaxmem failure : setvcpus, vcpupin, device attach/detach(*) "device attach/detach" is a command being proposed by Sunou-san now, but that will return failure because "xm block-attach/detach" is failed while the domain is not running. This might be Bug of Xen, so we should approach this to Xen. But also I think we can choose the way that has least impact of Xen Bug. What do you think about this ? Thanks, Saori Fukuta
Hi, I would like to talk more about this. Can I hear your comments ? Thanks, Saori Fukuta On Wed, 16 May 2007 10:07:06 +0900 Saori Fukuta wrote:
On Fri, 11 May 2007 14:06:07 -0400 Daniel Veillard wrote:
Can I ask something ? That means adding xmlDomainPtr to virDomainPtr, like this ?
struct _virDomain { unsigned int magic; /* specific value to check */ int uses; /* reference count */ virConnectPtr conn; /* pointer back to the connection */ xmlDomainPtr conf; <----this one char *name; /* the domain external name */ int id; /* the domain ID */ unsigned char uuid[VIR_UUID_BUFLEN]; /* the domain unique identifier */ };
I am not sure we can bind the two like this in general. That's something which would be possible if you use a configuration and then ask to start the domain (in a way similar to virDomainCreate() but from a configuration). But we can't expect to have a configuration everytime we get a Domain. If you start virsh and list the domains you will get the virDomainPtr of running domains, but you won't be able to guess in general where the config file for them would be. Does that make sense ?
Hmm, sorry but I do not really understand why the xmlDomainPtr is needed because I do not want to use "config file".
To give an example, I would like to achieve in a manner similar to the way the virt-manager remove the device. --- virtManager/domain.py:remove_device 1. dumpXML 2. modify the defined XML 3. define
But, there are the necessary information that is Xen-specific or should not be shown to all-user in sxp-format, and we should not have such information in XML. So I selected using sxp-format instead of XML, i.e. modifying the config at lower layer (xend_internal.c) that I proposed before. --- xend_internal.c 1. root = sexpr_get(domain->conn, "/xend/domain/%s?detail=1",domain->name); 2. snprintf(buf, sizeof(buf), "%lu", memory >> 10); 3. sexpr_chg_node(root, "domain/memory", buf, sizeof(buf)); /* new API */ 4. ret = xend_op(domain->conn, "", "op", "new", "config", buf, NULL);
Now, the result of command seems not to be consistent when the domain is not running. So I think if we can separate the API between running domain and not running domain, we get the consistent result.
I'm not sure I understand. Say you use virDomainSetMemory() with a very large value. I agree that if you target a running domain you may get an error due to lack of memory on the host, while running the same call on a virDomainPtr addressing a configuration file you are not likely to see such error raised. Are you concerned by this kind of difference in behaviour or something else ?
And my concern is that there are 2 kinds of return (some command returns success but some command returns failure) when executing the virsh command while the domain is not running, because of depending on Xen.
For example, there are following result for current Xen: success : setmem, setmaxmem failure : setvcpus, vcpupin, device attach/detach(*)
"device attach/detach" is a command being proposed by Sunou-san now, but that will return failure because "xm block-attach/detach" is failed while the domain is not running.
This might be Bug of Xen, so we should approach this to Xen. But also I think we can choose the way that has least impact of Xen Bug. What do you think about this ?
Thanks, Saori Fukuta
-- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Saori Fukuta wrote:
Why I did not use virDomainGetXMLDesc and virDomainDefineXML was some elements ware lost in current XML. For example, there is no "vncpasswd", "cpus", "localtime" in XML configuration with virDomainGetXMLDesc. So when I set the "vncpasswd" the "vncpasswd" will disappear after I define the XML configuration.
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere. How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Hi, Thank you for your comment ! On Fri, 11 May 2007 13:33:57 +0100 "Richard W.M. Jones" wrote:
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere.
How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain?
Well, I'm not sure about the meaning of "original config XML" but I do not think it is possible because the original config file might be not XML like a Xen config file. Xen does not have XML file. Is this right ? Thanks, Saori Fukuta
On Fri, May 11, 2007 at 10:59:37PM +0900, Saori Fukuta wrote:
Hi,
Thank you for your comment !
On Fri, 11 May 2007 13:33:57 +0100 "Richard W.M. Jones" wrote:
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere.
How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain?
Well, I'm not sure about the meaning of "original config XML" but I do not think it is possible because the original config file might be not XML like a Xen config file. Xen does not have XML file. Is this right ?
Yes I'm not sure that in general you can always hope to reconstruct the content of the configuration file just from a pointer to a running instance, Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Fri, May 11, 2007 at 02:07:50PM -0400, Daniel Veillard wrote:
On Fri, May 11, 2007 at 10:59:37PM +0900, Saori Fukuta wrote:
Hi,
Thank you for your comment !
On Fri, 11 May 2007 13:33:57 +0100 "Richard W.M. Jones" wrote:
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere.
How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain?
Well, I'm not sure about the meaning of "original config XML" but I do not think it is possible because the original config file might be not XML like a Xen config file. Xen does not have XML file. Is this right ?
Yes I'm not sure that in general you can always hope to reconstruct the content of the configuration file just from a pointer to a running instance,
If the guest was created by libvirt, then I consider it a bug if the XML dump does not allow re-creation in exact same config. If the guest was created by non-libvirt app, then there may be some xen specific bits we don't support in libvirt. So be it - there are some things we simply don't want to support. For any of the latter case, we can at least evaluate whether it makes sense to support them throughout libvirt, and/or accept patches. The main flaw I see currently is that the vncpassword is missing from the XML. We left that out for security obviously. We definitely have to exclude it if using a read-only connection. For a read-write connection we should probably continue to exclude it by default, since apps often write the XML dump to logfiles. We do fortunately have a (currently unused) flags argument to virDomainGetXMLDesc char * virDomainGetXMLDesc (virDomainPtr domain, int flags); I propose we add a flag LIBVIRT_XML_INCLUDE_SENSITIVE or something like that. If that flag were used, then we could include the password in the returned XML. Any apps using that flag would be aware that the XML should be treated as sensitive & thus not logged. The localtime option is something we should consider adding becasue that is relevant for any fullyvirt system, to specify whether the emulated BIOS should be set in localtime or UTC. QEMU/KVM also support a localtime option & folks running Windows guests frequently need it. Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
On Fri, May 11, 2007 at 07:15:33PM +0100, Daniel P. Berrange wrote:
On Fri, May 11, 2007 at 02:07:50PM -0400, Daniel Veillard wrote:
On Fri, May 11, 2007 at 10:59:37PM +0900, Saori Fukuta wrote:
Hi,
Thank you for your comment !
On Fri, 11 May 2007 13:33:57 +0100 "Richard W.M. Jones" wrote:
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere.
How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain?
Well, I'm not sure about the meaning of "original config XML" but I do not think it is possible because the original config file might be not XML like a Xen config file. Xen does not have XML file. Is this right ?
Yes I'm not sure that in general you can always hope to reconstruct the content of the configuration file just from a pointer to a running instance,
If the guest was created by libvirt, then I consider it a bug if the XML dump does not allow re-creation in exact same config.
If the guest was created by non-libvirt app, then there may be some xen specific bits we don't support in libvirt. So be it - there are some things we simply don't want to support. For any of the latter case, we can at least evaluate whether it makes sense to support them throughout libvirt, and/or accept patches.
The main flaw I see currently is that the vncpassword is missing from the XML. We left that out for security obviously. We definitely have to exclude it if using a read-only connection. For a read-write connection we should probably continue to exclude it by default, since apps often write the XML dump to logfiles. We do fortunately have a (currently unused) flags argument to virDomainGetXMLDesc
char * virDomainGetXMLDesc (virDomainPtr domain, int flags);
I propose we add a flag LIBVIRT_XML_INCLUDE_SENSITIVE or something like that. If that flag were used, then we could include the password in the returned XML. Any apps using that flag would be aware that the XML should be treated as sensitive & thus not logged.
On this subject of XML dumps for live VMs, there is the option that the live VM config will be different from the one on disk. Currently the virDomainGetXMLDesc will always give you the live config if the guest is running, and the disk config if the guest is not running. The option missing here is obviously a way to get the disk config if the guest is running. Now we have this convenient flags arg here still which we could use: char *livexml = virDomainGetXMLDesc(dom, 0); char *diskxml = virDomainGetXMLDesc(dom, VIR_DOMAIN_XML_INACTIVE); Once you have the real diskxml, then you can update it & use the virDomainDefineXML(dom, diskxml) to re-write it on disk, virDomainCreate(dom, diskxml) if you want to create a VM with the new config, without actually updating the config on disk Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Hi, Thank you for your comment. On Fri, 11 May 2007 19:29:48 +0100 "Daniel P. Berrange" wrote:
On Fri, May 11, 2007 at 07:15:33PM +0100, Daniel P. Berrange wrote:
If the guest was created by libvirt, then I consider it a bug if the XML dump does not allow re-creation in exact same config.
If the guest was created by non-libvirt app, then there may be some xen specific bits we don't support in libvirt. So be it - there are some things we simply don't want to support. For any of the latter case, we can at least evaluate whether it makes sense to support them throughout libvirt, and/or accept patches.
That means the libvirt basically supports a guest that was created by libvirt ( or virt-install ), right ? I think that would be a waste because we would miss an opportunity to get new customers. I personally want to support the guest was created by non-libvirt app too.
The main flaw I see currently is that the vncpassword is missing from the XML. We left that out for security obviously. We definitely have to exclude it if using a read-only connection. For a read-write connection we should probably continue to exclude it by default, since apps often write the XML dump to logfiles. We do fortunately have a (currently unused) flags argument to virDomainGetXMLDesc
char * virDomainGetXMLDesc (virDomainPtr domain, int flags);
I propose we add a flag LIBVIRT_XML_INCLUDE_SENSITIVE or something like that. If that flag were used, then we could include the password in the returned XML. Any apps using that flag would be aware that the XML should be treated as sensitive & thus not logged.
On this subject of XML dumps for live VMs, there is the option that the live VM config will be different from the one on disk. Currently the virDomainGetXMLDesc will always give you the live config if the guest is running, and the disk config if the guest is not running. The option missing here is obviously a way to get the disk config if the guest is running. Now we have this convenient flags arg here still which we could use:
char *livexml = virDomainGetXMLDesc(dom, 0); char *diskxml = virDomainGetXMLDesc(dom, VIR_DOMAIN_XML_INACTIVE);
Once you have the real diskxml, then you can update it & use the
virDomainDefineXML(dom, diskxml)
to re-write it on disk,
virDomainCreate(dom, diskxml)
if you want to create a VM with the new config, without actually updating the config on disk
It is good idea to use that flags. I would like to use your idea when the fix of domainDumpXML is needed. But I am afraid that I am not in a position to go further, because I have not had a discussion about this RFC still enough. This is a important things, so I keep your idea in my mind ! Thanks, Saori Fukuta
On Wed, May 16, 2007 at 03:28:14PM +0900, Saori Fukuta wrote:
Hi,
Thank you for your comment.
On Fri, 11 May 2007 19:29:48 +0100 "Daniel P. Berrange" wrote:
On Fri, May 11, 2007 at 07:15:33PM +0100, Daniel P. Berrange wrote:
If the guest was created by libvirt, then I consider it a bug if the XML dump does not allow re-creation in exact same config.
If the guest was created by non-libvirt app, then there may be some xen specific bits we don't support in libvirt. So be it - there are some things we simply don't want to support. For any of the latter case, we can at least evaluate whether it makes sense to support them throughout libvirt, and/or accept patches.
That means the libvirt basically supports a guest that was created by libvirt ( or virt-install ), right ?
That is almost correct. We support management of guests that were created by non-libvirt based applictions - if the config options they used are capable of being expressed in terms of libvirt XML.
I think that would be a waste because we would miss an opportunity to get new customers. I personally want to support the guest was created by non-libvirt app too.
That is almost correct. We do support management of guests that were created by non-libvirt based applictions - if the config options they used are capable of being expressed in terms of libvirt XML. Now there are certainly a number of Xen configuration options that we don't currently support in libvirt. We've perhaps got the 75% common case that people use. I don't think we'll ever get 100% because there are some things that are really very Xen specific, but that shouldn't stop us aiming to improve our coverage. There's plenty of scope for more work to let us deal with 95%+ of the Xen config options. IMHO, not hitting the remaining few % is a worthwhile tradeoff given the huge benefits of having a config representation which isn't tied to Xen, particularly if that remaining 5% is the type of niche edge case config rarely used. As always, I'd welcome suggestions & patches for representing more of the Xen config options in the generic XML format. The obvious big outstanding areas we've currently got are, USB (we've already agreed on format, now just need to implement it), serial/parallel devices (cf other thread suggesting a way to represent it), sound devices (no suggestions yet, but several end-user feature requests). Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Hi Dan, On Tue, 22 May 2007 00:36:31 +0100 "Daniel P. Berrange" wrote:
I think that would be a waste because we would miss an opportunity to get new customers. I personally want to support the guest was created by non-libvirt app too.
That is almost correct. We do support management of guests that were created by non-libvirt based applictions - if the config options they used are capable of being expressed in terms of libvirt XML.
Now there are certainly a number of Xen configuration options that we don't currently support in libvirt. We've perhaps got the 75% common case that people use. I don't think we'll ever get 100% because there are some things that are really very Xen specific, but that shouldn't stop us aiming to improve our coverage. There's plenty of scope for more work to let us deal with 95%+ of the Xen config options. IMHO, not hitting the remaining few % is a worthwhile tradeoff given the huge benefits of having a config representation which isn't tied to Xen, particularly if that remaining 5% is the type of niche edge case config rarely used.
Thank you for replying and I understand you. We will try to suggest about libvirt XML by other new thread like USB.
As always, I'd welcome suggestions & patches for representing more of the Xen config options in the generic XML format. The obvious big outstanding areas we've currently got are, USB (we've already agreed on format, now just need to implement it), serial/parallel devices (cf other thread suggesting a way to represent it), sound devices (no suggestions yet, but several end-user feature requests).
Okay. Thanks so much ! Saori.
Daniel Veillard wrote:
On Fri, May 11, 2007 at 10:59:37PM +0900, Saori Fukuta wrote:
Hi,
Thank you for your comment !
It sounds to me like the real solution would be to either fix virDomainGetXMLDesc to return these missing fields, or add another call to get them, or (for the password) to store it elsewhere.
How about loading the original config XML and the XML from the running domain (virDomainGetXMLDesc), and then updating the config XML with only certain fields from the running domain? Well, I'm not sure about the meaning of "original config XML" but I do not think it is possible because the original config file might be not XML
On Fri, 11 May 2007 13:33:57 +0100 "Richard W.M. Jones" wrote: like a Xen config file. Xen does not have XML file. Is this right ?
Yes I'm not sure that in general you can always hope to reconstruct the content of the configuration file just from a pointer to a running instance,
OK, so the problem is two-fold then: (1) libvirt virDomainGetXMLDesc should, if possible, return all the fields from the original XML that was passed in virDomainDefineXML. (2) If we just have a running instance, then perhaps not everything can be retrieved, but virDomainGetXMLDesc ought to retrieve as much as possible -- someone should file bugs for fields that they would like out which could be available. Is that right? Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Richard W.M. Jones -
Saori Fukuta