[Libvir] Persistent Configurations?
Howdy, Thought I'd try to start up a discussion on a topic that I know I've discussed a lot with people in the past. Xend currently stores its configurations in python files somewhere on the file system (usually in /etc/xen). This is problematic from a management tool perspective because the python files may contain arbitrary python code (requiring a python interpreter to read) and there's no central location which makes enumerating possible domains quite difficult. If you think about a typical use-case (or at least, my typical use-case :-)), a user is likely to have a larger number of domains than what are running (say a domain for a bunch of different distros for testing). It would be nice if any management tools could see those domains and easily start them up. I think we could achieve this with the following requirements: 1) Allow domain creation based on an opaque configuration object 2) Allow storing and retrieval of these configuration objects from the XenStore (using a user-specified path) I think this isolates the state well enough that it leaves the majority of the library stateless. Just some random thoughts... Regards, Anthony Liguori
On Tue, Jan 10, 2006 at 11:19:54AM -0600, Anthony Liguori wrote:
Howdy,
Thought I'd try to start up a discussion on a topic that I know I've discussed a lot with people in the past.
Xend currently stores its configurations in python files somewhere on the file system (usually in /etc/xen). This is problematic from a management tool perspective because the python files may contain arbitrary python code (requiring a python interpreter to read) and there's no central location which makes enumerating possible domains quite difficult.
Yeah my first reaction was to provide XML based configuration files that libvir would be able to understand, but I really think the configurations may come from very varied sources possibly databases. That's why I would rather have the library agnostic when it comes to what a configuration may look like.
If you think about a typical use-case (or at least, my typical use-case :-)), a user is likely to have a larger number of domains than what are running (say a domain for a bunch of different distros for testing). It would be nice if any management tools could see those domains and easily start them up.
What is needed to start a domain ? For Xen we know what is needed, at least at the moment (paths to kernels, images, block devices paths, network devices and I/Os), but this may change slightly with full virtualization, and other virtualization engine like QEmu have different ways since they emulate the bootstrap process too. To me we don't have a good answer to this question so that's one more reason to keep this as opaque as possible.
I think we could achieve this with the following requirements:
1) Allow domain creation based on an opaque configuration object 2) Allow storing and retrieval of these configuration objects from the XenStore (using a user-specified path)
I think this isolates the state well enough that it leaves the majority of the library stateless. Just some random thoughts...
Hum ... you will still need to put some decoding of this object inside the library to pass enough information to xend to start it (since I believe that we will need to use the xend API to create domains). This doesn't make the code agnostic to the format but it makes the API agnostic which is important too (though the versionning as xen evolves sounds a bit hellish, we'd better carefully design the versioning in that format :-) Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Daniel Veillard wrote:
Yeah my first reaction was to provide XML based configuration files that libvir would be able to understand, but I really think the configurations may come from very varied sources possibly databases. That's why I would rather have the library agnostic when it comes to what a configuration may look like.
I'll have to think about this one. My biggest fear is that if there are a bunch of management tools capable of creating domains you'll end up with a bunch of different places storing configs.
If you think about a typical use-case (or at least, my typical use-case :-)), a user is likely to have a larger number of domains than what are running (say a domain for a bunch of different distros for testing). It would be nice if any management tools could see those domains and easily start them up.
What is needed to start a domain ?
Required 1) name of domain 2) initial memory 3) maximum memory Optional 4) uuid 5) ssidref 6) poweroff, reboot, crash actions 7) number of vcpus 8) initial vcpu => cpu mapping Builder Specific Info Linux (misnomer, this is really any PV domain) Required 1) kernel Optional 2) ramdisk 3) root device 4) extra (additional kernel args) Vmx (another misnomer, really any FV domain) Required 1) device model path Optional 2) device model arguments 3) DISPLAY Device Specific Info VBD 1) backend ID 2) uname (file:/path/to/file) 3) dev (hda1) 4) mode (r, w, w!) VIF Required 1) backend ID 2) MAC Optional 3) bridge 4) ip 5) script 6) vifname PIO Range 1) starting address 2) enabling address Of course, this is subject to change as new features are added but I'm pretty sure this is pretty complete.
For Xen we know what is needed, at least at the moment (paths to kernels, images, block devices paths, network devices and I/Os), but this may change slightly with full virtualization, and other virtualization engine like QEmu have different ways since they emulate the bootstrap process too.
Fortunately, since FV uses the qemu device model, we could launch qemu pretty easily reusing this existing config format (since qemu already is launched anyway). However, IMHO, the VBD description Xen uses sucks so I'm not sure we want to adopt it as the universal model.
To me we don't have a good answer to this question so that's one more reason to keep this as opaque as possible.
opaqueness means easy backwards and forward compatibility so I'm definitely a fan :-)
I think we could achieve this with the following requirements:
1) Allow domain creation based on an opaque configuration object 2) Allow storing and retrieval of these configuration objects from the XenStore (using a user-specified path)
I think this isolates the state well enough that it leaves the majority of the library stateless. Just some random thoughts...
Hum ... you will still need to put some decoding of this object inside the library to pass enough information to xend to start it (since I believe that we will need to use the xend API to create domains). This doesn't make the code agnostic to the format but it makes the API agnostic which is important too (though the versionning as xen evolves sounds a bit hellish, we'd better carefully design the versioning in that format :-)
I definitely think it might be worthwhile to design a Xen configuration schema that we could keep consistent as things move forward. Some things would obviously map 1-1 like name but other things might require careful thinking (like block devices). Thoughts? Regards, Anthony Liguori
Daniel
On Tue, Jan 10, 2006 at 12:33:24PM -0600, Anthony Liguori wrote:
Daniel Veillard wrote:
Yeah my first reaction was to provide XML based configuration files that libvir would be able to understand, but I really think the configurations may come from very varied sources possibly databases. That's why I would rather have the library agnostic when it comes to what a configuration may look like.
I'll have to think about this one. My biggest fear is that if there are a bunch of management tools capable of creating domains you'll end up with a bunch of different places storing configs.
I expect my QA engineers and the developpers to store their configs in different places on the test cluster :-) you really expect people to use a unified storage ? I didn't expect it myself.
What is needed to start a domain ?
Required 1) name of domain 2) initial memory 3) maximum memory
Optional 4) uuid 5) ssidref 6) poweroff, reboot, crash actions 7) number of vcpus 8) initial vcpu => cpu mapping
Builder Specific Info
Linux (misnomer, this is really any PV domain)
Required 1) kernel
Optional 2) ramdisk 3) root device 4) extra (additional kernel args)
Vmx (another misnomer, really any FV domain)
Required 1) device model path
Optional 2) device model arguments 3) DISPLAY
Device Specific Info
VBD
1) backend ID 2) uname (file:/path/to/file) 3) dev (hda1) 4) mode (r, w, w!)
VIF
Required 1) backend ID 2) MAC
Optional 3) bridge 4) ip 5) script 6) vifname
PIO Range
1) starting address 2) enabling address
Of course, this is subject to change as new features are added but I'm pretty sure this is pretty complete.
Good to have an exhaustive list. Thanks :-)
For Xen we know what is needed, at least at the moment (paths to kernels, images, block devices paths, network devices and I/Os), but this may change slightly with full virtualization, and other virtualization engine like QEmu have different ways since they emulate the bootstrap process too.
Fortunately, since FV uses the qemu device model, we could launch qemu pretty easily reusing this existing config format (since qemu already is launched anyway). However, IMHO, the VBD description Xen uses sucks so I'm not sure we want to adopt it as the universal model.
To me we don't have a good answer to this question so that's one more reason to keep this as opaque as possible.
opaqueness means easy backwards and forward compatibility so I'm definitely a fan :-)
:-)
I definitely think it might be worthwhile to design a Xen configuration schema that we could keep consistent as things move forward. Some things would obviously map 1-1 like name but other things might require careful thinking (like block devices).
Thoughts?
I can't help myself thinking of XML descriptions, but what we actually use are sexpr and that linking with libxml2 should be avoided if possible (in general the linux kernel guys aren't fans of XML and libxml2). Daniel -- Daniel Veillard | Red Hat http://redhat.com/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, 2006-01-10 at 16:12 -0500, Daniel Veillard wrote:
On Tue, Jan 10, 2006 at 12:33:24PM -0600, Anthony Liguori wrote:
Daniel Veillard wrote:
Yeah my first reaction was to provide XML based configuration files that libvir would be able to understand, but I really think the configurations may come from very varied sources possibly databases. That's why I would rather have the library agnostic when it comes to what a configuration may look like.
I'll have to think about this one. My biggest fear is that if there are a bunch of management tools capable of creating domains you'll end up with a bunch of different places storing configs.
I expect my QA engineers and the developpers to store their configs in different places on the test cluster :-) you really expect people to use a unified storage ? I didn't expect it myself.
There should be (at least) suggested and by distributions supported way how to share domain configurations between more tools. I think we can support both -- unified storage (/etc/something) and API that allows to use customized storages (e.g. databases). Karel -- Karel Zak <kzak@redhat.com>
participants (3)
-
Anthony Liguori -
Daniel Veillard -
Karel Zak