[libvirt] xml format for openvz driver
Hello! I made review of domain XML format for driver in libvirt. And I have several questions and additions. For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future. For tag domain/os: need to add "ostemplate" desirable "config" For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely. For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network? Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type? Thanks! Evgeniy
On Tue, Jul 22, 2008 at 05:21:41PM +0400, Evgeniy Sokolov wrote:
Hello!
Hi Evgeniy
I made review of domain XML format for driver in libvirt. And I have several questions and additions.
For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future.
Hum ... the id is usually added as @id in the domain - assuming it is running. The decision to go for the numerical id for the <name> value was that it was supposed to be permanent and no extra high level naming scheme would appear. IIRC the 'id' name was the one of the subdirectory for the OpenVZ container. How is the new name support added on top of that ? Unless the directory names can now be allocated as name I don't see how the mapping is done. If the new external name is as good as the old id then just replace the id with the external name in <name> otherwise i wonder what the value of this new naming scheme is and would ignore it
For tag domain/os: need to add "ostemplate" desirable "config"
For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely.
Hum, could you describe those new fields a bit more ?
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
http://libvirt.org/formatdomain.html#elementsNICS and http://libvirt.org/formatnetwork.html#examplesRoute in general in libvirt the networking capabilities are not described per domain but as a separate set of networks with their own definitions. Maybe the OpenVZ driver would have to dynamically add/remove them as domain are instanciated. It would be good to see how the LXC containers plans things too, if we need to extend the model, they should be kept as compatible as possible.
Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type?
Hum, i don't know how i would express that in libvirt Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
On Tue, Jul 22, 2008 at 05:21:41PM +0400, Evgeniy Sokolov wrote:
Hello!
Hi Evgeniy
I made review of domain XML format for driver in libvirt. And I have several questions and additions.
For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future.
Hum ... the id is usually added as @id in the domain - assuming it is running. The decision to go for the numerical id for the <name> value was that it was supposed to be permanent and no extra high level naming scheme would appear. IIRC the 'id' name was the one of the subdirectory for the OpenVZ container. How is the new name support added on top of that ? Unless the directory names can now be allocated as name I don't see how the mapping is done. 'id' is name of subdirectory and is prefix of config file 'name' is saved in config and then can be used.
# vzctl set 105 --name test_vm --save Name test_vm assigned Saved parameters for VE 105 # vzctl status test_vm VEID 105 exist mounted running # grep test_vm /etc/sysconfig/vz-scripts/105.conf NAME="test_vm" container is still in the same directory /vz/root/105.
If the new external name is as good as the old id then just replace the id with the external name in <name> otherwise i wonder what the value of this new naming scheme is and would ignore it
'id' is requred. Other choice is to generate id during creation of container.
For tag domain/os: need to add "ostemplate" desirable "config"
For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely.
Hum, could you describe those new fields a bit more ?
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
http://libvirt.org/formatdomain.html#elementsNICS and http://libvirt.org/formatnetwork.html#examplesRoute
I think it is not the same. OpenVZ has simple way to add/remove IPs to VM # vzctl set 150 --ipadd 2.2.2.2 --save Saved parameters for VE 150 It mean that traffic for all VMs is going via virtual adapter from VM to harware node and then spread over via eth.
in general in libvirt the networking capabilities are not described per domain but as a separate set of networks with their own definitions. Maybe the OpenVZ driver would have to dynamically add/remove them as domain are instanciated.
i.e. create device and then attach them to VM?
It would be good to see how the LXC containers plans things too, if we need to extend the model, they should be kept as compatible as possible.
Also, bridge may be used. It looks like # vzctl set 105 --netif_add eth15,00:11:00:00:00:11,node_eth15,22:11:00:00:00:11 --save It can be desribed as <interface type='bridge'> <source bridge='eth15'/> <mac address='00:11:00:00:00:11'/> </interface> Host device name and host MAC are optional and will be generated.
Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type?
Hum, i don't know how i would express that in libvirt
Daniel
On Tue, Jul 22, 2008 at 05:21:41PM +0400, Evgeniy Sokolov wrote:
Hello!
Hi Evgeniy
I made review of domain XML format for driver in libvirt. And I have several questions and additions.
For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future.
Hum ... the id is usually added as @id in the domain - assuming it is running. The decision to go for the numerical id for the <name> value was that it was supposed to be permanent and no extra high level naming scheme would appear. IIRC the 'id' name was the one of the subdirectory for the OpenVZ container. How is the new name support added on top of that ? Unless the directory names can now be allocated as name I don't see how the mapping is done.
If the new external name is as good as the old id then just replace the id with the external name in <name> otherwise i wonder what the value of this new naming scheme is and would ignore it
For tag domain/os: need to add "ostemplate" desirable "config"
For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely.
Hum, could you describe those new fields a bit more ? It is disk quota for container. OpenVZ mount /vz/root/<ID> into VM as /. It can be changed on running container (increase/decrease). diskinodes is quota for inodes.
Inside VM: [root@fedora-minimal /]# df Filesystem 1K-blocks Used Available Use% Mounted on simfs 1048576 96424 952152 10% / [root@fedora-minimal /]# df -i Filesystem Inodes IUsed IFree IUse% Mounted on simfs 200000 6813 193187 4% /
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
http://libvirt.org/formatdomain.html#elementsNICS and http://libvirt.org/formatnetwork.html#examplesRoute
in general in libvirt the networking capabilities are not described per domain but as a separate set of networks with their own definitions. Maybe the OpenVZ driver would have to dynamically add/remove them as domain are instanciated. It would be good to see how the LXC containers plans things too, if we need to extend the model, they should be kept as compatible as possible.
Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type?
Hum, i don't know how i would express that in libvirt
Daniel
On Tue, Jul 22, 2008 at 05:21:41PM +0400, Evgeniy Sokolov wrote:
Hello!
I made review of domain XML format for driver in libvirt. And I have several questions and additions.
For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future.
For tag domain/os: need to add "ostemplate" desirable "config"
Can you explain exactly what the semantics of the two desired attributes are used for ? If I'm understanding previous OpenVZ discussions correctly, the template is basically a master tar.gz archive, which is unpacked to form the container's 'root filesystem' ? But where does 'config' fit into this. For the filesystem stuff, for LXC we've got a section under <devices> whcih looks like <filesystem type="mount"> <source dir='/some/path'/> <target dir='/'/> </filessytem> Which says 'make /some/path be the root filesystem, using a bind mount'. I could imagine representing the idea of a template with a slight variation like <filesystem type="template"> <source name="template name"/> <target dir='/'/> </filesystem>
For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely.
Does OpenVZ actually attach disks to containers ? I thought it was just filesystem level virtualization, not actual block devices in the guest ? Anyway, this seems like quota management to me, so perhaps adding a <quota type="size" max="10000"/> And <quota type="inodes" max="100"/>
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config. Does OpenVZ support bridging, or NAT for containers ?
Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type?
The latter case is basically hardware device assignment. This isn't something we currently have a way to represent in libvirt, but we certainly need something. I don't think it belongs in the <interface> tag though - for device passthrough we want something more generic, allowing specification of devices based on subsystem, or class. eg Based on subsystem naming, to pass through PCI network card with PCI device number 00:1d.2 <host subsystem='pci' dev='00:1d.2'/> Or based on class specific naming <host class='network' dev='eth1'/> Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Hello!
I made review of domain XML format for driver in libvirt. And I have several questions and additions.
For tag domain: need to add "vmid" or "id" - currenly tag "name" is used for ID. OpenVZ has mandatory parameter ID, but it also support optional parameter "name", which is not implemented for openvz driver now. I plan to support of "name" in future.
For tag domain/os: need to add "ostemplate" desirable "config"
Can you explain exactly what the semantics of the two desired attributes are used for ?
"config" is name of sample config for container. It contains many parameters http://wiki.openvz.org/UBC_parameters_table It is desirable because of many customers like to use config.
If I'm understanding previous OpenVZ discussions correctly, the template is basically a master tar.gz archive, which is unpacked to form the container's 'root filesystem' ? But where does 'config' fit into this. Yes, you template is "ostemplate". It is pre created template of container. Usually, template correspond to OS name centos-4-i386 debian-3.1-ia64 gentoo-20060317-amd64 etc
For the filesystem stuff, for LXC we've got a section under <devices> whcih looks like
<filesystem type="mount"> <source dir='/some/path'/> <target dir='/'/> </filessytem>
Which says 'make /some/path be the root filesystem, using a bind mount'.
I could imagine representing the idea of a template with a slight variation like
<filesystem type="template"> <source name="template name"/> <target dir='/'/> </filesystem>
Yes, we can. But template name is not property of file system.
For tag domain/devices/disk: need to add "diskspace" desirable "diskinodes" - it is optional because of "disknodes" are over very rarely.
Does OpenVZ actually attach disks to containers ? I thought it was just filesystem level virtualization, not actual block devices in the guest ?
It is filesystem level virtualization.
Anyway, this seems like quota management to me, so perhaps adding a
<quota type="size" max="10000"/>
And
<quota type="inodes" max="100"/> Good idea. To add into "filesystem" tag.
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config.
Does OpenVZ support bridging, or NAT for containers ? bridging is supported. NAT can be configured via iptables.
Also, OpenVZ may move network adapter to VM (for example, eth1), adapter becomes inaccessible on harware node. How to describe it? Is it ethernet type?
The latter case is basically hardware device assignment. This isn't something we currently have a way to represent in libvirt, but we certainly need something. I don't think it belongs in the <interface> tag though - for device passthrough we want something more generic, allowing specification of devices based on subsystem, or class.
eg Based on subsystem naming, to pass through PCI network card with PCI device number 00:1d.2
<host subsystem='pci' dev='00:1d.2'/>
Or based on class specific naming
<host class='network' dev='eth1'/> This case is more match for OpenVZ network assignment.
Daniel
On Tue, Jul 22, 2008 at 08:41:36PM +0400, Evgeniy Sokolov wrote:
For tag domain/os: need to add "ostemplate" desirable "config"
Can you explain exactly what the semantics of the two desired attributes are used for ?
"config" is name of sample config for container. It contains many parameters http://wiki.openvz.org/UBC_parameters_table It is desirable because of many customers like to use config.
The problem including a 'config' parameter is that this data becomes totally opaque to the management app. Given a piece of XML describing a domain, including a "config=xxxx' parameter, there's no way for the app to find out what the underlying resources settings are.
For the filesystem stuff, for LXC we've got a section under <devices> whcih looks like
<filesystem type="mount"> <source dir='/some/path'/> <target dir='/'/> </filessytem>
Which says 'make /some/path be the root filesystem, using a bind mount'.
I could imagine representing the idea of a template with a slight variation like
<filesystem type="template"> <source name="template name"/> <target dir='/'/> </filesystem>
Yes, we can. But template name is not property of file system.
Well, actually I'd argue that it is. The "source" element refers to the resource in the host OS that forms the source of the filesystem. So for a root filesystem instantiated from a template, I think template name is the most applicable.
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network?
We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config.
Does OpenVZ support bridging, or NAT for containers ? bridging is supported. NAT can be configured via iptables.
If it supports bridging that is sufficient. The libvirt networking APIs, allow us to implement NAT for any VM in terms of the generic bridge support. Basically libvirt creates a bridge device 'virbr0' and sets up NAT rules for that device. A guest VM simply needs to be connected to virbr0, and then NAT automagically works for it. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Tue, Jul 22, 2008 at 08:41:36PM +0400, Evgeniy Sokolov wrote:
For tag domain/os: need to add "ostemplate" desirable "config" Can you explain exactly what the semantics of the two desired attributes are used for ?
"config" is name of sample config for container. It contains many parameters http://wiki.openvz.org/UBC_parameters_table It is desirable because of many customers like to use config.
The problem including a 'config' parameter is that this data becomes totally opaque to the management app. Given a piece of XML describing a domain, including a "config=xxxx' parameter, there's no way for the app to find out what the underlying resources settings are. Can we add to XML some parameters that is in config file? It is because of OpenVZ has no one parameter "memory". It consist of many different
parameters which can be configured separately: kernel memory, application memory, etc I suppose it will be not generic. How can we desing XML to tune more memory limits?
For the filesystem stuff, for LXC we've got a section under <devices> whcih looks like
<filesystem type="mount"> <source dir='/some/path'/> <target dir='/'/> </filessytem>
Which says 'make /some/path be the root filesystem, using a bind mount'.
I could imagine representing the idea of a template with a slight variation like
<filesystem type="template"> <source name="template name"/> <target dir='/'/> </filesystem>
Yes, we can. But template name is not property of file system.
Well, actually I'd argue that it is. The "source" element refers to the resource in the host OS that forms the source of the filesystem. So for a root filesystem instantiated from a template, I think template name is the most applicable.
Sounds good from such point of view. Tag "target" is useless for OpenVZ.
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network? We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config.
Does OpenVZ support bridging, or NAT for containers ? bridging is supported. NAT can be configured via iptables.
If it supports bridging that is sufficient. The libvirt networking APIs, allow us to implement NAT for any VM in terms of the generic bridge support. Basically libvirt creates a bridge device 'virbr0' and sets up NAT rules for that device. A guest VM simply needs to be connected to virbr0, and then NAT automagically works for it.
Do I correctly undertand? For NAT we shoud use tag "interface" with type "bridge" and <target dev="virbr0" />
On Wed, Jul 23, 2008 at 01:33:25PM +0400, Evgeniy Sokolov wrote:
On Tue, Jul 22, 2008 at 08:41:36PM +0400, Evgeniy Sokolov wrote:
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network? We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config.
Does OpenVZ support bridging, or NAT for containers ? bridging is supported. NAT can be configured via iptables.
If it supports bridging that is sufficient. The libvirt networking APIs, allow us to implement NAT for any VM in terms of the generic bridge support. Basically libvirt creates a bridge device 'virbr0' and sets up NAT rules for that device. A guest VM simply needs to be connected to virbr0, and then NAT automagically works for it.
Do I correctly undertand? For NAT we shoud use tag "interface" with type "bridge" and <target dev="virbr0" />
No, you'd have something like <target type='network'> <source network='default'/> </target> And inside your driver you will call virNetworkLookupByName(...) and then virNetworkGetBridge() to find the actual bridge device name. Take a look at the QEMU driver 's qemudNetworkIfaceConnect() method and how it deals with VIR_DOMAIN_NET_TYPE_NETWORK vs the alternative VIR_DOMAIN_NET_TYPE_BRIDGE Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
I just want to summarize XML for OpenVZ will looks like <domain type='openvz'> <name>209</name> <uuid>8dea22b31d52d8f32516782e98ab8fa0</uuid> <devices> <filesystem type="template"> <source name="fedora-core-5-i386" /> <quota type="size" max="10000"/> <quota type="inodes" max="200000"/> </filesystem> <interface type='bridge'> //for bridge <source bridge='eth10'/> <mac address='00:16:3e:5d:c7:9e'/> </interface> <interface type='bridge'> //for NAT <source bridge='eth11'/> <target type='network'> <source network='default'/> </target> </interface> <interface type='network'> //for phisical device <host class='network' dev='eth1'/> </interface> <interface type='route'> //for routing network <mac address='00:16:3e:34:21:9e'/> <ip address="192.168.122.1" /> <ip address="192.168.122.15" /> </interface> </devices> </domain> Please, correct if it is necessary.
On Wed, Jul 23, 2008 at 01:33:25PM +0400, Evgeniy Sokolov wrote:
On Tue, Jul 22, 2008 at 08:41:36PM +0400, Evgeniy Sokolov wrote:
For tag domain/devices/interface: How to describe, if want to add ip addresses for routing network? We'll probably want todo something based on <interface type='ethernet' which is a generic catch all config.
Does OpenVZ support bridging, or NAT for containers ? bridging is supported. NAT can be configured via iptables. If it supports bridging that is sufficient. The libvirt networking APIs, allow us to implement NAT for any VM in terms of the generic bridge support. Basically libvirt creates a bridge device 'virbr0' and sets up NAT rules for that device. A guest VM simply needs to be connected to virbr0, and then NAT automagically works for it. Do I correctly undertand? For NAT we shoud use tag "interface" with type "bridge" and <target dev="virbr0" />
No, you'd have something like
<target type='network'> <source network='default'/> </target>
And inside your driver you will call virNetworkLookupByName(...) and then virNetworkGetBridge() to find the actual bridge device name. Take a look at the QEMU driver 's qemudNetworkIfaceConnect() method and how it deals with VIR_DOMAIN_NET_TYPE_NETWORK vs the alternative VIR_DOMAIN_NET_TYPE_BRIDGE
Regards, Daniel
On Thu, Jul 24, 2008 at 11:48:16AM +0400, Evgeniy Sokolov wrote:
I just want to summarize XML for OpenVZ will looks like
<domain type='openvz'> <name>209</name> <uuid>8dea22b31d52d8f32516782e98ab8fa0</uuid> <devices> <filesystem type="template"> <source name="fedora-core-5-i386" />
THis will also get an element: <target dir="/"/>
<quota type="size" max="10000"/> <quota type="inodes" max="200000"/> </filesystem>
<interface type='bridge'> //for bridge <source bridge='eth10'/> <mac address='00:16:3e:5d:c7:9e'/> </interface>
This is good.
<interface type='bridge'> //for NAT <source bridge='eth11'/> <target type='network'> <source network='default'/> </target> </interface>
I think you mis-understood how I described this previously. The libvirt NAT networking works like this: <interface type='network'> <source network='default'/> <mac address="11:22:33:44:55:66"/> </interface>
<interface type='network'> //for phisical device <host class='network' dev='eth1'/> </interface>
I'm not sure about this - I think I'd still like host device passthough to be an explicit type of device: <host class='network' subsys='pci'> <source name='eth0'/> <address domain="0000" bus="00" slot="1f" function="3"/> </host> With either the 'class' or 'subsys' properties being required. Given one, libvirt can fill in the other as needed. If 'class' is provided, then the <source> element should be used too. If 'subsys' is provided, then <address> provides the hardware device address. This information will all be available directly from the host device enumeration APIs, so we'll want to have a clearly defined mapping between the domain XML for a host device and the host device XML http://www.redhat.com/archives/libvir-list/2008-April/msg00005.html
<interface type='route'> //for routing network <mac address='00:16:3e:34:21:9e'/> <ip address="192.168.122.1" /> <ip address="192.168.122.15" /> </interface>
This is basically the generic type='ethernet' that we already have, but with a few more properties available, so I don't thing there's need for a new type='route' Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
Evgeniy Sokolov