Re: [libvirt] migration of vnlink VMs

On Fri, Apr 29, 2011 at 04:12:55PM -0400, Laine Stump wrote: IMHO having a 'mode' here is throwing away the main reason for using type=network in the first place - namely independance from this host config element. None of this really feels right to me. With this proposed schema, there is basically nothing in common between the existing functionality for <network> and this new functionality except for the <name> and <uuid> elements. Apps which know how to deal with existing <network> schema will have no ability to interpret this new data at all. Quite probably they will mis-interpet it as providing an isolated virtual network, with no IP addr set, since this design isn't actually changing any attribute value that they currently look for. Either we need to make this align with the existing schema, or we need to put this under a completely separate set of APIs. I think we can likely do better with the schema design and achieve the former. I don't really like the fact that this design has special cased the num(intefaces) == 1 case to have a completely different XML schema. eg we have this: <source dev='eth0' mode='vepa'/> And this <source mode='vepa'> <pool> <interface name='eth10' maxConnect='1'/> </pool> both meaning the same thing. There should only be one representation in the schema for this kind of thing. As mentioned above, I think this design is wrong, because it is not taking any account of the current schema for <network> which defines the various routed modes. Currently <network> supports 3 connectivity modes - Non-routed network, separate subnet (no <forward> element present) - Routed network, separate subnet with NAT (<forward mode='nat'/>) - Routed network, separate subnet (<forward mode='route'/>) Following on from this, I can see another couple of routed modes - Routed network, IP subnetting - Routed network, separate subnet with VPN And the core goal here is to replae type=bridge and type=direct from the domain XML, which means we're adding several bridging modes - Bridged network, eth + bridge + tap (akin to type=bridge) - Bridged network, eth + macvtap (akin to type=direct) - Bridged network, sriov eth + bridge + tap (akin to type=bridge) - Bridged network, sriov eth + macvtap (akin to type=direct) The macvtap can be in 4 modes, so perhaps it is probably better to consider them separately - Bridged network, eth + bridge + tap - Bridged network, eth + macvtap + vepa - Bridged network, eth + macvtap + private - Bridged network, eth + macvtap + passthrough - Bridged network, eth + macvtap + bridge - Bridged network, sriov eth + bridge + tap - Bridged network, sriov eth + macvtap + vepa - Bridged network, sriov eth + macvtap + private - Bridged network, sriov eth + macvtap + passthrough - Bridged network, sriov eth + macvtap + bridge I can also perhaps imagine another VPN mode: - Bridged network, with VPN The current routed modes can route to anywhere, or be restricted to a particular network interface eg with <forward dev='eth0'/>. It only allows for a single interface, though even for routed modes it could be desirable to list multiple devs. The other big distinction is that the <network> modes which do routing, include interface configuration data (ie the IP addrs & bridge name) which is configured on the fly. It looks like with the bridged modes, you're assuming the app has statically configured the interfaces via the virInterface APIs already, and this just points to an existing configured interface. This isn't neccessarily a bad thing, just an observation of a significant difference. So if we ignore the <ip> and <domain> elements from the current <network> schema, then there are a handful of others which we need to have a plan for <forward mode='nat|route'/> (omitted completely for isolated networks) <bridge name="virbr0" /> (auto-generated/filled if omitted) <mac address='....'/> (auto-generated/filled if omitted) The <forward> element can have an optional dev= attribute. I think the key attribute is the <forward> mode= attribute. I think we should be adding further values to that attribute for the new network modes we want to support. We should also make use of the dev= attribute on <forward> where practical, and/or extend it. We could expand the list of <foward> mode values in a flat list - route - nat - bridge (brctl) - vepa - private - passthru - bridge (macvtap) NB: really need to avoid using 'bridge' in terminology, since all 5 of the last options are really 'bridge'. Or we could introduce a extra attribute, and have a 2 level list - <forward layer='link'/> (for all ethernet layer bridging) - <forward layer='network'/> (for all IP layer bridging aka routing) So the current modes would be <forward layer='network' mode='route|nat'/> And new bridging modes would be <forward layer='link' mode='bridge-brctl|vepa|private|passthru|bridge-macvtap'/> For the brctl/macvtap modes, the dev= attribute on <forward> could point to the NIC being used, while with brctl modes, <bridge> would also be present. In the SRIOV case, we potentiallly need a list of interfaces. For this we probably want to use <forward dev='eth0'> <interface dev='eth0'/> <interface dev='eth1'/> <interface dev='eth2'/> ... </forward> NB, the first interface is always to be listed both as a dev= attribute (for compat with existing apps) *and* as a child <interface> element (for apps knowing the new schema). The maxConnect= attribute from your examples above is an interesting thing. I'm not sure whether that is neccessarily a good idea. It feels similar to VMWare's "port group" idea, but I don't think having a simple 'maxConnect=' attribute is sufficient to let us represent the vmware port group idea. I think we might need an more explicit element eg <portgroup count='5'> <interface dev='eth2'/> </portgroup> eg, so this associates a port group which allows 5 clients (VM NICs) with the uplink provided by eth2 (which is assumed to be listed under <forward>). So a complete SRIOV example might be <network> <name>Foo</name> <forward dev='eth0' layer='link' mode='vepa'> <interface dev='eth0'/> <interface dev='eth1'/> <interface dev='eth2'/> ... </forward> <portgroup count='10'> <interface dev='eth0'/> </portgroup> <portgroup count='5'> <interface dev='eth1'/> </portgroup> <portgroup count='5'> <interface dev='eth2'/> </portgroup> </network> The <virtualport> parameters for VEPA/VNLink could either be stored at the top level under <network>, or inside <portgroup> or both. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|