3:12 p.m.
To "cut to the chase", go down to the end of the message, where I
outline the proposed XML changes to support this. If everyone approves
of it, I'll make the RNG based on that description and update the
parser, then worry about filling in the functionality.
On 04/29/2011 09:29 AM, Oved Ourfalli wrote:
See my comments below.
Thank you,
Oved
This is very useful, Oved! Thanks for the replies.
----- Original Message -----
> From: "Laine Stump"<laine(a)laine.org>
> To: libvir-list(a)redhat.com
> Sent: Friday, April 29, 2011 3:45:50 PM
> Subject: Re: [libvirt] migration of vnlink VMs
> On 04/28/2011 04:15 AM, Oved Ourfalli wrote:
>>> From: "Laine Stump"<lstump(a)redhat.com
>>> On 04/27/2011 09:58 AM, Oved Ourfalli wrote:
>>>> Laine, hello.
>>>>
>>>> We read your proposal for abstraction of guest<--> host network
>>>> connection in libvirt.
>>>>
>>>> You has an open issue there regarding the vepa/vnlink attributes:
>>>> "3) What about the parameters in the<virtualport> element
that are
>>>> currently used by vepa/vnlink. Do those belong with the host, or
>>>> with the guest?"
>>>>
>>>> The parameters for the virtualport element should be on the guest,
>>>> and not the host, because a specific interface can run multiple
>>>> profiles,
>>> Are you talking about host interface or guest interface? If you
>>> mean
>>> that multiple different profiles can be used when connecting to a
>>> particular switch - as long as there are only a few different
>>> profiles,
>>> rather than each guest having its own unique profile, then it still
>>> seems better to have the port profile live with the network
>>> definition
>>> (and just define multiple networks, one for each port profile).
>>>
>> The profile names can be changed regularly, so it looks like it will
>> be better to put them in the guest level, so that the network host
>> file won't have to be changed on all hosts once something has
>> changed in the profiles.
>>
>> Also, you will have a duplication of data, writing all the profile
>> name on all the hosts that are connected to the vn-link/vepa switch.
> But is it potentially the same for many/all guests, or is it
> necessarily
> different for every guest? If it's the former, then do you have more
> guests, or more hosts?
>
I guess it will be the same for many guests. There will be some profiles, and each group
of guests will use the same profile, according to its demands.
Except instanceid, as you point out below. Since there is at least one
exception to the "same for all guests", we do need to keep <virtualport>
with the guest, but since many are changed, we should also supply it
with the network. Things are starting to take shape now :-)
>>>> so it will be a mistake to define a profile to be
interface
>>>> specific on the host. Moreover, putting it in the guest level
>>>> will
>>>> enable us in the future (if supported by libvirt/qemu) to
>>>> migrate
>>>> a vm from a host with vepa/vnlink interfaces, to another host
>>>> with
>>>> a bridge, for example.
>>> It seems to me like doing exactly the opposite would make it easier
>>> to
>>> migrate to a host that used a different kind of switching (from
>>> vepa
>>> to
>>> vnlink, or from a bridged interface to vepa, etc), since the port
>>> profile required for a particular host's network would be at the
>>> host
>>> waiting to be used.
>> You are right, but we would want to have the option to prevent that
>> from happening in case we wouldn't want to allow it.
>> We can make the ability to migrate between different network types
>> configurable, and we would like an easy way to tell libvirt -
>> "please allow/don't allow it".
> I *think* what you're getting at is this situation:
>
> HostA has a group of interfaces that are connected to a vepa-capable
> switch, HostB has a group of interfaces connected to a vnlink-capable
> switch. Guest1 is allowed to connect either via a vnlink switch or a
> vepa switch, but Guest2 should only use vepa.
>
> In that case, HostA would have a network that had a pool of interfaces
> and type "vepa", while HostB would have a pool of interfaces and a
> type
> "vnlink". Guest1 could be accommodated by giving both networks the
> same
> name, or Guest2 could be accommodated by giving each network a
> different
> name (when migrating, if the dest. host doesn't have the desired
> network, the migration would fail). However, using just the network
> naming, it wouldn't be possible to allow both.
>
> I don't think keeping the virtualport parameters only with the guest
> would help (or hurt) this though. What would be needed would be to
> have
> the information about network type *optionally* specified in the guest
> interface config (as well as in the network config); if present the
> migration would only succeed if the given network on the dest host
> matched the given type (and parameters, if any) in the guest config.
>
That would be great. It will enable the flexibility we need
>>>> So, in the networks at the host level you will have:
>>>> <network type='direct'>
>>>> <name>red-network</name>
>>>> <source mode='vepa'>
>>>> <pool>
>>>> <interface>
>>>> <name>eth0</name>
>>>> .....
>>>> </interface>
>>>> <interface>
>>>> <name>eth4</name>
>>>> .....
>>>> </interface>
>>>> <interface>
>>>> <name>eth18</name>
>>>> .....
>>>> </interface>
>>>> </pool>
I'm assuming there will never be a case when an interface pool might
need to be used by *both* a vepa network and a vnlink network (doesn't
really make sense - it will be physically connected to a switch that
supports vnlink, or one that supports vepa; by definition it can't be
connected to both (I don't foresee any switch that supports both!). So
defining the pool inside network should work okay. Since each interface
will always have a name (and just one name), I think it makes sense to
make that an attribute rather than subelement.
>>>> </source>
>>>> </network>
>>>>
>>>> And in the guest you will have (for vepa):
>>>> <interface type='network'>
>>>> <source network='red-network'/>
and if you want to force connecting only to vepa or vnlink, you can do:
<source network='red-network' mode='vepa'/>
for example.
>>>> <virtualport type="802.1Qbg">
>>>> <parameters managerid="11" typeid="1193047"
typeidversion="2"
>>>> instanceid="09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f"/>
>>>> </virtualport>
Likewise, anything that's specified here MUST either match what is in
the <network> definition on the host, or be unspecified on the host. The
final virtualport definition used will be the OR of the two.
>>>> </interface>
>>>>
>>>> Or (for vnlink):
>>>> <interface type='network'>
>>>> <source network='red-network'/>
>>>> <virtualport type="802.1Qbh">
>>>> <parameters profile_name="profile1"/>
>>>> </virtualport>
>>>> </interface>
> What would the interface for a 2nd guest of each type look like? Could
> it be identical? Or might some parameters change for every single
> guest?
For vn-link it will be the same, just the profile_name.
As for vepa, the instanceid is vm specific so it should be on the guest (taken from
http://libvirt.org/formatdomain.html):
"managerid - The VSI Manager ID identifies the database containing the VSI type and
instance definitions. This is an integer value and the value 0 is reserved.
typeid - The VSI Type ID identifies a VSI type characterizing the network access. VSI
types are typically managed by network administrator. This is an integer value.
typeidversion - The VSI Type Version allows multiple versions of a VSI Type. This is an
integer value.
instanceid - The VSI Instance ID Identifier is generated when a VSI instance (i.e. a
virtual interface of a virtual machine) is created. This is a globally unique
identifier."
That's what we know on vepa an vn-link now. I guess that when we'll have the
possibility to test these environments we will learn more on them.
> Perhaps it would be best to have virtualport parameters on both
> network
> and guest interface XML, and merge the two to arrive at what's used
> (the
> network definition could contain all the attributes that would be
> common
> to all guests using that network on that host, and the guest interface
> definition would contain extra parameters specific to that host. In
> the
> case of a parameter being specified in both places, if they were not
> identical, the migration would fail).
>
Sounds good.
Okay, here's a brief description of what I *think* will work. I'll build
up the RNG based on this pseudo-xml:
For the <interface> definition in the guest XML, the main change will be
that <source .. mode='something'> will be valid (but optional) when
interface type='network' - in this case, it will just be used to match
against the source mode of the network on the host. <virtualport> will
also become valid for type='network', and will serve two purposes:
1) if there is a mismatch with the virtualport on the host network, the
migrate/start will fail.
2) It will be ORed with <virtualport> on the host network to arrive at
the virtualport settings actually used.
For example:
<interface type='network'>
<source network='red-network' mode='vepa'/>
<virtualport type='802.1Qbg'>
<parameters instanceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/>
</virtualport>
<mac address='xx:xx:.....'/>
</interface>
(NB: if "mode" isn't specified, and the host network is actually a
bridge or virtual network, the contents of virtualport will be ignored.)
<network> will be expanded by giving it an optional "type" attribute
(which will default to 'virtual'), <source> subelement, and
<virtualport> subelement. When type='bridge', you can specify source
exactly as you would in a domain <interface> definition:
<network type='bridge'>
<name>red-network</name>
<source bridge='br0'/>
</network>
When type='direct', again you can specify source and virtualport pretty
much as you would in an interface definition:
<network type='direct'>
<name>red-network</name>
<source dev='eth0' mode='vepa'/>
<virtualport type='802.1Qbg'>
<parameters managerid="11" typeid="1193047"
typeidversion="2"
instanceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/>
</virtualport>
</network>
However, dev would be optional - if not specified, we would expect a
pool of interfaces to be defined within source, eg:
<network type='direct'>
<name>red-network</name>
<source mode='vepa'>
<pool>
<interface name='eth10' maxConnect='1'/>
<interface name='eth11' maxConnect='1'/>
<interface name='eth12' maxConnect='1'/>
<interface name='eth13' maxConnect='1'/>
<interface name='eth14' maxConnect='1'/>
<interface name='eth25' maxConnect='5'/>
</pool>
</source>
<virtualport ...... />
</network>
at connect time, source dev would be allocated from the pool of
interfaces, round robin, with each interface having at most maxConnect
connections to guests at any given time.
Again, <virtualport> is optional, and if specified would have the same
purpose as <virtualport> in the interface definition.
Does this look like it covers everything?
BTW, for all the people asking about sectunnel, openvswitch, and vde -
can you see how those would fit in with this? In particular, do you see
any conflicts? (It's easy to add more stuff on later if something is
just missing, but much more problematic if I put something in that is
just plain wrong).