6:42 a.m.
Hi List,
Like most of us, I've bought my actual computer with an Ethernet
interface card, that I can connect to a wall jack at will. It's quite
easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via
libvirt, of course. That's useful, since an admin may need to disconnect
a running VM from a network temporarily, without resorting to
hot-unplugging its nic.
How should the domxml represent this situation? An interface that is
connected to a specific LAN1 bridge is
<interface type='bridge'>
<source bridge='LAN1'>
<target dev='vnet1'/>
<model type='virtio'/>
</interface>
Would the following make sense for an interface with no bridge at the
source?
<interface type='bridge'>
<source/>
<target dev='vnet1'/>
<model type='virtio'/>
</interface>
And how about the recommended config for <interface>s, which is
type='network'? Would it make sense to predefine
<network>
<name>null-bridge-network</name>
<forward mode='bridge'>
<bridge/>
</network>
so that
<interface type='network'>
<source network='null-bridge-network'/>
</interface>
means "keep this interface dangling out there"?
Regards,
Dan.
11:42 a.m.
On 06/20/2013 07:42 AM, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet
interface card, that I can connect to a wall jack at will. It's quite
easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via
libvirt, of course. That's useful, since an admin may need to disconnect
a running VM from a network temporarily, without resorting to
hot-unplugging its nic.
Well, there is already '<link state="up|down"/>', but of course
that
still requires that the tap be connected to *something*, it's just made
to appear to the guest that it's not connected (by using qemu's set_link
command), and I think what you really want is something to leave an
interface truly "plugged into nothing".
How should the domxml represent this situation? An interface that is
connected to a specific LAN1 bridge is
<interface type='bridge'>
<source bridge='LAN1'>
<target dev='vnet1'/>
<model type='virtio'/>
</interface>
Would the following make sense for an interface with no bridge at the
source?
<interface type='bridge'>
<source/>
<target dev='vnet1'/>
<model type='virtio'/>
</interface>
Since I think it would be a bad idea for "lack of <source>" to indicate
connection to some default bridge my opinion is that simple omission of
<source> should be enough to indicate a disconnected tap.
And how about the recommended config for <interface>s, which is
type='network'? Would it make sense to predefine
<network>
<name>null-bridge-network</name>
<forward mode='bridge'>
<bridge/>
</network>
so that
<interface type='network'>
<source network='null-bridge-network'/>
</interface>
means "keep this interface dangling out there"?
Or just allow no <source> again.
Hmm. networks do create a problem (which is really just an extension of
the idea of moving a running interface from one network to another) -
all this discussion is assuming that the networks are implemented in a
fashion that uses a tap device to connect the qemu process to the rest
of the network. However, there are at least two other types of networks:
1) those that use a macvtap device, 2) those that use PCI passthrough
device assignment. In both of those cases it isn't possible to have an
interface that is moved from one network to another, nor to completely
disconnect it while it is still running; in the case of PCI passthrough,
the "connection" is in the actual hardware, and in the case of macvtap
the physical device used for connection can only be specified when the
macvtap device is created, and can't be modified later (without
destroying the macvtap device and creating a new one).
At any rate, I guess attempts to do that will just have to return a
VIR_ERR_CONFIG_UNSUPPORTED.
To summarize all that:
1) I think we can just use an omission of the <source> to indicate that
an interface shouldn't be connected to anything, both for type='bridge'
and type='network'.
2) If the interface type is anything else, omission of <source> will
lead to VIR_ERR_CONFIG_UNSUPPORTED.
3) For type='network', omission of <source> will imply that any eventual
connection will be via a tap interface, so a later attempt to modify
<source> to connect to a macvtap or pci passthrough network will again
give VIR_ERR_CONFIG_UNSUPPORTED.
4) assuming neither (2) nor (3) is encountered, this will lead to a tap
device being created and given to qemu, but no connection being made.
libvirt is already able to change the bridge that a tap device is
attached to, so it shouldn't be difficult to update that code to accept
"nothing" as an option.
5) I don't think we need to support "<bridge/>" in networks.
4:29 a.m.
On Thu, Jun 20, 2013 at 12:42:14PM -0400, Laine Stump wrote:
To summarize all that:
1) I think we can just use an omission of the <source> to indicate that
an interface shouldn't be connected to anything, both for type='bridge'
and type='network'.
I think that it pretty dangerous. There are two many places in our code
which will be assuming a non-NULL bridge or network name. We should not
risk NULL-pointer crashes by changing this to allow NULLs. Also this will
be incompatible apps parsing the XML which expect that the source is
mandatory.
It is also semantically dubious - if there is no <source> then what
is the difference between type=bridge & type=network ?
If we want a disconnected netwrk, then we'd need type='none' IMHO.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:31 a.m.
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet
interface card, that I can connect to a wall jack at will. It's quite
easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via
libvirt, of course. That's useful, since an admin may need to disconnect
a running VM from a network temporarily, without resorting to
hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want
this so that you can make oVirt configure bridging behind libvirts
back with Quantum. As explained in the bug, this is a really bad way
todo this & should not be required - OpenStack Nova demonstrates you
can do the right thing wrt Quantum using exisiting libvirt config.
So I'm not inclined to support this disconnected mode.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
7:34 a.m.
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
> Hi List,
>
> Like most of us, I've bought my actual computer with an Ethernet
> interface card, that I can connect to a wall jack at will. It's quite
> easy to disconnect the Ethernet cable from the wall, as well.
>
> I would like to expose this behavior to virtual computers, too. Via
> libvirt, of course. That's useful, since an admin may need to disconnect
> a running VM from a network temporarily, without resorting to
> hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want
this so that you can make oVirt configure bridging behind libvirts
back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface>
was opened with the VM-connected-to-no-bridge in mind.
Since libvirt is lacking this feature, oVirt has introduced an ugly
hack: a dummy bridge, to which a disconnected VM is moved to. The
interface fo this VM had to be set with <link state=off> to avoid
inter-VM communication.
Only then came the Quantum use case.
As explained in the bug, this is a really bad way
todo this & should not be required - OpenStack Nova demonstrates you
can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both
quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have
an ensure_vlan_bridge() method.
I'm no OpenStack expert, but I think that the nicest separation between
nova's and quantum's domains is the tap device (obviously only for
networks which use tap devices). Quantum should create the Linux bridge
and its underlying connectivity (or even worse for ovs with security
groups...), and Nova should connect a newly-created VM to it. Otherwise,
Nova would have to reimplement just about any extension that is
introduced to Quantum.
In particular, I worry about the mapping of a network to a physical
device. Quantum's linuxbridge does it according to a preconfigured
cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif
driver support something like this? From where does it collect this
information?
Anyway, would you suggest oVirt to implement its own
ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge
quantum plugin?
So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the
virtual world as well. I would like to discuss the domxml schema for
representing it, and then, hopefully find the menpower to implement it
outside the core libvirt team. So please, reconsider.
Regards,
Dan.
5:22 a.m.
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
> On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
> > Hi List,
> >
> > Like most of us, I've bought my actual computer with an Ethernet
> > interface card, that I can connect to a wall jack at will. It's quite
> > easy to disconnect the Ethernet cable from the wall, as well.
> >
> > I would like to expose this behavior to virtual computers, too. Via
> > libvirt, of course. That's useful, since an admin may need to disconnect
> > a running VM from a network temporarily, without resorting to
> > hot-unplugging its nic.
>
> In the bug you filed requested this feature, you said that you want
> this so that you can make oVirt configure bridging behind libvirts
> back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface>
was opened with the VM-connected-to-no-bridge in mind.
Since libvirt is lacking this feature, oVirt has introduced an ugly
hack: a dummy bridge, to which a disconnected VM is moved to. The
interface fo this VM had to be set with <link state=off> to avoid
inter-VM communication.
Only then came the Quantum use case.
Sigh. In the BZ, your justification only mentioned that this is
required for integration with Quantum, never that it was required
separately.
> As explained in the bug, this is a really bad way
> todo this & should not be required - OpenStack Nova demonstrates you
> can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both
quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have
an ensure_vlan_bridge() method.
I'm no OpenStack expert, but I think that the nicest separation between
nova's and quantum's domains is the tap device (obviously only for
networks which use tap devices). Quantum should create the Linux bridge
and its underlying connectivity (or even worse for ovs with security
groups...), and Nova should connect a newly-created VM to it. Otherwise,
Nova would have to reimplement just about any extension that is
introduced to Quantum.
While there are many, many ways to configure a physical network,
there are a small, finite number of ways that you can connect a
virtual machine to a physical network. Thus while there can be
many different quantum plugins, Nova only needs to know about a
handful of VM configuration rules.
In particular, I worry about the mapping of a network to a physical
device. Quantum's linuxbridge does it according to a preconfigured
cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif
driver support something like this? From where does it collect this
information?
Anyway, would you suggest oVirt to implement its own
ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge
quantum plugin?
I'm not sure you're looking at current codebase. As of Grizzly the
quantum specific VIF plugin (and all other existing VIF plugins)
are deprecated in favour of LibvirtGenericVIFDriver. This defines
(currently) 4 different types of VIF configuration, linux bridge,
openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a
set of metadata associated with it describing the configuration
requirements, which is used to configure the VM interface appropriately.
This clean separation isolates Nova from any knowledge of Quantum,
and vica-verca.
> So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the
virtual world as well. I would like to discuss the domxml schema for
representing it, and then, hopefully find the menpower to implement it
outside the core libvirt team. So please, reconsider.
The XML schema is easy enough - it is just a new <interface type=none>.
Ideally we would want some kind of support in QEMU for this, concept
so that we don't have to have a hidden dangling tap device
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3:07 p.m.
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
> > On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
> > > Hi List,
> > >
> > > Like most of us, I've bought my actual computer with an Ethernet
> > > interface card, that I can connect to a wall jack at will. It's quite
> > > easy to disconnect the Ethernet cable from the wall, as well.
> > >
> > > I would like to expose this behavior to virtual computers, too. Via
> > > libvirt, of course. That's useful, since an admin may need to
disconnect
> > > a running VM from a network temporarily, without resorting to
> > > hot-unplugging its nic.
> >
> > In the bug you filed requested this feature, you said that you want
> > this so that you can make oVirt configure bridging behind libvirts
> > back with Quantum.
>
> This is not exact. Bug 878481 - define a disconnected <interface>
> was opened with the VM-connected-to-no-bridge in mind.
>
> Since libvirt is lacking this feature, oVirt has introduced an ugly
> hack: a dummy bridge, to which a disconnected VM is moved to. The
> interface fo this VM had to be set with <link state=off> to avoid
> inter-VM communication.
>
> Only then came the Quantum use case.
Sigh. In the BZ, your justification only mentioned that this is
required for integration with Quantum, never that it was required
separately.
Verifying this statement is left as an excercise to the
reader of https://bugzilla.redhat.com/show_bug.cgi?id=878481#c0 .
> > As explained in the bug, this is a really bad way
> > todo this & should not be required - OpenStack Nova demonstrates you
> > can do the right thing wrt Quantum using exisiting libvirt config.
>
> I'm not sure that this is THE right thing. At the momement, both
> quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have
> an ensure_vlan_bridge() method.
>
>
> I'm no OpenStack expert, but I think that the nicest separation between
> nova's and quantum's domains is the tap device (obviously only for
> networks which use tap devices). Quantum should create the Linux bridge
> and its underlying connectivity (or even worse for ovs with security
> groups...), and Nova should connect a newly-created VM to it. Otherwise,
> Nova would have to reimplement just about any extension that is
> introduced to Quantum.
While there are many, many ways to configure a physical network,
there are a small, finite number of ways that you can connect a
virtual machine to a physical network. Thus while there can be
many different quantum plugins, Nova only needs to know about a
handful of VM configuration rules.
> In particular, I worry about the mapping of a network to a physical
> device. Quantum's linuxbridge does it according to a preconfigured
> cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif
> driver support something like this? From where does it collect this
> information?
>
> Anyway, would you suggest oVirt to implement its own
> ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge
> quantum plugin?
I'm not sure you're looking at current codebase. As of Grizzly the
quantum specific VIF plugin (and all other existing VIF plugins)
are deprecated in favour of LibvirtGenericVIFDriver. This defines
(currently) 4 different types of VIF configuration, linux bridge,
openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a
set of metadata associated with it describing the configuration
requirements, which is used to configure the VM interface appropriately.
This clean separation isolates Nova from any knowledge of Quantum,
and vica-verca.
I'm looking at the tip of master branch, both in
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L342
and
https://github.com/openstack/quantum/blob/master/quantum/plugins/linuxbri...
My spefcific painpoint is that the vif driver calls
linux_net.LinuxBridgeInterfaceDriver.ensure_vlan_bridge() with no
regards of the configuration of the quantum agent.
True, a specific configuration parameter of a specific agent of a
quantum plugin is not of the business of Nova. But this suggests that
connecting a bridge to an external NIC is not its business, either.
> > So I'm not inclined to support this disconnected mode.
>
> Disconnected mode exists in actuality. It has valid use cases in the
> virtual world as well. I would like to discuss the domxml schema for
> representing it, and then, hopefully find the menpower to implement it
> outside the core libvirt team. So please, reconsider.
The XML schema is easy enough - it is just a new <interface type=none>.
Ideally we would want some kind of support in QEMU for this, concept
so that we don't have to have a hidden dangling tap device
That would be cool indeed. It would make it possible to
virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to
implement <interface type=none> via a dangling tap? Wouldn't it be fine
to limit changing type=none to type=network only to bridge-based
networks?
Regards,
Dan.
4:32 p.m.
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
>> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
>>> So I'm not inclined to support this disconnected mode.
>> Disconnected mode exists in actuality. It has valid use cases in the
>> virtual world as well. I would like to discuss the domxml schema for
>> representing it, and then, hopefully find the menpower to implement it
>> outside the core libvirt team. So please, reconsider.
> The XML schema is easy enough - it is just a new <interface type=none>.
> Ideally we would want some kind of support in QEMU for this, concept
> so that we don't have to have a hidden dangling tap device
That would be cool indeed. It would make it possible to
virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to
implement <interface type=none> via a dangling tap? Wouldn't it be fine
to limit changing type=none to type=network only to bridge-based
networks?
Well, that *is* how virDomainUpdateDevice behaves when switching from
one network connection to another - if the source and destination are
both implemented with tap, it works, otherwise it returns
OPERATION_UNSUPPORTED.
1:39 a.m.
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
> On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
>> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
>>> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
>>>> So I'm not inclined to support this disconnected mode.
>>> Disconnected mode exists in actuality. It has valid use cases in the
>>> virtual world as well. I would like to discuss the domxml schema for
>>> representing it, and then, hopefully find the menpower to implement it
>>> outside the core libvirt team. So please, reconsider.
>> The XML schema is easy enough - it is just a new <interface type=none>.
>> Ideally we would want some kind of support in QEMU for this, concept
>> so that we don't have to have a hidden dangling tap device
> That would be cool indeed. It would make it possible to
> virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
>
> Until we have something like that in qemu, would it be reasonable to
> implement <interface type=none> via a dangling tap? Wouldn't it be fine
> to limit changing type=none to type=network only to bridge-based
> networks?
Well, that *is* how virDomainUpdateDevice behaves when switching from
one network connection to another - if the source and destination are
both implemented with tap, it works, otherwise it returns
OPERATION_UNSUPPORTED.
My question is slightly different: it's about switching from one
interface type (=none) to another (=network), not between two networks.
I am asking whether it would be fine to implement type=none with tap,
given this unsupportedness.
Dan.
4:27 a.m.
On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote:
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
> On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
> > On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
> >> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
> >>> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
> >>>> So I'm not inclined to support this disconnected mode.
> >>> Disconnected mode exists in actuality. It has valid use cases in the
> >>> virtual world as well. I would like to discuss the domxml schema for
> >>> representing it, and then, hopefully find the menpower to implement it
> >>> outside the core libvirt team. So please, reconsider.
> >> The XML schema is easy enough - it is just a new <interface
type=none>.
> >> Ideally we would want some kind of support in QEMU for this, concept
> >> so that we don't have to have a hidden dangling tap device
> > That would be cool indeed. It would make it possible to
> > virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
> >
> > Until we have something like that in qemu, would it be reasonable to
> > implement <interface type=none> via a dangling tap? Wouldn't it be
fine
> > to limit changing type=none to type=network only to bridge-based
> > networks?
>
> Well, that *is* how virDomainUpdateDevice behaves when switching from
> one network connection to another - if the source and destination are
> both implemented with tap, it works, otherwise it returns
> OPERATION_UNSUPPORTED.
My question is slightly different: it's about switching from one
interface type (=none) to another (=network), not between two networks.
I am asking whether it would be fine to implement type=none with tap,
given this unsupportedness.
No, it doesn't really work properly. Not all type=network configs
are based on tap devices. We need to be able to ask QEMU to remove
the netdev backend, without affecting the guest device, and then
add in a new netdev backend.
We should have asked for this when we first did dynamic network
re-configuration, but we took the easy way out back then. We need
to fix this properly so that all possible config changes work.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
9:43 a.m.
On 07/03/2013 05:27 AM, Daniel P. Berrange wrote:
On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote:
> On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
>> On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
>>> On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
>>>> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
>>>>> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
>>>>>> So I'm not inclined to support this disconnected mode.
>>>>> Disconnected mode exists in actuality. It has valid use cases in the
>>>>> virtual world as well. I would like to discuss the domxml schema for
>>>>> representing it, and then, hopefully find the menpower to implement
it
>>>>> outside the core libvirt team. So please, reconsider.
>>>> The XML schema is easy enough - it is just a new <interface
type=none>.
>>>> Ideally we would want some kind of support in QEMU for this, concept
>>>> so that we don't have to have a hidden dangling tap device
>>> That would be cool indeed. It would make it possible to
>>> virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
>>>
>>> Until we have something like that in qemu, would it be reasonable to
>>> implement <interface type=none> via a dangling tap? Wouldn't it be
fine
>>> to limit changing type=none to type=network only to bridge-based
>>> networks?
>> Well, that *is* how virDomainUpdateDevice behaves when switching from
>> one network connection to another - if the source and destination are
>> both implemented with tap, it works, otherwise it returns
>> OPERATION_UNSUPPORTED.
> My question is slightly different: it's about switching from one
> interface type (=none) to another (=network), not between two networks.
>
> I am asking whether it would be fine to implement type=none with tap,
> given this unsupportedness.
No, it doesn't really work properly. Not all type=network configs
are based on tap devices. We need to be able to ask QEMU to remove
the netdev backend, without affecting the guest device, and then
add in a new netdev backend.
We should have asked for this when we first did dynamic network
re-configuration, but we took the easy way out back then. We need
to fix this properly so that all possible config changes work.
Actually we did ask for that (basically, for the ability to change the
frontend of a network device without changing the backend), and stefanha
kindly produced a patch that *kind of* did it, but due to internal
limitations of qemu, it wasn't fully functional; it worked for some
limited cases but not for everything. If I recall correctly, the split
between frontend and backend in qemu wasn't as clear cut and total as
the commands led us to believe, and even once Stefan's patch was in
place, switching from tap to macvtap (or vice versa) still didn't work
properly (am I remembering that correctly, Stefan?).
2:34 a.m.
On Wed, Jul 03, 2013 at 10:43:22AM -0400, Laine Stump wrote:
On 07/03/2013 05:27 AM, Daniel P. Berrange wrote:
> On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote:
>> On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
>>> On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
>>>> On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
>>>>> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
>>>>>> On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange
wrote:
>>>>>>> So I'm not inclined to support this disconnected mode.
>>>>>> Disconnected mode exists in actuality. It has valid use cases in
the
>>>>>> virtual world as well. I would like to discuss the domxml schema
for
>>>>>> representing it, and then, hopefully find the menpower to
implement it
>>>>>> outside the core libvirt team. So please, reconsider.
>>>>> The XML schema is easy enough - it is just a new <interface
type=none>.
>>>>> Ideally we would want some kind of support in QEMU for this,
concept
>>>>> so that we don't have to have a hidden dangling tap device
>>>> That would be cool indeed. It would make it possible to
>>>> virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
>>>>
>>>> Until we have something like that in qemu, would it be reasonable to
>>>> implement <interface type=none> via a dangling tap? Wouldn't
it be fine
>>>> to limit changing type=none to type=network only to bridge-based
>>>> networks?
>>> Well, that *is* how virDomainUpdateDevice behaves when switching from
>>> one network connection to another - if the source and destination are
>>> both implemented with tap, it works, otherwise it returns
>>> OPERATION_UNSUPPORTED.
>> My question is slightly different: it's about switching from one
>> interface type (=none) to another (=network), not between two networks.
>>
>> I am asking whether it would be fine to implement type=none with tap,
>> given this unsupportedness.
> No, it doesn't really work properly. Not all type=network configs
> are based on tap devices. We need to be able to ask QEMU to remove
> the netdev backend, without affecting the guest device, and then
> add in a new netdev backend.
>
> We should have asked for this when we first did dynamic network
> re-configuration, but we took the easy way out back then. We need
> to fix this properly so that all possible config changes work.
Actually we did ask for that (basically, for the ability to change the
frontend of a network device without changing the backend), and stefanha
kindly produced a patch that *kind of* did it, but due to internal
limitations of qemu, it wasn't fully functional; it worked for some
limited cases but not for everything. If I recall correctly, the split
between frontend and backend in qemu wasn't as clear cut and total as
the commands led us to believe, and even once Stefan's patch was in
place, switching from tap to macvtap (or vice versa) still didn't work
properly (am I remembering that correctly, Stefan?).
Hi Laine,
I sent a patch for testing in October 2012 but did not get a response:
http://comments.gmane.org/gmane.comp.emulators.qemu/177516
This patch makes "netdev_del netdev0" disconnect a NIC from its backend
and deletes the backend. The link will be down and the guest will be
notified if the emulated NIC supports link state change interrupts.
A new backend can be added later and this will bring the link back up.
The (solvable) limitation is when the NIC has told the guest to use
vnet_hdr offload:
The current code is not safe with virtio-net + tap, where the
virtio-net device reports offload features from the tap device to the
guest. If you try to switch to a -netdev user or -netdev socket
device, those offload capabilities are not present and network
communication will fail.
The issue with virtio-net is that there is no way to disable vnet_hdr
offload without resetting the device. This means QEMU would need to
emulate vnet_hdr offload itself to generate the packets that are
compatible with -netdev user or other non-vnet_hdr backends.
This is doable but someone needs to volunteer to implement it. Here is
the vnet_hdr structure:
/* This is the first element of the scatter-gather list. If you don't
* specify GSO or CSUM features, you can simply ignore the header. */
struct virtio_net_hdr {
#define VIRTIO_NET_HDR_F_NEEDS_CSUM 1 // Use csum_start, csum_offset
#define VIRTIO_NET_HDR_F_DATA_VALID 2 // Csum is valid
__u8 flags;
#define VIRTIO_NET_HDR_GSO_NONE 0 // Not a GSO frame
#define VIRTIO_NET_HDR_GSO_TCPV4 1 // GSO frame, IPv4 TCP (TSO)
#define VIRTIO_NET_HDR_GSO_UDP 3 // GSO frame, IPv4 UDP (UFO)
#define VIRTIO_NET_HDR_GSO_TCPV6 4 // GSO frame, IPv6 TCP
#define VIRTIO_NET_HDR_GSO_ECN 0x80 // TCP has ECN set
__u8 gso_type;
__u16 hdr_len; /* Ethernet + IP + tcp/udp hdrs */
__u16 gso_size; /* Bytes to append to hdr_len per frame */
__u16 csum_start; /* Position to start checksumming from */
__u16 csum_offset; /* Offset after that to place checksum */
};
Stefan
10:53 p.m.
On Tue, Jul 2, 2013 at 3:07 PM, Dan Kenigsberg <danken(a)redhat.com> wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
> On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
> > On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
> > > On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
> > > > Hi List,
> > > >
> > > > Like most of us, I've bought my actual computer with an Ethernet
> > > > interface card, that I can connect to a wall jack at will. It's
quite
> > > > easy to disconnect the Ethernet cable from the wall, as well.
> > > >
> > > > I would like to expose this behavior to virtual computers, too. Via
> > > > libvirt, of course. That's useful, since an admin may need to
disconnect
> > > > a running VM from a network temporarily, without resorting to
> > > > hot-unplugging its nic.
> > >
> > > In the bug you filed requested this feature, you said that you want
> > > this so that you can make oVirt configure bridging behind libvirts
> > > back with Quantum.
> >
> > This is not exact. Bug 878481 - define a disconnected <interface>
> > was opened with the VM-connected-to-no-bridge in mind.
> >
> > Since libvirt is lacking this feature, oVirt has introduced an ugly
> > hack: a dummy bridge, to which a disconnected VM is moved to. The
> > interface fo this VM had to be set with <link state=off> to avoid
> > inter-VM communication.
> >
> > Only then came the Quantum use case.
>
> Sigh. In the BZ, your justification only mentioned that this is
> required for integration with Quantum, never that it was required
> separately.
Verifying this statement is left as an excercise to the
reader of https://bugzilla.redhat.com/show_bug.cgi?id=878481#c0 .
>
> > > As explained in the bug, this is a really bad way
> > > todo this & should not be required - OpenStack Nova demonstrates you
> > > can do the right thing wrt Quantum using exisiting libvirt config.
> >
> > I'm not sure that this is THE right thing. At the momement, both
> > quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver
have
> > an ensure_vlan_bridge() method.
> >
> >
> > I'm no OpenStack expert, but I think that the nicest separation between
> > nova's and quantum's domains is the tap device (obviously only for
> > networks which use tap devices). Quantum should create the Linux bridge
> > and its underlying connectivity (or even worse for ovs with security
> > groups...), and Nova should connect a newly-created VM to it.
Otherwise,
> > Nova would have to reimplement just about any extension that is
> > introduced to Quantum.
>
> While there are many, many ways to configure a physical network,
> there are a small, finite number of ways that you can connect a
> virtual machine to a physical network. Thus while there can be
> many different quantum plugins, Nova only needs to know about a
> handful of VM configuration rules.
>
> > In particular, I worry about the mapping of a network to a physical
> > device. Quantum's linuxbridge does it according to a preconfigured
> > cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif
> > driver support something like this? From where does it collect this
> > information?
> >
> > Anyway, would you suggest oVirt to implement its own
> > ensure_vlan_bridge()? Or use the vif driver code? Or that of
linuxbridge
> > quantum plugin?
>
> I'm not sure you're looking at current codebase. As of Grizzly the
> quantum specific VIF plugin (and all other existing VIF plugins)
> are deprecated in favour of LibvirtGenericVIFDriver. This defines
> (currently) 4 different types of VIF configuration, linux bridge,
> openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a
> set of metadata associated with it describing the configuration
> requirements, which is used to configure the VM interface appropriately.
> This clean separation isolates Nova from any knowledge of Quantum,
> and vica-verca.
I'm looking at the tip of master branch, both in
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L342
and
https://github.com/openstack/quantum/blob/master/quantum/plugins/linuxbri...
My spefcific painpoint is that the vif driver calls
linux_net.LinuxBridgeInterfaceDriver.ensure_vlan_bridge() with no
regards of the configuration of the quantum agent.
True, a specific configuration parameter of a specific agent of a
quantum plugin is not of the business of Nova. But this suggests that
connecting a bridge to an external NIC is not its business, either.
>
> > > So I'm not inclined to support this disconnected mode.
> >
> > Disconnected mode exists in actuality. It has valid use cases in the
> > virtual world as well. I would like to discuss the domxml schema for
> > representing it, and then, hopefully find the menpower to implement it
> > outside the core libvirt team. So please, reconsider.
>
> The XML schema is easy enough - it is just a new <interface type=none>.
> Ideally we would want some kind of support in QEMU for this, concept
> so that we don't have to have a hidden dangling tap device
That would be cool indeed. It would make it possible to
virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to
implement <interface type=none> via a dangling tap? Wouldn't it be fine
to limit changing type=none to type=network only to bridge-based
networks?
Regards,
Dan.
Did anything come of <interface type='none'> or allowing
<source> to be
missing? While going through the vmx parser (VMWare), I noticed a good
quantity of vmx files provided to me by different people have 2 ethernet
devices defined. Both defined as bridged devices and both with their own
unique MAC address however the 2nd one will not have an associated
network/bridge device. libvirt will generate the following:
<interface type='bridge'>
<mac address='00:0c:29:3b:64:f4'/>
<source bridge=''/>
</interface>
But that just doesn't seem correct and I was trying to figure out what a
more correct syntax would be.
--
Doug Goldstein
3922
days inactive
3986
days old
12 comments
5 participants
participants (5)
-
Dan Kenigsberg -
Daniel P. Berrange -
Doug Goldstein -
Laine Stump -
Stefan Hajnoczi