[libvirt] configuring a disconnected <interface>
Hi List, Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well. I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic. How should the domxml represent this situation? An interface that is connected to a specific LAN1 bridge is <interface type='bridge'> <source bridge='LAN1'> <target dev='vnet1'/> <model type='virtio'/> </interface> Would the following make sense for an interface with no bridge at the source? <interface type='bridge'> <source/> <target dev='vnet1'/> <model type='virtio'/> </interface> And how about the recommended config for <interface>s, which is type='network'? Would it make sense to predefine <network> <name>null-bridge-network</name> <forward mode='bridge'> <bridge/> </network> so that <interface type='network'> <source network='null-bridge-network'/> </interface> means "keep this interface dangling out there"? Regards, Dan.
On 06/20/2013 07:42 AM, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
Well, there is already '<link state="up|down"/>', but of course that still requires that the tap be connected to *something*, it's just made to appear to the guest that it's not connected (by using qemu's set_link command), and I think what you really want is something to leave an interface truly "plugged into nothing".
How should the domxml represent this situation? An interface that is connected to a specific LAN1 bridge is
<interface type='bridge'> <source bridge='LAN1'> <target dev='vnet1'/> <model type='virtio'/> </interface>
Would the following make sense for an interface with no bridge at the source?
<interface type='bridge'> <source/> <target dev='vnet1'/> <model type='virtio'/> </interface>
Since I think it would be a bad idea for "lack of <source>" to indicate connection to some default bridge my opinion is that simple omission of <source> should be enough to indicate a disconnected tap.
And how about the recommended config for <interface>s, which is type='network'? Would it make sense to predefine
<network> <name>null-bridge-network</name> <forward mode='bridge'> <bridge/> </network>
so that
<interface type='network'> <source network='null-bridge-network'/> </interface>
means "keep this interface dangling out there"?
Or just allow no <source> again. Hmm. networks do create a problem (which is really just an extension of the idea of moving a running interface from one network to another) - all this discussion is assuming that the networks are implemented in a fashion that uses a tap device to connect the qemu process to the rest of the network. However, there are at least two other types of networks: 1) those that use a macvtap device, 2) those that use PCI passthrough device assignment. In both of those cases it isn't possible to have an interface that is moved from one network to another, nor to completely disconnect it while it is still running; in the case of PCI passthrough, the "connection" is in the actual hardware, and in the case of macvtap the physical device used for connection can only be specified when the macvtap device is created, and can't be modified later (without destroying the macvtap device and creating a new one). At any rate, I guess attempts to do that will just have to return a VIR_ERR_CONFIG_UNSUPPORTED. To summarize all that: 1) I think we can just use an omission of the <source> to indicate that an interface shouldn't be connected to anything, both for type='bridge' and type='network'. 2) If the interface type is anything else, omission of <source> will lead to VIR_ERR_CONFIG_UNSUPPORTED. 3) For type='network', omission of <source> will imply that any eventual connection will be via a tap interface, so a later attempt to modify <source> to connect to a macvtap or pci passthrough network will again give VIR_ERR_CONFIG_UNSUPPORTED. 4) assuming neither (2) nor (3) is encountered, this will lead to a tap device being created and given to qemu, but no connection being made. libvirt is already able to change the bridge that a tap device is attached to, so it shouldn't be difficult to update that code to accept "nothing" as an option. 5) I don't think we need to support "<bridge/>" in networks.
On Thu, Jun 20, 2013 at 12:42:14PM -0400, Laine Stump wrote:
To summarize all that:
1) I think we can just use an omission of the <source> to indicate that an interface shouldn't be connected to anything, both for type='bridge' and type='network'.
I think that it pretty dangerous. There are two many places in our code which will be assuming a non-NULL bridge or network name. We should not risk NULL-pointer crashes by changing this to allow NULLs. Also this will be incompatible apps parsing the XML which expect that the source is mandatory. It is also semantically dubious - if there is no <source> then what is the difference between type=bridge & type=network ? If we want a disconnected netwrk, then we'd need type='none' IMHO. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want this so that you can make oVirt configure bridging behind libvirts back with Quantum. As explained in the bug, this is a really bad way todo this & should not be required - OpenStack Nova demonstrates you can do the right thing wrt Quantum using exisiting libvirt config. So I'm not inclined to support this disconnected mode. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want this so that you can make oVirt configure bridging behind libvirts back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface> was opened with the VM-connected-to-no-bridge in mind. Since libvirt is lacking this feature, oVirt has introduced an ugly hack: a dummy bridge, to which a disconnected VM is moved to. The interface fo this VM had to be set with <link state=off> to avoid inter-VM communication. Only then came the Quantum use case.
As explained in the bug, this is a really bad way todo this & should not be required - OpenStack Nova demonstrates you can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have an ensure_vlan_bridge() method. I'm no OpenStack expert, but I think that the nicest separation between nova's and quantum's domains is the tap device (obviously only for networks which use tap devices). Quantum should create the Linux bridge and its underlying connectivity (or even worse for ovs with security groups...), and Nova should connect a newly-created VM to it. Otherwise, Nova would have to reimplement just about any extension that is introduced to Quantum. In particular, I worry about the mapping of a network to a physical device. Quantum's linuxbridge does it according to a preconfigured cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif driver support something like this? From where does it collect this information? Anyway, would you suggest oVirt to implement its own ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge quantum plugin?
So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider. Regards, Dan.
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want this so that you can make oVirt configure bridging behind libvirts back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface> was opened with the VM-connected-to-no-bridge in mind.
Since libvirt is lacking this feature, oVirt has introduced an ugly hack: a dummy bridge, to which a disconnected VM is moved to. The interface fo this VM had to be set with <link state=off> to avoid inter-VM communication.
Only then came the Quantum use case.
Sigh. In the BZ, your justification only mentioned that this is required for integration with Quantum, never that it was required separately.
As explained in the bug, this is a really bad way todo this & should not be required - OpenStack Nova demonstrates you can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have an ensure_vlan_bridge() method.
I'm no OpenStack expert, but I think that the nicest separation between nova's and quantum's domains is the tap device (obviously only for networks which use tap devices). Quantum should create the Linux bridge and its underlying connectivity (or even worse for ovs with security groups...), and Nova should connect a newly-created VM to it. Otherwise, Nova would have to reimplement just about any extension that is introduced to Quantum.
While there are many, many ways to configure a physical network, there are a small, finite number of ways that you can connect a virtual machine to a physical network. Thus while there can be many different quantum plugins, Nova only needs to know about a handful of VM configuration rules.
In particular, I worry about the mapping of a network to a physical device. Quantum's linuxbridge does it according to a preconfigured cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif driver support something like this? From where does it collect this information?
Anyway, would you suggest oVirt to implement its own ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge quantum plugin?
I'm not sure you're looking at current codebase. As of Grizzly the quantum specific VIF plugin (and all other existing VIF plugins) are deprecated in favour of LibvirtGenericVIFDriver. This defines (currently) 4 different types of VIF configuration, linux bridge, openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a set of metadata associated with it describing the configuration requirements, which is used to configure the VM interface appropriately. This clean separation isolates Nova from any knowledge of Quantum, and vica-verca.
So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider.
The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want this so that you can make oVirt configure bridging behind libvirts back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface> was opened with the VM-connected-to-no-bridge in mind.
Since libvirt is lacking this feature, oVirt has introduced an ugly hack: a dummy bridge, to which a disconnected VM is moved to. The interface fo this VM had to be set with <link state=off> to avoid inter-VM communication.
Only then came the Quantum use case.
Sigh. In the BZ, your justification only mentioned that this is required for integration with Quantum, never that it was required separately.
Verifying this statement is left as an excercise to the reader of https://bugzilla.redhat.com/show_bug.cgi?id=878481#c0 .
As explained in the bug, this is a really bad way todo this & should not be required - OpenStack Nova demonstrates you can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have an ensure_vlan_bridge() method.
I'm no OpenStack expert, but I think that the nicest separation between nova's and quantum's domains is the tap device (obviously only for networks which use tap devices). Quantum should create the Linux bridge and its underlying connectivity (or even worse for ovs with security groups...), and Nova should connect a newly-created VM to it. Otherwise, Nova would have to reimplement just about any extension that is introduced to Quantum.
While there are many, many ways to configure a physical network, there are a small, finite number of ways that you can connect a virtual machine to a physical network. Thus while there can be many different quantum plugins, Nova only needs to know about a handful of VM configuration rules.
In particular, I worry about the mapping of a network to a physical device. Quantum's linuxbridge does it according to a preconfigured cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif driver support something like this? From where does it collect this information?
Anyway, would you suggest oVirt to implement its own ensure_vlan_bridge()? Or use the vif driver code? Or that of linuxbridge quantum plugin?
I'm not sure you're looking at current codebase. As of Grizzly the quantum specific VIF plugin (and all other existing VIF plugins) are deprecated in favour of LibvirtGenericVIFDriver. This defines (currently) 4 different types of VIF configuration, linux bridge, openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a set of metadata associated with it describing the configuration requirements, which is used to configure the VM interface appropriately. This clean separation isolates Nova from any knowledge of Quantum, and vica-verca.
I'm looking at the tip of master branch, both in https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L342 and https://github.com/openstack/quantum/blob/master/quantum/plugins/linuxbridge... My spefcific painpoint is that the vif driver calls linux_net.LinuxBridgeInterfaceDriver.ensure_vlan_bridge() with no regards of the configuration of the quantum agent. True, a specific configuration parameter of a specific agent of a quantum plugin is not of the business of Nova. But this suggests that connecting a bridge to an external NIC is not its business, either.
So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider.
The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device
That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one. Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks? Regards, Dan.
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
So I'm not inclined to support this disconnected mode. Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider. The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks?
Well, that *is* how virDomainUpdateDevice behaves when switching from one network connection to another - if the source and destination are both implemented with tap, it works, otherwise it returns OPERATION_UNSUPPORTED.
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
So I'm not inclined to support this disconnected mode. Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider. The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks?
Well, that *is* how virDomainUpdateDevice behaves when switching from one network connection to another - if the source and destination are both implemented with tap, it works, otherwise it returns OPERATION_UNSUPPORTED.
My question is slightly different: it's about switching from one interface type (=none) to another (=network), not between two networks. I am asking whether it would be fine to implement type=none with tap, given this unsupportedness. Dan.
On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote:
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
So I'm not inclined to support this disconnected mode. Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider. The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks?
Well, that *is* how virDomainUpdateDevice behaves when switching from one network connection to another - if the source and destination are both implemented with tap, it works, otherwise it returns OPERATION_UNSUPPORTED.
My question is slightly different: it's about switching from one interface type (=none) to another (=network), not between two networks.
I am asking whether it would be fine to implement type=none with tap, given this unsupportedness.
No, it doesn't really work properly. Not all type=network configs are based on tap devices. We need to be able to ask QEMU to remove the netdev backend, without affecting the guest device, and then add in a new netdev backend. We should have asked for this when we first did dynamic network re-configuration, but we took the easy way out back then. We need to fix this properly so that all possible config changes work. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 07/03/2013 05:27 AM, Daniel P. Berrange wrote:
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote: > So I'm not inclined to support this disconnected mode. Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider. The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks? Well, that *is* how virDomainUpdateDevice behaves when switching from one network connection to another - if the source and destination are both implemented with tap, it works, otherwise it returns OPERATION_UNSUPPORTED. My question is slightly different: it's about switching from one interface type (=none) to another (=network), not between two networks.
I am asking whether it would be fine to implement type=none with tap, given this unsupportedness. No, it doesn't really work properly. Not all type=network configs are based on tap devices. We need to be able to ask QEMU to remove
On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote: the netdev backend, without affecting the guest device, and then add in a new netdev backend.
We should have asked for this when we first did dynamic network re-configuration, but we took the easy way out back then. We need to fix this properly so that all possible config changes work.
Actually we did ask for that (basically, for the ability to change the frontend of a network device without changing the backend), and stefanha kindly produced a patch that *kind of* did it, but due to internal limitations of qemu, it wasn't fully functional; it worked for some limited cases but not for everything. If I recall correctly, the split between frontend and backend in qemu wasn't as clear cut and total as the commands led us to believe, and even once Stefan's patch was in place, switching from tap to macvtap (or vice versa) still didn't work properly (am I remembering that correctly, Stefan?).
On Wed, Jul 03, 2013 at 10:43:22AM -0400, Laine Stump wrote:
On 07/03/2013 05:27 AM, Daniel P. Berrange wrote:
On Tue, Jul 02, 2013 at 05:32:20PM -0400, Laine Stump wrote:
On 07/02/2013 04:07 PM, Dan Kenigsberg wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote: > On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote: >> So I'm not inclined to support this disconnected mode. > Disconnected mode exists in actuality. It has valid use cases in the > virtual world as well. I would like to discuss the domxml schema for > representing it, and then, hopefully find the menpower to implement it > outside the core libvirt team. So please, reconsider. The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks? Well, that *is* how virDomainUpdateDevice behaves when switching from one network connection to another - if the source and destination are both implemented with tap, it works, otherwise it returns OPERATION_UNSUPPORTED. My question is slightly different: it's about switching from one interface type (=none) to another (=network), not between two networks.
I am asking whether it would be fine to implement type=none with tap, given this unsupportedness. No, it doesn't really work properly. Not all type=network configs are based on tap devices. We need to be able to ask QEMU to remove
On Wed, Jul 03, 2013 at 09:39:57AM +0300, Dan Kenigsberg wrote: the netdev backend, without affecting the guest device, and then add in a new netdev backend.
We should have asked for this when we first did dynamic network re-configuration, but we took the easy way out back then. We need to fix this properly so that all possible config changes work.
Actually we did ask for that (basically, for the ability to change the frontend of a network device without changing the backend), and stefanha kindly produced a patch that *kind of* did it, but due to internal limitations of qemu, it wasn't fully functional; it worked for some limited cases but not for everything. If I recall correctly, the split between frontend and backend in qemu wasn't as clear cut and total as the commands led us to believe, and even once Stefan's patch was in place, switching from tap to macvtap (or vice versa) still didn't work properly (am I remembering that correctly, Stefan?).
Hi Laine, I sent a patch for testing in October 2012 but did not get a response: http://comments.gmane.org/gmane.comp.emulators.qemu/177516 This patch makes "netdev_del netdev0" disconnect a NIC from its backend and deletes the backend. The link will be down and the guest will be notified if the emulated NIC supports link state change interrupts. A new backend can be added later and this will bring the link back up. The (solvable) limitation is when the NIC has told the guest to use vnet_hdr offload: The current code is not safe with virtio-net + tap, where the virtio-net device reports offload features from the tap device to the guest. If you try to switch to a -netdev user or -netdev socket device, those offload capabilities are not present and network communication will fail. The issue with virtio-net is that there is no way to disable vnet_hdr offload without resetting the device. This means QEMU would need to emulate vnet_hdr offload itself to generate the packets that are compatible with -netdev user or other non-vnet_hdr backends. This is doable but someone needs to volunteer to implement it. Here is the vnet_hdr structure: /* This is the first element of the scatter-gather list. If you don't * specify GSO or CSUM features, you can simply ignore the header. */ struct virtio_net_hdr { #define VIRTIO_NET_HDR_F_NEEDS_CSUM 1 // Use csum_start, csum_offset #define VIRTIO_NET_HDR_F_DATA_VALID 2 // Csum is valid __u8 flags; #define VIRTIO_NET_HDR_GSO_NONE 0 // Not a GSO frame #define VIRTIO_NET_HDR_GSO_TCPV4 1 // GSO frame, IPv4 TCP (TSO) #define VIRTIO_NET_HDR_GSO_UDP 3 // GSO frame, IPv4 UDP (UFO) #define VIRTIO_NET_HDR_GSO_TCPV6 4 // GSO frame, IPv6 TCP #define VIRTIO_NET_HDR_GSO_ECN 0x80 // TCP has ECN set __u8 gso_type; __u16 hdr_len; /* Ethernet + IP + tcp/udp hdrs */ __u16 gso_size; /* Bytes to append to hdr_len per frame */ __u16 csum_start; /* Position to start checksumming from */ __u16 csum_offset; /* Offset after that to place checksum */ }; Stefan
On Tue, Jul 2, 2013 at 3:07 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Mon, Jun 24, 2013 at 11:22:16AM +0100, Daniel P. Berrange wrote:
On Sun, Jun 23, 2013 at 03:34:06PM +0300, Dan Kenigsberg wrote:
On Fri, Jun 21, 2013 at 04:31:53AM -0500, Daniel P. Berrange wrote:
On Thu, Jun 20, 2013 at 02:42:31PM +0300, Dan Kenigsberg wrote:
Hi List,
Like most of us, I've bought my actual computer with an Ethernet interface card, that I can connect to a wall jack at will. It's quite easy to disconnect the Ethernet cable from the wall, as well.
I would like to expose this behavior to virtual computers, too. Via libvirt, of course. That's useful, since an admin may need to disconnect a running VM from a network temporarily, without resorting to hot-unplugging its nic.
In the bug you filed requested this feature, you said that you want this so that you can make oVirt configure bridging behind libvirts back with Quantum.
This is not exact. Bug 878481 - define a disconnected <interface> was opened with the VM-connected-to-no-bridge in mind.
Since libvirt is lacking this feature, oVirt has introduced an ugly hack: a dummy bridge, to which a disconnected VM is moved to. The interface fo this VM had to be set with <link state=off> to avoid inter-VM communication.
Only then came the Quantum use case.
Sigh. In the BZ, your justification only mentioned that this is required for integration with Quantum, never that it was required separately.
Verifying this statement is left as an excercise to the reader of https://bugzilla.redhat.com/show_bug.cgi?id=878481#c0 .
As explained in the bug, this is a really bad way todo this & should not be required - OpenStack Nova demonstrates you can do the right thing wrt Quantum using exisiting libvirt config.
I'm not sure that this is THE right thing. At the momement, both quantum's linuxbridge plugin and nova's LinuxBridgeInterfaceDriver have an ensure_vlan_bridge() method.
I'm no OpenStack expert, but I think that the nicest separation between nova's and quantum's domains is the tap device (obviously only for networks which use tap devices). Quantum should create the Linux bridge and its underlying connectivity (or even worse for ovs with security groups...), and Nova should connect a newly-created VM to it.
Nova would have to reimplement just about any extension that is introduced to Quantum.
While there are many, many ways to configure a physical network, there are a small, finite number of ways that you can connect a virtual machine to a physical network. Thus while there can be many different quantum plugins, Nova only needs to know about a handful of VM configuration rules.
In particular, I worry about the mapping of a network to a physical device. Quantum's linuxbridge does it according to a preconfigured cfg.CONF.LINUX_BRIDGE.physical_interface_mappings. Does Nova's vif driver support something like this? From where does it collect this information?
Anyway, would you suggest oVirt to implement its own ensure_vlan_bridge()? Or use the vif driver code? Or that of
Otherwise, linuxbridge
quantum plugin?
I'm not sure you're looking at current codebase. As of Grizzly the quantum specific VIF plugin (and all other existing VIF plugins) are deprecated in favour of LibvirtGenericVIFDriver. This defines (currently) 4 different types of VIF configuration, linux bridge, openvswitch, 802.qbg and 802.qbh. Each of these VIF types has a set of metadata associated with it describing the configuration requirements, which is used to configure the VM interface appropriately. This clean separation isolates Nova from any knowledge of Quantum, and vica-verca.
I'm looking at the tip of master branch, both in https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L342 and
https://github.com/openstack/quantum/blob/master/quantum/plugins/linuxbridge...
My spefcific painpoint is that the vif driver calls linux_net.LinuxBridgeInterfaceDriver.ensure_vlan_bridge() with no regards of the configuration of the quantum agent.
True, a specific configuration parameter of a specific agent of a quantum plugin is not of the business of Nova. But this suggests that connecting a bridge to an external NIC is not its business, either.
So I'm not inclined to support this disconnected mode.
Disconnected mode exists in actuality. It has valid use cases in the virtual world as well. I would like to discuss the domxml schema for representing it, and then, hopefully find the menpower to implement it outside the core libvirt team. So please, reconsider.
The XML schema is easy enough - it is just a new <interface type=none>. Ideally we would want some kind of support in QEMU for this, concept so that we don't have to have a hidden dangling tap device
That would be cool indeed. It would make it possible to virDomainUpdateDevice() from a tap-based connectivity to non-tap one.
Until we have something like that in qemu, would it be reasonable to implement <interface type=none> via a dangling tap? Wouldn't it be fine to limit changing type=none to type=network only to bridge-based networks?
Regards, Dan.
Did anything come of <interface type='none'> or allowing <source> to be missing? While going through the vmx parser (VMWare), I noticed a good quantity of vmx files provided to me by different people have 2 ethernet devices defined. Both defined as bridged devices and both with their own unique MAC address however the 2nd one will not have an associated network/bridge device. libvirt will generate the following: <interface type='bridge'> <mac address='00:0c:29:3b:64:f4'/> <source bridge=''/> </interface> But that just doesn't seem correct and I was trying to figure out what a more correct syntax would be. -- Doug Goldstein
participants (5)
-
Dan Kenigsberg -
Daniel P. Berrange -
Doug Goldstein -
Laine Stump -
Stefan Hajnoczi