[libvirt-users] How does the libvirt deal with the vnet mac address
How does the libvirt deal with the vnet mac address? Greetings, if I establish a network for the VM (hypervisor is KVM) using bridge in the virt-manager , a vnet0 device is created . There are some relationships about mac address between the vnet0 device in the hypervisor and the ethX device in the VM, for example : the mac address of vnet0 is FE:54:00:84:E3:62 the mac address of ethX in the VM is 52:54:00:84:E3:62 two mac addresses above are almost the same except the first part of the address . but if I created a tap device manually , tunctl -t tap0 -u root brctl addif br0 tap0 and add tap0 to the VM, I will find that mac address between the tap0 device in the hypervisor and the ethX device in the VM will totally different . so I think that libvirt must do something about the mac address handling, could you please kindly tell me something about this ? I have found a function in libvirt-0.10.2.8\src\util\ virnetdevtap.c int virNetDevTapCreateInBridgePort(const char *brname, char **ifname, const virMacAddrPtr macaddr, const unsigned char *vmuuid, int *tapfd, virNetDevVPortProfilePtr virtPortProfile, virNetDevVlanPtr virtVlan, unsigned int flags) { …. virMacAddr tapmac; if (virNetDevTapCreate(ifname, tapfd, flags) < 0) return -1; /* We need to set the interface MAC before adding it * to the bridge, because the bridge assumes the lowest * MAC of all enslaved interfaces & we don't want it * seeing the kernel allocate random MAC for the TAP * device before we set our static MAC. */ virMacAddrSet(&tapmac, macaddr); if (!(flags & VIR_NETDEV_TAP_CREATE_USE_MAC_FOR_BRIDGE)) { if (macaddr->addr[0] == 0xFE) { /* For normal use, the tap device's MAC address cannot * match the MAC address used by the guest. This results * in "received packet on vnetX with own address as source * address" error logs from the kernel. */ virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Unable to use MAC address starting with " "reserved value 0xFE - '%02X:%02X:%02X:%02X:%02X:%02X' - "), macaddr->addr[0], macaddr->addr[1], macaddr->addr[2], macaddr->addr[3], macaddr->addr[4], macaddr->addr[5]); goto error; } tapmac.addr[0] = 0xFE; /* Discourage bridge from using TAP dev MAC */ //the first part of mac address is set to 0xFE. } if (virNetDevSetMAC(*ifname, &tapmac) < 0) goto error; …. } How does the libvirt establish the arp table in the hypervisor if the vnet0 device in the hypervisor and the ethX device in the VM is different? If I want to create tap device manually , how should I deal with the mac address ?I have setup the mac address of the tap0 device in the hypervisor and the ethX device in the VM in the same way with libvirt , but the network of VM cannot work. weihua wh.h@foxmail.com wh.h@foxmail.com
On Sun, Apr 26, 2015 at 10:51:34AM +0800, wh.h@foxmail.com wrote:
How does the libvirt deal with the vnet mac address?
Greetings, if I establish a network for the VM (hypervisor is KVM) using bridge in the virt-manager , a vnet0 device is created . There are some relationships about mac address between the vnet0 device in the hypervisor and the ethX device in the VM, for example : the mac address of vnet0 is FE:54:00:84:E3:62 the mac address of ethX in the VM is 52:54:00:84:E3:62 two mac addresses above are almost the same except the first part of the address . but if I created a tap device manually , tunctl -t tap0 -u root brctl addif br0 tap0 and add tap0 to the VM, I will find that mac address between the tap0 device in the hypervisor and the ethX device in the VM will totally different . so I think that libvirt must do something about the mac address handling, could you please kindly tell me something about this ?
When first created, the kernel assigns the tap device a completely random MAC address. This bears no relation to the MAC address that is used in the guest OS. When you create a bridge device it initially has a MAC address of all zeros, and when you add NIC devices to the bridge, its MAC address gets update to the numerically lowest MAC address of all the NICs. The problem is that when the kernel assigns MAC addresses randomly, one of these random MAC address might be numerically lower than the bridge's current MAC address. So the effect is that when you start/stop guests, and their TAP devices get added/removed from the bridge, the bridge's own MAC address will occassionally change which is a bad thing. So deal with this, libvirt will set all guest TAP devices so that they have a MAC address with 0xFE as the first byte. The real physical NIC added to the bridge is thus guaranteed to have a smaller MAC address, and so the bridge will permanently use the MAC address of the physical NIC, which is what we want. For bridges which do not have any physical NIC, libvirt will create a dummy TAP device, not connected to any guest, and give it a small MAC address. This ensures again ensures the bridge MAC address won't change when guests start/stop.
How does the libvirt establish the arp table in the hypervisor if the vnet0 device in the hypervisor and the ethX device in the VM is different?
The MAC address of the TAP device is actually totally irrelevant for the ARP table maintenance. If a packet arrives on the bridge and the IP doesn't have a ARP table mapping, the bridge will just send it to all connected TAP devices. When a packet arrives from a guest TAP, the source MAC address will be used to populate the ARP table. In neither case does the MAC address of the TAP device itself have any involvement. The only time the TAP device MAC address has any effect is when the kerenel auto-assigns a MAC to the bridge device as explained above.
If I want to create tap device manually , how should I deal with the mac address ?I have setup the mac address of the tap0 device in the hypervisor and the ethX device in the VM in the same way with libvirt , but the network of VM cannot work.
As mentioned above, the TAP device MAC can be pretty much anything, but we'd recommend using 0xFE as the first byte. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 04/27/2015 04:59 AM, Daniel P. Berrange wrote:
On Sun, Apr 26, 2015 at 10:51:34AM +0800, wh.h@foxmail.com wrote:
How does the libvirt deal with the vnet mac address?
Greetings, if I establish a network for the VM (hypervisor is KVM) using bridge in the virt-manager , a vnet0 device is created . There are some relationships about mac address between the vnet0 device in the hypervisor and the ethX device in the VM, for example : the mac address of vnet0 is FE:54:00:84:E3:62 the mac address of ethX in the VM is 52:54:00:84:E3:62 two mac addresses above are almost the same except the first part of the address . but if I created a tap device manually , tunctl -t tap0 -u root brctl addif br0 tap0 and add tap0 to the VM, I will find that mac address between the tap0 device in the hypervisor and the ethX device in the VM will totally different . so I think that libvirt must do something about the mac address handling, could you please kindly tell me something about this ? When first created, the kernel assigns the tap device a completely random MAC address. This bears no relation to the MAC address that is used in the guest OS.
When you create a bridge device it initially has a MAC address of all zeros, and when you add NIC devices to the bridge, its MAC address gets update to the numerically lowest MAC address of all the NICs. The problem is that when the kernel assigns MAC addresses randomly, one of these random MAC address might be numerically lower than the bridge's current MAC address. So the effect is that when you start/stop guests, and their TAP devices get added/removed from the bridge, the bridge's own MAC address will occassionally change which is a bad thing.
So deal with this, libvirt will set all guest TAP devices so that they have a MAC address with 0xFE as the first byte. The real physical NIC added to the bridge is thus guaranteed to have a smaller MAC address, and so the bridge will permanently use the MAC address of the physical NIC, which is what we want.
For bridges which do not have any physical NIC, libvirt will create a dummy TAP device, not connected to any guest, and give it a small MAC address. This ensures again ensures the bridge MAC address won't change when guests start/stop.
How does the libvirt establish the arp table in the hypervisor if the vnet0 device in the hypervisor and the ethX device in the VM is different? The MAC address of the TAP device is actually totally irrelevant for the ARP table maintenance.
If a packet arrives on the bridge and the IP doesn't have a ARP table mapping, the bridge will just send it to all connected TAP devices.
When a packet arrives from a guest TAP, the source MAC address will be used to populate the ARP table.
In neither case does the MAC address of the TAP device itself have any involvement.
The only time the TAP device MAC address has any effect is when the kerenel auto-assigns a MAC to the bridge device as explained above.
If I want to create tap device manually , how should I deal with the mac address ?I have setup the mac address of the tap0 device in the hypervisor and the ethX device in the VM in the same way with libvirt , but the network of VM cannot work. As mentioned above, the TAP device MAC can be pretty much anything, but we'd recommend using 0xFE as the first byte.
Along with everything that Dan has explained, it's important to also know that it is essential the tap device's MAC address be different from the MAC address used by the guest. The reason is that the tap device will not forward a packet to the other side of itself if it sees a destination MAC address matching its own - it will think that the packet must be intended for local delivery; this is another reason that libvirt replaces the 1st byte of the guest address with 0xFE when setting the tap device address. (This is in contrast to macvtap devices, which *do* match the MAC address used by the guest.) (...and I'm still curious why you are choosing to do all of this yourself rather than just letting libvirt deal with it. Is there some functionality you need that libvirt can't supply?)
From: Laine Stump Date: 2015-04-27 21:41 To: libvirt-users CC: Daniel P. Berrange; wh.h@foxmail.com Subject: Re: [libvirt-users] How does the libvirt deal with the vnet mac address On 04/27/2015 04:59 AM, Daniel P. Berrange wrote:
On Sun, Apr 26, 2015 at 10:51:34AM +0800, wh.h@foxmail.com wrote:
How does the libvirt deal with the vnet mac address?
Greetings, if I establish a network for the VM (hypervisor is KVM) using bridge in the virt-manager , a vnet0 device is created . There are some relationships about mac address between the vnet0 device in the hypervisor and the ethX device in the VM, for example : the mac address of vnet0 is FE:54:00:84:E3:62 the mac address of ethX in the VM is 52:54:00:84:E3:62 two mac addresses above are almost the same except the first part of the address . but if I created a tap device manually , tunctl -t tap0 -u root brctl addif br0 tap0 and add tap0 to the VM, I will find that mac address between the tap0 device in the hypervisor and the ethX device in the VM will totally different . so I think that libvirt must do something about the mac address handling, could you please kindly tell me something about this ? When first created, the kernel assigns the tap device a completely random MAC address. This bears no relation to the MAC address that is used in the guest OS.
When you create a bridge device it initially has a MAC address of all zeros, and when you add NIC devices to the bridge, its MAC address gets update to the numerically lowest MAC address of all the NICs. The problem is that when the kernel assigns MAC addresses randomly, one of these random MAC address might be numerically lower than the bridge's current MAC address. So the effect is that when you start/stop guests, and their TAP devices get added/removed from the bridge, the bridge's own MAC address will occassionally change which is a bad thing.
So deal with this, libvirt will set all guest TAP devices so that they have a MAC address with 0xFE as the first byte. The real physical NIC added to the bridge is thus guaranteed to have a smaller MAC address, and so the bridge will permanently use the MAC address of the physical NIC, which is what we want.
For bridges which do not have any physical NIC, libvirt will create a dummy TAP device, not connected to any guest, and give it a small MAC address. This ensures again ensures the bridge MAC address won't change when guests start/stop.
How does the libvirt establish the arp table in the hypervisor if the vnet0 device in the hypervisor and the ethX device in the VM is different? The MAC address of the TAP device is actually totally irrelevant for the ARP table maintenance.
If a packet arrives on the bridge and the IP doesn't have a ARP table mapping, the bridge will just send it to all connected TAP devices.
When a packet arrives from a guest TAP, the source MAC address will be used to populate the ARP table.
In neither case does the MAC address of the TAP device itself have any involvement.
The only time the TAP device MAC address has any effect is when the kerenel auto-assigns a MAC to the bridge device as explained above.
If I want to create tap device manually , how should I deal with the mac address ?I have setup the mac address of the tap0 device in the hypervisor and the ethX device in the VM in the same way with libvirt , but the network of VM cannot work. As mentioned above, the TAP device MAC can be pretty much anything, but we'd recommend using 0xFE as the first byte.
Along with everything that Dan has explained, it's important to also know that it is essential the tap device's MAC address be different from the MAC address used by the guest. The reason is that the tap device will not forward a packet to the other side of itself if it sees a destination MAC address matching its own - it will think that the packet must be intended for local delivery; this is another reason that libvirt replaces the 1st byte of the guest address with 0xFE when setting the tap device address. (This is in contrast to macvtap devices, which *do* match the MAC address used by the guest.) (...and I'm still curious why you are choosing to do all of this yourself rather than just letting libvirt deal with it. Is there some functionality you need that libvirt can't supply?) /**********************************************/ Thanks a lot for your reply! The reason is that only the command scripts can be integrated to the upper layer software which is designed by workmates. I remain puzzled about : "it's important to also know that it is essential the tap device's MAC address be different from the MAC address used by the guest. The reason is that the tap device will not forward a packet to the other side of itself if it sees a destination MAC address matching its own" Does "the other side" mean : one side is ethx in the VM, the other side is tap device in the hypervisor? But , these two devices is actually the same one . If package is received in the ethx device in the VM , the package also arrives at the tap device in the hypervisor at the same time , doesn't it ? weihua wh.h@foxmail.com
(I'm not sure what the settings/commands are in the email application you use ("foxmail" apparently), but please configure it / use it such that you *reply* to messages rather than adding text onto a *forwarded* copy of the message you mean to reply to - in a "reply", everything in the original message has "> " (or something similar) added to the beginning of each line so that you can insert your own text in the middle and it will be easily recognizeable (no markers). Also, sending your messages in plain text format would be greatly appreciated (some readers of the mailing list will not be able to read your messages since they are in html-only, so you are lowering the visibility of your questions.) On 04/28/2015 09:48 AM, wh.h@foxmail.com wrote:
I remain puzzled about :
Laine Stump wrote:
> it's important to also > know that it is essential the tap device's MAC address be different from > the MAC address used by the guest. The reason is that the tap device > will not forward a packet to the other side of itself if it sees a > destination MAC address matching its own - it will think that the packet > must be intended for local delivery; this is another reason that libvirt > replaces the 1st byte of the guest address with 0xFE when setting the > tap device address.
Does "the other side" mean : one side is ethx in the VM, the other side is tap device in the hypervisor? But , these two devices is actually the same one . If package is received in the ethx device in the VM , the package also arrives at the tap device in the hypervisor at the same time , doesn't it ?
Well, not exactly, but I think I replied before I had enough coffee. You can disregard that entire paragraph, as it was based on some old memory of [something, not sure what right now] and experiments I performed just now verified that it is *not* the case for tap devices.
From: Daniel P. Berrange Date: 2015-04-27 16:59 To: wh.h@foxmail.com CC: libvirt-users Subject: Re: [libvirt-users] How does the libvirt deal with the vnet mac address On Sun, Apr 26, 2015 at 10:51:34AM +0800, wh.h@foxmail.com wrote:
How does the libvirt deal with the vnet mac address?
Greetings, if I establish a network for the VM (hypervisor is KVM) using bridge in the virt-manager , a vnet0 device is created . There are some relationships about mac address between the vnet0 device in the hypervisor and the ethX device in the VM, for example : the mac address of vnet0 is FE:54:00:84:E3:62 the mac address of ethX in the VM is 52:54:00:84:E3:62 two mac addresses above are almost the same except the first part of the address . but if I created a tap device manually , tunctl -t tap0 -u root brctl addif br0 tap0 and add tap0 to the VM, I will find that mac address between the tap0 device in the hypervisor and the ethX device in the VM will totally different . so I think that libvirt must do something about the mac address handling, could you please kindly tell me something about this ?
When first created, the kernel assigns the tap device a completely random MAC address. This bears no relation to the MAC address that is used in the guest OS. When you create a bridge device it initially has a MAC address of all zeros, and when you add NIC devices to the bridge, its MAC address gets update to the numerically lowest MAC address of all the NICs. The problem is that when the kernel assigns MAC addresses randomly, one of these random MAC address might be numerically lower than the bridge's current MAC address. So the effect is that when you start/stop guests, and their TAP devices get added/removed from the bridge, the bridge's own MAC address will occassionally change which is a bad thing. So deal with this, libvirt will set all guest TAP devices so that they have a MAC address with 0xFE as the first byte. The real physical NIC added to the bridge is thus guaranteed to have a smaller MAC address, and so the bridge will permanently use the MAC address of the physical NIC, which is what we want. For bridges which do not have any physical NIC, libvirt will create a dummy TAP device, not connected to any guest, and give it a small MAC address. This ensures again ensures the bridge MAC address won't change when guests start/stop.
How does the libvirt establish the arp table in the hypervisor if the vnet0 device in the hypervisor and the ethX device in the VM is different?
The MAC address of the TAP device is actually totally irrelevant for the ARP table maintenance. If a packet arrives on the bridge and the IP doesn't have a ARP table mapping, the bridge will just send it to all connected TAP devices. When a packet arrives from a guest TAP, the source MAC address will be used to populate the ARP table. In neither case does the MAC address of the TAP device itself have any involvement. The only time the TAP device MAC address has any effect is when the kerenel auto-assigns a MAC to the bridge device as explained above.
If I want to create tap device manually , how should I deal with the mac address ?I have setup the mac address of the tap0 device in the hypervisor and the ethX device in the VM in the same way with libvirt , but the network of VM cannot work.
As mentioned above, the TAP device MAC can be pretty much anything, but we'd recommend using 0xFE as the first byte. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| /**********************************************/ Thanks very much for your reply! -- weihua wh.h@foxmail.com
participants (3)
-
Daniel P. Berrange -
Laine Stump -
wh.h＠foxmail.com