Dear Team,

I am reaching out regarding an issue I encountered with libvirt and MAC address conflicts. Below is a summary of the situation: 1. Initially, the vNIC's MAC address was different from the target VM's MAC address.

2. After modifying the vNIC's MAC address to match the VM's MAC address, the network was interrupted. 3. After rebooting the VM, the vNIC's MAC address was automatically modified again.

I have observed the following kernel logs during this process: Dec 24 16:59:40 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered disabled stateDec 24 16:59:40 zstack-manager kernel: device vnic43.0 left promiscuous modeDec 24 16:59:40 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered disabled stateDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered blocking stateDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered disabled stateDec 24 17:00:11 zstack-manager kernel: device vnic43.0 entered promiscuous modeDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered blocking stateDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered forwarding state I am looking to understand the underlying code that handles the automatic modification of the vNIC's MAC address after the conflict and how the network interruption occurs. Can you help direct me to the relevant code segment or provide any insights into this behavior?

Thank you for your assistance. Best regards,

To further clarify - when you say "vNIC" do you mean the tap device that is connected to the bridge on the host side, and "target VM's MAC address" is the MAC address of the network device inside the guest (VM)?

"If no target dev is specified, libvirt will create a new standard tap device with a name of the pattern 'vnetN', where 'N' is replaced with a number. If a target dev is specified and that device doesn't exist, then a new standard tap device will be created with the exact dev name given."

"Any time you stop/destroy a guest, any tap device(s) associated with that guest will be destroyed, and the next time you start the guest, new tap devices will be created."

(Somehow I never received the original of this message into my libvirt folder. Possibly my email client mistakenly decided it was spam...) On 2/3/25 8:36 AM, Martin Kletzander wrote: > On Tue, Dec 24, 2024 at 05:26:29PM +0800, Xuda Zhang wrote: >> Dear Team, >> > > Hi, not sure if this is still relevant, but ... > >> I am reaching out regarding an issue I encountered with libvirt and MAC >> address conflicts. Below is a summary of the situation: >> >> 1. Initially, the vNIC's MAC address was different from the target VM's >> MAC address. > > you are talking about a vNIC from the host's point of view and "target > VM" as seen from the guest? I'm just trying to make sure I understand. To further clarify - when you say "vNIC" do you mean the tap device that is connected to the bridge on the host side, and "target VM's MAC address" is the MAC address of the network device inside the guest (VM)? Assuming that is the case, the explanation is this: The tap device on the host *must* have a different MAC address than the device in the guest. If the two MAC addresses are the same, then when the host's network stack see a packet destined for that MAC address, it will think to itself "Hey - that's the MAC address of an interface on *this* machine*, so I don't need to forward it anywhere!" and then attempt to deliver the packet locally (i.e. send it up to IP on the host). You would of course see this as "guest networking doesn't work". The reason that you see the tap device MAC address go back to "normal" when you stop and restart the guest is because any time you stop/destroy a guest, any tap device(s) associated with that guest will be destroyed, and the next time you start the guest, *new* tap devices will be created. And whenever libvirt creates a tap device, it automatically makes a MAC address that is just "guest interface MAC address, except replace the first byte with 0xFE". This is done to guarantee that the tap device and guest interface have different MAC addresses, but they are similar to help make it more obvious which tap device is used by which guest interface (because the MACs are *almost* the same). So everything that you see is normal and, for the most part, necessary. I'm surprised that you care about the MAC address of the tap device. It really doesn't matter what it is except that it must be different from the MAC in the guest. > >> 2. After modifying the vNIC's MAC address to match the VM's MAC >> address, >> the network was interrupted. >> 3. After rebooting the VM, the vNIC's MAC address was automatically >> modified again. >> > > Are you using some filtering (nwfilter) on the libvirt network? > >> I have observed the following kernel logs during this process: >> >> Dec 24 16:59:40 zstack-manager kernel: br_enp2s0: port 14(vnic43.0) >> entered disabled stateDec 24 16:59:40 zstack-manager kernel: device >> vnic43.0 left promiscuous modeDec 24 16:59:40 zstack-manager kernel: >> br_enp2s0: port 14(vnic43.0) entered disabled stateDec 24 17:00:11 >> zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered blocking >> stateDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port >> 14(vnic43.0) entered disabled stateDec 24 17:00:11 zstack-manager >> kernel: device vnic43.0 entered promiscuous modeDec 24 17:00:11 >> zstack-manager kernel: br_enp2s0: port 14(vnic43.0) entered blocking >> stateDec 24 17:00:11 zstack-manager kernel: br_enp2s0: port >> 14(vnic43.0) entered forwarding state >> >> I am looking to understand the underlying code that handles the automatic >> modification of the vNIC's MAC address after the conflict and how the >> network interruption occurs. Can you help direct me to the relevant code >> segment or provide any insights into this behavior? See above for the explanation of why the MAC address of the tap is changed, and why networking stops if you change it to match the guest MAC. The code that sets the MAC address is in virNetDevTapCreateInBridgePort(), src/util/virnetdevtap.c:638 in current upstream sources - this is done unconditionally every time a tap device is created. >> > > What conflict are you talking precisely? Is something having the same > MAC address as the VM? > >> Thank you for your assistance. >> >> Best regards,

But again, already-existing tap devices can't be used for interface type='bridge' or type='network' (which also connects the tap to a bridge).

But again, already-existing tap devices can't be used for interface type='bridge' or type='network' (which also connects the tap to a bridge).

On 2/5/25 1:39 AM, Xuda Zhang wrote: > Hi Laine, > > You mentioned: > > But again, already-existing tap devices can't be used for interface > type='bridge' or type='network' (which also connects the tap to a > bridge). > > However, the documentation does not *explicitly *state that already- > existing tap devices *cannot *be used for interface type='bridge' or > interface type='network'. Any reasonably large piece of software has many many behaviors that aren't explicitly stated. > I also conducted another test, which confirmed that whenever the VM is > shut down, the existing tap device is deleted. > This suggests that the correct understanding is *recreation *rather than > modification. That's basically what I told you. It's unfortunate that I ever brought up the idea of pre-existing tap devices, since they are only allowed in one very narrow use case. > Can I conclude that, in a bridge network type, libvirt is expected to > create and manage the tap device automatically? > This would mean that the tap device is always created by libvirt, its > lifecycle is tied to the VM (guest OS), and it is deleted when the VM > stops. Correct. > Additionally, each time the VM starts, a new tap device is created with > slight variations, likely derived from the VM's MAC address. I guess you mean "... with a MAC address that is the same as the guest interface's MAC except the first byte is changed to 0xFE." If so then yes. > Would you agree with this assessment? Yes. > Best regards, > > Xuda Zhang > > > > Laine Stump <laine(a)laine.org <mailto:laine@laine.org>> 于2025年2月5日周 > 三 14:03写道： > > On 2/4/25 11:04 PM, Xuda Zhang wrote: > > > Could you help confirm the exact behavior in this case? Specifically: > > > > 1. If a target tap device already exists, does libvirt modify > the MAC > > address instead of recreating the device? > > Sorry, I needed to qualify that detail a bit (and actually I probably > shouldn't have even brought it up, since it doesn't apply to tap > devices > used to connect to a bridge device) - *only for > "<interface type='ethernet' managed='no'>" * an already-existing tap > device can be specified in the XML (with <target dev='blah'/>). > type='ethernet' is used for tap devices that aren't connected to any > bridge (all communication with the guest must then be *routed* by the > host at the IP level, rather than being bridged). > > And a detail that I misremembered - when libvirt uses a pre-existing > tap > device, it assumes that the creator of the tap already set the MAC > appropriately, so it doesn't modify it. > > But again, already-existing tap devices can't be used for interface > type='bridge' or type='network' (which also connects the tap to a > bridge). > > > 2. Under what circumstances does libvirt destroy and recreate > the tap > > device instead of modifying its attributes? > > Another detail that I've forgotten over the long time since I last > looked at this code. libvirt doesn't explicitly delete the tap device, > it just closes the device. In the case of tap devices that libvirt > itself created, they are automatically deleted when they are closed. > > In the case of pre-existing tap devices (which again doesn't apply in > your use case), 1) libvirt assumes that the creator of the pre-existing > device has already set the MAC address to something appropriate, so it > doesn't attempt to change it, and 2) again libvirt won't explicitly > delete the tap device. If it was created as a persistent device, then > closing it doesn't cause it to be auto-deleted, but if the original > creator of the tap device didn't create it as persistent, and no other > process has an open handle for the device, then again closing the > device > will auto-delete it. > > But again, for your use case (where the tap is connected to a bridge) > the creator of the tap device is always libvirt, and so it will always > be auto-deleted when libvirt closes the final handle it has open on the > device. > > > Looking forward to your insights! > > > > Best regards, > > Xuda Zhang >