Re: Could you please help with questions about the net failover feature

On 2020-07-06 10:01, Laine Stump wrote: > On 7/6/20 5:10 AM, Yalan Zhang wrote: >> Hi Laine, >> >> For the feature testing before, I only test the linux bridge setting as >> in 2), it works. >> Now I tried 1), to use macvtap bridge mode connected to the PF, it can >> not work as the hostdev interface can not get dhcp ip address on the >> guest. >> Check on host, the /var/log/messages and dmesg both says: >> >> "Jul  6 04:54:45 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 >> Spoofed packets detected >> ...... >> Jul  6 04:56:17 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 >> Spoofed packets detected >> Jul  6 04:56:54 dell-per730-xx kernel: ixgbe 0000:82:00.1 enp130s0f1: 1 >> Spoofed packets detected >> " >> (enp130s0f1 is the PF's interface name, and 0000:82:00.1 is the PF's pci >> address) >> # rpm -q kernel >> kernel-4.18.0-193.4.1.el8_2.x86_64 >> >> Could you please help to confirm if this is a kernel bug?  Thank you >> very much! > Interesting. I'm not sure if this is expected behavior, or if it's improper > behavior and it just hasn't been tested before (obviously based on my > earlier recommendation, I think it *should* be able to work like this, and I > *thought* I had tried it, but maybe I just imagined it :-/). > > I'm Cc'ing Stefan Assmann to see if he has an opinion on whether or not this > should work. For his convenience, here is a summary of the config: The setup > is that there is a bridge-mode macvtap interface on the PF, and one of the > VF's has been given the same MAC address as the macvtap. the macvtap > interface is connected to an emulated NIC in the guest, and the VF is > assigned to the guest with VFIO.

Hi Laine, I'm on PTO till the 20th. Ccing Ken as ixgbe maintainer. Stefan > I'll try this today on my setup, which uses I350 cards (igb driver). > > >> >> >> You have two choices for the backup virtio interface: >> >> 1) it can be a macvtap device connected to the PF of the same SRIOV device. >> >> 2) it can be a standard tap device connected to a Linux host bridge >> (created outside libvirt in the host system network config) that is >> attached to the PF (or alternately one of the VFs that isn't being used >> for VMs, or to another physical ethernet adapter on the host that is >> connected to the same network. >> >> >> >> >> ------- >> Best Regards, >> Yalan Zhang >> IRC: yalzhang >> >> >> On Sun, Mar 22, 2020 at 6:50 AM Laine Stump <laine(a)redhat.com >> <mailto:laine@redhat.com>> wrote: >> >> On 3/21/20 1:08 AM, Yalan Zhang wrote: >> >> > In my understanding, the standby and primary hostdev interface >> may be in >> > different subnet. >> >> There is only one hostdev device in the team pair (that will be the one >> with <teaming type='transient'.../> since it needs to be unplugged >> during migration). The other device must be a virtio device (the one >> with <teaming type='persistent'/>). And no, they cannot be on different >> subnets. They must both connect into the same ethernet "collision >> domain", such that the guest could assign the same IP address to either >> of them and be able to communicate on the network. >> >> There is some explanation of the use case for this option. and some >> example config, here: >> >> https://www.libvirt.org/formatdomain.html#elementsTeaming >> >> > I'm not sure whether it is correct. Could you please help to >> explain? >> > Thank you in advance. >> > >> > For example, primary hostdev is connected to vf-pool with >> <pf='eth0'/>, >> > while the standby is connected to NAT network with " forward >> dev='eth0'". >> > The standby interface will get ip as 192.168.122.x, but after >> NAT, it >> > will be in the same subnet of the vf. >>  > >> > So after the VF is unplugged, the packet will still broadcast in the >> > same subnet, and the vm will get the packet as the standby share the >> > same mac. Right? >> >> No, not right :-) >> >> The VF of an SRIOV network adapter is connected directly to the >> physical >> network, and will have an IP address that is on that network. Tap >> devices plugged into the default network (or any other libvirt network >> based on a bridge device that is created/managed by libvirt) have no >> direct connection to the physical network, and are on a different >> subnet. The fact that traffic from the guest *seems* to be coming from >> an IP on the physical subnet is meaningless. The *guest* needs to be >> able to use both NICs using the same IP address, and anything plugged >> into the default network will need to have an IP address on a different >> subnet from the perspective of the guest. >> >> You have two choices for the backup virtio interface: >> >> 1) it can be a macvtap device connected to the PF of the same SRIOV >> device. >> >> 2) it can be a standard tap device connected to a Linux host bridge >> (created outside libvirt in the host system network config) that is >> attached to the PF (or alternately one of the VFs that isn't being used >> for VMs, or to another physical ethernet adapter on the host that is >> connected to the same network. >> >> >> It is simplest to have the same name refer to the connection on the >> source and destination hosts of a migration. That can be handled by >> creating a libvirt network to refer to the bridge device created >> outside >> libvirt (or to the PF directly if you're going to use macvtap. >> >> For example, if you're going to use macvtap, and the PF's name on the >> host is ens4f0, you'd just create this network: >> >>    <network> >>      <name>persistent-net</name> >>      <forward mode='bridge'> >>        <interface dev='ens4f0'/> >>      </forward> >>     <network> >> >> any guest interface with this: >> >>       <interface type='network'> >>         <source network='persistent-net'/> >>         <mac address='00:11:22:33:44:55'/> >>         <model type='virtio'/> >>         <teaming type='persistent'/> >>         <alias name='ua-backup0'/> >>       </interface> >> >> will get a macvtap device that's connected to ens4f0 in bridge mode. >> >> Or, if your host has a bridge device called br0 that is directly >> attached to the physical network (in whatever manner, it doesn't >> matter), you can define the network this way: >> >>    <network> >>      <name>persistent-net</name> >>      <bridge name='br0'/> >>      <forward mode='bridge'/> >>     <network> >> >> The XML for the guest interface would be the same. >> >> Then for the vfio (transient) interface, you could also define a >> network: >> >>     <network> >>       <name>transient-net</name> >>       <forward mode='hostdev'> >>         <pf dev='ens4f0'/> >>       </forward> >>     </network> >> >> and instead of using <interface type='hostdev'> in the guest config, >> you >> would use this: >> >> >>      <interface type='network'> >>        <source network='transient-net'/> >>        <model type='virtio'/>    [1] >>        <mac address='00:11:22:33:44:55'/> >>        <teaming type='transient' persistent='ua-backup0'/> >>     </interface> >> >> Even if the device names change on the other host (the destination of >> the migration), as long as the other host has networks named >> "persistent-net" and "transient-net" that are of similar types (macvtap >> or bridged for persistent-net, and hostdev for transient-net) then >> libvirt will be able to migrate the guest from one host to the other >> with no mangling of the XML required. >> >>