11:42 p.m.
On a hypervisor with multiple interfaces, we'd like to make those interfaces available
to each of multiple VMs.
We have both bridged interfaces and SRIOV interfaces, no NAT interfaces.
When we create a vm in virsh using an xml file with no network elements, but having
interface elements, we get the
interfaces in the vm, but the ordering of eth devices does not match the ordering on the
hypervisor and we aren't sure
the ordering is repeatable either. In other words, eth0 in the HV is thingA and eth1 is
thingB, but on the VM eth0 is
thingC eth1 is thingD and eth4 is thingA so forth.
we have 12 interfaces so ordering or naming is important and confusing.
We've tried to use the PCI bus numbers as seen in lspci on the HV, but no luck.
Do we need to start using network elements in the vm's xml file or separately to
control the ordering? Whats the usual
practice here?
Sample xml:
<interface type='hostdev'>
<source dev='eth3' mode='vepa'>
<address type='pci' domain='0x0000' bus='0x16'
slot='0x10' function='0x1'/>
</source>
<target dev='macvtap0'/>
</interface>
<interface type='bridge'>
<source bridge='br4'/>
<target dev='vnet0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
Thanks!
--
John Fisher
12:27 a.m.
@Gao
see the other thread Managing eths in vm from hv too.
We weren't able to order based on vnets, though I agree its seems intuitive. But our
test wasn't thorough. Eric says:
"Libvirt passes the devices to the qemu command line in the same order that they
appear in the XML. It is not libvirt's
fault if qemu then takes the command line arguments and presents it to the guest in such a
way that the guest can
initialize the devices in the order in which a bus scan finds them, rather than in the
order they were presented on the
command line."
as far as I can tell the bridges are always setup first, then the direct VF connections.
Whether this is caused by our
hardware or Qemu or PCI I have no idea.
Now, I am trying the idea of editing the /etc/udev/rules.d/70* file to move the MAC
addresses around. Results are mixed.
very puzzling.
John
On 12/08/2012 08:21 AM, Gao Yongwei wrote:
2012/12/7 john fisher <john(a)jpfisher.net <mailto:john@jpfisher.net>>
<target dev='vnet0'/>
I think the vnet* can affect the order in vm.
--
John Fisher
2:38 a.m.
Gao you were right! see below....
On 12/08/2012 08:21 AM, Gao Yongwei wrote:
2012/12/7 john fisher <john(a)jpfisher.net <mailto:john@jpfisher.net>>
<target dev='vnet0'/>
I think the vnet* can affect the order in vm.
For the benefit of 1) others and 2) me when I forget how this works-
I did find a solution in formatting the xml file.
If you leave the vnets out completely, see file below, the generic xml file will
cooperate with libvirt and qemu and
order the VM's eth devices as they are ordered on the hypervisor.
(note: the macvtap entries seen below may also not be needed, sound and usb not tested)
## sample xml file for libvirt 1.0.0 showing some bridges and some SRIOV ports too ##
<domain type='kvm' id='1'>
<name>sample</name>
<hostname>sample</hostname>
<memory unit='KiB'>524288</memory>
<currentMemory unit='KiB'>524288</currentMemory>
<vcpu placement='static'>1</vcpu>
<os>
<type arch='x86_64' machine='pc-1.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/oa4-vm-sample-cli.qcow2'/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
</disk>
<controller type='usb' index='0'>
<alias name='usb0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
</controller>
<interface type='bridge'>
<source bridge='br4'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br5'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br6'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br7'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br8'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br9'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br10'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br11'/>
<model type='virtio'/>
</interface>
<interface type='bridge'>
<source bridge='br250'/>
<model type='virtio'/>
</interface>
<interface type='hostdev'>
<source dev='eth0' mode='vepa'>
<address type='pci' domain='0x0000'
bus='0x02' slot='0x10' function='0x0'/>
</source>
<target dev='macvtap1'/>
<model type='virtio'/>
</interface>
<interface type='hostdev'>
<source dev='eth1' mode='vepa'>
<address type='pci' domain='0x0000'
bus='0x02' slot='0x10' function='0x1'/>
</source>
<target dev='macvtap1'/>
<model type='virtio'/>
</interface>
<interface type='hostdev'>
<source dev='eth2' mode='vepa'>
<address type='pci' domain='0x0000'
bus='0x16' slot='0x10' function='0x0'/>
</source>
<target dev='macvtap0'/>
</interface>
<interface type='hostdev'>
<source dev='eth3' mode='vepa'>
<address type='pci' domain='0x0000'
bus='0x16' slot='0x10' function='0x1'/>
</source>
<target dev='macvtap0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/1'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='5900' autoport='yes'
listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
<sound model='ich6'>
<alias name='sound0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
</sound>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x0'/>
</memballoon>
</devices>
<seclabel type='none'/>
</domain>
--
John Fisher
4595
days inactive
4607
days old
3 comments
2 participants
participants (2)
-
Gao Yongwei -
john fisher