1:35 a.m.
List,
I have a number of Windows 2016 servers I am deploying, but I’m having some I/O
performance issues. I have done all of the obvious things like virtio drivers, but am
finding there is more performance to be found with hyper-v extensions, how we virtualize
the hardware clock, and iothreads. I’m using ZVOLs to back the VM, and I’m using 4k block
sizes, which seems to offer the best 4k random read/write performance (mail and database
workloads), but maybe I’m missing something at this layer too.
Questions:
1. Does my VM config look reasonable for the latest releases of windows? Are there
features I should be using that will help performance?
2. Why does the hypervclock timer make so much performance difference in windows VMs?
3. Does my virtualized CPU model make sense? I defined Haswell-noTSX-IBRS and libvirt
added the features.
4. Which kernel branch offers the best stability and performance?
5. Are there performance gains in using UEFI booting the windows guest and defining
“<blockio logical_block_size='4096' physical_block_size='4096'/>”?
Perhaps better block size consistency through to the zvol?
Here is my setup:
48 core Haswell CPU
192G Ram
Linux 4.14.61 or 4.9.114 (testing both)
ZFS file system on optane SSD drive or ZFS file system on dumb HBA with 8 spindles of 15k
disks (testing both)
4k block size zvol for virtual machines
32G arc cache
Here is my VM:
<domain type='kvm' id='12'>
<name>testvm</name>
<memory unit='KiB'>33554432</memory>
<currentMemory unit='KiB'>33554432</currentMemory>
<vcpu placement='static'>12</vcpu>
<iothreads>1</iothreads>
<os>
<type arch='x86_64' machine='pc-i440fx-2.12'>hvm</type>
<boot dev='cdrom'/>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<hyperv>
<relaxed state='on'/>
<vapic state='on'/>
<spinlocks state='on' retries='8191'/>
<vpindex state='on'/>
<runtime state='on'/>
<synic state='on'/>
<reset state='on'/>
<vendor_id state='on' value='KVM Hv'/>
</hyperv>
</features>
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Haswell-noTSX-IBRS</model>
<topology sockets='1' cores='6' threads='2'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='f16c'/>
<feature policy='require' name='rdrand'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
<feature policy='disable' name='spec-ctrl'/>
<feature policy='require' name='xsaveopt'/>
<feature policy='require' name='abm'/>
</cpu>
<clock offset='localtime'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='yes'/>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'
io='native' ioeventfd='on' iothread='1'/>
<source dev='/dev/zvol/datastore/vm/testvm-vda'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<alias name='ide0-1-0'/>
<address type='drive' controller='0' bus='1'
target='0' unit='0'/>
</disk>
<controller type='ide' index='0'>
<alias name='ide'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
</controller>
<controller type='usb' index='0' model='piix3-uhci'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
</controller>
<interface type='bridge'>
<source bridge='lan'/>
<target dev='vnet0'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
<channel type='unix'>
<source mode='bind'
path='/var/lib/libvirt/qemu/channel/target/domain-12-testvm/org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0'
state='connected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0'
port='1'/>
</channel>
<input type='tablet' bus='usb'>
<alias name='input0'/>
<address type='usb' bus='0' port='1'/>
</input>
<input type='mouse' bus='ps2'>
<alias name='input1'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input2'/>
</input>
<graphics type='vnc' port='5901' autoport='no'
listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<video>
<model type='cirrus' vram='16384' heads='1'
primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
</video>
<memballoon model='none'/>
</devices>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+0:+100</label>
<imagelabel>+0:+100</imagelabel>
</seclabel>
</domain>