[libvirt-users] Tunnelled migrate Windows7 VMs halted
[moderator note: I'm forwarding a stripped down version of the original mail which was rejected in the moderator queue. I stripped the 3.3 megabyte .tar.bz2 of the log file attachment, which is inappropriate for a technical list. Either trim the log to the relevant portion, or host the log externally and have your list email merely give a URL of the externally-hosted file]
ForwardedMessage.eml
Subject: Tunnelled migrate Windows7 VMs halted From: 邓林文 <dlworld0@163.com> Date: 04/25/2017 11:12 PM
To: libvirt-users@redhat.com
I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although the status is running.
[root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system
But when migrated with qemu native mode, the VM runs well.
[root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system
System Info: Release: Centos 7.2 Kernel: 3.10.0-327.28.3.el7.x86_64 Qemu: qemu-kvm-rhev-2.3.0 Libvirt: libvirt-1.2.17/libvirt-2.0.0 CPU: AMD Opteron(TM) Processor 6212
As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
Does anyone have some sugestions?
Thanks, Linwen Deng
vm.xml
<domain type='kvm' id='8'> <name>i-000000ac</name> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="12.0.0-4"/> <nova:name>c7-vm15-test15-7</nova:name> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> <nova:flavor name="oeflavor-4-2048-20"> <nova:memory>2048</nova:memory> <nova:disk>20</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>4</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> </nova:owner> </nova:instance> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <hugepages> <page size='2048' unit='KiB' nodeset='0'/> </hugepages> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <shares>4096</shares> <vcpupin vcpu='0' cpuset='4-7'/> <vcpupin vcpu='1' cpuset='4-7'/> <emulatorpin cpuset='4-7'/> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Fedora Project</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>12.0.0-4</entry> <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> <boot dev='fd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='2' cores='1' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/opt/ssd/win7.qcow2'/> <backingStore type='file' index='1'> <format type='raw'/> <source file='/opt/ssd/_base/win7.qcow2'/> <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> <serial>8164a82d-6066-46d7-8a92-8391620fc58c</serial> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> <serial type='file'> <source path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/> <target port='0'/> <alias name='serial0'/> </serial> <serial type='pty'> <source path='/dev/pts/2'/> <target port='1'/> <alias name='serial1'/> </serial> <console type='file'> <source path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0' state='connected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <graphics type='spice' port='5901' autoport='yes' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> <image compression='auto_glz'/> <streaming mode='filter'/> <mouse mode='client'/> </graphics> <sound model='ich6'> <alias name='sound0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </sound> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <stats period='10'/> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </memballoon> </devices> <seclabel type='none' model='none'/> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+0:+107</label> <imagelabel>+0:+107</imagelabel> </seclabel> </domain>
-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU has crashed, or something else ?
[root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system
But when migrated with qemu native mode, the VM runs well.
[root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in both cases QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
System Info: Release: Centos 7.2 Kernel: 3.10.0-327.28.3.el7.x86_64 Qemu: qemu-kvm-rhev-2.3.0 Libvirt: libvirt-1.2.17/libvirt-2.0.0
Does the 2 libvirt versions mean you are live-migrating between two different versions of CentOS ? If so, this also likely means two different versions of QEMU involved.
CPU: AMD Opteron(TM) Processor 6212
As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
Does anyone have some sugestions?
Thanks, Linwen Deng
vm.xml
<domain type='kvm' id='8'> <name>i-000000ac</name> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="12.0.0-4"/> <nova:name>c7-vm15-test15-7</nova:name> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> <nova:flavor name="oeflavor-4-2048-20"> <nova:memory>2048</nova:memory> <nova:disk>20</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>4</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> </nova:owner> </nova:instance> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <hugepages> <page size='2048' unit='KiB' nodeset='0'/> </hugepages> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <shares>4096</shares> <vcpupin vcpu='0' cpuset='4-7'/> <vcpupin vcpu='1' cpuset='4-7'/> <emulatorpin cpuset='4-7'/> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Fedora Project</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>12.0.0-4</entry> <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> <boot dev='fd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='2' cores='1' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock>
If nothing else, this guest is incorrectly configured for running the Windows OS. I suspect you forget to set the Glance image property needed to tell Nova that it is windows. Specifically it is missing hyperv paravirt <timer>, and many hyperv paravirt <feature> elements. This will make Windows guests unstable and liable to hit BSOD Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange@redhat.com> wrote:
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU has crashed, or something else ?
I can still connect to the guest OS and see the windows opened before migrate on the target, but cann't click any more, and there's no effect when send "ctrl+alt+del" keys by virsh.
[root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system
But when migrated with qemu native mode, the VM runs well.
[root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in both cases QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
When I destroy the guest and restart, the guest OS works well. The attachment is a stripped version log.
System Info: Release: Centos 7.2 Kernel: 3.10.0-327.28.3.el7.x86_64 Qemu: qemu-kvm-rhev-2.3.0 Libvirt: libvirt-1.2.17/libvirt-2.0.0
Does the 2 libvirt versions mean you are live-migrating between two different versions of CentOS ? If so, this also likely means two different versions of QEMU involved.
CPU: AMD Opteron(TM) Processor 6212
As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
Does anyone have some sugestions?
Thanks, Linwen Deng
vm.xml
<domain type='kvm' id='8'> <name>i-000000ac</name> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="12.0.0-4"/> <nova:name>c7-vm15-test15-7</nova:name> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> <nova:flavor name="oeflavor-4-2048-20"> <nova:memory>2048</nova:memory> <nova:disk>20</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>4</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> </nova:owner> </nova:instance> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <hugepages> <page size='2048' unit='KiB' nodeset='0'/> </hugepages> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <shares>4096</shares> <vcpupin vcpu='0' cpuset='4-7'/> <vcpupin vcpu='1' cpuset='4-7'/> <emulatorpin cpuset='4-7'/> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Fedora Project</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>12.0.0-4</entry> <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> <boot dev='fd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='2' cores='1' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock>
If nothing else, this guest is incorrectly configured for running the Windows OS. I suspect you forget to set the Glance image property needed to tell Nova that it is windows.
Specifically it is missing hyperv paravirt <timer>, and many hyperv paravirt <feature> elements.
This will make Windows guests unstable and liable to hit BSOD
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange@redhat.com> wrote:
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU has crashed, or something else ?
[root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system
But when migrated with qemu native mode, the VM runs well.
[root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in both cases QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
System Info: Release: Centos 7.2 Kernel: 3.10.0-327.28.3.el7.x86_64 Qemu: qemu-kvm-rhev-2.3.0 Libvirt: libvirt-1.2.17/libvirt-2.0.0
Does the 2 libvirt versions mean you are live-migrating between two different versions of CentOS ? If so, this also likely means two different versions of QEMU involved.
Migrate between 2 same version, and tried to upgrade the libvirt version.
CPU: AMD Opteron(TM) Processor 6212
As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
Does anyone have some sugestions?
Thanks, Linwen Deng
vm.xml
<domain type='kvm' id='8'> <name>i-000000ac</name> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="12.0.0-4"/> <nova:name>c7-vm15-test15-7</nova:name> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> <nova:flavor name="oeflavor-4-2048-20"> <nova:memory>2048</nova:memory> <nova:disk>20</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>4</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> </nova:owner> </nova:instance> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <memoryBacking> <hugepages> <page size='2048' unit='KiB' nodeset='0'/> </hugepages> </memoryBacking> <vcpu placement='static'>2</vcpu> <cputune> <shares>4096</shares> <vcpupin vcpu='0' cpuset='4-7'/> <vcpupin vcpu='1' cpuset='4-7'/> <emulatorpin cpuset='4-7'/> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Fedora Project</entry> <entry name='product'>OpenStack Nova</entry> <entry name='version'>12.0.0-4</entry> <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> <boot dev='cdrom'/> <boot dev='hd'/> <boot dev='fd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='2' cores='1' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock>
If nothing else, this guest is incorrectly configured for running the Windows OS. I suspect you forget to set the Glance image property needed to tell Nova that it is windows.
Specifically it is missing hyperv paravirt <timer>, and many hyperv paravirt <feature> elements.
This will make Windows guests unstable and liable to hit BSOD
I have tried to update the xml, the problem also exists, although there's one success in 5 tries. But the guest can accept ctrl-alt-delete, further operations still not available. <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> <timer name='hypervclock' present='yes'/> </clock>
Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
participants (3)
-
Daniel P. Berrange -
Eric Blake -
邓林文