8:51 a.m.
[moderator note: I'm forwarding a stripped down version of the original
mail which was rejected in the moderator queue. I stripped the 3.3
megabyte .tar.bz2 of the log file attachment, which is inappropriate for
a technical list. Either trim the log to the relevant portion, or host
the log externally and have your list email merely give a URL of the
externally-hosted file]
ForwardedMessage.eml
Subject:
Tunnelled migrate Windows7 VMs halted
From:
邓林文 <dlworld0(a)163.com>
Date:
04/25/2017 11:12 PM
To:
libvirt-users(a)redhat.com
I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although
the status is running.
[root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac
qemu+tcp://192.168.65.13/system
But when migrated with qemu native mode, the VM runs well.
[root@test15 ~]# virsh migrate --live --p2p i-000000ac
qemu+tcp://192.168.65.13/system
System Info:
Release: Centos 7.2
Kernel: 3.10.0-327.28.3.el7.x86_64
Qemu: qemu-kvm-rhev-2.3.0
Libvirt: libvirt-1.2.17/libvirt-2.0.0
CPU: AMD Opteron(TM) Processor 6212
As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
Does anyone have some sugestions?
Thanks,
Linwen Deng
vm.xml
<domain type='kvm' id='8'>
<name>i-000000ac</name>
<uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid>
<metadata>
<nova:instance
xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0">
<nova:package version="12.0.0-4"/>
<nova:name>c7-vm15-test15-7</nova:name>
<nova:creationTime>2017-04-14 06:48:54</nova:creationTime>
<nova:flavor name="oeflavor-4-2048-20">
<nova:memory>2048</nova:memory>
<nova:disk>20</nova:disk>
<nova:swap>0</nova:swap>
<nova:ephemeral>0</nova:ephemeral>
<nova:vcpus>4</nova:vcpus>
</nova:flavor>
<nova:owner>
<nova:user
uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user>
<nova:project
uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project>
</nova:owner>
</nova:instance>
</metadata>
<memory unit='KiB'>2097152</memory>
<currentMemory unit='KiB'>2097152</currentMemory>
<memoryBacking>
<hugepages>
<page size='2048' unit='KiB' nodeset='0'/>
</hugepages>
</memoryBacking>
<vcpu placement='static'>2</vcpu>
<cputune>
<shares>4096</shares>
<vcpupin vcpu='0' cpuset='4-7'/>
<vcpupin vcpu='1' cpuset='4-7'/>
<emulatorpin cpuset='4-7'/>
</cputune>
<resource>
<partition>/machine</partition>
</resource>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>Fedora Project</entry>
<entry name='product'>OpenStack Nova</entry>
<entry name='version'>12.0.0-4</entry>
<entry
name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry>
<entry
name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry>
<entry name='family'>Virtual Machine</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64'
machine='pc-i440fx-rhel7.2.0'>hvm</type>
<boot dev='cdrom'/>
<boot dev='hd'/>
<boot dev='fd'/>
<smbios mode='sysinfo'/>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-model'>
<model fallback='allow'/>
<topology sockets='2' cores='1' threads='1'/>
</cpu>
<clock offset='utc'>
<timer name='pit' tickpolicy='delay'/>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='none'/>
<source file='/opt/ssd/win7.qcow2'/>
<backingStore type='file' index='1'>
<format type='raw'/>
<source file='/opt/ssd/_base/win7.qcow2'/>
<backingStore/>
</backingStore>
<target dev='vda' bus='virtio'/>
<serial>8164a82d-6066-46d7-8a92-8391620fc58c</serial>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x07' function='0x0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<alias name='usb'/>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<alias name='usb'/>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<alias name='usb'/>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>
</controller>
<serial type='file'>
<source
path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<serial type='pty'>
<source path='/dev/pts/2'/>
<target port='1'/>
<alias name='serial1'/>
</serial>
<console type='file'>
<source
path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'
state='connected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0'
port='1'/>
</channel>
<input type='tablet' bus='usb'>
<alias name='input0'/>
</input>
<input type='mouse' bus='ps2'>
<alias name='input1'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input2'/>
</input>
<graphics type='vnc' port='5900' autoport='yes'
listen='0.0.0.0' keymap='en-us'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<graphics type='spice' port='5901' autoport='yes'
listen='0.0.0.0' keymap='en-us'>
<listen type='address' address='0.0.0.0'/>
<image compression='auto_glz'/>
<streaming mode='filter'/>
<mouse mode='client'/>
</graphics>
<sound model='ich6'>
<alias name='sound0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
</sound>
<video>
<model type='qxl' ram='65536' vram='65536'
vgamem='16384' primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<stats period='10'/>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x08' function='0x0'/>
</memballoon>
</devices>
<seclabel type='none' model='none'/>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+0:+107</label>
<imagelabel>+0:+107</imagelabel>
</seclabel>
</domain>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
8:59 a.m.
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
>
> I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted
> on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU
has crashed, or something else ?
>
>
> [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac
qemu+tcp://192.168.65.13/system
>
>
> But when migrated with qemu native mode, the VM runs well.
>
>
> [root@test15 ~]# virsh migrate --live --p2p i-000000ac
qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in
both cases
QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
> System Info:
> Release: Centos 7.2
> Kernel: 3.10.0-327.28.3.el7.x86_64
> Qemu: qemu-kvm-rhev-2.3.0
> Libvirt: libvirt-1.2.17/libvirt-2.0.0
Does the 2 libvirt versions mean you are live-migrating between two different
versions of CentOS ? If so, this also likely means two different versions
of QEMU involved.
> CPU: AMD Opteron(TM) Processor 6212
>
>
> As CPUFrequency may lead windows migrate halt, I have disabled Power Management.
>
>
> Does anyone have some sugestions?
>
>
> Thanks,
> Linwen Deng
>
>
> vm.xml
>
>
> <domain type='kvm' id='8'>
> <name>i-000000ac</name>
> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid>
> <metadata>
> <nova:instance
xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0">
> <nova:package version="12.0.0-4"/>
> <nova:name>c7-vm15-test15-7</nova:name>
> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime>
> <nova:flavor name="oeflavor-4-2048-20">
> <nova:memory>2048</nova:memory>
> <nova:disk>20</nova:disk>
> <nova:swap>0</nova:swap>
> <nova:ephemeral>0</nova:ephemeral>
> <nova:vcpus>4</nova:vcpus>
> </nova:flavor>
> <nova:owner>
> <nova:user
uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user>
> <nova:project
uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project>
> </nova:owner>
> </nova:instance>
> </metadata>
> <memory unit='KiB'>2097152</memory>
> <currentMemory unit='KiB'>2097152</currentMemory>
> <memoryBacking>
> <hugepages>
> <page size='2048' unit='KiB' nodeset='0'/>
> </hugepages>
> </memoryBacking>
> <vcpu placement='static'>2</vcpu>
> <cputune>
> <shares>4096</shares>
> <vcpupin vcpu='0' cpuset='4-7'/>
> <vcpupin vcpu='1' cpuset='4-7'/>
> <emulatorpin cpuset='4-7'/>
> </cputune>
> <resource>
> <partition>/machine</partition>
> </resource>
> <sysinfo type='smbios'>
> <system>
> <entry name='manufacturer'>Fedora Project</entry>
> <entry name='product'>OpenStack Nova</entry>
> <entry name='version'>12.0.0-4</entry>
> <entry
name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry>
> <entry
name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry>
> <entry name='family'>Virtual Machine</entry>
> </system>
> </sysinfo>
> <os>
> <type arch='x86_64'
machine='pc-i440fx-rhel7.2.0'>hvm</type>
> <boot dev='cdrom'/>
> <boot dev='hd'/>
> <boot dev='fd'/>
> <smbios mode='sysinfo'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> </features>
> <cpu mode='host-model'>
> <model fallback='allow'/>
> <topology sockets='2' cores='1' threads='1'/>
> </cpu>
> <clock offset='utc'>
> <timer name='pit' tickpolicy='delay'/>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='hpet' present='no'/>
> </clock>
If nothing else, this guest is incorrectly configured for running
the Windows OS. I suspect you forget to set the Glance image
property needed to tell Nova that it is windows.
Specifically it is missing hyperv paravirt <timer>, and many hyperv
paravirt <feature> elements.
This will make Windows guests unstable and liable to hit BSOD
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
9:49 a.m.
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange(a)redhat.com> wrote:
When I destroy the guest and restart, the guest OS works well.
The attachment is a stripped version log.
>> > System Info:
>> > Release: Centos 7.2
>> > Kernel: 3.10.0-327.28.3.el7.x86_64
>> > Qemu: qemu-kvm-rhev-2.3.0
>> > Libvirt: libvirt-1.2.17/libvirt-2.0.0
>Does the 2 libvirt versions mean you are live-migrating
between two different
>versions of CentOS ? If so, this also likely means two different versions
>of QEMU involved.
>> > CPU: AMD Opteron(TM) Processor 6212
>> >
>> >
>> > As CPUFrequency may lead windows migrate halt, I have disabled Power
Management.
>> >
>> >
>> > Does anyone have some sugestions?
>> >
>> >
>> > Thanks,
>> > Linwen Deng
>> >
>> >
>> > vm.xml
>> >
>> >
>> > <domain type='kvm' id='8' >>
> <name>i-000000ac</name >>
> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid >> > <metadata >>
> <nova:instance
xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0" >> > <nova:package
version="12.0.0-4"/ >> >
<nova:name>c7-vm15-test15-7</nova:name >>
> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime >> > <nova:flavor
name="oeflavor-4-2048-20" >>
> <nova:memory>2048</nova:memory >>
> <nova:disk>20</nova:disk >>
> <nova:swap>0</nova:swap >>
> <nova:ephemeral>0</nova:ephemeral >>
> <nova:vcpus>4</nova:vcpus >>
> </nova:flavor >> >
<nova:owner >> > <nova:user
uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user >> > <nova:project
uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project >> > </nova:owner >> > </nova:instance >> > </metadata >>
> <memory unit='KiB'>2097152</memory >>
> <currentMemory unit='KiB'>2097152</currentMemory >> > <memoryBacking >>
> <hugepages >> > <page
size='2048' unit='KiB' nodeset='0'/ >>
> </hugepages >> >
</memoryBacking >> > <vcpu
placement='static'>2</vcpu >>
> <cputune >> >
<shares>4096</shares >> > <vcpupin
vcpu='0' cpuset='4-7'/ >>
> <vcpupin vcpu='1' cpuset='4-7'/ >>
> <emulatorpin cpuset='4-7'/ >>
> </cputune >> > <resource >> > <partition>/machine</partition >> > </resource >>
> <sysinfo type='smbios' >>
> <system >> > <entry
name='manufacturer'>Fedora Project</entry >>
> <entry name='product'>OpenStack Nova</entry >> > <entry
name='version'>12.0.0-4</entry >>
> <entry
name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry >> > <entry
name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry >> > <entry name='family'>Virtual
Machine</entry >> > </system >> > </sysinfo >>
> <os >> > <type
arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type >> > <boot dev='cdrom'/ >> > <boot dev='hd'/ >> > <boot dev='fd'/ >> > <smbios mode='sysinfo'/ >> > </os >>
> <features >> > <acpi/ >> > <apic/ >>
> </features >> > <cpu
mode='host-model' >> > <model
fallback='allow'/ >> > <topology
sockets='2' cores='1' threads='1'/ >>
> </cpu >> > <clock
offset='utc' >> > <timer
name='pit' tickpolicy='delay'/ >>
> <timer name='rtc' tickpolicy='catchup'/ >> > <timer name='hpet'
present='no'/ >> > </clock >If nothing else, this guest is
incorrectly configured for running
>the Windows OS. I suspect you forget to set the Glance image
>property needed to tell Nova that it is windows.
>Specifically it is missing hyperv paravirt <timer>,
and many hyperv
>paravirt <feature> elements.
>This will make Windows guests unstable and liable to hit
BSOD
>Regards,
>Daniel
>--
>|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
>|: https://libvirt.org -o- https://fstop138.berrange.com :|
>|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
> >
> > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted
> > on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU
has crashed, or something else ?
I can still connect to the guest OS and see the windows opened before migrate on the
target, but cann't click any more,
and there's no effect when send "ctrl+alt+del" keys by virsh.
> >
> >
> > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac
qemu+tcp://192.168.65.13/system
> >
> >
> > But when migrated with qemu native mode, the VM runs well.
> >
> >
> > [root@test15 ~]# virsh migrate --live --p2p i-000000ac
qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in both cases
QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
11:04 a.m.
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange(a)redhat.com> wrote:
Migrate between 2 same version, and tried to upgrade the libvirt version.
>> > CPU: AMD Opteron(TM) Processor 6212
>> >
>> >
>> > As CPUFrequency may lead windows migrate halt, I have disabled Power
Management.
>> >
>> >
>> > Does anyone have some sugestions?
>> >
>> >
>> > Thanks,
>> > Linwen Deng
>> >
>> >
>> > vm.xml
>> >
>> >
>> > <domain type='kvm' id='8' >>
> <name>i-000000ac</name >>
> <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid >> > <metadata >>
> <nova:instance
xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0" >> > <nova:package
version="12.0.0-4"/ >> >
<nova:name>c7-vm15-test15-7</nova:name >>
> <nova:creationTime>2017-04-14 06:48:54</nova:creationTime >> > <nova:flavor
name="oeflavor-4-2048-20" >>
> <nova:memory>2048</nova:memory >>
> <nova:disk>20</nova:disk >>
> <nova:swap>0</nova:swap >>
> <nova:ephemeral>0</nova:ephemeral >>
> <nova:vcpus>4</nova:vcpus >>
> </nova:flavor >> >
<nova:owner >> > <nova:user
uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user >> > <nova:project
uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project >> > </nova:owner >> > </nova:instance >> > </metadata >>
> <memory unit='KiB'>2097152</memory >>
> <currentMemory unit='KiB'>2097152</currentMemory >> > <memoryBacking >>
> <hugepages >> > <page
size='2048' unit='KiB' nodeset='0'/ >>
> </hugepages >> >
</memoryBacking >> > <vcpu
placement='static'>2</vcpu >>
> <cputune >> >
<shares>4096</shares >> > <vcpupin
vcpu='0' cpuset='4-7'/ >>
> <vcpupin vcpu='1' cpuset='4-7'/ >>
> <emulatorpin cpuset='4-7'/ >>
> </cputune >> > <resource >> > <partition>/machine</partition >> > </resource >>
> <sysinfo type='smbios' >>
> <system >> > <entry
name='manufacturer'>Fedora Project</entry >>
> <entry name='product'>OpenStack Nova</entry >> > <entry
name='version'>12.0.0-4</entry >>
> <entry
name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry >> > <entry
name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry >> > <entry name='family'>Virtual
Machine</entry >> > </system >> > </sysinfo >>
> <os >> > <type
arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type >> > <boot dev='cdrom'/ >> > <boot dev='hd'/ >> > <boot dev='fd'/ >> > <smbios mode='sysinfo'/ >> > </os >>
> <features >> > <acpi/ >> > <apic/ >>
> </features >> > <cpu
mode='host-model' >> > <model
fallback='allow'/ >> > <topology
sockets='2' cores='1' threads='1'/ >>
> </cpu >> > <clock
offset='utc' >> > <timer
name='pit' tickpolicy='delay'/ >>
> <timer name='rtc' tickpolicy='catchup'/ >> > <timer name='hpet'
present='no'/ >> > </clock >If nothing else, this guest is
incorrectly configured for running
>the Windows OS. I suspect you forget to set the Glance image
>property needed to tell Nova that it is windows.
>Specifically it is missing hyperv paravirt <timer>,
and many hyperv
>paravirt <feature> elements.
>This will make Windows guests unstable and liable to hit
BSOD
I have tried to update the xml, the problem also exists, although there's one
success in 5 tries. But the guest can accept ctrl-alt-delete, further operations still not
available.
<features <acpi/ <apic/
<hyperv <relaxed
state='on'/ <vapic
state='on'/ <spinlocks
state='on' retries='8191'/
</hyperv </features <cpu mode='host-model' <model fallback='allow'/ <topology sockets='1' cores='2'
threads='1'/ </cpu <clock offset='utc'
<timer name='pit' tickpolicy='delay'/
<timer name='rtc' tickpolicy='catchup'/
<timer name='hpet' present='no'/
<timer name='hypervclock' present='yes'/
</clock
>Regards,
>Daniel
>--
>|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
>|: https://libvirt.org -o- https://fstop138.berrange.com :|
>|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:
> >
> > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted
> > on the target although the status is running.
What do you mean by halted ? The guest OS has shutdown, or QEMU
has crashed, or something else ?
> >
> >
> > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac
qemu+tcp://192.168.65.13/system
> >
> >
> > But when migrated with qemu native mode, the VM runs well.
> >
> >
> > [root@test15 ~]# virsh migrate --live --p2p i-000000ac
qemu+tcp://192.168.65.13/system
From QEMU's POV there's no difference between these modes - in both cases
QEMU is just getting a file descriptor from libvirt.
So any problems with the guest after migration are
> > System Info:
> > Release: Centos 7.2
> > Kernel: 3.10.0-327.28.3.el7.x86_64
> > Qemu: qemu-kvm-rhev-2.3.0
> > Libvirt: libvirt-1.2.17/libvirt-2.0.0
Does the 2 libvirt versions mean you are live-migrating between two different
versions of CentOS ? If so, this also likely means two different versions
of QEMU involved.
2766
days inactive
2767
days old
3 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Eric Blake -
邓林文