[libvirt-users] Help? Running into problems with migrateToURI2() and virDomainDefCheckABIStability()
7:46 a.m.
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some
comments on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things
(numatune, vcpu mask, etc.) that may need adjustment during the
migration. We found that migrateToURI2() mostly works if we use XML
created by copying the domain XML from the running instance and
modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in
libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is
failing even though we haven't touched the serial console XML:
2014-02-10 19:41:07.196 1264 ERROR nova.virt.libvirt.driver [-]
[instance: 42a841a8-034f-4565-937e-adb396a2ec48] Live Migration failure:
unsupported configuration: Target domain console count 1 does not match
source 0
On occasion it passes if the devices/serial/source element happens to be
the same after migration, but this seems to be a fluke.
If we remove the devices/serial/source element in the XML it seems to
make the serial console check work on the first migration, but upon
doing subsequent migration it fails with a console count error:
2014-02-07 16:11:32.459 28759 ERROR nova.virt.libvirt.driver [-]
[instance: 1d7c470d-4337-450a-8a40-d0fb375ca2c1] Live Migration failure:
unsupported configuration: Target domain console count 0 does not match
source 1
Are we doing something wrong? Is there an issue with the console check
code? Is there a better way to do what we're trying to do?
Any help would be appreciated...
Thanks,
Chris
6:45 a.m.
On 02/10/2014 06:46 PM, Chris Friesen wrote:
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some comments
on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things (numatune,
vcpu mask, etc.) that may need adjustment during the migration. We found that
migrateToURI2() mostly works if we use XML created by copying the domain XML
from the running instance and modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in
libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing
even though we haven't touched the serial console XML:
2014-02-10 19:41:07.196 1264 ERROR nova.virt.libvirt.driver [-] [instance:
42a841a8-034f-4565-937e-adb396a2ec48] Live Migration failure: unsupported
configuration: Target domain console count 1 does not match source 0
On occasion it passes if the devices/serial/source element happens to be the
same after migration, but this seems to be a fluke.
If we remove the devices/serial/source element in the XML it seems to make the
serial console check work on the first migration, but upon doing subsequent
migration it fails with a console count error:
2014-02-07 16:11:32.459 28759 ERROR nova.virt.libvirt.driver [-] [instance:
1d7c470d-4337-450a-8a40-d0fb375ca2c1] Live Migration failure: unsupported
configuration: Target domain console count 0 does not match source 1
Are we doing something wrong? Is there an issue with the console check code?
Is there a better way to do what we're trying to do?
Any help would be appreciated...
I think we will have to see the XML your passing before and after edit to
figure out what's going wrong. Also, showing all the arguments you are
invoking the migration APIs with.
This could have to do with some back compatible stuff we do with <console>
handling: basically, <console type='pty'/> implies <serial
type='pty'/> and
vice versa. So if one is available in the XML, libvirt will force add another.
If you clear out the <serial> bit, then pass that to libvirt, it will just
fill it in with the matching console details. So you might need to remove
<serial> and the matching <console> piece.
- Cole
2:03 a.m.
On 02/11/2014 04:45 PM, Cole Robinson wrote:
On 02/10/2014 06:46 PM, Chris Friesen wrote:
> Hi,
>
> We've run into a problem with libvirt 1.1.2 and are looking for some comments
> on whether this is a bug or design intent.
>
> We're trying to use migrateToURI() but we're using a few things (numatune,
> vcpu mask, etc.) that may need adjustment during the migration. We found that
> migrateToURI2() mostly works if we use XML created by copying the domain XML
> from the running instance and modifying the appropriate sections.
>
> The problem that we're seeing is that the serial console checking in
> libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing
> even though we haven't touched the serial console XML:
I think we will have to see the XML your passing before and after
edit to
figure out what's going wrong. Also, showing all the arguments you are
invoking the migration APIs with.
Okay, this is going to be a long message. :)
We're calling it as:
dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth)
Here are the supplied arguments:
duri = u'qemu+tcp://compute-0/system'
muri = u'tcp://compute-0'
flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER',
'VIR_MIGRATE_LIVE']
logical_sum = 19, (logical OR of flaglist)
CONF.live_migration_bandwidth = 0
Grabbing original XML prior to migration:
dom_xml = etree.fromstring(dom.XMLDesc(0))
xml_str = etree.tostring(dom_xml)
xml_str (BEFORE) =
<domain type="qemu" id="2">
<name>instance-00000006</name>
<uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid>
<dpdk>
<process type="secondary"/>
<file prefix="vs"/>
<cpu list="0"/>
<memory channels="4"/>
</dpdk>
<memory unit="KiB">262144</memory>
<currentMemory unit="KiB">262144</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement="static" cpuset="1-2">2</vcpu>
<cputune>
<vcpupin vcpu="0" cpuset="1"/>
<vcpupin vcpu="1" cpuset="2"/>
<emulatorpin cpuset="1-2"/>
</cputune>
<numatune>
<memory mode="preferred" nodeset="0"/>
</numatune>
<sysinfo type="smbios">
<system>
<entry name="manufacturer">OpenStack Foundation</entry>
<entry name="product">OpenStack Nova</entry>
<entry name="version">2013.2.2</entry>
<entry
name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry>
<entry
name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry>
</system>
</sysinfo>
<os>
<type arch="x86_64"
machine="pc-i440fx-1.4">hvm</type>
<boot dev="hd"/>
<smbios mode="sysinfo"/>
</os>
<features>
<acpi/>
<apic/>
</features>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2" cache="none"/>
<source
file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/>
<target dev="vda" bus="virtio"/>
<alias name="virtio-disk0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x05" function="0x0"/>
</disk>
<controller type="usb" index="0">
<alias name="usb0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x01" function="0x2"/>
</controller>
<controller type="pci" index="0"
model="pci-root">
<alias name="pci.0"/>
</controller>
<interface type="vswitch">
<mac address="fa:16:3e:fb:34:6b"/>
<source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/>
<target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/>
<model type="e1000"/>
<alias name="net0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x03" function="0x0"/>
</interface>
<interface type="vswitch">
<mac address="fa:16:3e:d5:b9:50"/>
<source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/>
<target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/>
<model type="e1000"/>
<alias name="net1"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x04" function="0x0"/>
</interface>
<serial type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
<alias name="input0"/>
</input>
<input type="mouse" bus="ps2"/>
<graphics type="vnc" port="5900" autoport="yes"
listen="0.0.0.0" keymap="en-us">
<listen type="address" address="0.0.0.0"/>
</graphics>
<video>
<model type="cirrus" vram="9216" heads="1"/>
<alias name="video0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x02" function="0x0"/>
</video>
<memballoon model="virtio">
<alias name="balloon0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x06" function="0x0"/>
</memballoon>
</devices>
<seclabel type="none"/>
</domain>
This is the modified XML, specifically updating vcpu, cputune, numatune.
In this case they actually have the same values as above but there are
whitespace differences:
xml_str (MODIFIED) =
<domain type="qemu" id="2">
<name>instance-00000006</name>
<uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid>
<dpdk>
<process type="secondary"/>
<file prefix="vs"/>
<cpu list="0"/>
<memory channels="4"/>
</dpdk>
<memory unit="KiB">262144</memory>
<currentMemory unit="KiB">262144</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement="static" cpuset="1,2">2</vcpu>
<cputune>
<vcpupin vcpu="0" cpuset="1"/><vcpupin vcpu="1"
cpuset="2"/><emulatorpin cpuset="1,2"/></cputune>
<numatune>
<memory mode="preferred" nodeset="0"/></numatune>
<sysinfo type="smbios">
<system>
<entry name="manufacturer">OpenStack Foundation</entry>
<entry name="product">OpenStack Nova</entry>
<entry name="version">2013.2.2</entry>
<entry
name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry>
<entry
name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry>
</system>
</sysinfo>
<os>
<type arch="x86_64"
machine="pc-i440fx-1.4">hvm</type>
<boot dev="hd"/>
<smbios mode="sysinfo"/>
</os>
<features>
<acpi/>
<apic/>
</features>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2" cache="none"/>
<source
file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/>
<target dev="vda" bus="virtio"/>
<alias name="virtio-disk0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x05" function="0x0"/>
</disk>
<controller type="usb" index="0">
<alias name="usb0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x01" function="0x2"/>
</controller>
<controller type="pci" index="0"
model="pci-root">
<alias name="pci.0"/>
</controller>
<interface type="vswitch">
<mac address="fa:16:3e:fb:34:6b"/>
<source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/>
<target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/>
<model type="e1000"/>
<alias name="net0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x03" function="0x0"/>
</interface>
<interface type="vswitch">
<mac address="fa:16:3e:d5:b9:50"/>
<source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/>
<target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/>
<model type="e1000"/>
<alias name="net1"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x04" function="0x0"/>
</interface>
<serial type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
<alias name="input0"/>
</input>
<input type="mouse" bus="ps2"/>
<graphics type="vnc" port="5900" autoport="yes"
listen="0.0.0.0" keymap="en-us">
<listen type="address" address="0.0.0.0"/>
</graphics>
<video>
<model type="cirrus" vram="9216" heads="1"/>
<alias name="video0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x02" function="0x0"/>
</video>
<memballoon model="virtio">
<alias name="balloon0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x06" function="0x0"/>
</memballoon>
</devices>
<seclabel type="none"/>
</domain>
Calling migrateToURI2 then gives the following error :
2014-02-12 03:37:17.738 1251 ERROR nova.virt.libvirt.driver [-] [instance:
2e13869b-edc0-40dc-92e3-1f2ffde60cce] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we remove the this section :
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
Then we get the following error :
2014-02-12 04:19:12.689 1278 ERROR nova.virt.libvirt.driver [-] [instance:
90335c32-a7ae-4c99-8e5a-cd177f6b3c22] Live Migration failure: unsupported configuration:
Target domain serial port count 1 does not match source 2
If we remove just the following line:
<source path="/dev/pts/1"/>
so the modified portion looks like this,
<serial type="file">
<source
path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
Then we get this error :
2014-02-12 04:29:06.402 5175 ERROR nova.virt.libvirt.driver [-] [instance:
00b99f94-436e-406b-8459-83e77758f154] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element like this:
<serial type="file">
<source
path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
<console type="file">
<source
path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
Then we get this error:
2014-02-12 04:36:45.686 5358 ERROR nova.virt.libvirt.driver [-] [instance:
b1103349-8c9f-4315-9144-03fc22f09be1] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element and also remove
<console type='file'> section completely like this :
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
Then this successfully migrates!
However, it then fails on subsequent migration back to the original host. Prior to
migrating
back the domain XML serial/console section looks like:
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
After modifying empty <serial type='pty'> element and removing the
<console type='file'> section
it looks like this :
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
But this give the following error when migrating back:
2014-02-12 06:00:19.831 7125 ERROR nova.virt.libvirt.driver [-] [instance:
8cccb4ad-2abc-4513-b8fd-bfb811301677] Live Migration failure: unsupported configuration:
Target domain console count 0 does not match source 1
Chris
2:35 a.m.
On 02/12/2014 01:03 PM, Chris Friesen wrote:
On 02/11/2014 04:45 PM, Cole Robinson wrote:
> On 02/10/2014 06:46 PM, Chris Friesen wrote:
>> Hi,
>>
>> We've run into a problem with libvirt 1.1.2 and are looking for some
comments
>> on whether this is a bug or design intent.
>>
>> We're trying to use migrateToURI() but we're using a few things
(numatune,
>> vcpu mask, etc.) that may need adjustment during the migration. We found that
>> migrateToURI2() mostly works if we use XML created by copying the domain XML
>> from the running instance and modifying the appropriate sections.
>>
>> The problem that we're seeing is that the serial console checking in
>> libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing
>> even though we haven't touched the serial console XML:
> I think we will have to see the XML your passing before and after edit to
> figure out what's going wrong. Also, showing all the arguments you are
> invoking the migration APIs with.
Okay, this is going to be a long message. :)
We're calling it as:
dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth)
Here are the supplied arguments:
duri = u'qemu+tcp://compute-0/system'
muri = u'tcp://compute-0'
flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER',
'VIR_MIGRATE_LIVE']
logical_sum = 19, (logical OR of flaglist)
CONF.live_migration_bandwidth = 0
Grabbing original XML prior to migration:
dom_xml = etree.fromstring(dom.XMLDesc(0))
xml_str = etree.tostring(dom_xml)
xml_str (BEFORE) =
<domain type="qemu" id="2">
<name>instance-00000006</name>
<uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid>
<dpdk>
<process type="secondary"/>
<file prefix="vs"/>
<cpu list="0"/>
<memory channels="4"/>
</dpdk>
<memory unit="KiB">262144</memory>
<currentMemory unit="KiB">262144</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement="static" cpuset="1-2">2</vcpu>
<cputune>
<vcpupin vcpu="0" cpuset="1"/>
<vcpupin vcpu="1" cpuset="2"/>
<emulatorpin cpuset="1-2"/>
</cputune>
<numatune>
<memory mode="preferred" nodeset="0"/>
</numatune>
<sysinfo type="smbios">
<system>
<entry name="manufacturer">OpenStack Foundation</entry>
<entry name="product">OpenStack Nova</entry>
<entry name="version">2013.2.2</entry>
<entry
name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry>
<entry
name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry>
</system>
</sysinfo>
<os>
<type arch="x86_64"
machine="pc-i440fx-1.4">hvm</type>
<boot dev="hd"/>
<smbios mode="sysinfo"/>
</os>
<features>
<acpi/>
<apic/>
</features>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2"
cache="none"/>
<source
file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/>
<target dev="vda" bus="virtio"/>
<alias name="virtio-disk0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x05" function="0x0"/>
</disk>
<controller type="usb" index="0">
<alias name="usb0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x01" function="0x2"/>
</controller>
<controller type="pci" index="0"
model="pci-root">
<alias name="pci.0"/>
</controller>
<interface type="vswitch">
<mac address="fa:16:3e:fb:34:6b"/>
<source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/>
<target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/>
<model type="e1000"/>
<alias name="net0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x03" function="0x0"/>
</interface>
<interface type="vswitch">
<mac address="fa:16:3e:d5:b9:50"/>
<source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/>
<target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/>
<model type="e1000"/>
<alias name="net1"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x04" function="0x0"/>
</interface>
<serial type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
<alias name="input0"/>
</input>
<input type="mouse" bus="ps2"/>
<graphics type="vnc" port="5900" autoport="yes"
listen="0.0.0.0" keymap="en-us">
<listen type="address" address="0.0.0.0"/>
</graphics>
<video>
<model type="cirrus" vram="9216" heads="1"/>
<alias name="video0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x02" function="0x0"/>
</video>
<memballoon model="virtio">
<alias name="balloon0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x06" function="0x0"/>
</memballoon>
</devices>
<seclabel type="none"/>
</domain>
This is the modified XML, specifically updating vcpu, cputune, numatune.
In this case they actually have the same values as above but there are
whitespace differences:
xml_str (MODIFIED) =
<domain type="qemu" id="2">
<name>instance-00000006</name>
<uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid>
<dpdk>
<process type="secondary"/>
<file prefix="vs"/>
<cpu list="0"/>
<memory channels="4"/>
</dpdk>
<memory unit="KiB">262144</memory>
<currentMemory unit="KiB">262144</currentMemory>
<memoryBacking>
<hugepages/>
</memoryBacking>
<vcpu placement="static" cpuset="1,2">2</vcpu>
<cputune>
<vcpupin vcpu="0" cpuset="1"/><vcpupin
vcpu="1" cpuset="2"/><emulatorpin
cpuset="1,2"/></cputune>
<numatune>
<memory mode="preferred" nodeset="0"/></numatune>
<sysinfo type="smbios">
<system>
<entry name="manufacturer">OpenStack Foundation</entry>
<entry name="product">OpenStack Nova</entry>
<entry name="version">2013.2.2</entry>
<entry
name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry>
<entry
name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry>
</system>
</sysinfo>
<os>
<type arch="x86_64"
machine="pc-i440fx-1.4">hvm</type>
<boot dev="hd"/>
<smbios mode="sysinfo"/>
</os>
<features>
<acpi/>
<apic/>
</features>
<clock offset="utc"/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type="file" device="disk">
<driver name="qemu" type="qcow2"
cache="none"/>
<source
file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/>
<target dev="vda" bus="virtio"/>
<alias name="virtio-disk0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x05" function="0x0"/>
</disk>
<controller type="usb" index="0">
<alias name="usb0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x01" function="0x2"/>
</controller>
<controller type="pci" index="0"
model="pci-root">
<alias name="pci.0"/>
</controller>
<interface type="vswitch">
<mac address="fa:16:3e:fb:34:6b"/>
<source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/>
<target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/>
<model type="e1000"/>
<alias name="net0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x03" function="0x0"/>
</interface>
<interface type="vswitch">
<mac address="fa:16:3e:d5:b9:50"/>
<source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/>
<target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/>
<model type="e1000"/>
<alias name="net1"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x04" function="0x0"/>
</interface>
<serial type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
<alias name="input0"/>
</input>
<input type="mouse" bus="ps2"/>
<graphics type="vnc" port="5900" autoport="yes"
listen="0.0.0.0" keymap="en-us">
<listen type="address" address="0.0.0.0"/>
</graphics>
<video>
<model type="cirrus" vram="9216" heads="1"/>
<alias name="video0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x02" function="0x0"/>
</video>
<memballoon model="virtio">
<alias name="balloon0"/>
<address type="pci" domain="0x0000" bus="0x00"
slot="0x06" function="0x0"/>
</memballoon>
</devices>
<seclabel type="none"/>
</domain>
Calling migrateToURI2 then gives the following error :
2014-02-12 03:37:17.738 1251 ERROR nova.virt.libvirt.driver [-] [instance:
2e13869b-edc0-40dc-92e3-1f2ffde60cce] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we remove the this section :
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
Then we get the following error :
2014-02-12 04:19:12.689 1278 ERROR nova.virt.libvirt.driver [-] [instance:
90335c32-a7ae-4c99-8e5a-cd177f6b3c22] Live Migration failure: unsupported configuration:
Target domain serial port count 1 does not match source 2
If we remove just the following line:
<source path="/dev/pts/1"/>
so the modified portion looks like this,
<serial type="file">
<source
path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
<input type="tablet" bus="usb">
Then we get this error :
2014-02-12 04:29:06.402 5175 ERROR nova.virt.libvirt.driver [-] [instance:
00b99f94-436e-406b-8459-83e77758f154] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element like this:
<serial type="file">
<source
path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
<console type="file">
<source
path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
Then we get this error:
2014-02-12 04:36:45.686 5358 ERROR nova.virt.libvirt.driver [-] [instance:
b1103349-8c9f-4315-9144-03fc22f09be1] Live Migration failure: unsupported configuration:
Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element and also remove
<console type='file'> section completely like this :
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
Then this successfully migrates!
However, it then fails on subsequent migration back to the original host. Prior to
migrating
back the domain XML serial/console section looks like:
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
<source path="/dev/pts/1"/>
<target port="1"/>
<alias name="serial1"/>
</serial>
<console type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target type="serial" port="0"/>
<alias name="serial0"/>
</console>
After modifying empty <serial type='pty'> element and removing the
<console type='file'> section
it looks like this :
<serial type="file">
<source
path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/>
<target port="0"/>
<alias name="serial0"/>
</serial>
<serial type="pty">
</serial>
But this give the following error when migrating back:
2014-02-12 06:00:19.831 7125 ERROR nova.virt.libvirt.driver [-] [instance:
8cccb4ad-2abc-4513-b8fd-bfb811301677] Live Migration failure: unsupported configuration:
Target domain console count 0 does not match source 1
Thanks for the detailed info. Can you put that into an upstream libvirt bug
report? http://libvirt.org/bugs.html
Does this sound familiar to any other libvirt devs?
- Cole
6:17 p.m.
On Wed, Feb 12, 2014 at 12:03:11PM -0600, Chris Friesen wrote:
On 02/11/2014 04:45 PM, Cole Robinson wrote:
> On 02/10/2014 06:46 PM, Chris Friesen wrote:
>> Hi,
>>
>> We've run into a problem with libvirt 1.1.2 and are looking for some
comments
>> on whether this is a bug or design intent.
>>
>> We're trying to use migrateToURI() but we're using a few things
(numatune,
>> vcpu mask, etc.) that may need adjustment during the migration. We found that
>> migrateToURI2() mostly works if we use XML created by copying the domain XML
>> from the running instance and modifying the appropriate sections.
>>
>> The problem that we're seeing is that the serial console checking in
>> libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing
>> even though we haven't touched the serial console XML:
> I think we will have to see the XML your passing before and after edit to
> figure out what's going wrong. Also, showing all the arguments you are
> invoking the migration APIs with.
Okay, this is going to be a long message. :)
We're calling it as:
dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth)
Here are the supplied arguments:
duri = u'qemu+tcp://compute-0/system'
muri = u'tcp://compute-0'
flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER',
'VIR_MIGRATE_LIVE']
logical_sum = 19, (logical OR of flaglist)
CONF.live_migration_bandwidth = 0
Grabbing original XML prior to migration:
dom_xml = etree.fromstring(dom.XMLDesc(0))
Can you retry this whole thing but pass VIR_DOMAIN_XML_MIGRATABLE instead
of 0.
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
3916
days inactive
3919
days old
4 comments
3 participants
participants (3)
-
Chris Friesen -
Cole Robinson -
Daniel P. Berrange