[libvirt-users] Help? Running into problems with migrateToURI2() and virDomainDefCheckABIStability()
Hi, We've run into a problem with libvirt 1.1.2 and are looking for some comments on whether this is a bug or design intent. We're trying to use migrateToURI() but we're using a few things (numatune, vcpu mask, etc.) that may need adjustment during the migration. We found that migrateToURI2() mostly works if we use XML created by copying the domain XML from the running instance and modifying the appropriate sections. The problem that we're seeing is that the serial console checking in libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing even though we haven't touched the serial console XML: 2014-02-10 19:41:07.196 1264 ERROR nova.virt.libvirt.driver [-] [instance: 42a841a8-034f-4565-937e-adb396a2ec48] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0 On occasion it passes if the devices/serial/source element happens to be the same after migration, but this seems to be a fluke. If we remove the devices/serial/source element in the XML it seems to make the serial console check work on the first migration, but upon doing subsequent migration it fails with a console count error: 2014-02-07 16:11:32.459 28759 ERROR nova.virt.libvirt.driver [-] [instance: 1d7c470d-4337-450a-8a40-d0fb375ca2c1] Live Migration failure: unsupported configuration: Target domain console count 0 does not match source 1 Are we doing something wrong? Is there an issue with the console check code? Is there a better way to do what we're trying to do? Any help would be appreciated... Thanks, Chris
On 02/10/2014 06:46 PM, Chris Friesen wrote:
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some comments on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things (numatune, vcpu mask, etc.) that may need adjustment during the migration. We found that migrateToURI2() mostly works if we use XML created by copying the domain XML from the running instance and modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing even though we haven't touched the serial console XML:
2014-02-10 19:41:07.196 1264 ERROR nova.virt.libvirt.driver [-] [instance: 42a841a8-034f-4565-937e-adb396a2ec48] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0
On occasion it passes if the devices/serial/source element happens to be the same after migration, but this seems to be a fluke.
If we remove the devices/serial/source element in the XML it seems to make the serial console check work on the first migration, but upon doing subsequent migration it fails with a console count error:
2014-02-07 16:11:32.459 28759 ERROR nova.virt.libvirt.driver [-] [instance: 1d7c470d-4337-450a-8a40-d0fb375ca2c1] Live Migration failure: unsupported configuration: Target domain console count 0 does not match source 1
Are we doing something wrong? Is there an issue with the console check code? Is there a better way to do what we're trying to do?
Any help would be appreciated...
I think we will have to see the XML your passing before and after edit to figure out what's going wrong. Also, showing all the arguments you are invoking the migration APIs with. This could have to do with some back compatible stuff we do with <console> handling: basically, <console type='pty'/> implies <serial type='pty'/> and vice versa. So if one is available in the XML, libvirt will force add another. If you clear out the <serial> bit, then pass that to libvirt, it will just fill it in with the matching console details. So you might need to remove <serial> and the matching <console> piece. - Cole
On 02/11/2014 04:45 PM, Cole Robinson wrote:
On 02/10/2014 06:46 PM, Chris Friesen wrote:
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some comments on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things (numatune, vcpu mask, etc.) that may need adjustment during the migration. We found that migrateToURI2() mostly works if we use XML created by copying the domain XML from the running instance and modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing even though we haven't touched the serial console XML:
I think we will have to see the XML your passing before and after edit to figure out what's going wrong. Also, showing all the arguments you are invoking the migration APIs with.
Okay, this is going to be a long message. :) We're calling it as: dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth) Here are the supplied arguments: duri = u'qemu+tcp://compute-0/system' muri = u'tcp://compute-0' flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER', 'VIR_MIGRATE_LIVE'] logical_sum = 19, (logical OR of flaglist) CONF.live_migration_bandwidth = 0 Grabbing original XML prior to migration: dom_xml = etree.fromstring(dom.XMLDesc(0)) xml_str = etree.tostring(dom_xml) xml_str (BEFORE) = <domain type="qemu" id="2"> <name>instance-00000006</name> <uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid> <dpdk> <process type="secondary"/> <file prefix="vs"/> <cpu list="0"/> <memory channels="4"/> </dpdk> <memory unit="KiB">262144</memory> <currentMemory unit="KiB">262144</currentMemory> <memoryBacking> <hugepages/> </memoryBacking> <vcpu placement="static" cpuset="1-2">2</vcpu> <cputune> <vcpupin vcpu="0" cpuset="1"/> <vcpupin vcpu="1" cpuset="2"/> <emulatorpin cpuset="1-2"/> </cputune> <numatune> <memory mode="preferred" nodeset="0"/> </numatune> <sysinfo type="smbios"> <system> <entry name="manufacturer">OpenStack Foundation</entry> <entry name="product">OpenStack Nova</entry> <entry name="version">2013.2.2</entry> <entry name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry> <entry name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry> </system> </sysinfo> <os> <type arch="x86_64" machine="pc-i440fx-1.4">hvm</type> <boot dev="hd"/> <smbios mode="sysinfo"/> </os> <features> <acpi/> <apic/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2" cache="none"/> <source file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/> <target dev="vda" bus="virtio"/> <alias name="virtio-disk0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/> </disk> <controller type="usb" index="0"> <alias name="usb0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/> </controller> <controller type="pci" index="0" model="pci-root"> <alias name="pci.0"/> </controller> <interface type="vswitch"> <mac address="fa:16:3e:fb:34:6b"/> <source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/> <target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/> <model type="e1000"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/> </interface> <interface type="vswitch"> <mac address="fa:16:3e:d5:b9:50"/> <source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/> <target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/> <model type="e1000"/> <alias name="net1"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/> </interface> <serial type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb"> <alias name="input0"/> </input> <input type="mouse" bus="ps2"/> <graphics type="vnc" port="5900" autoport="yes" listen="0.0.0.0" keymap="en-us"> <listen type="address" address="0.0.0.0"/> </graphics> <video> <model type="cirrus" vram="9216" heads="1"/> <alias name="video0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/> </video> <memballoon model="virtio"> <alias name="balloon0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/> </memballoon> </devices> <seclabel type="none"/> </domain> This is the modified XML, specifically updating vcpu, cputune, numatune. In this case they actually have the same values as above but there are whitespace differences: xml_str (MODIFIED) = <domain type="qemu" id="2"> <name>instance-00000006</name> <uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid> <dpdk> <process type="secondary"/> <file prefix="vs"/> <cpu list="0"/> <memory channels="4"/> </dpdk> <memory unit="KiB">262144</memory> <currentMemory unit="KiB">262144</currentMemory> <memoryBacking> <hugepages/> </memoryBacking> <vcpu placement="static" cpuset="1,2">2</vcpu> <cputune> <vcpupin vcpu="0" cpuset="1"/><vcpupin vcpu="1" cpuset="2"/><emulatorpin cpuset="1,2"/></cputune> <numatune> <memory mode="preferred" nodeset="0"/></numatune> <sysinfo type="smbios"> <system> <entry name="manufacturer">OpenStack Foundation</entry> <entry name="product">OpenStack Nova</entry> <entry name="version">2013.2.2</entry> <entry name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry> <entry name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry> </system> </sysinfo> <os> <type arch="x86_64" machine="pc-i440fx-1.4">hvm</type> <boot dev="hd"/> <smbios mode="sysinfo"/> </os> <features> <acpi/> <apic/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2" cache="none"/> <source file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/> <target dev="vda" bus="virtio"/> <alias name="virtio-disk0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/> </disk> <controller type="usb" index="0"> <alias name="usb0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/> </controller> <controller type="pci" index="0" model="pci-root"> <alias name="pci.0"/> </controller> <interface type="vswitch"> <mac address="fa:16:3e:fb:34:6b"/> <source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/> <target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/> <model type="e1000"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/> </interface> <interface type="vswitch"> <mac address="fa:16:3e:d5:b9:50"/> <source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/> <target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/> <model type="e1000"/> <alias name="net1"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/> </interface> <serial type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb"> <alias name="input0"/> </input> <input type="mouse" bus="ps2"/> <graphics type="vnc" port="5900" autoport="yes" listen="0.0.0.0" keymap="en-us"> <listen type="address" address="0.0.0.0"/> </graphics> <video> <model type="cirrus" vram="9216" heads="1"/> <alias name="video0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/> </video> <memballoon model="virtio"> <alias name="balloon0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/> </memballoon> </devices> <seclabel type="none"/> </domain> Calling migrateToURI2 then gives the following error : 2014-02-12 03:37:17.738 1251 ERROR nova.virt.libvirt.driver [-] [instance: 2e13869b-edc0-40dc-92e3-1f2ffde60cce] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0 If we remove the this section : <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> Then we get the following error : 2014-02-12 04:19:12.689 1278 ERROR nova.virt.libvirt.driver [-] [instance: 90335c32-a7ae-4c99-8e5a-cd177f6b3c22] Live Migration failure: unsupported configuration: Target domain serial port count 1 does not match source 2 If we remove just the following line: <source path="/dev/pts/1"/> so the modified portion looks like this, <serial type="file"> <source path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb"> Then we get this error : 2014-02-12 04:29:06.402 5175 ERROR nova.virt.libvirt.driver [-] [instance: 00b99f94-436e-406b-8459-83e77758f154] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0 If we modify it with empty <serial type='pty'> element like this: <serial type="file"> <source path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial> <console type="file"> <source path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> Then we get this error: 2014-02-12 04:36:45.686 5358 ERROR nova.virt.libvirt.driver [-] [instance: b1103349-8c9f-4315-9144-03fc22f09be1] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0 If we modify it with empty <serial type='pty'> element and also remove <console type='file'> section completely like this : <serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial> Then this successfully migrates! However, it then fails on subsequent migration back to the original host. Prior to migrating back the domain XML serial/console section looks like: <serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> After modifying empty <serial type='pty'> element and removing the <console type='file'> section it looks like this : <serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial> But this give the following error when migrating back: 2014-02-12 06:00:19.831 7125 ERROR nova.virt.libvirt.driver [-] [instance: 8cccb4ad-2abc-4513-b8fd-bfb811301677] Live Migration failure: unsupported configuration: Target domain console count 0 does not match source 1 Chris
On 02/12/2014 01:03 PM, Chris Friesen wrote:
On 02/11/2014 04:45 PM, Cole Robinson wrote:
On 02/10/2014 06:46 PM, Chris Friesen wrote:
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some comments on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things (numatune, vcpu mask, etc.) that may need adjustment during the migration. We found that migrateToURI2() mostly works if we use XML created by copying the domain XML from the running instance and modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing even though we haven't touched the serial console XML:
I think we will have to see the XML your passing before and after edit to figure out what's going wrong. Also, showing all the arguments you are invoking the migration APIs with.
Okay, this is going to be a long message. :)
We're calling it as:
dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth)
Here are the supplied arguments: duri = u'qemu+tcp://compute-0/system' muri = u'tcp://compute-0' flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER', 'VIR_MIGRATE_LIVE'] logical_sum = 19, (logical OR of flaglist) CONF.live_migration_bandwidth = 0
Grabbing original XML prior to migration:
dom_xml = etree.fromstring(dom.XMLDesc(0)) xml_str = etree.tostring(dom_xml)
xml_str (BEFORE) = <domain type="qemu" id="2"> <name>instance-00000006</name> <uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid> <dpdk> <process type="secondary"/> <file prefix="vs"/> <cpu list="0"/> <memory channels="4"/> </dpdk> <memory unit="KiB">262144</memory> <currentMemory unit="KiB">262144</currentMemory> <memoryBacking> <hugepages/> </memoryBacking> <vcpu placement="static" cpuset="1-2">2</vcpu> <cputune> <vcpupin vcpu="0" cpuset="1"/> <vcpupin vcpu="1" cpuset="2"/> <emulatorpin cpuset="1-2"/> </cputune> <numatune> <memory mode="preferred" nodeset="0"/> </numatune> <sysinfo type="smbios"> <system> <entry name="manufacturer">OpenStack Foundation</entry> <entry name="product">OpenStack Nova</entry> <entry name="version">2013.2.2</entry> <entry name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry> <entry name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry> </system> </sysinfo> <os> <type arch="x86_64" machine="pc-i440fx-1.4">hvm</type> <boot dev="hd"/> <smbios mode="sysinfo"/> </os> <features> <acpi/> <apic/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2" cache="none"/> <source file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/> <target dev="vda" bus="virtio"/> <alias name="virtio-disk0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/> </disk> <controller type="usb" index="0"> <alias name="usb0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/> </controller> <controller type="pci" index="0" model="pci-root"> <alias name="pci.0"/> </controller> <interface type="vswitch"> <mac address="fa:16:3e:fb:34:6b"/> <source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/> <target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/> <model type="e1000"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/> </interface> <interface type="vswitch"> <mac address="fa:16:3e:d5:b9:50"/> <source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/> <target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/> <model type="e1000"/> <alias name="net1"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/> </interface> <serial type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb"> <alias name="input0"/> </input> <input type="mouse" bus="ps2"/> <graphics type="vnc" port="5900" autoport="yes" listen="0.0.0.0" keymap="en-us"> <listen type="address" address="0.0.0.0"/> </graphics> <video> <model type="cirrus" vram="9216" heads="1"/> <alias name="video0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/> </video> <memballoon model="virtio"> <alias name="balloon0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/> </memballoon> </devices> <seclabel type="none"/> </domain>
This is the modified XML, specifically updating vcpu, cputune, numatune. In this case they actually have the same values as above but there are whitespace differences:
xml_str (MODIFIED) = <domain type="qemu" id="2"> <name>instance-00000006</name> <uuid>2e13869b-edc0-40dc-92e3-1f2ffde60cce</uuid> <dpdk> <process type="secondary"/> <file prefix="vs"/> <cpu list="0"/> <memory channels="4"/> </dpdk> <memory unit="KiB">262144</memory> <currentMemory unit="KiB">262144</currentMemory> <memoryBacking> <hugepages/> </memoryBacking> <vcpu placement="static" cpuset="1,2">2</vcpu> <cputune> <vcpupin vcpu="0" cpuset="1"/><vcpupin vcpu="1" cpuset="2"/><emulatorpin cpuset="1,2"/></cputune> <numatune> <memory mode="preferred" nodeset="0"/></numatune> <sysinfo type="smbios"> <system> <entry name="manufacturer">OpenStack Foundation</entry> <entry name="product">OpenStack Nova</entry> <entry name="version">2013.2.2</entry> <entry name="serial">771a1b10-d2f7-4ebb-b21e-4040560732f5</entry> <entry name="uuid">2e13869b-edc0-40dc-92e3-1f2ffde60cce</entry> </system> </sysinfo> <os> <type arch="x86_64" machine="pc-i440fx-1.4">hvm</type> <boot dev="hd"/> <smbios mode="sysinfo"/> </os> <features> <acpi/> <apic/> </features> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2" cache="none"/> <source file="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/disk"/> <target dev="vda" bus="virtio"/> <alias name="virtio-disk0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/> </disk> <controller type="usb" index="0"> <alias name="usb0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/> </controller> <controller type="pci" index="0" model="pci-root"> <alias name="pci.0"/> </controller> <interface type="vswitch"> <mac address="fa:16:3e:fb:34:6b"/> <source network="4db95d31-64f1-4f10-9138-5fe68c0d77ac"/> <target dev="7bc76b0e-b12a-4153-8ce8-d3d145c51e6f"/> <model type="e1000"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/> </interface> <interface type="vswitch"> <mac address="fa:16:3e:d5:b9:50"/> <source network="8512742c-af3e-4787-b2d8-b0b9ab160297"/> <target dev="33ab1061-7fa4-4250-99a4-dd7eb4029aa1"/> <model type="e1000"/> <alias name="net1"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x04" function="0x0"/> </interface> <serial type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/2e13869b-edc0-40dc-92e3-1f2ffde60cce/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb"> <alias name="input0"/> </input> <input type="mouse" bus="ps2"/> <graphics type="vnc" port="5900" autoport="yes" listen="0.0.0.0" keymap="en-us"> <listen type="address" address="0.0.0.0"/> </graphics> <video> <model type="cirrus" vram="9216" heads="1"/> <alias name="video0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/> </video> <memballoon model="virtio"> <alias name="balloon0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/> </memballoon> </devices> <seclabel type="none"/> </domain>
Calling migrateToURI2 then gives the following error : 2014-02-12 03:37:17.738 1251 ERROR nova.virt.libvirt.driver [-] [instance: 2e13869b-edc0-40dc-92e3-1f2ffde60cce] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0
If we remove the this section : <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial>
Then we get the following error : 2014-02-12 04:19:12.689 1278 ERROR nova.virt.libvirt.driver [-] [instance: 90335c32-a7ae-4c99-8e5a-cd177f6b3c22] Live Migration failure: unsupported configuration: Target domain serial port count 1 does not match source 2
If we remove just the following line: <source path="/dev/pts/1"/>
so the modified portion looks like this, <serial type="file"> <source path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/00b99f94-436e-406b-8459-83e77758f154/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console> <input type="tablet" bus="usb">
Then we get this error :
2014-02-12 04:29:06.402 5175 ERROR nova.virt.libvirt.driver [-] [instance: 00b99f94-436e-406b-8459-83e77758f154] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element like this:
<serial type="file"> <source path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial> <console type="file"> <source path="/etc/nova/instances/b1103349-8c9f-4315-9144-03fc22f09be1/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console>
Then we get this error:
2014-02-12 04:36:45.686 5358 ERROR nova.virt.libvirt.driver [-] [instance: b1103349-8c9f-4315-9144-03fc22f09be1] Live Migration failure: unsupported configuration: Target domain console count 1 does not match source 0
If we modify it with empty <serial type='pty'> element and also remove <console type='file'> section completely like this :
<serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial>
Then this successfully migrates!
However, it then fails on subsequent migration back to the original host. Prior to migrating back the domain XML serial/console section looks like:
<serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> <source path="/dev/pts/1"/> <target port="1"/> <alias name="serial1"/> </serial> <console type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target type="serial" port="0"/> <alias name="serial0"/> </console>
After modifying empty <serial type='pty'> element and removing the <console type='file'> section it looks like this :
<serial type="file"> <source path="/etc/nova/instances/8cccb4ad-2abc-4513-b8fd-bfb811301677/console.log"/> <target port="0"/> <alias name="serial0"/> </serial> <serial type="pty"> </serial>
But this give the following error when migrating back:
2014-02-12 06:00:19.831 7125 ERROR nova.virt.libvirt.driver [-] [instance: 8cccb4ad-2abc-4513-b8fd-bfb811301677] Live Migration failure: unsupported configuration: Target domain console count 0 does not match source 1
Thanks for the detailed info. Can you put that into an upstream libvirt bug report? http://libvirt.org/bugs.html Does this sound familiar to any other libvirt devs? - Cole
On Wed, Feb 12, 2014 at 12:03:11PM -0600, Chris Friesen wrote:
On 02/11/2014 04:45 PM, Cole Robinson wrote:
On 02/10/2014 06:46 PM, Chris Friesen wrote:
Hi,
We've run into a problem with libvirt 1.1.2 and are looking for some comments on whether this is a bug or design intent.
We're trying to use migrateToURI() but we're using a few things (numatune, vcpu mask, etc.) that may need adjustment during the migration. We found that migrateToURI2() mostly works if we use XML created by copying the domain XML from the running instance and modifying the appropriate sections.
The problem that we're seeing is that the serial console checking in libvirt/src/conf/domain_conf.c::virDomainDefCheckABIStability() is failing even though we haven't touched the serial console XML:
I think we will have to see the XML your passing before and after edit to figure out what's going wrong. Also, showing all the arguments you are invoking the migration APIs with.
Okay, this is going to be a long message. :)
We're calling it as:
dom.migrateToURI2(duri, muri, xml_str, logical_sum, None, CONF.live_migration_bandwidth)
Here are the supplied arguments: duri = u'qemu+tcp://compute-0/system' muri = u'tcp://compute-0' flaglist = ['VIR_MIGRATE_UNDEFINE_SOURCE', 'VIR_MIGRATE_PEER2PEER', 'VIR_MIGRATE_LIVE'] logical_sum = 19, (logical OR of flaglist) CONF.live_migration_bandwidth = 0
Grabbing original XML prior to migration:
dom_xml = etree.fromstring(dom.XMLDesc(0))
Can you retry this whole thing but pass VIR_DOMAIN_XML_MIGRATABLE instead of 0. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (3)
-
Chris Friesen -
Cole Robinson -
Daniel P. Berrange