Re: [Libvirt-cim] Query regarding Xen_ComputerSystem.RequestStateChange() and Live migration
Subba Rao, Sandeep M (STSD) wrote:
Hi Kaitlin,
I'm not sure what you mean by "goes back to host1".
So, if we switch off the VM after the live migration, the VM loses the assocation with target. Virsh shows as follows after shutdown:
[root@RHEL54-host1 ~]# virsh list --all
Id Name State ---------------------------------- 0 Domain-0 running - liveVm shut off
[root@RHEL54-host2 ~]# virsh list --all Id Name State ---------------------------------- 0 Domain-0 running - localVm shut off
Ah, I see what you mean. libvirt does not define the guest on the target during a migration. So before you shutdown the guest, you'll need to define it so that the config persists. That said, libvirt-cim will define the guest so that when the guest is migrated over, it's configuration persists after the guest is shutdown.
Ah, so libvirt-cim is cleaning the guest up on the source host properly. That is good to hear.
libvirt-cim is cleaning the guest up on the source host properly, like you said looks like libvirt has some issues and we will follow up accordingly.
Regarding the stop VM operation -- RequestStateChange() function.
We tried the stop operation on the VM. The stop operation through libvirt-cim reported success i.e the RequestStateChange() function returned a 0, but the VM is still running on the host.
Attaching the libvirt-cim log file.
The operation was tried on the VM 'Win'. Details are as below:
[root@RHEL54 ~]# cat /etc/issue Red Hat Enterprise Linux Server release 5.4 Beta (Tikanga) Kernel \r on an \m
[root@RHEL54 ~]# virsh list --all Id Name State ---------------------------------- 0 Domain-0 running 1 Win idle - HVM shut off - SLES shut off - VM shut off
virsh # dominfo 1 Id: 1 Name: Win UUID: 090df819-8ac6-f4d3-784b-4309a938080a OS Type: hvm State: no state CPU(s): 1 CPU time: 55.0s Max memory: 278528 kB Used memory: 270208 kB Autostart: disable
The stop operation doesn't seem to work through xm and virsh. However, our concern is that the RequestStateChange() function returns 0 immediately. This is the case even when the operation succeeds.
So if xm is failing to shutdown the guest, then libvirt-cim won't be able to shutdown the guest either. However, I agree with you that there is a bug in libvirt-cim here. We should check to make sure the guest is properly shutdown before returning from RequestStateChange(). We don't currently do that. Currently, we just return whether libvirt reports a success or a failure.
Please let us know if you need any information.
Regards, Sandeep
-----Original Message----- From: Kaitlin Rupert [mailto:kaitlin@linux.vnet.ibm.com] Sent: Friday, October 09, 2009 1:41 AM To: Velayudham, Madan Ganesh (STSD / ESS SW) Cc: Deepti B Kalakeri; List for discussion and development of libvirt CIM; Subba Rao, Sandeep M (STSD) Subject: Re: [Libvirt-cim] Query regarding Xen_ComputerSystem.RequestStateChange() and Live migration
Velayudham, Madan Ganesh (STSD / ESS SW) wrote:
Thanks Kaitlin,
Here it is:
[root@RHEL54-host1 ~]# virsh list --all Id Name State ---------------------------------- 0 Domain-0 running - liveVm shut off
This is why libvirt-cim reports the guest on host1. The definition of the guest still exists here.
[root@RHEL54-host2 ~]# virsh list --all Id Name State ---------------------------------- 0 Domain-0 running 14 liveVm idle - localVm shut off
If I shutdown the liveVm on host2, the VM goes back to host1!! Does it indicate any problems?
I'm not sure what you mean by "goes back to host1".
Please note that above mentioned was the case when we invoked the live migration through virsh command.
My mistake - I didn't realize you were launching the migration through libvirt. So it sounds like libvirt doesn't remove the definition from the source system in RHEL 5.4. I believe this is the desired behavior, but you'll have to check with the libvirt team.
I enabled the logging as mentioned, and invoked live migration through libvirt-CIM - in which case - the migration completed successfully, and the VM was reported only on the target; Attached is the log.
Ah, so libvirt-cim is cleaning the guest up on the source host properly. That is good to hear.
Now it appears to be - if we start using virsh command, then the inconsistency is seen.
Please let us know your comments.
We'll capture logs for the stop issue (in another setup) and send soon.
Thanks and Regards, Madan (_MaGa_) Madan Ganesh Velayudham Office: +91 80 2516 5333
-----Original Message----- From: Kaitlin Rupert [mailto:kaitlin@linux.vnet.ibm.com] Sent: Thursday, October 08, 2009 9:48 PM To: Velayudham, Madan Ganesh (STSD / ESS SW) Cc: Deepti B Kalakeri; List for discussion and development of libvirt CIM; Subba Rao, Sandeep M (STSD) Subject: Re: [Libvirt-cim] Query regarding Xen_ComputerSystem.RequestStateChange() and Live migration
Velayudham, Madan Ganesh (STSD / ESS SW) wrote:
Thank you Kaitlin and Deepti,
Hi Madan,
Can you run "virsh list --all" on both hosts. "virsh list" just shows active guests, while "virsh list --all" shows active and inactive guests.
Mu guess is that liveVM is still defined on RHEL54-host1. Can you enable the provider debug and then rerun the migration? I'm wondering if the providers print an error as to why the guest isn't being undefined from the source system.
To enable the provider debug, do the following:
export CU_DEBUG="/var/log/libvirt-cim.log" /etc/init.d/tog-pegasus restart
The VM "liveVm" on shared storage is being reported on both the hosts (source and destination), here is what we receive through "Xen_HostedDepenency" association:
[root@RHEL54-host1 ~]# wbemcli ain -ac Xen_HostedDependency 'http://root:password@localhost:5988/root/virt:Xen_HostSystem.CreationClassName="Xen_HostSystem",Name="RHEL54-host1"' RHEL54-host1/root/virt:Xen_ComputerSystem.CreationClassName="Xen_ComputerSystem",Name="liveVm" RHEL54-host1/root/virt:Xen_ComputerSystem.CreationClassName="Xen_ComputerSystem",Name="Domain-0"
[root@RHEL54-host1 ~]# virsh list Id Name State ---------------------------------- 0 Domain-0 running
[root@RHEL54-host2 ~]# wbemcli ain -ac Xen_HostedDependency 'http://root:password@localhost:5988/root/virt:Xen_HostSystem.CreationClassName="Xen_HostSystem",Name="RHEL54-host2"' RHEL54-host2/root/virt:Xen_ComputerSystem.CreationClassName="Xen_ComputerSystem",Name="localVm" RHEL54-host2/root/virt:Xen_ComputerSystem.CreationClassName="Xen_ComputerSystem",Name="Domain-0" RHEL54-host2/root/virt:Xen_ComputerSystem.CreationClassName="Xen_ComputerSystem",Name="liveVm"
[root@RHEL54-host2 ~]# virsh list Id Name State ---------------------------------- 0 Domain-0 running 7 liveVm idle
As you notice, virsh command is listing properly, but the association is not updated properly and results in both hosts.
On the RequestStateChange, we did not capture the message of stop failure. However, here is what we observe - we invoke the RequestStateChange method, it returns 0 - indicating success; But the VM will not start or stop due to insufficient resources. We assuming that RequestStatechange method to be synchronous call if it does not provide CIM_Job to track the status of start or stop. Generally starting a Windows VM take more than a minute. Where is the insufficient resources printing out from? Can you paste the errors you see? Can you also include the accompanying provider debug?
Either way, libvirt-cim should not return a success in this case. So it sounds like we have a bug here. My guess is that the following is happening:
1) libvirt-cim asks libvirt to stop the guest 2) libvirt returns that the operation is successful, but the operation really fails 3) libvirt-cim doesn't verify that the guest was really stopped / started, so it doesn't catch the failure
Thanks again for your help,
Best Regards, Madan
-----Original Message----- From: Deepti B Kalakeri [mailto:deeptik@linux.vnet.ibm.com] Sent: Thursday, October 08, 2009 1:13 PM To: Velayudham, Madan Ganesh (STSD / ESS SW) Cc: List for discussion and development of libvirt CIM; Kaitlin Rupert; Deepti B Kalakeri Subject: Re: [Libvirt-cim] Query regarding Xen_ComputerSystem.RequestStateChange() and Live migration
Subba Rao, Sandeep M (STSD) wrote:
Hi Kaitlin,
Hope you are doing fine.
We need a clarification regarding the CIM_ConcereteJob support part of Xen_ComputerSystem.RequestStateChange() method to start or stop a Virtual Machine. After we invoke the method, we were expecting a return value of 4096 (Operation Scheduled) with the reference to Job to track the status.
AFAIK, the method would not return 4096, it would return success or failure only.
Instead we are noticing that the method returns 0 - indicating success, however the Virtual machine does not stop because of memory or other errors.
Can you elaborate on what operation you are doing here ? What errors are you seeing ?
Can you clarify whether this is a known issue in libvirt-CIM? Also recommend how to track the invoked job?
Once the RqquestedStateChange() is successful you should be able to see the new state value in the Xen_ComputerSystem.EnabledState.
The OS is an RHEL 5.4 and the libvirt and xen versions are as follows:
[root@RHEL54 ~]# cat /etc/issue Red Hat Enterprise Linux Server release 5.4 Beta (Tikanga) Kernel \r on an \m
[root@RHEL54 ~]# uname -a Linux RHEL54 2.6.18-160.el5xen #1 SMP Mon Jul 27 18:01:58 EDT 2009 i686 i686 i386 GNU/Linux [root@RHEL54 ~]# rpm -qa | grep libvirt libvirt-0.6.3-17.el5 libvirt-cim-0.5.5-2.el5 libvirt-python-0.6.3-17.el5 [root@RHEL54 ~]# rpm -qa | grep xen kernel-xen-devel-2.6.18-160.el5 kernel-xen-2.6.18-160.el5 xen-3.0.3-92.el5 xen-libs-3.0.3-92.el5 [root@RHEL54 ~]# rpm -qa | grep sblim sblim-cmpi-base-1.5.5-34.el5 [root@RHEL54 ~]#
We also need clarification for live migration. We notice that Xen_ComputerSystem instance for the Virtual machine which is live migrated is being reported on the source and target host. Can you clarify us how to determine the actual host where the VM is associated? We were hoping that HostedDependency association will be helpful, but did not work.
Once the live migration of the guest using libvirt-cim is successful you should not see any information of the VM on the host. I dont think there is any means from which you can track information regarding the source from which the guest was migrated using libivrt-cim providers. Xen_HostedDependency will give information of the VM's the particular Server hosts.
Appreciate any help in this regard.
Sandeep
_______________________________________________ Libvirt-cim mailing list Libvirt-cim@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-cim
-- Kaitlin Rupert IBM Linux Technology Center kaitlin@linux.vnet.ibm.com
participants (1)
-
Kaitlin Rupert