what memory state do you refer to exactly?

If you refer to port profile info, then there is no copying involved:  the source host disassociates the vnic port profile and  the destination host re-associates the port profile on the new vnic. /Chris > -----Original Message----- > From: libvir-list-bounces(a)redhat.com [mailto:libvir-list-bounces@redhat.com] On Behalf Of Jagath > Weerasinghe > Sent: Friday, April 13, 2012 9:16 AM > To: libvir-list(a)redhat.com > Subject: [libvirt] VN-Link vNIC memory state copying on VM Migration > > Hi All, > > I am new to libvirt. And want to know how the VM migration > occurs in VN-Link (IEEE802.1Qbh). As far as I know, > the memory state of vNICs in M81KR VIC has to be copied > and moved to the destination vNIC on VM migration. > Is that correct? If so, could you please tell me how this > has been implemented in libvirt? > > Thanks > Jagath > > -- > libvir-list mailing list > libvir-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list

> >  what memory state do you refer to exactly? > What I meant by the memory state is the traffic being > transferred through the vNIC when VM migration is triggered. Since you are worried about the traffic I assume we are talking about _live_ migration.

Libvirt qemu migration V3 takes place in 5 phases. (If you search the mailing list archive for "migration v3" you will find a number of 3Ds) This is a simplified description of the algorithm:      +--+      |VM|   +--------+           +---------+   |Src host|           |dst host |   +--------+           +---------+   Phase1: BEGIN   - Get XML of VM     and tx it to dst                         Phase2: PREPARE                         - create (paused) VM based on XML   Phase3: PERFORM   - Copy RAM to dst   - Pause VM   ..................................................                         Phase4: FINISH                         - Apply port profiles   ..................................................                         - Start VM   Phase5: CONFIRM   - Kill VM Libvirt on the dst host orchestrates everything using RPCs. Dst host can return error/success codes (the two libvirt also exchange data, .. see cookies) During phases 1/2/3 the VM is still running on the src host and hence able to rx/tx traffic. At the end of Phase 3, when qemu is done copying the RAM from source host to destination host (*), qemu puts the VM in pause state on the source host.

(*) See the migration parameter 'migrate-setmaxdowntime' In phase 4 libvirt applies the port profile to the interfaces that need it and starts the VM (which had been created and put in pause state in phase 2). There is therefore a small period during which both VMs (the one on the source host and the one on the destination host) are in pause state (see dotted lines above). Phase 4 will start the VM on the dst host, Phase 5 will kill the (paused) VM on the src host. The amount of time during which both VMs are in pause state depends on a number of factors, including: - maxdowntime:  the smaller you configure this value and                - the longer will take the migration to complete, but                - the smaller the pause duration will be - bandwidth available on the interface/s used to carry the migration  traffic, and amount of RAM assigned to the VM (since you need to  copy it) - time taken to complete the port profile associations on the dst host  (which depends on how loaded the switches are) - ... Hope this clarifies a bit. /Chris > If this memory state (or traffic) is not copied and moved to the > destination vNIC how the smooth communication, which is > guaranteed on VM migration, is achieved? > > Thanks > Jagath > > > > > If you refer to port profile info, then there is no copying involved: > >  the source host disassociates the vnic port profile and > >  the destination host re-associates the port profile on the new vnic. > > > > /Chris > > > >> -----Original Message----- > >> From: libvir-list-bounces(a)redhat.com > > [mailto:libvir-list-bounces@redhat.com] On Behalf Of Jagath > >> Weerasinghe > >> Sent: Friday, April 13, 2012 9:16 AM > >> To: libvir-list(a)redhat.com > >> Subject: [libvirt] VN-Link vNIC memory state copying on VM Migration > >> > >> Hi All, > >> > >> I am new to libvirt. And want to know how the VM migration > >> occurs in VN-Link (IEEE802.1Qbh). As far as I know, > >> the memory state of vNICs in M81KR VIC has to be copied > >> and moved to the destination vNIC on VM migration. > >> Is that correct? If so, could you please tell me how this > >> has been implemented in libvirt? > >> > >> Thanks > >> Jagath > >> > >> -- > >> libvir-list mailing list > >> libvir-list(a)redhat.com > >> https://www.redhat.com/mailman/listinfo/libvir-list > > -- > libvir-list mailing list > libvir-list(a)redhat.com > https://www.redhat.com/mailman/listinfo/libvir-list

> OK. In phase 3, the RAM used by VM in source host is copied to the > destination. Then what about the memory state of vNIC's in the M81KR VIC. > Isn't there any memory for traffic passing through vNICs and also for > vNIC statistics?  What I am talking about is copying of this memory to the > destination vNIC. No, that data (statistics, etc) are not copied to the dst host/vnic. However, you can find those statistics/counters on the switch (which are preserved across live migrations).

Thanks /Chris > > (*) See the migration parameter 'migrate-setmaxdowntime' > > > > In phase 4 libvirt applies the port profile to the interfaces > > that need it and starts the VM (which had been created and put in > > pause state in phase 2). > > There is therefore a small period during which both VMs (the one > > on the source host and the one on the destination host) are in pause > > state (see dotted lines above). > > Phase 4 will start the VM on the dst host, > > Phase 5 will kill the (paused) VM on the src host. > > > > The amount of time during which both VMs are in pause state depends on > > a number of factors, including: > > - maxdowntime:  the smaller you configure this value and > >                - the longer will take the migration to complete, but > >                - the smaller the pause duration will be > > - bandwidth available on the interface/s used to carry the migration > >  traffic, and amount of RAM assigned to the VM (since you need to > >  copy it) > > - time taken to complete the port profile associations on the dst host > >  (which depends on how loaded the switches are) > > - ... > > > > > > Hope this clarifies a bit. > > /Chris > > > >> If this memory state (or traffic) is not copied and moved to the > >> destination vNIC how the smooth communication, which is > >> guaranteed on VM migration, is achieved? > >> > >> Thanks > >> Jagath > >> > >> > > >> > If you refer to port profile info, then there is no copying involved: > >> >  the source host disassociates the vnic port profile and > >> >  the destination host re-associates the port profile on the new vnic. > >> > > >> > /Chris > >> > > >> >> -----Original Message----- > >> >> From: libvir-list-bounces(a)redhat.com > >> > [mailto:libvir-list-bounces@redhat.com] On Behalf Of Jagath > >> >> Weerasinghe > >> >> Sent: Friday, April 13, 2012 9:16 AM > >> >> To: libvir-list(a)redhat.com > >> >> Subject: [libvirt] VN-Link vNIC memory state copying on VM Migration > >> >> > >> >> Hi All, > >> >> > >> >> I am new to libvirt. And want to know how the VM migration > >> >> occurs in VN-Link (IEEE802.1Qbh). As far as I know, > >> >> the memory state of vNICs in M81KR VIC has to be copied > >> >> and moved to the destination vNIC on VM migration. > >> >> Is that correct? If so, could you please tell me how this > >> >> has been implemented in libvirt? > >> >> > >> >> Thanks > >> >> Jagath > >> >> > >> >> -- > >> >> libvir-list mailing list > >> >> libvir-list(a)redhat.com > >> >> https://www.redhat.com/mailman/listinfo/libvir-list > >> > >> -- > >> libvir-list mailing list > >> libvir-list(a)redhat.com > >> https://www.redhat.com/mailman/listinfo/libvir-list

> -----Original Message----- > From: Jagath Weerasinghe [mailto:jagfoss@gmail.com] > Sent: Tuesday, April 17, 2012 8:19 AM > To: Christian Benvenuti (benve) > Cc: libvir-list(a)redhat.com > Subject: Re: [libvirt] VN-Link vNIC memory state copying on VM Migration > > Chris > > >> OK. In phase 3, the RAM used by VM in source host is copied to the > >> destination. Then what about the memory state of vNIC's in the M81KR VIC. > >> Isn't there any memory for traffic passing through vNICs and also for > >> vNIC statistics?  What I am talking about is copying of this memory to the > >> destination vNIC. > > > > No, that data (statistics, etc) are not copied to the dst host/vnic. > > However, you can find those statistics/counters on the switch (which are > > preserved across live migrations). > > OK. I think this operation is the same for both IEEE802.1qbh (VN-Link) > and IEEE802.1qbg. Is that correct? I assume so. > One more thing. Could you please tell me where the port profile > association/dissociation messages of libvirt are converted to > VIC protocol messages. Is this done by CNA it self? Yes, the NIC firmware takes care of it. /Chris > >> > (*) See the migration parameter 'migrate-setmaxdowntime' > >> > > >> > In phase 4 libvirt applies the port profile to the interfaces > >> > that need it and starts the VM (which had been created and put in > >> > pause state in phase 2). > >> > There is therefore a small period during which both VMs (the one > >> > on the source host and the one on the destination host) are in pause > >> > state (see dotted lines above). > >> > Phase 4 will start the VM on the dst host, > >> > Phase 5 will kill the (paused) VM on the src host. > >> > > >> > The amount of time during which both VMs are in pause state depends on > >> > a number of factors, including: > >> > - maxdowntime:  the smaller you configure this value and > >> >                - the longer will take the migration to complete, but > >> >                - the smaller the pause duration will be > >> > - bandwidth available on the interface/s used to carry the migration > >> >  traffic, and amount of RAM assigned to the VM (since you need to > >> >  copy it) > >> > - time taken to complete the port profile associations on the dst host > >> >  (which depends on how loaded the switches are) > >> > - ... > >> > > >> > > >> > Hope this clarifies a bit. > >> > /Chris > >> > > >> >> If this memory state (or traffic) is not copied and moved to the > >> >> destination vNIC how the smooth communication, which is > >> >> guaranteed on VM migration, is achieved? > >> >> > >> >> Thanks > >> >> Jagath > >> >> > >> >> > > >> >> > If you refer to port profile info, then there is no copying involved: > >> >> >  the source host disassociates the vnic port profile and > >> >> >  the destination host re-associates the port profile on the new vnic. > >> >> > > >> >> > /Chris > >> >> > > >> >> >> -----Original Message----- > >> >> >> From: libvir-list-bounces(a)redhat.com > >> >> > [mailto:libvir-list-bounces@redhat.com] On Behalf Of Jagath > >> >> >> Weerasinghe > >> >> >> Sent: Friday, April 13, 2012 9:16 AM > >> >> >> To: libvir-list(a)redhat.com > >> >> >> Subject: [libvirt] VN-Link vNIC memory state copying on VM Migration > >> >> >> > >> >> >> Hi All, > >> >> >> > >> >> >> I am new to libvirt. And want to know how the VM migration > >> >> >> occurs in VN-Link (IEEE802.1Qbh). As far as I know, > >> >> >> the memory state of vNICs in M81KR VIC has to be copied > >> >> >> and moved to the destination vNIC on VM migration. > >> >> >> Is that correct? If so, could you please tell me how this > >> >> >> has been implemented in libvirt? > >> >> >> > >> >> >> Thanks > >> >> >> Jagath > >> >> >> > >> >> >> -- > >> >> >> libvir-list mailing list > >> >> >> libvir-list(a)redhat.com > >> >> >> https://www.redhat.com/mailman/listinfo/libvir-list > >> >> > >> >> -- > >> >> libvir-list mailing list > >> >> libvir-list(a)redhat.com > >> >> https://www.redhat.com/mailman/listinfo/libvir-list