[libvirt-users] migrate local storage to ceph | exchanging the storage system
TLDR: Why is virsh migrate --persistent --live domain qemu+ssh://root@host/system --xml domain.ceph.xml not persistent and what could i do about it? Hi, after years of beeing pleased with local storage and migrating the complete storage from one host to another, it was time for ceph. After setting up a cluster and testing it, its time now for moving a lot of VMs on that type of storage, without a disturbance if possible. So I 'virsh dumpxml domain > domain.ceph.xml', edit the file, replace // ---- old ---- // <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='domain.img'/> <backingStore/> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> // ---- with new ---- // <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='libvirt'> <secret type='ceph' uuid='53e4ba03-5ad0-4f1e-b69e-897afba272ff'/> </auth> <source protocol='rbd' name='libvirt/domain'> <host name='cephmon0' port='6789'/> <host name='cephmon1' port='6789'/> <host name='cephmon2' port='6789'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> Now i'm calling: root@host_a:~# virsh migrate --verbose --p2p --copy-storage-all --persistent --change-protection --abort-on-error --undefinesource --live domain qemu+ssh://root@host_b/system --xml domain.ceph.xml This takes its time to copy everything into ceph, but when its over, i never have to migrate Terabytes of Storage again. But its not over. The new domain definition on host_b is not persistent and after a shutdown, local storage is used again. What do i have todo, to make this change persistent, or is there a better way for doing this? (i am using libvirt 1.2.9) Thanks in advance Björn Lässig -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
-----Oorspronkelijk bericht----- Van: libvirt-users-bounces@redhat.com [mailto:libvirt-users- bounces@redhat.com] Namens Björn Lässig Verzonden: vrijdag 27 mei 2016 10:10 Aan: libvirt-users@redhat.com Onderwerp: [libvirt-users] migrate local storage to ceph | exchanging the storage system
TLDR: Why is virsh migrate --persistent --live domain qemu+ssh://root@host/system --xml domain.ceph.xml not persistent and what could i do about it?
Hi, after years of beeing pleased with local storage and migrating the complete storage from one host to another, it was time for ceph.
After setting up a cluster and testing it, its time now for moving a lot of VMs on that type of storage, without a disturbance if possible.
So I 'virsh dumpxml domain > domain.ceph.xml', edit the file, replace // ---- old ---- // <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='domain.img'/> <backingStore/> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk>
// ---- with new ---- // <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='libvirt'> <secret type='ceph' uuid='53e4ba03-5ad0-4f1e-b69e-897afba272ff'/> </auth> <source protocol='rbd' name='libvirt/domain'> <host name='cephmon0' port='6789'/> <host name='cephmon1' port='6789'/> <host name='cephmon2' port='6789'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk>
Now i'm calling:
root@host_a:~# virsh migrate --verbose --p2p --copy-storage-all --persistent -- change-protection --abort-on-error --undefinesource --live domain qemu+ssh://root@host_b/system --xml domain.ceph.xml
Weird: The domain should be persistent You can make an transient domain persistent by doing a 'virsh define domain.ceph.xml'
This takes its time to copy everything into ceph, but when its over, i never have to migrate Terabytes of Storage again.
But its not over. The new domain definition on host_b is not persistent and after a shutdown, local storage is used again.
What do i have todo, to make this change persistent, or is there a better way for doing this?
(i am using libvirt 1.2.9)
Thanks in advance Björn Lässig -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On 05/30/2016 09:07 AM, Dominique Ramaekers wrote:
root@host_a:~# virsh migrate --verbose --p2p --copy-storage-all --persistent -- change-protection --abort-on-error --undefinesource --live domain qemu+ssh://root@host_b/system --xml domain.ceph.xml
Weird: The domain should be persistent
Well, the domain is persistent. But the changes i did to domain.ceph.xml to migrate the storage to ceph are not.
You can make an transient domain persistent by doing a 'virsh define domain.ceph.xml'
This works for one vm, but this needs a shutdown/reboot sequence. It is really painful to coordinate this for every VM, so i'd like to skip this. Björn Lässig
-----Oorspronkelijk bericht----- Van: Björn Lässig [mailto:b.laessig@pengutronix.de] Verzonden: maandag 30 mei 2016 15:35 Aan: Dominique Ramaekers; libvirt-users@redhat.com Onderwerp: Re: [libvirt-users] migrate local storage to ceph | exchanging the storage system
On 05/30/2016 09:07 AM, Dominique Ramaekers wrote:
root@host_a:~# virsh migrate --verbose --p2p --copy-storage-all --persistent -- change-protection --abort-on-error --undefinesource --live domain qemu+ssh://root@host_b/system --xml domain.ceph.xml
Weird: The domain should be persistent
Well, the domain is persistent. But the changes i did to domain.ceph.xml to migrate the storage to ceph are not. More weird: the configuration in domain.ceph.xml should be applied and the domain should be persistent... I use the same technic without trouble...
Maybe you can write down the precise procedure of how you prepare your migration? The source of your problem can be in there... How do you create domain.ceph.xml, Whats the content of dumpxml and whats the content fo dumpxml --inactive,... Things like that...
You can make an transient domain persistent by doing a 'virsh define domain.ceph.xml'
This works for one vm, but this needs a shutdown/reboot sequence. It is really painful to coordinate this for every VM, so i'd like to skip this.
No, you don't need a shutdown/reboot sequence to make a domain persistent. You can make a running domain persistent with a define. You can also make a running domain transendient with undefined.
Björn Lässig
participants (2)
-
Björn Lässig -
Dominique Ramaekers