[libvirt-users] Incremental Backups

Hello. Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups? Another way could be mount the image and rsync its contents. Does that makes sense? Also i read in qemu changelog: --- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. --- Is this supported by libvirt? thanks and best regards thomas

Hi Thomas, I'm using this script which leverages external snapshots and blockpull: https://github.com/dguerri/LibVirtKvm-scripts It's very simple and on ubuntu requires some tweaks on apparmour configuration. Hth Cheers, Davide -- Davide Guerri http://about.me/davide_guerri
On 20 Sep 2013, at 15:31, Thomas Stein <himbeere@meine-oma.de> wrote:
Hello.
Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups?
Another way could be mount the image and rsync its contents. Does that makes sense?
Also i read in qemu changelog:
--- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. ---
Is this supported by libvirt?
thanks and best regards thomas
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

On 2013-09-20 15:41, Davide Guerri wrote:
Hi Thomas,
Hello Davide.
I'm using this script which leverages external snapshots and blockpull: https://github.com/dguerri/LibVirtKvm-scripts
Yeah. I saw that and tried this already. But it needs to be adjusted to libvirtd-1.1.2 i think. Can you do this? I fiddled a litte bit around but wasn't realy successfull. hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-local/ -d serve.lordcritical [DEB] qemu-img version '1.6.0' is supported [DEB] KVM version '1.6.0,' is supported [DEB] Snapshot for domain 'serve.lordcritical' requested [DEB] Using timestamp '20130920-151742' [DEB] Snapshotting block devices for 'serve.lordcritical' using suffix 'bimg-20130920-151742' [VER] Snapshot for block devices of 'serve.lordcritical' successful [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742'. At the moment i try: https://bitbucket.org/guilhemfr/virt-back cheers t.
It's very simple and on ubuntu requires some tweaks on apparmour configuration.
Hth
Cheers, Davide
--
Davide Guerri http://about.me/davide_guerri
On 20 Sep 2013, at 15:31, Thomas Stein <himbeere@meine-oma.de> wrote:
Hello.
Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups?
Another way could be mount the image and rsync its contents. Does that makes sense?
Also i read in qemu changelog:
--- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. ---
Is this supported by libvirt?
thanks and best regards thomas
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

Hi
hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-local/ -d serve.lordcritical [DEB] qemu-img version '1.6.0' is supported [DEB] KVM version '1.6.0,' is supported [DEB] Snapshot for domain 'serve.lordcritical' requested [DEB] Using timestamp '20130920-151742' [DEB] Snapshotting block devices for 'serve.lordcritical' using suffix 'bimg-20130920-151742' [VER] Snapshot for block devices of 'serve.lordcritical' successful [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742'.
Will you please send me the output of the following? qemu-img info /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742 Thanks!
At the moment i try:
Seems very interesting and promising. I will try it! Cheers, D.

On 2013-09-20 16:05, Davide Guerri wrote:
Hi
hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-local/ -d serve.lordcritical [DEB] qemu-img version '1.6.0' is supported [DEB] KVM version '1.6.0,' is supported [DEB] Snapshot for domain 'serve.lordcritical' requested [DEB] Using timestamp '20130920-151742' [DEB] Snapshotting block devices for 'serve.lordcritical' using suffix 'bimg-20130920-151742' [VER] Snapshot for block devices of 'serve.lordcritical' successful [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742'.
Will you please send me the output of the following?
qemu-img info /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742
Looks okay actually. hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-blockcopy/ serve.lordcritical [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511'. hn LibVirtKvm-scripts # qemu-img info /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511 image: /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511 file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 196K cluster_size: 65536 backing file: /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170121 backing file format: qcow2 hn LibVirtKvm-scripts # cheers t.
Thanks!
At the moment i try:
Seems very interesting and promising. I will try it!
Cheers, D.

Hi Thomas, thanks for reporting this issue ;) The script should be fixed now: https://github.com/dguerri/LibVirtKvm-scripts/ Cheers, Davide. On 20 Sep 2013, at 17:07, Thomas Stein <himbeere@meine-oma.de> wrote:
On 2013-09-20 16:05, Davide Guerri wrote:
Hi
hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-local/ -d serve.lordcritical [DEB] qemu-img version '1.6.0' is supported [DEB] KVM version '1.6.0,' is supported [DEB] Snapshot for domain 'serve.lordcritical' requested [DEB] Using timestamp '20130920-151742' [DEB] Snapshotting block devices for 'serve.lordcritical' using suffix 'bimg-20130920-151742' [VER] Snapshot for block devices of 'serve.lordcritical' successful [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742'. Will you please send me the output of the following? qemu-img info /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-151742
Looks okay actually.
hn LibVirtKvm-scripts # ./fi-backup.sh -b /opt/virt-backup-blockcopy/ serve.lordcritical [ERR] Error getting backing file for '/var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511'. hn LibVirtKvm-scripts # qemu-img info /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511 image: /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170511 file format: qcow2 virtual size: 40G (42949672960 bytes) disk size: 196K cluster_size: 65536 backing file: /var/lib/libvirt/images/serve.lordcritical.bimg-20130920-170121 backing file format: qcow2 hn LibVirtKvm-scripts #
cheers t.
Thanks!
At the moment i try: https://bitbucket.org/guilhemfr/virt-back Seems very interesting and promising. I will try it! Cheers, D.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

Hi Davide,
Hi Thomas, I'm using this script which leverages external snapshots and blockpull: https://github.com/dguerri/LibVirtKvm-scripts [1]
It's very simple and on ubuntu requires some tweaks on apparmour configuration.
Would it be hard to adapt this script for LVM-backed VMs? []s, Fernando Lozano Links: ------ [1] https://github.com/dguerri/LibVirtKvm-scripts

Hi Fernando, unfortunately that requires a totally different approach. My script is very basic and will only work for qcow2 images. Sorry. Best, Davide. 2013/9/20 <fernando@lozano.eti.br>
**
Hi Davide,
Hi Thomas, I'm using this script which leverages external snapshots and blockpull: https://github.com/dguerri/LibVirtKvm-scripts
It's very simple and on ubuntu requires some tweaks on apparmour configuration.
Would it be hard to adapt this script for LVM-backed VMs?
[]s, Fernando Lozano
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

Hello. Another question regarding incremental backups. How can i roll back to a specific version? Example. I have: 1. a base image -rw-r--r-- 1 qemu qemu 39538851840 Sep 23 11:09 serve.lordcritical.qcow2 2. two external snapshots -rw------- 1 qemu qemu 1179648 Sep 23 11:17 daily0.serve.lordcritical.qcow2 -rw------- 1 qemu qemu 179175424 Sep 23 14:29 daily1.serve.lordcritical.qcow2 At the moment daily1 is used as the active image. hn ~ # virsh domblklist serve.lordcritical Target Source ------------------------------------------------ vda /var/lib/libvirt/images/daily1.serve.lordcritical.qcow2 hn ~ # How do i switch back to daily0? Do i just to have to change the disk path in the domains xml file and restart? Can i do this somehow live? thanks and best regards t.

Le vendredi 20 septembre 2013 à 15:31 +0200, Thomas Stein a écrit :
Hello.
Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups?
You can take a look at this script: http://gitweb.firewall-services.com/?p=virt-backup;a=blob_plain;f=virt-backu... I'm using it on several servers. You can either take a full backup (raw dd of your LV), or incremential backups. Incrementials are done this way: - pause the VM - take an LVM snapshot of each of its disk - resume the VM - use chunkfs to mount the LVM snapshots as if it was a lot of small files - now you can rsync the fuse mount point using rsync, and only modified blocks will be transfered. If you use something like BackupPC or rsnapshot, it'll also only store on disk the modified blocks Regards, Daniel
Another way could be mount the image and rsync its contents. Does that makes sense?
Also i read in qemu changelog:
--- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. ---
Is this supported by libvirt?
thanks and best regards thomas
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- Daniel Berteaud FIREWALL-SERVICES SARL. Société de Services en Logiciels Libres Technopôle Montesquieu 33650 MARTILLAC Tel : 05 56 64 15 32 Fax : 05 56 64 15 32 Web : http://www.firewall-services.com

Am 23.09.13 16:00, schrieb Daniel Berteaud:
Le vendredi 20 septembre 2013 à 15:31 +0200, Thomas Stein a écrit :
Hello.
Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups?
You can take a look at this script: http://gitweb.firewall-services.com/?p=virt-backup;a=blob_plain;f=virt-backu...
Hello Daniel. Thank you but unfortunately i don't have a LVM setup. cheers t.
I'm using it on several servers. You can either take a full backup (raw dd of your LV), or incremential backups. Incrementials are done this way:
- pause the VM - take an LVM snapshot of each of its disk - resume the VM - use chunkfs to mount the LVM snapshots as if it was a lot of small files - now you can rsync the fuse mount point using rsync, and only modified blocks will be transfered. If you use something like BackupPC or rsnapshot, it'll also only store on disk the modified blocks
Regards, Daniel
Another way could be mount the image and rsync its contents. Does that makes sense?
Also i read in qemu changelog:
--- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. ---
Is this supported by libvirt?
thanks and best regards thomas
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

Le lundi 23 septembre 2013 à 16:28 +0200, Thomas Stein a écrit :
Am 23.09.13 16:00, schrieb Daniel Berteaud:
Le vendredi 20 septembre 2013 à 15:31 +0200, Thomas Stein a écrit :
Hello.
Is someone performing incremental backups via libvirt for qemu/kvm machines? I'm still having a hard time to find a nice procedure. I mean is it possible to make a full backup of an image on monday a do the next days of the week incremental backups?
You can take a look at this script: http://gitweb.firewall-services.com/?p=virt-backup;a=blob_plain;f=virt-backu...
Hello Daniel.
Thank you but unfortunately i don't have a LVM setup.
I'm actually working on support for image based virtual disk (LVM still needed on the host where the image are stored) Regards, Daniel
cheers t.
I'm using it on several servers. You can either take a full backup (raw dd of your LV), or incremential backups. Incrementials are done this way:
- pause the VM - take an LVM snapshot of each of its disk - resume the VM - use chunkfs to mount the LVM snapshots as if it was a lot of small files - now you can rsync the fuse mount point using rsync, and only modified blocks will be transfered. If you use something like BackupPC or rsnapshot, it'll also only store on disk the modified blocks
Regards, Daniel
Another way could be mount the image and rsync its contents. Does that makes sense?
Also i read in qemu changelog:
--- Support for a new block device background job. Started by drive-backup, it will backup a disk's content to a new file. Unlike drive-mirror, the new file will include the source disk's content at the time the backup job was started. Atomic backup of multiple disks is supported using the "transaction" QMP command. ---
Is this supported by libvirt?
thanks and best regards thomas
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- Daniel Berteaud FIREWALL-SERVICES SARL. Société de Services en Logiciels Libres Technopôle Montesquieu 33650 MARTILLAC Tel : 05 56 64 15 32 Fax : 05 56 64 15 32 Web : http://www.firewall-services.com
participants (4)
-
Daniel Berteaud
-
Davide Guerri
-
fernando@lozano.eti.br
-
Thomas Stein