[libvirt-users] KVM incremental backup using CBT
Hi Looking in to implementing (CBT like) delta backup for KVM. The following looks promising..(last paragraph) http://wiki.qemu.org/Features/Snapshots2 * In the last para, there is a mention of copy the blocks from the disk using dirty-bitmap as reference. How to accomplish this ? block-mirror with bitmap or is there a qemu-img command ? some details would be appreciated. // backup software now reads foo.img using t0_dirty.dbmp to perform incremental backup, when finished * The backup after few runs in the backup store would be base image + bunch of delta blocks ? Will this be same as base disk and bunch of deltas ? or there is some special way to merge these ? * I am assuming this scheme (snapshot, bitmap, block merge etc.) should work with base disk in raw (non-qcow) format as well ? Right ? i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux block device? * Are there libvirt-api/verbs for doing some of this or we will have to sue the qemu-monitor-command ? * What version of qemu/kvm will have the core capabilities and which min libvirt version would be sufficient ? Is there a better way to do incremental backup / CBT like backup than one mentioned here ? Thanks /Jd
On 10/10/2014 11:37 AM, Jd wrote:
Hi Looking in to implementing (CBT like) delta backup for KVM.
Not quite sure what you mean by CBT.
The following looks promising..(last paragraph) http://wiki.qemu.org/Features/Snapshots2
Libvirt hasn't yet been patched to take advantage of all the latest qemu features. Patches are welcome. But libvirt already knows how to create external snapshots and do blockpull (backing files pulled into the active image so the backing isn't needed any more) and commit (active files pushed into the backing files), and even though it is not as full-featured as where we'd like to get, it can already do quite a bit.
* In the last para, there is a mention of copy the blocks from the disk using dirty-bitmap as reference. How to accomplish this ? block-mirror with bitmap or is there a qemu-img command ? some details would be appreciated.
block-backup is a new QMP command that libvirt would need to expose; I don't know whether it is better to overload the existing virDomainBlockCopy() API for it, or to add a new API.
// backup software now reads foo.img using t0_dirty.dbmp to perform incremental backup, when finished
* The backup after few runs in the backup store would be base image + bunch of delta blocks ? Will this be same as base disk and bunch of deltas ? or there is some special way to merge these ?
* I am assuming this scheme (snapshot, bitmap, block merge etc.) should work with base disk in raw (non-qcow) format as well ? Right ? i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux block device?
Yes, you can use qcow2 to store deltas on top of any underlying storage device.
* Are there libvirt-api/verbs for doing some of this or we will have to sue the qemu-monitor-command ?
If you come up with a scenario where the only way to do something is via a raw qemu-monitor-command, please report it as a bug (or better yet provide a patch) so that a stable libvirt API can be written to do the same task.
* What version of qemu/kvm will have the core capabilities and which min libvirt version would be sufficient ?
Until patches materialize, it's hard to predict the future.
Is there a better way to do incremental backup / CBT like backup than one mentioned here ?
Thanks /Jd
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
-- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake wrote:
On 10/10/2014 11:37 AM, Jd wrote:
Hi Looking in to implementing (CBT like) delta backup for KVM.
Not quite sure what you mean by CBT.
The following looks promising..(last paragraph) http://wiki.qemu.org/Features/Snapshots2
Libvirt hasn't yet been patched to take advantage of all the latest qemu features. Patches are welcome. But libvirt already knows how to create external snapshots and do blockpull (backing files pulled into the active image so the backing isn't needed any more) and commit (active files pushed into the backing files), and even though it is not as full-featured as where we'd like to get, it can already do quite a bit.
To expand an example of what Eric wrote here about external snapshots, some commands to try: 1. Check the current acitve disk image in use: $ virsh domblklist vm1 2. Create an external live snapshot: $ virsh snapshot-create-as --domain vm1 sn1 \ --diskspec vda,file=/export/images/sn1.qcow2 \ --disk-only --atomic --no-metadata Note (thanks Eric Blake): Above, if you have QEMU guest agent installed in your virtual machine, try --quiesce option with 'snapshot-create-as' to ensure you have a consistent disk state. Now, you have a disk image chain: base <-sn1 (live QEMU) 3. Take a backup of the original disk in backround: $ rsync -avh --progress /export/images/base.img \ /export/images/copy.img 4. Commit the sn1 contents back into base: $ virsh blockcommit vm1 vda --active --verbose --pivot Now, the chain is: base (live QEMU) with contents from sn1 live committed into base. 5. Check the current acitve disk image in use again: $ virsh domblklist vm1 -- /kashyap
Thanks Eric.. inline. On 10/10/14, 6:32 PM, Eric Blake wrote:
On 10/10/2014 11:37 AM, Jd wrote:
Hi Looking in to implementing (CBT like) delta backup for KVM. Not quite sure what you mean by CBT. Sorry vmware term.. CBT -> changed block tracking. (dirty block)
The following looks promising..(last paragraph) http://wiki.qemu.org/Features/Snapshots2
Libvirt hasn't yet been patched to take advantage of all the latest qemu features. Patches are welcome. But libvirt already knows how to create external snapshots and do blockpull (backing files pulled into the active image so the backing isn't needed any more) and commit (active files pushed into the backing files), and even though it is not as full-featured as where we'd like to get, it can already do quite a bit.
* In the last para, there is a mention of copy the blocks from the disk using dirty-bitmap as reference. How to accomplish this ? block-mirror with bitmap or is there a qemu-img command ? some details would be appreciated.
block-backup is a new QMP command that libvirt would need to expose; I don't know whether it is better to overload the existing virDomainBlockCopy() API for it, or to add a new API.
So this seems to be already in qemu. How can I try this with KVM context. Do I need to build from source ? or some version of KVM already has this ?
// backup software now reads foo.img using t0_dirty.dbmp to perform
incremental backup, when finished
* The backup after few runs in the backup store would be base image + bunch of delta blocks ? Will this be same as base disk and bunch of deltas ? or there is some special way to merge these ?
* I am assuming this scheme (snapshot, bitmap, block merge etc.) should work with base disk in raw (non-qcow) format as well ? Right ? i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux block device? Yes, you can use qcow2 to store deltas on top of any underlying storage device. Glad to get confirmation.
* Are there libvirt-api/verbs for doing some of this or we will have to sue the qemu-monitor-command ?
If you come up with a scenario where the only way to do something is via a raw qemu-monitor-command, please report it as a bug (or better yet provide a patch) so that a stable libvirt API can be written to do the same task.
* What version of qemu/kvm will have the core capabilities and which min libvirt version would be sufficient ?
Until patches materialize, it's hard to predict the future.
Could you clarify a bit more .. I am not too familiar with how things move from qemu --> kvm and how the libvirt releases are aligned with qemu or kvm releases if they are. I want to have an environment where I can try this end to end. I have ubuntu 14.04.x release.
Is there a better way to do incremental backup / CBT like backup than one mentioned here ?
Thanks /Jd
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Thanks Kashyap, The command line examples makes thing quite clear :) rsync for image (large) files create a new file for every little change, that was the reason I started looking in to using dirty bitmaps. /Jd On 10/11/14, 1:13 AM, Kashyap Chamarthy wrote:
On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake wrote:
On 10/10/2014 11:37 AM, Jd wrote:
Hi Looking in to implementing (CBT like) delta backup for KVM. Not quite sure what you mean by CBT.
The following looks promising..(last paragraph) http://wiki.qemu.org/Features/Snapshots2
Libvirt hasn't yet been patched to take advantage of all the latest qemu features. Patches are welcome. But libvirt already knows how to create external snapshots and do blockpull (backing files pulled into the active image so the backing isn't needed any more) and commit (active files pushed into the backing files), and even though it is not as full-featured as where we'd like to get, it can already do quite a bit. To expand an example of what Eric wrote here about external snapshots, some commands to try:
1. Check the current acitve disk image in use:
$ virsh domblklist vm1
2. Create an external live snapshot:
$ virsh snapshot-create-as --domain vm1 sn1 \ --diskspec vda,file=/export/images/sn1.qcow2 \ --disk-only --atomic --no-metadata
Note (thanks Eric Blake): Above, if you have QEMU guest agent installed in your virtual machine, try --quiesce option with 'snapshot-create-as' to ensure you have a consistent disk state.
Now, you have a disk image chain:
base <-sn1 (live QEMU)
3. Take a backup of the original disk in backround:
$ rsync -avh --progress /export/images/base.img \ /export/images/copy.img
4. Commit the sn1 contents back into base:
$ virsh blockcommit vm1 vda --active --verbose --pivot
Now, the chain is:
base (live QEMU)
with contents from sn1 live committed into base.
5. Check the current acitve disk image in use again:
$ virsh domblklist vm1
On 10/12/2014 10:14 PM, Jd wrote:
So this seems to be already in qemu. How can I try this with KVM context. Do I need to build from source ? or some version of KVM already has this ?
KVM is the kernel technology that allows user-space virtualization. qemu is the user-space program that takes advantage of kvm; sometimes, the binary is named qemu-kvm to express the relationship. Many people treat the terms interchangeably these days. If your distro ships qemu 2.1, you already have all that features that libvirt currently can drive on top of kvm.
* What version of qemu/kvm will have the core capabilities and which min libvirt version would be sufficient ?
Until patches materialize, it's hard to predict the future. Could you clarify a bit more .. I am not too familiar with how things move from qemu --> kvm and how the libvirt releases are aligned with qemu or kvm releases if they are.
You shouldn't be worried about qemu/kvm interactions as much as qemu/libvirt interactions. Right now, there are features in qemu that have not yet been exposed in libvirt; the features will be added as soon as someone volunteers to write the patches, but given the nature of volunteer work, it is hard to predict when that will be. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 10/12/2014 10:19 PM, Jd wrote: [please don't top-post on technical lists]
Thanks Kashyap, The command line examples makes thing quite clear :) rsync for image (large) files create a new file for every little change, that was the reason I started looking in to using dirty bitmaps.
Upstream qemu still doesn't have persistent dirty bitmap tracking working yet; once patches land there, then we also need to get libvirt to interact with those new qemu features. But yes, dirty bitmap tracking, so that you can track which portions of a file have changed rather than copying the entire file, is something that is technically feasible once the patches materialize. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On 10/13/2014 03:56 AM, Thomas Stein wrote:
Hello.
blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?
virsh # blockcommit puppet-test vda --active --verbose --pivot error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at least libvirt 1.2.8 to play with active commit. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Am 13.10.14 16:25, schrieb Eric Blake:
On 10/13/2014 03:56 AM, Thomas Stein wrote:
Hello.
blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?
virsh # blockcommit puppet-test vda --active --verbose --pivot error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at least libvirt 1.2.8 to play with active commit.
Ah cool. Good to know. 1.2.6 running at the moment. thanks t.
On Monday 13 October 2014 16:35:15 Thomas Stein wrote:
Am 13.10.14 16:25, schrieb Eric Blake:
On 10/13/2014 03:56 AM, Thomas Stein wrote:
Hello.
blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?
virsh # blockcommit puppet-test vda --active --verbose --pivot error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at least libvirt 1.2.8 to play with active commit.
Ah cool. Good to know. 1.2.6 running at the moment.
Nice. It works. Just upgraded to 1.2.9. virsh # snapshot-create-as --domain one sn1 --diskspec vda,file=/var/lib/libvirt/images/export/sn1.qcow2 --disk-only --atomic --no- metadata Domain snapshot sn1 created virsh # domblklist one Target Source ------------------------------------------------ vda /var/lib/libvirt/images/export/sn1.qcow2 virsh # blockcommit one vda --active --verbose --pivot Block Commit: [100 %] Successfully pivoted virsh # domblklist one Target Source ------------------------------------------------ vda /opt/libvirt/ssdimages/one.qcow2 virsh # It's a charming way to make a backup. Is there anything usefull i can make with the snapshot i created during this procedure? cheers t.
thanks t.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On 10/13/14, 7:23 AM, Eric Blake wrote:
On 10/12/2014 10:19 PM, Jd wrote:
[please don't top-post on technical lists]
Thanks Kashyap, The command line examples makes thing quite clear :) rsync for image (large) files create a new file for every little change, that was the reason I started looking in to using dirty bitmaps. Upstream qemu still doesn't have persistent dirty bitmap tracking working yet; once patches land there, then we also need to get libvirt to interact with those new qemu features. But yes, dirty bitmap tracking, so that you can track which portions of a file have changed rather than copying the entire file, is something that is technically feasible once the patches materialize.
Thank you. I want to try this out in my environment for testing, what would you suggest ? I am ok with using qemu-monitor commands in the interim till things materializes in the libvirt level. * Where can I start ? qemu : http://wiki.qemu.org/ChangeLog/2.1 ? kvm : do I need specific kvm or the following will work ? (see at the end) libvirt :update to 1.2.8 at least as you recommended to thomas. * Where can I get relevant patches etc? block-backup, anything else? * With some modifications to following procedure, can I get it a test environment for testing incremental backup ? http://kashyapc.com/2014/03/10/notes-for-building-kvm-based-virtualization-c... Thanks /Jd ==== KVM Info === sudo modinfo kvm filename: /lib/modules/3.13.0-24-generic/kernel/arch/x86/kvm/kvm.ko license: GPL author: Qumranet srcversion: 57170F28A3DA376BDCD5012 depends: intree: Y vermagic: 3.13.0-24-generic SMP mod_unload modversions signer: Magrathea: Glacier signing key sig_key: 69:B0:D0:A7:9B:85:D9:06:21:70:6E:8D:06:60:4D:73:0B:35:9F:C0 sig_hashalgo: sha512 parm: ignore_msrs:bool parm: min_timer_period_us:uint parm: tsc_tolerance_ppm:uint parm: allow_unsafe_assigned_interrupts:Enable device assignment on platforms without interrupt remapping support. (bool)
On Mon, Oct 13, 2014 at 08:11:44PM +0200, Thomas Stein wrote:
On Monday 13 October 2014 16:35:15 Thomas Stein wrote:
Am 13.10.14 16:25, schrieb Eric Blake:
On 10/13/2014 03:56 AM, Thomas Stein wrote:
Hello.
blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?
virsh # blockcommit puppet-test vda --active --verbose --pivot error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at least libvirt 1.2.8 to play with active commit.
Ah cool. Good to know. 1.2.6 running at the moment.
Nice. It works. Just upgraded to 1.2.9.
Right, I forgot to note that you need 1.2.9.
virsh # snapshot-create-as --domain one sn1 --diskspec vda,file=/var/lib/libvirt/images/export/sn1.qcow2 --disk-only --atomic --no- metadata Domain snapshot sn1 created
virsh # domblklist one Target Source ------------------------------------------------ vda /var/lib/libvirt/images/export/sn1.qcow2
virsh # blockcommit one vda --active --verbose --pivot Block Commit: [100 %] Successfully pivoted virsh # domblklist one Target Source ------------------------------------------------ vda /opt/libvirt/ssdimages/one.qcow2
virsh #
It's a charming way to make a backup. Is there anything usefull i can make with the snapshot i created during this procedure?
Referring to the example I noted earlier, once you commited the contents of sn1 into base, sn1 (the snapshot) is invalidated. So, I'm afraid you can't do anything useful with it any further. However, you might want to explore the other mechanism libvirt provides 'virsh blockpull' which merges base images into top images - in that case, your intermediate images won't be invalidated. Some old notes here: https://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html Search for 'blockpull'. Hope that helps a bit. -- /kashyap
On Saturday 11 October 2014 10:13:14 Kashyap Chamarthy wrote:
2. Create an external live snapshot:
$ virsh snapshot-create-as --domain vm1 sn1 \ --diskspec vda,file=/export/images/sn1.qcow2 \ --disk-only --atomic --no-metadata
Note (thanks Eric Blake): Above, if you have QEMU guest agent installed in your virtual machine, try --quiesce option with 'snapshot-create-as' to ensure you have a consistent disk state.
About the --quiesce option. Do i need to do something inside the vm? The most commonly would probably be a sql server running inside the vm. Do i need to tell the sql server something about the --quiesce option i use? I read this article here which suggests such a procedure. Okay, it's vmware, but... Is that right? http://www.virtuallifestyle.nl/2013/03/back-up-mysql-on-linux-without-stoppi... thanks and cheers t.
On 10/29/2014 01:07 PM, Thomas Stein wrote:
About the --quiesce option. Do i need to do something inside the vm? The most commonly would probably be a sql server running inside the vm. Do i need to tell the sql server something about the --quiesce option i use? I read this article here which suggests such a procedure. Okay, it's vmware, but... Is that right?
For --quiesce to work, you have to have qemu-guest-agent installed and running in your guest, and also inform libvirt via the domain XML to establish a channel to the guest agent. Furthermore, if you want your sql database to be at a stable point, you can install freeze hooks in your guest that get invoked prior to freezing the file system and just after thawing it (that is, qemu-guest-agent already has documented hooks that let you do any additional prep work beyond just freezing the file system). A quick google search found this: http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Guid... Sadly, it didn't spell out the name of where you install hook scripts into the guest. But this commit is pretty telling: https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h... -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Wednesday 29 October 2014 13:29:25 Eric Blake wrote:
On 10/29/2014 01:07 PM, Thomas Stein wrote:
About the --quiesce option. Do i need to do something inside the vm? The most commonly would probably be a sql server running inside the vm. Do i need to tell the sql server something about the --quiesce option i use? I read this article here which suggests such a procedure. Okay, it's vmware, but... Is that right?
For --quiesce to work, you have to have qemu-guest-agent installed and running in your guest, and also inform libvirt via the domain XML to establish a channel to the guest agent. Furthermore, if you want your sql database to be at a stable point, you can install freeze hooks in your guest that get invoked prior to freezing the file system and just after thawing it (that is, qemu-guest-agent already has documented hooks that let you do any additional prep work beyond just freezing the file system).
A quick google search found this: http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Guid e/QEMU_Guest_Agent_Overview.html
Sadly, it didn't spell out the name of where you install hook scripts into the guest. But this commit is pretty telling: https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h ook
Thank you very much Eric. The hole thing now raises a few more questions. But maybe i will ask them later. :-) cheers t.
A quick google search found this: http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Gu id e/QEMU_Guest_Agent_Overview.html
Sadly, it didn't spell out the name of where you install hook scripts into the guest. But this commit is pretty telling: https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze -h ook
After a little bit of fiddling i've got this running. Here is what's neccessary in case someone wanna try this too. qemu-guest.-agent has to run with the option: "-F /etc/qemu/fsfreeze-hook". This file contains as pointed out by Eric: https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h... Then you need to have a directory called /etc/qemu/fsfreeze-hook.d. Inside this directory this mysql-flush script: https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h... I had to adjust this script a little for mariadb 10.0.14. But after that everything works as expected. Here the output: Sun 2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush freeze + MYSQL='mysql -uroot -ppassword' + FIFO=/tmp/mysql-flush.fifo + case "$1" in + mkfifo /tmp/mysql-flush.fifo + flush_and_wait + printf 'FLUSH TABLES WITH READ LOCK \G\n' + mysql -uroot -ppassword + read ++ echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' ++ mysql -uroot -ppassword ++ tail -1 ++ cut -f 2 + '[' 0 -gt 0 ']' ++ mktemp /tmp/mysql-flush.XXXXXX + INNODB_STATUS=/tmp/mysql-flush.eTdvA4 + '[' 0 -ne 0 ']' + trap 'rm -f /tmp/mysql-flush.eTdvA4' SIGINT + : + printf 'SHOW ENGINE INNODB STATUS \G' + mysql -uroot -ppassword ++ grep 'Log sequence number' /tmp/mysql-flush.eTdvA4 tr -s ' ' ++ cut '-d ' -f4 + LOG_CURRENT=12242169543 ++ grep 'Log flushed up to' /tmp/mysql-flush.eTdvA4 tr -s ' ' ++ cut '-d ' -f7 + LOG_FLUSHED=12242169543 + '[' 12242169543 = 12242169543 ']' + break + rm -f /tmp/mysql-flush.eTdvA4 Sun 2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished with status=0 Sun 2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush thaw + MYSQL='mysql -uroot -ppassword' + FIFO=/tmp/mysql-flush.fifo + case "$1" in + '[' '!' -p /tmp/mysql-flush.fifo ']' + echo + rm -f /tmp/mysql-flush.fifo + printf 'UNLOCK TABLES \G\n' Sun 2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished with status=0 Just to be sure i activated the general.log for mysql: 141102 13:46:14 6 Connect root@localhost as anonymous on 6 Query select @@version_comment limit 1 6 Query FLUSH TABLES WITH READ LOCK 7 Connect root@localhost as anonymous on 7 Query select @@version_comment limit 1 7 Query SHOW STATUS LIKE "Key_blocks_not_flushed" 7 Quit 8 Connect root@localhost as anonymous on 8 Query select @@version_comment limit 1 8 Query SHOW ENGINE INNODB STATUS 8 Quit 6 Query UNLOCK TABLES 6 Quit Nice! cheers t.
On 11/02/2014 02:13 PM, Thomas Stein wrote:
After a little bit of fiddling i've got this running. Here is what's neccessary in case someone wanna try this too.
That's awesome news! Thanks for sharing.
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h...
I had to adjust this script a little for mariadb 10.0.14. But after that everything works as expected.
Do you want to post your version of the script for others to learn from? -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Am 03.11.14 09:39, schrieb Eric Blake:
On 11/02/2014 02:13 PM, Thomas Stein wrote:
After a little bit of fiddling i've got this running. Here is what's neccessary in case someone wanna try this too.
That's awesome news! Thanks for sharing.
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h...
I had to adjust this script a little for mariadb 10.0.14. But after that everything works as expected.
Do you want to post your version of the script for others to learn from?
Sure. Here we go: ---- #!/bin/bash -x # Flush MySQL tables to the disk before the filesystem is freezed. # At the same time, this keeps a read lock while the filesystem is freezed # in order to avoid write accesses by the other clients. MYSQL="mysql -uroot -ppassword" FIFO=/tmp/mysql-flush.fifo flush_and_wait() { printf "FLUSH TABLES WITH READ LOCK \\G\n" read < $FIFO printf "UNLOCK TABLES \\G\n" } case "$1" in freeze) mkfifo $FIFO || exit 1 flush_and_wait | $MYSQL & # wait until every block is flushed while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\ $MYSQL | tail -1 | cut -f 2)" -gt 0 ]; do sleep 1 done # for InnoDB, wait until every log is flushed INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX) [ $? -ne 0 ] && exit 2 trap "rm -f $INNODB_STATUS" SIGINT while :; do printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL > $INNODB_STATUS LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS tr -s ' ' | cut -d' ' -f4) LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS tr -s ' ' | cut -d' ' -f7) [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break sleep 1 done rm -f $INNODB_STATUS ;; thaw) [ ! -p $FIFO ] && exit 1 echo > $FIFO rm -f $FIFO ;; esac ---- cheers t.
participants (4)
-
Eric Blake -
Jd -
Kashyap Chamarthy -
Thomas Stein