12:37 p.m.
Hi
Looking in to implementing (CBT like) delta backup for KVM.
The following looks promising..(last paragraph)
http://wiki.qemu.org/Features/Snapshots2
* In the last para, there is a mention of copy the blocks from the
disk using dirty-bitmap as reference. How to accomplish this ?
block-mirror with bitmap or is there a qemu-img command ? some details
would be appreciated.
// backup software now reads foo.img using t0_dirty.dbmp to perform incremental backup,
when finished
* The backup after few runs in the backup store would be base
image + bunch of delta blocks ? Will this be same as base disk and bunch
of deltas ? or there is some special way to merge these ?
* I am assuming this scheme (snapshot, bitmap, block merge etc.)
should work with base disk in raw (non-qcow) format as well ? Right ?
i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux
block device?
* Are there libvirt-api/verbs for doing some of this or we will
have to sue the qemu-monitor-command ?
* What version of qemu/kvm will have the core capabilities and
which min libvirt version would be sufficient ?
Is there a better way to do incremental backup / CBT like backup
than one mentioned here ?
Thanks
/Jd
8:32 p.m.
On 10/10/2014 11:37 AM, Jd wrote:
Hi
Looking in to implementing (CBT like) delta backup for KVM.
Not quite sure what you mean by CBT.
The following looks promising..(last paragraph)
http://wiki.qemu.org/Features/Snapshots2
Libvirt hasn't yet been patched to take advantage of all the latest qemu
features. Patches are welcome. But libvirt already knows how to create
external snapshots and do blockpull (backing files pulled into the
active image so the backing isn't needed any more) and commit (active
files pushed into the backing files), and even though it is not as
full-featured as where we'd like to get, it can already do quite a bit.
* In the last para, there is a mention of copy the blocks from
the
disk using dirty-bitmap as reference. How to accomplish this ?
block-mirror with bitmap or is there a qemu-img command ? some details
would be appreciated.
block-backup is a new QMP command that libvirt would need to expose; I
don't know whether it is better to overload the existing
virDomainBlockCopy() API for it, or to add a new API.
// backup software now reads foo.img using t0_dirty.dbmp to perform
incremental backup, when finished
* The backup after few runs in the backup store would be base image
+ bunch of delta blocks ? Will this be same as base disk and bunch of
deltas ? or there is some special way to merge these ?
* I am assuming this scheme (snapshot, bitmap, block merge etc.)
should work with base disk in raw (non-qcow) format as well ? Right ?
i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux
block device?
Yes, you can use qcow2 to store deltas on top of any underlying storage
device.
* Are there libvirt-api/verbs for doing some of this or we will
have to sue the qemu-monitor-command ?
If you come up with a scenario where the only way to do something is via
a raw qemu-monitor-command, please report it as a bug (or better yet
provide a patch) so that a stable libvirt API can be written to do the
same task.
* What version of qemu/kvm will have the core capabilities and
which min libvirt version would be sufficient ?
Until patches materialize, it's hard to predict the future.
Is there a better way to do incremental backup / CBT like backup
than one mentioned here ?
Thanks
/Jd
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
3:13 a.m.
On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake wrote:
On 10/10/2014 11:37 AM, Jd wrote:
> Hi
> Looking in to implementing (CBT like) delta backup for KVM.
Not quite sure what you mean by CBT.
>
> The following looks promising..(last paragraph)
> http://wiki.qemu.org/Features/Snapshots2
>
Libvirt hasn't yet been patched to take advantage of all the latest qemu
features. Patches are welcome. But libvirt already knows how to create
external snapshots and do blockpull (backing files pulled into the
active image so the backing isn't needed any more) and commit (active
files pushed into the backing files), and even though it is not as
full-featured as where we'd like to get, it can already do quite a bit.
To expand an example of what Eric wrote here about external snapshots,
some commands to try:
1. Check the current acitve disk image in use:
$ virsh domblklist vm1
2. Create an external live snapshot:
$ virsh snapshot-create-as --domain vm1 sn1 \
--diskspec vda,file=/export/images/sn1.qcow2 \
--disk-only --atomic --no-metadata
Note (thanks Eric Blake): Above, if you have QEMU guest agent
installed in your virtual machine, try --quiesce option with
'snapshot-create-as' to ensure you have a consistent disk state.
Now, you have a disk image chain:
base <-sn1 (live QEMU)
3. Take a backup of the original disk in backround:
$ rsync -avh --progress /export/images/base.img \
/export/images/copy.img
4. Commit the sn1 contents back into base:
$ virsh blockcommit vm1 vda --active --verbose --pivot
Now, the chain is:
base (live QEMU)
with contents from sn1 live committed into base.
5. Check the current acitve disk image in use again:
$ virsh domblklist vm1
--
/kashyap
11:19 p.m.
Thanks Kashyap,
The command line examples makes thing quite clear :)
rsync for image (large) files create a new file for every little
change, that was the reason I started looking in to using dirty bitmaps.
/Jd
On 10/11/14, 1:13 AM, Kashyap Chamarthy wrote:
On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake wrote:
> On 10/10/2014 11:37 AM, Jd wrote:
>> Hi
>> Looking in to implementing (CBT like) delta backup for KVM.
> Not quite sure what you mean by CBT.
>
>> The following looks promising..(last paragraph)
>> http://wiki.qemu.org/Features/Snapshots2
>>
> Libvirt hasn't yet been patched to take advantage of all the latest qemu
> features. Patches are welcome. But libvirt already knows how to create
> external snapshots and do blockpull (backing files pulled into the
> active image so the backing isn't needed any more) and commit (active
> files pushed into the backing files), and even though it is not as
> full-featured as where we'd like to get, it can already do quite a bit.
To expand an example of what Eric wrote here about external snapshots,
some commands to try:
1. Check the current acitve disk image in use:
$ virsh domblklist vm1
2. Create an external live snapshot:
$ virsh snapshot-create-as --domain vm1 sn1 \
--diskspec vda,file=/export/images/sn1.qcow2 \
--disk-only --atomic --no-metadata
Note (thanks Eric Blake): Above, if you have QEMU guest agent
installed in your virtual machine, try --quiesce option with
'snapshot-create-as' to ensure you have a consistent disk state.
Now, you have a disk image chain:
base <-sn1 (live QEMU)
3. Take a backup of the original disk in backround:
$ rsync -avh --progress /export/images/base.img \
/export/images/copy.img
4. Commit the sn1 contents back into base:
$ virsh blockcommit vm1 vda --active --verbose --pivot
Now, the chain is:
base (live QEMU)
with contents from sn1 live committed into base.
5. Check the current acitve disk image in use again:
$ virsh domblklist vm1
9:25 a.m.
On 10/13/2014 03:56 AM, Thomas Stein wrote:
Hello.
> blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?
virsh # blockcommit puppet-test vda --active --verbose --pivot
error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at
least libvirt 1.2.8 to play with active commit.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
9:35 a.m.
Am 13.10.14 16:25, schrieb Eric Blake:
On 10/13/2014 03:56 AM, Thomas Stein wrote:
> Hello.
>
>> blockcommit vm1 vda --active --verbose --pivot
>
> That's where libvirt stops working at the moment, right?
>
> virsh # blockcommit puppet-test vda --active --verbose --pivot
> error: unsupported flags (0x4) in function qemuDomainBlockCommit
Which version of libvirtd is running on the server side? You'll need at
least libvirt 1.2.8 to play with active commit.
Ah cool. Good to know. 1.2.6 running at the moment.
thanks
t.
1:11 p.m.
On Monday 13 October 2014 16:35:15 Thomas Stein wrote:
Am 13.10.14 16:25, schrieb Eric Blake:
> On 10/13/2014 03:56 AM, Thomas Stein wrote:
>> Hello.
>>
>>> blockcommit vm1 vda --active --verbose --pivot
>>
>> That's where libvirt stops working at the moment, right?
>>
>> virsh # blockcommit puppet-test vda --active --verbose --pivot
>> error: unsupported flags (0x4) in function qemuDomainBlockCommit
>
> Which version of libvirtd is running on the server side? You'll need at
> least libvirt 1.2.8 to play with active commit.
Ah cool. Good to know. 1.2.6 running at the moment.
Nice. It works. Just upgraded to 1.2.9.
virsh # snapshot-create-as --domain one sn1 --diskspec
vda,file=/var/lib/libvirt/images/export/sn1.qcow2 --disk-only --atomic --no-
metadata
Domain snapshot sn1 created
virsh # domblklist one
Target Source
------------------------------------------------
vda /var/lib/libvirt/images/export/sn1.qcow2
virsh # blockcommit one vda --active --verbose --pivot
Block Commit: [100 %]
Successfully pivoted
virsh # domblklist one
Target Source
------------------------------------------------
vda /opt/libvirt/ssdimages/one.qcow2
virsh #
It's a charming way to make a backup. Is there anything usefull i can make
with the snapshot i created during this procedure?
cheers
t.
thanks
t.
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users
8:55 a.m.
On Mon, Oct 13, 2014 at 08:11:44PM +0200, Thomas Stein wrote:
On Monday 13 October 2014 16:35:15 Thomas Stein wrote:
> Am 13.10.14 16:25, schrieb Eric Blake:
> > On 10/13/2014 03:56 AM, Thomas Stein wrote:
> >> Hello.
> >>
> >>> blockcommit vm1 vda --active --verbose --pivot
> >>
> >> That's where libvirt stops working at the moment, right?
> >>
> >> virsh # blockcommit puppet-test vda --active --verbose --pivot
> >> error: unsupported flags (0x4) in function qemuDomainBlockCommit
> >
> > Which version of libvirtd is running on the server side? You'll need at
> > least libvirt 1.2.8 to play with active commit.
>
> Ah cool. Good to know. 1.2.6 running at the moment.
Nice. It works. Just upgraded to 1.2.9.
Right, I forgot to note that you need 1.2.9.
virsh # snapshot-create-as --domain one sn1 --diskspec
vda,file=/var/lib/libvirt/images/export/sn1.qcow2 --disk-only --atomic --no-
metadata
Domain snapshot sn1 created
virsh # domblklist one
Target Source
------------------------------------------------
vda /var/lib/libvirt/images/export/sn1.qcow2
virsh # blockcommit one vda --active --verbose --pivot
Block Commit: [100 %]
Successfully pivoted
virsh # domblklist one
Target Source
------------------------------------------------
vda /opt/libvirt/ssdimages/one.qcow2
virsh #
It's a charming way to make a backup. Is there anything usefull i can
make with the snapshot i created during this procedure?
Referring to the example I noted earlier, once you commited the contents
of sn1 into base, sn1 (the snapshot) is invalidated. So, I'm afraid you
can't do anything useful with it any further.
However, you might want to explore the other mechanism libvirt provides
'virsh blockpull' which merges base images into top images - in that
case, your intermediate images won't be invalidated.
Some old notes here:
https://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-handout.html
Search for 'blockpull'.
Hope that helps a bit.
--
/kashyap
9:23 a.m.
On 10/12/2014 10:19 PM, Jd wrote:
[please don't top-post on technical lists]
Thanks Kashyap,
The command line examples makes thing quite clear :)
rsync for image (large) files create a new file for every little
change, that was the reason I started looking in to using dirty bitmaps.
Upstream qemu still doesn't have persistent dirty bitmap tracking
working yet; once patches land there, then we also need to get libvirt
to interact with those new qemu features. But yes, dirty bitmap
tracking, so that you can track which portions of a file have changed
rather than copying the entire file, is something that is technically
feasible once the patches materialize.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
1:16 p.m.
On 10/13/14, 7:23 AM, Eric Blake wrote:
On 10/12/2014 10:19 PM, Jd wrote:
[please don't top-post on technical lists]
> Thanks Kashyap,
> The command line examples makes thing quite clear :)
> rsync for image (large) files create a new file for every little
> change, that was the reason I started looking in to using dirty bitmaps.
Upstream qemu still doesn't have persistent dirty bitmap tracking
working yet; once patches land there, then we also need to get libvirt
to interact with those new qemu features. But yes, dirty bitmap
tracking, so that you can track which portions of a file have changed
rather than copying the entire file, is something that is technically
feasible once the patches materialize.
Thank you.
I want to try this out in my environment for testing, what would you
suggest ? I am ok with using qemu-monitor commands in the interim till
things materializes in the libvirt level.
* Where can I start ?
qemu : http://wiki.qemu.org/ChangeLog/2.1 ?
kvm : do I need specific kvm or the following will work ? (see at
the end)
libvirt :update to 1.2.8 at least as you recommended to thomas.
* Where can I get relevant patches etc? block-backup, anything else?
* With some modifications to following procedure, can I get it a test
environment for testing incremental backup ?
http://kashyapc.com/2014/03/10/notes-for-building-kvm-based-virtualizatio...
Thanks
/Jd
==== KVM Info ===
sudo modinfo kvm
filename: /lib/modules/3.13.0-24-generic/kernel/arch/x86/kvm/kvm.ko
license: GPL
author: Qumranet
srcversion: 57170F28A3DA376BDCD5012
depends:
intree: Y
vermagic: 3.13.0-24-generic SMP mod_unload modversions
signer: Magrathea: Glacier signing key
sig_key: 69:B0:D0:A7:9B:85:D9:06:21:70:6E:8D:06:60:4D:73:0B:35:9F:C0
sig_hashalgo: sha512
parm: ignore_msrs:bool
parm: min_timer_period_us:uint
parm: tsc_tolerance_ppm:uint
parm: allow_unsafe_assigned_interrupts:Enable device
assignment on platforms without interrupt remapping support. (bool)
2:07 p.m.
On Saturday 11 October 2014 10:13:14 Kashyap Chamarthy wrote:
2. Create an external live snapshot:
$ virsh snapshot-create-as --domain vm1 sn1 \
--diskspec vda,file=/export/images/sn1.qcow2 \
--disk-only --atomic --no-metadata
Note (thanks Eric Blake): Above, if you have QEMU guest agent
installed in your virtual machine, try --quiesce option with
'snapshot-create-as' to ensure you have a consistent disk state.
About the --quiesce option. Do i need to do something inside the vm? The most
commonly would probably be a sql server running inside the vm. Do i need to
tell the sql server something about the --quiesce option i use? I read this
article here which suggests such a procedure. Okay, it's vmware, but... Is
that right?
http://www.virtuallifestyle.nl/2013/03/back-up-mysql-on-linux-without-sto...
thanks and cheers
t.
2:29 p.m.
On 10/29/2014 01:07 PM, Thomas Stein wrote:
About the --quiesce option. Do i need to do something inside the vm?
The most
commonly would probably be a sql server running inside the vm. Do i need to
tell the sql server something about the --quiesce option i use? I read this
article here which suggests such a procedure. Okay, it's vmware, but... Is
that right?
For --quiesce to work, you have to have qemu-guest-agent installed and
running in your guest, and also inform libvirt via the domain XML to
establish a channel to the guest agent. Furthermore, if you want your
sql database to be at a stable point, you can install freeze hooks in
your guest that get invoked prior to freezing the file system and just
after thawing it (that is, qemu-guest-agent already has documented hooks
that let you do any additional prep work beyond just freezing the file
system).
A quick google search found this:
http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_G...
Sadly, it didn't spell out the name of where you install hook scripts
into the guest. But this commit is pretty telling:
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreez...
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
4:05 p.m.
On Wednesday 29 October 2014 13:29:25 Eric Blake wrote:
On 10/29/2014 01:07 PM, Thomas Stein wrote:
> About the --quiesce option. Do i need to do something inside the vm? The
> most commonly would probably be a sql server running inside the vm. Do i
> need to tell the sql server something about the --quiesce option i use? I
> read this article here which suggests such a procedure. Okay, it's
> vmware, but... Is that right?
For --quiesce to work, you have to have qemu-guest-agent installed and
running in your guest, and also inform libvirt via the domain XML to
establish a channel to the guest agent. Furthermore, if you want your
sql database to be at a stable point, you can install freeze hooks in
your guest that get invoked prior to freezing the file system and just
after thawing it (that is, qemu-guest-agent already has documented hooks
that let you do any additional prep work beyond just freezing the file
system).
A quick google search found this:
http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Guid
e/QEMU_Guest_Agent_Overview.html
Sadly, it didn't spell out the name of where you install hook scripts
into the guest. But this commit is pretty telling:
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-h
ook
Thank you very much Eric.
The hole thing now raises a few more questions. But maybe i will ask them
later. :-)
cheers
t.
7:13 a.m.
> A quick google search found this:
> http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Gu
> id e/QEMU_Guest_Agent_Overview.html
>
> Sadly, it didn't spell out the name of where you install hook scripts
> into the guest. But this commit is pretty telling:
> https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze
> -h ook
After a little bit of fiddling i've got this running. Here is what's
neccessary in case someone wanna try this too.
qemu-guest.-agent has to run with the option: "-F /etc/qemu/fsfreeze-hook".
This file contains as pointed out by Eric:
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreez...
Then you need to have a directory called /etc/qemu/fsfreeze-hook.d. Inside
this directory this mysql-flush script:
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreez...
I had to adjust this script a little for mariadb 10.0.14. But after that
everything works as expected.
Here the output:
Sun 2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush
freeze
+ MYSQL='mysql -uroot -ppassword'
+ FIFO=/tmp/mysql-flush.fifo
+ case "$1" in
+ mkfifo /tmp/mysql-flush.fifo
+ flush_and_wait
+ printf 'FLUSH TABLES WITH READ LOCK \G\n'
+ mysql -uroot -ppassword
+ read
++ echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"'
++ mysql -uroot -ppassword
++ tail -1
++ cut -f 2
+ '[' 0 -gt 0 ']'
++ mktemp /tmp/mysql-flush.XXXXXX
+ INNODB_STATUS=/tmp/mysql-flush.eTdvA4
+ '[' 0 -ne 0 ']'
+ trap 'rm -f /tmp/mysql-flush.eTdvA4' SIGINT
+ :
+ printf 'SHOW ENGINE INNODB STATUS \G'
+ mysql -uroot -ppassword
++ grep 'Log sequence number' /tmp/mysql-flush.eTdvA4 tr -s ' '
++ cut '-d ' -f4
+ LOG_CURRENT=12242169543
++ grep 'Log flushed up to' /tmp/mysql-flush.eTdvA4 tr -s ' '
++ cut '-d ' -f7
+ LOG_FLUSHED=12242169543
+ '[' 12242169543 = 12242169543 ']'
+ break
+ rm -f /tmp/mysql-flush.eTdvA4
Sun 2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished
with status=0
Sun 2 Nov 13:46:14 CET 2014: execute /etc/qemu/fsfreeze-hook.d/mysql-flush
thaw
+ MYSQL='mysql -uroot -ppassword'
+ FIFO=/tmp/mysql-flush.fifo
+ case "$1" in
+ '[' '!' -p /tmp/mysql-flush.fifo ']'
+ echo
+ rm -f /tmp/mysql-flush.fifo
+ printf 'UNLOCK TABLES \G\n'
Sun 2 Nov 13:46:14 CET 2014: /etc/qemu/fsfreeze-hook.d/mysql-flush finished
with status=0
Just to be sure i activated the general.log for mysql:
141102 13:46:14 6 Connect root@localhost as anonymous on
6 Query select @@version_comment limit 1
6 Query FLUSH TABLES WITH READ LOCK
7 Connect root@localhost as anonymous on
7 Query select @@version_comment limit 1
7 Query SHOW STATUS LIKE "Key_blocks_not_flushed"
7 Quit
8 Connect root@localhost as anonymous on
8 Query select @@version_comment limit 1
8 Query SHOW ENGINE INNODB STATUS
8 Quit
6 Query UNLOCK TABLES
6 Quit
Nice!
cheers
t.
2:39 a.m.
On 11/02/2014 02:13 PM, Thomas Stein wrote:
After a little bit of fiddling i've got this running. Here is
what's
neccessary in case someone wanna try this too.
That's awesome news! Thanks for sharing.
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreez...
I had to adjust this script a little for mariadb 10.0.14. But after that
everything works as expected.
Do you want to post your version of the script for others to learn from?
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
4:28 a.m.
Am 03.11.14 09:39, schrieb Eric Blake:
On 11/02/2014 02:13 PM, Thomas Stein wrote:
> After a little bit of fiddling i've got this running. Here is what's
> neccessary in case someone wanna try this too.
That's awesome news! Thanks for sharing.
>
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreez...
>
> I had to adjust this script a little for mariadb 10.0.14. But after that
> everything works as expected.
Do you want to post your version of the script for others to learn from?
Sure. Here we go:
----
#!/bin/bash -x
# Flush MySQL tables to the disk before the filesystem is freezed.
# At the same time, this keeps a read lock while the filesystem is freezed
# in order to avoid write accesses by the other clients.
MYSQL="mysql -uroot -ppassword"
FIFO=/tmp/mysql-flush.fifo
flush_and_wait() {
printf "FLUSH TABLES WITH READ LOCK \\G\n"
read < $FIFO
printf "UNLOCK TABLES \\G\n"
}
case "$1" in
freeze)
mkfifo $FIFO || exit 1
flush_and_wait | $MYSQL &
# wait until every block is flushed
while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"'
|\
$MYSQL | tail -1 | cut -f 2)" -gt 0 ]; do
sleep 1
done
# for InnoDB, wait until every log is flushed
INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
[ $? -ne 0 ] && exit 2
trap "rm -f $INNODB_STATUS" SIGINT
while :; do
printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL >
$INNODB_STATUS
LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS tr
-s ' ' | cut -d' ' -f4)
LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS tr -s
' ' | cut -d' ' -f7)
[ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
sleep 1
done
rm -f $INNODB_STATUS
;;
thaw)
[ ! -p $FIFO ] && exit 1
echo > $FIFO
rm -f $FIFO
;;
esac
----
cheers
t.
11:14 p.m.
Thanks Eric.. inline.
On 10/10/14, 6:32 PM, Eric Blake wrote:
On 10/10/2014 11:37 AM, Jd wrote:
> Hi
> Looking in to implementing (CBT like) delta backup for KVM.
Not quite sure what you mean by CBT.
Sorry vmware term.. CBT -> changed block
tracking. (dirty block)
> The following looks promising..(last paragraph)
> http://wiki.qemu.org/Features/Snapshots2
>
Libvirt hasn't yet been patched to take advantage of all the latest qemu
features. Patches are welcome. But libvirt already knows how to create
external snapshots and do blockpull (backing files pulled into the
active image so the backing isn't needed any more) and commit (active
files pushed into the backing files), and even though it is not as
full-featured as where we'd like to get, it can already do quite a bit.
> * In the last para, there is a mention of copy the blocks from the
> disk using dirty-bitmap as reference. How to accomplish this ?
> block-mirror with bitmap or is there a qemu-img command ? some details
> would be appreciated.
block-backup is a new QMP command that libvirt would need to expose; I
don't know whether it is better to overload the existing
virDomainBlockCopy() API for it, or to add a new API.
So this seems to be already in qemu. How can I try this with KVM
context. Do I need to build from source ? or some version of KVM already
has this ?
> // backup software now reads foo.img using t0_dirty.dbmp to perform
> incremental backup, when finished
>
>
>
> * The backup after few runs in the backup store would be base image
> + bunch of delta blocks ? Will this be same as base disk and bunch of
> deltas ? or there is some special way to merge these ?
>
> * I am assuming this scheme (snapshot, bitmap, block merge etc.)
> should work with base disk in raw (non-qcow) format as well ? Right ?
> i.e. will it work when the storage disk is iscsi, lvm, fc ? std linux
> block device?
Yes, you can use qcow2 to store deltas on top of any underlying storage
device.
Glad to get confirmation.
> * Are there libvirt-api/verbs for doing some of this or we will
> have to sue the qemu-monitor-command ?
If you come up with a scenario where the only way to do something is via
a raw qemu-monitor-command, please report it as a bug (or better yet
provide a patch) so that a stable libvirt API can be written to do the
same task.
> * What version of qemu/kvm will have the core capabilities and
> which min libvirt version would be sufficient ?
Until patches materialize, it's hard to predict the future.
Could you clarify a
bit more .. I am not too familiar with how things
move from qemu --> kvm and how the libvirt releases are aligned with
qemu or kvm releases if they are.
I want to have an environment where I can try this end to end. I have
ubuntu 14.04.x release.
> Is there a better way to do incremental backup / CBT like backup
> than one mentioned here ?
>
> Thanks
> /Jd
>
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users(a)redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>
9:21 a.m.
On 10/12/2014 10:14 PM, Jd wrote:
So this seems to be already in qemu. How can I try this with KVM
context. Do I need to build from source ? or some version of KVM already
has this ?
KVM is the kernel technology that allows user-space virtualization.
qemu is the user-space program that takes advantage of kvm; sometimes,
the binary is named qemu-kvm to express the relationship. Many people
treat the terms interchangeably these days. If your distro ships qemu
2.1, you already have all that features that libvirt currently can drive
on top of kvm.
>> * What version of qemu/kvm will have the core
capabilities and
>> which min libvirt version would be sufficient ?
> Until patches materialize, it's hard to predict the future.
Could you clarify a bit more .. I am not too familiar with how things
move from qemu --> kvm and how the libvirt releases are aligned with
qemu or kvm releases if they are.
You shouldn't be worried about qemu/kvm interactions as much as
qemu/libvirt interactions. Right now, there are features in qemu that
have not yet been exposed in libvirt; the features will be added as soon
as someone volunteers to write the patches, but given the nature of
volunteer work, it is hard to predict when that will be.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
3553
days inactive
3577
days old
18 comments
4 participants
participants (4)
-
Eric Blake -
Jd -
Kashyap Chamarthy -
Thomas Stein