[libvirt] RFC backup API
Hi all, It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities. Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API. virDomainCreateBackup - which creates a backup full/incremental of all/selected disks, virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup. It looks like backup management functions, except create one, shouldn't be or might not be bound to a particular domain and we could possibly leverage storage pool API with some extension. Specifically, volume definition could be extended with necessary meta data related to backups. The *question* is: if the whole idea about this new API as described above or something similar makes sense? If yes, then let's find out requirements for it (if any) and I will try to prepare a patch set with the first RFC implementation to discuss the API in more details. Looking forward for your opinions on the matter. Maxim
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API.
virDomainCreateBackup - which creates a backup full/incremental of all/selected disks, virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.
It looks like backup management functions, except create one, shouldn't be or might not be bound to a particular domain and we could possibly leverage storage pool API with some extension. Specifically, volume definition could be extended with necessary meta data related to backups.
The *question* is: if the whole idea about this new API as described above or something similar makes sense?
Well we certainly don't want applications going directly to the QMP monitor command to achieve this. So given demand for this kind of feature, some kind of facility will be needed in libvirt. The question is just what any API would look like.
If yes, then let's find out requirements for it (if any) and I will try to prepare a patch set with the first RFC implementation to discuss the API in more details. Looking forward for your opinions on the matter.
Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this area ? If they have any existing models, it'd be desirable to look at them to align our APIs where appropriate. Also how does this fit in with recent discussions on QEMU mailing list about enabling 3rd party vendor backup software by having QEMU expose an NBD server to do efficient sparse backups. That suggested a very different kind of API where libvirt would not manage the backups, but just provide an API to allow an NBD target to be enabled for the 3rd party to manage. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
21.03.2016 13:36, Daniel P. Berrange пишет:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API.
virDomainCreateBackup - which creates a backup full/incremental of all/selected disks, virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.
It looks like backup management functions, except create one, shouldn't be or might not be bound to a particular domain and we could possibly leverage storage pool API with some extension. Specifically, volume definition could be extended with necessary meta data related to backups.
The *question* is: if the whole idea about this new API as described above or something similar makes sense? Well we certainly don't want applications going directly to the QMP monitor command to achieve this. So given demand for this kind of feature, some kind of facility will be needed in libvirt. The question is just what any API would look like.
If yes, then let's find out requirements for it (if any) and I will try to prepare a patch set with the first RFC implementation to discuss the API in more details. Looking forward for your opinions on the matter. Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this area ? If they have any existing models, it'd be desirable to look at them to align our APIs where appropriate.
As far as I know VirtualBox doesn't have any backup API, at least there is no reference to it at [1]. What I know about VMWare is that they use an approach that involves temporary snapshot creation, providing an access to quiesced disk data and then deleting this temporary snapshot [2]. Thus, their approach is close to what libvirt can have with 3d party managed backups, i.e. setup an NBD server on qemu side and let 3d party backup software access it from outside. [1] http://download.virtualbox.org/virtualbox/SDKRef.pdf [2] http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vddk.pg.doc/vddkBkupV...
Also how does this fit in with recent discussions on QEMU mailing list about enabling 3rd party vendor backup software by having QEMU expose an NBD server to do efficient sparse backups. That suggested a very different kind of API where libvirt would not manage the backups, but just provide an API to allow an NBD target to be enabled for the 3rd party to manage.
If you mean this thread [3] then as far as I understand, there is an agreement that NDB protocol gets extended a bit in such a way that external clients are able to get dirty block bitmap and this shouldn't be exposed in libvirt API. For such scenarios libvirt can have the following calls: virDomainStartBackup, virDomainFinishBackup When an external backup is started, qemu starts NBD server, allocates CBT (changed block tracking) bitmap and returns allocated NBD port to the caller. After that 3d party backup software is responcible for reading appropriate portion of data according to requested CBT bitmap. [3] http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg03075.html
Regards, Daniel
Thus, we have two ways of creating backups. The first is external (3d party managed), which is backed by QMP "blockdev-backup" command, which doesn't need from my point of view more than two calls mentioned above. The second is a another one, which is backed by QMP "drive-backup" command. My first letter covered only the latter case and I believe that this backup management API will be very useful. What is unclear still is: if there should be disk backup facilities only or whole VM backup as well, should libvirt be able to restore deleted VMs, should it be able to list created backups and so on. As for me, using storage pools with some extensions of storage pool API could perfectly solve this task. What do you think? Maxim
On Tue, Mar 22, 2016 at 02:19:10PM +0300, Maxim Nestratov wrote:
21.03.2016 13:36, Daniel P. Berrange пишет:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API.
virDomainCreateBackup - which creates a backup full/incremental of all/selected disks, virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.
It looks like backup management functions, except create one, shouldn't be or might not be bound to a particular domain and we could possibly leverage storage pool API with some extension. Specifically, volume definition could be extended with necessary meta data related to backups.
The *question* is: if the whole idea about this new API as described above or something similar makes sense? Well we certainly don't want applications going directly to the QMP monitor command to achieve this. So given demand for this kind of feature, some kind of facility will be needed in libvirt. The question is just what any API would look like.
If yes, then let's find out requirements for it (if any) and I will try to prepare a patch set with the first RFC implementation to discuss the API in more details. Looking forward for your opinions on the matter. Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this area ? If they have any existing models, it'd be desirable to look at them to align our APIs where appropriate.
As far as I know VirtualBox doesn't have any backup API, at least there is no reference to it at [1]. What I know about VMWare is that they use an approach that involves temporary snapshot creation, providing an access to quiesced disk data and then deleting this temporary snapshot [2]. Thus, their approach is close to what libvirt can have with 3d party managed backups, i.e. setup an NBD server on qemu side and let 3d party backup software access it from outside.
[1] http://download.virtualbox.org/virtualbox/SDKRef.pdf [2] http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vddk.pg.doc/vddkBkupV...
Also how does this fit in with recent discussions on QEMU mailing list about enabling 3rd party vendor backup software by having QEMU expose an NBD server to do efficient sparse backups. That suggested a very different kind of API where libvirt would not manage the backups, but just provide an API to allow an NBD target to be enabled for the 3rd party to manage.
If you mean this thread [3] then as far as I understand, there is an agreement that NDB protocol gets extended a bit in such a way that external clients are able to get dirty block bitmap and this shouldn't be exposed in libvirt API.
For such scenarios libvirt can have the following calls:
virDomainStartBackup, virDomainFinishBackup
When an external backup is started, qemu starts NBD server, allocates CBT (changed block tracking) bitmap and returns allocated NBD port to the caller. After that 3d party backup software is responcible for reading appropriate portion of data according to requested CBT bitmap.
[3] http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg03075.html
Yes, that is the thread I was referring to.
Thus, we have two ways of creating backups. The first is external (3d party managed), which is backed by QMP "blockdev-backup" command, which doesn't need from my point of view more than two calls mentioned above. The second is a another one, which is backed by QMP "drive-backup" command. My first letter covered only the latter case and I believe that this backup management API will be very useful. What is unclear still is: if there should be disk backup facilities only or whole VM backup as well, should libvirt be able to restore deleted VMs, should it be able to list created backups and so on. As for me, using storage pools with some extensions of storage pool API could perfectly solve this task. What do you think?
Ok, yes, there is clearly two completely separate modes of dealing with backups, externally managed and internally (to libvirt) managed. I can understand the desire to support both modes of operating backups. I was wondering what the difference is between doing a backup of the VM vs taking a snapshot of the VM. At a high level they feel pretty much the same, just no memory is snapshot for backups. IIUC though at the disk level, they pretty much inverted, a snapshot switches the running qcow2 file for the VM to point to a new qcow2 overlay, while a backup never touchs the VM disk, always using separate qcow2 images. So I can see why we'd want explicit backup support separately from snapshot support. At an API level it feels like the design of our snapshot APIs would map fairly naturally onto the new backups APIs, so getting consistency between the two is desirable IMHO. In particular the snapshot API for creating a snapshot allows an XML document to be fed in which describes how each disk is to be snapshot. I think we would need the same kind of flexibility for backups, to avoid having to hardcode the fact that backups always use qcow2. ie if a VM is using RBD, we want mgmt apps to be able to indicate that the backup should use a particular RBD volume too. Of course it should also be possible for an RBD backed guest to be able to save its backups to local raw/qcow2 files. We should also be able to indicate that some disks should be skipped for pupose of backups. So backup creation clearly needs a high level of configurability in general. We don't have to implement the full support matrix but the API design should allow for it in the future. There's a question as to whether should have allow for some default backup location if none is specified. eg perhaps we should always just store backups in /var/spool/backups/libvirt/qemu by default if the user/app didn't provide an explicit list of target volumes to hold the backup. This would allow the backup API to have better ease of use in the simple case Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
23.03.2016 13:30, Daniel P. Berrange пишет:
On Tue, Mar 22, 2016 at 02:19:10PM +0300, Maxim Nestratov wrote:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API.
virDomainCreateBackup - which creates a backup full/incremental of all/selected disks, virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.
It looks like backup management functions, except create one, shouldn't be or might not be bound to a particular domain and we could possibly leverage storage pool API with some extension. Specifically, volume definition could be extended with necessary meta data related to backups.
The *question* is: if the whole idea about this new API as described above or something similar makes sense? Well we certainly don't want applications going directly to the QMP monitor command to achieve this. So given demand for this kind of feature, some kind of facility will be needed in libvirt. The question is just what any API would look like.
If yes, then let's find out requirements for it (if any) and I will try to prepare a patch set with the first RFC implementation to discuss the API in more details. Looking forward for your opinions on the matter. Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this area ? If they have any existing models, it'd be desirable to look at them to align our APIs where appropriate. As far as I know VirtualBox doesn't have any backup API, at least there is no reference to it at [1]. What I know about VMWare is that they use an approach that involves temporary snapshot creation, providing an access to quiesced disk data and
21.03.2016 13:36, Daniel P. Berrange пишет: then deleting this temporary snapshot [2]. Thus, their approach is close to what libvirt can have with 3d party managed backups, i.e. setup an NBD server on qemu side and let 3d party backup software access it from outside.
[1] http://download.virtualbox.org/virtualbox/SDKRef.pdf [2] http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vddk.pg.doc/vddkBkupV...
Also how does this fit in with recent discussions on QEMU mailing list about enabling 3rd party vendor backup software by having QEMU expose an NBD server to do efficient sparse backups. That suggested a very different kind of API where libvirt would not manage the backups, but just provide an API to allow an NBD target to be enabled for the 3rd party to manage. If you mean this thread [3] then as far as I understand, there is an agreement that NDB protocol gets extended a bit in such a way that external clients are able to get dirty block bitmap and this shouldn't be exposed in libvirt API.
For such scenarios libvirt can have the following calls:
virDomainStartBackup, virDomainFinishBackup
When an external backup is started, qemu starts NBD server, allocates CBT (changed block tracking) bitmap and returns allocated NBD port to the caller. After that 3d party backup software is responcible for reading appropriate portion of data according to requested CBT bitmap.
[3] http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg03075.html Yes, that is the thread I was referring to.
Thus, we have two ways of creating backups. The first is external (3d party managed), which is backed by QMP "blockdev-backup" command, which doesn't need from my point of view more than two calls mentioned above. The second is a another one, which is backed by QMP "drive-backup" command. My first letter covered only the latter case and I believe that this backup management API will be very useful. What is unclear still is: if there should be disk backup facilities only or whole VM backup as well, should libvirt be able to restore deleted VMs, should it be able to list created backups and so on. As for me, using storage pools with some extensions of storage pool API could perfectly solve this task. What do you think? Ok, yes, there is clearly two completely separate modes of dealing with backups, externally managed and internally (to libvirt) managed. I can understand the desire to support both modes of operating backups.
I was wondering what the difference is between doing a backup of the VM vs taking a snapshot of the VM. At a high level they feel pretty much the same, just no memory is snapshot for backups. IIUC though at the disk level, they pretty much inverted, a snapshot switches the running qcow2 file for the VM to point to a new qcow2 overlay, while a backup never touchs the VM disk, always using separate qcow2 images. So I can see why we'd want explicit backup support separately from snapshot support.
Yeah, although they feel like the same, they are not. Backups are separate from VMs' disks and should be stored separately from the storage a VM runs from.
At an API level it feels like the design of our snapshot APIs would map fairly naturally onto the new backups APIs, so getting consistency between the two is desirable IMHO.
Ok, I'll take it into account.
In particular the snapshot API for creating a snapshot allows an XML document to be fed in which describes how each disk is to be snapshot. I think we would need the same kind of flexibility for backups, to avoid having to hardcode the fact that backups always use qcow2. ie if a VM is using RBD, we want mgmt apps to be able to indicate that the backup should use a particular RBD volume too. Of course it should also be possible for an RBD backed guest to be able to save its backups to local raw/qcow2 files. We should also be able to indicate that some disks should be skipped for pupose of backups. So backup creation clearly needs a high level of configurability in general. We don't have to implement the full support matrix but the API design should allow for it in the future.
There's a question as to whether should have allow for some default backup location if none is specified. eg perhaps we should always just store backups in /var/spool/backups/libvirt/qemu by default if the user/app didn't provide an explicit list of target volumes to hold the backup. This would allow the backup API to have better ease of use in the simple case
I agree. It perfectly makes sense. Also we can have a new libvirtd.conf parameter for this purpose.
Regards, Daniel
To sum up the above, I'll start designing this new API paying attention to mentioned conserns and will get back to the thread with the first RFC version or additional questions, whichever comes earlier. Best, Maxim
On 03/23/2016 04:30 AM, Daniel P. Berrange wrote:
Ok, yes, there is clearly two completely separate modes of dealing with backups, externally managed and internally (to libvirt) managed. I can understand the desire to support both modes of operating backups.
I was wondering what the difference is between doing a backup of the VM vs taking a snapshot of the VM. At a high level they feel pretty much the same, just no memory is snapshot for backups. IIUC though at the disk level, they pretty much inverted, a snapshot switches the running qcow2 file for the VM to point to a new qcow2 overlay, while a backup never touchs the VM disk, always using separate qcow2 images. So I can see why we'd want explicit backup support separately from snapshot support.
At an API level it feels like the design of our snapshot APIs would map fairly naturally onto the new backups APIs, so getting consistency between the two is desirable IMHO.
Yes, it would be nice to have virDomainSnapshot and virDomainBlockCopy more integrated with one another, particularly since XML gives us the flexibility to state more details about where a backup will be created.
In particular the snapshot API for creating a snapshot allows an XML document to be fed in which describes how each disk is to be snapshot. I think we would need the same kind of flexibility for backups, to avoid having to hardcode the fact that backups always use qcow2. ie if a VM is using RBD, we want mgmt apps to be able to indicate that the backup should use a particular RBD volume too. Of course it should also be possible for an RBD backed guest to be able to save its backups to local raw/qcow2 files. We should also be able to indicate that some disks should be skipped for pupose of backups. So backup creation clearly needs a high level of configurability in general. We don't have to implement the full support matrix but the API design should allow for it in the future.
There's a question as to whether should have allow for some default backup location if none is specified. eg perhaps we should always just store backups in /var/spool/backups/libvirt/qemu by default if the user/app didn't provide an explicit list of target volumes to hold the backup. This would allow the backup API to have better ease of use in the simple case
I'd really like us to add a notion of a default pool to the <domain> definition; if omitted, then libvirt chooses where to carve out storage for backups, snapshots, and other operations; but if present, then the user can specify a storage pool that will be used for all large-size operations that might be done on the domain. Even things like managed saves should use the per-domain storage pool rather than blindly assuming that /var/run/libvirt or /etc/libvirt/ is the right place to use. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Hello all, Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome. -------------------------------------------------------------------------------- virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Create a new backup of a domain based on the backup xml contained in xmlDesc. - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure enum virBackupCreateFlags { to be defined } An example of backup xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Start a new unmanaged backup of a domain based on the backup xml contained in xmlDesc. This function starts 'blockdev-backup' QMP command to block devices exposed as nbd servers by qemu. It is for 3d party managed backups. - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure An example of backup xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <disks> <disk name='hda'> <target> ... nbd server parameters for 'nbd-server-start' command followed by 'blockdev-backup' and 'nbd-server-add' QMP commands OR some other suitable target </target> </disk> <disk name='hdb'> <target> ... nbd server parameters </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- int virDomainBackupList(virDomainPtr domain, virBackupPtr ** backups, unsigned int flags) Collect the list of domain backups for the given domain, and allocate an array to store those objects - domain: a domain object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure enum virBackupListFlags { VIR_BACKUP_LIST_FULL = 1, VIR_BACKUP_LIST_LATEST = 2, ... } -------------------------------------------------------------------------------- int virBackupListChildren(virBackupPtr backup, virBackupPtr ** backups, unsigned int flags) Collect the list of child backups for the given backup, and allocate an array to store them - backup: a backup object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure -------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags) Restore a domain from a given backup. - xmlDesc: string containing an XML description of what to restore. It can repeate complete backup xml or specify some part of disks to restore. - flags: bitwise-OR of virBackupRestoreFlags - returns: 0 on success or -1 on failure enum virBackupRestoreFlags { VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, ... } An example of restore xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <disks> <disk name='vda'> <target> ... volume, file, or any possible target, (if absent, reuses current disk target) </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- int virBackupDelete(virBackupPtr backup, unsigned int flags) Delete the specified backup - backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on success or -1 on failure enum virBackupDeleteFlags { VIR_BACKUP_DELETE_ALL = 1, VIR_BACKUP_DELETE_ALL_CHILDREN = 2, ... } -------------------------------------------------------------------------------- int virBackupFree(virBackupPtr backup) Free the backup object. The backup itself is not modified. The data structure is freed and should not be used thereafter. - backup: a backup object - returns: 0 on success or -1 on failure -------------------------------------------------------------------------------- char* virBackupGetXMLDesc(virBackupPtr backup, unsigned int flags) Provide an XML description of the backup - backup: a backup object - flags: bitwise-OR of virDomainXMLFlags - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of error. The caller must free() the returned value An example of dumped xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <creationTime>1270477159</creationTime> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- virBackupPtr virBackupLookupByUUID(const char * uuid, unsigned int flags) Try to lookup a backup by its UUID - uuid: a backup uuid - flags: for future use - returns: an opaque virBackupPtr on success or NULL on failure -------------------------------------------------------------------------------- virDomainPtr virBackupGetDomain(virBackupPtr backup) Provide the domain pointer associated with a backup. The reference counter on the domain is not increased by this call - backup: a backup object - returns: a domain object on success or NULL on failure -------------------------------------------------------------------------------- int virBackupFinishJob(virBackupPtr backup, unsigned int flags) Finish async job associated with specified backup - backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on if the job for the specified backup was finished or cancelled successfully or -1 otherwise enum virBackupFinishJobFlags { VIR_BACKUP_FINISH_JOB_CANCEL = 1, VIR_BACKUP_FINISH_JOB_CHECK = 2, ... } -------------------------------------------------------------------------------- int virBackupGetParameters(virBackupPtr backup, virTypedParameterPtr params, int * nparams, unsigned int flags) Get backup parameters. On input, @nparams gives the size of the @params array; on output, @nparams gives how many slots were filled with parameter information, which might be less but will not exceed the input value - backup: a backup object - params: pointer to backup parameter object (return value, allocated by the caller) - nparams: pointer to number of blkio parameters; input and output - flags: for future use - returns: 0 on success or -1 on failure -------------------------------------------------------------------------------- int virDomainSetDefaultTarget(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Set default store target for backup, snapshot, managed saved state metadata - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virDomainDeviceModifyFlags - returns: 0 on success or -1 on failure Also, virDomainUndefineFlagsValues should be expanded with VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should take into account this flag when undefining domains. Best regards, Maxim
08.04.2016 23:47, Maxim Nestratov пишет:
Hello all,
Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome.
Any opinion on that?
08.04.2016 23:47, Maxim Nestratov пишет:
Hello all,
Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome.
--------------------------------------------------------------------------------
virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Create a new backup of a domain based on the backup xml contained in xmlDesc.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
enum virBackupCreateFlags {
to be defined }
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
--------------------------------------------------------------------------------
virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Start a new unmanaged backup of a domain based on the backup xml contained in xmlDesc. This function starts 'blockdev-backup' QMP command to block devices exposed as nbd servers by qemu. It is for 3d party managed backups.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <disks> <disk name='hda'> <target> ... nbd server parameters for 'nbd-server-start' command followed by 'blockdev-backup' and 'nbd-server-add' QMP commands OR some other suitable target </target> </disk> <disk name='hdb'> <target> ... nbd server parameters </target> </disk> </disks> </domainbackup>
--------------------------------------------------------------------------------
int virDomainBackupList(virDomainPtr domain, virBackupPtr ** backups, unsigned int flags)
Collect the list of domain backups for the given domain, and allocate an array to store those objects
- domain: a domain object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
enum virBackupListFlags {
VIR_BACKUP_LIST_FULL = 1, VIR_BACKUP_LIST_LATEST = 2, ...
}
--------------------------------------------------------------------------------
int virBackupListChildren(virBackupPtr backup, virBackupPtr ** backups, unsigned int flags)
Collect the list of child backups for the given backup, and allocate an array to store them
- backup: a backup object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
--------------------------------------------------------------------------------
int virBackupRestore(const char * xmlDesc, unsigned int flags)
Restore a domain from a given backup.
- xmlDesc: string containing an XML description of what to restore. It can repeate complete backup xml or specify some part of disks to restore. - flags: bitwise-OR of virBackupRestoreFlags - returns: 0 on success or -1 on failure
enum virBackupRestoreFlags {
VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, ... }
An example of restore xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <disks> <disk name='vda'> <target> ... volume, file, or any possible target, (if absent, reuses current disk target) </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
--------------------------------------------------------------------------------
int virBackupDelete(virBackupPtr backup, unsigned int flags)
Delete the specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on success or -1 on failure
enum virBackupDeleteFlags {
VIR_BACKUP_DELETE_ALL = 1, VIR_BACKUP_DELETE_ALL_CHILDREN = 2, ... }
--------------------------------------------------------------------------------
int virBackupFree(virBackupPtr backup)
Free the backup object. The backup itself is not modified. The data structure is freed and should not be used thereafter.
- backup: a backup object - returns: 0 on success or -1 on failure
--------------------------------------------------------------------------------
char* virBackupGetXMLDesc(virBackupPtr backup, unsigned int flags)
Provide an XML description of the backup
- backup: a backup object - flags: bitwise-OR of virDomainXMLFlags - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of error. The caller must free() the returned value
An example of dumped xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <creationTime>1270477159</creationTime> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
--------------------------------------------------------------------------------
virBackupPtr virBackupLookupByUUID(const char * uuid, unsigned int flags)
Try to lookup a backup by its UUID
- uuid: a backup uuid - flags: for future use - returns: an opaque virBackupPtr on success or NULL on failure
--------------------------------------------------------------------------------
virDomainPtr virBackupGetDomain(virBackupPtr backup)
Provide the domain pointer associated with a backup. The reference counter on the domain is not increased by this call
- backup: a backup object - returns: a domain object on success or NULL on failure
--------------------------------------------------------------------------------
int virBackupFinishJob(virBackupPtr backup, unsigned int flags)
Finish async job associated with specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on if the job for the specified backup was finished or cancelled successfully or -1 otherwise
enum virBackupFinishJobFlags {
VIR_BACKUP_FINISH_JOB_CANCEL = 1, VIR_BACKUP_FINISH_JOB_CHECK = 2, ... }
--------------------------------------------------------------------------------
int virBackupGetParameters(virBackupPtr backup, virTypedParameterPtr params, int * nparams, unsigned int flags)
Get backup parameters. On input, @nparams gives the size of the @params array; on output, @nparams gives how many slots were filled with parameter information, which might be less but will not exceed the input value
- backup: a backup object - params: pointer to backup parameter object (return value, allocated by the caller) - nparams: pointer to number of blkio parameters; input and output - flags: for future use - returns: 0 on success or -1 on failure
--------------------------------------------------------------------------------
int virDomainSetDefaultTarget(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Set default store target for backup, snapshot, managed saved state metadata
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virDomainDeviceModifyFlags - returns: 0 on success or -1 on failure
Also, virDomainUndefineFlagsValues should be expanded with VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should take into account this flag when undefining domains.
Best regards, Maxim
-- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Any chances to get comments on that?
On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote:
Hello all,
Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome.
-------------------------------------------------------------------------------- virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Nitpick, lets call it virDomainBackupPtr rather than just virBackupPtr. Likewise use virDomainBackup as name prefix for other data types and method names
Create a new backup of a domain based on the backup xml contained in xmlDesc.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
enum virBackupCreateFlags {
to be defined }
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Start a new unmanaged backup of a domain based on the backup xml contained in xmlDesc. This function starts 'blockdev-backup' QMP command to block devices exposed as nbd servers by qemu. It is for 3d party managed backups.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <disks> <disk name='hda'> <target> ... nbd server parameters for 'nbd-server-start' command followed by 'blockdev-backup' and 'nbd-server-add' QMP commands OR some other suitable target </target> </disk> <disk name='hdb'> <target> ... nbd server parameters </target> </disk> </disks> </domainbackup>
I'm assuming an 'unmanaged' backup would not be shown when calling virDomainBackupList later on ? Also with the QEMU built-in NBD server, you just have a single nbd-server-start to do, and that one server will export multiple disks as named exports. So you wouldn't repeat the NBD server parameters for all disks.
-------------------------------------------------------------------------------- int virDomainBackupList(virDomainPtr domain, virBackupPtr ** backups, unsigned int flags)
Collect the list of domain backups for the given domain, and allocate an array to store those objects
- domain: a domain object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
enum virBackupListFlags {
VIR_BACKUP_LIST_FULL = 1, VIR_BACKUP_LIST_LATEST = 2, ...
}
-------------------------------------------------------------------------------- int virBackupListChildren(virBackupPtr backup, virBackupPtr ** backups, unsigned int flags)
Collect the list of child backups for the given backup, and allocate an array to store them
- backup: a backup object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
-------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags)
For the "managed backups" case it feels like this should be accepting a virDomainBackupPtr rather than an xmlDesc string. For the "unmanaged backups", then IIUC, we would need to have a virDOmainPtr and the xmlDesc. So probably want a separate API for that. virBackupRestoreUnmanaged ?
Restore a domain from a given backup.
- xmlDesc: string containing an XML description of what to restore. It can repeate complete backup xml or specify some part of disks to restore. - flags: bitwise-OR of virBackupRestoreFlags - returns: 0 on success or -1 on failure
enum virBackupRestoreFlags {
VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, ... }
An example of restore xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <disks> <disk name='vda'> <target> ... volume, file, or any possible target, (if absent, reuses current disk target) </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- int virBackupDelete(virBackupPtr backup, unsigned int flags)
Delete the specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on success or -1 on failure
enum virBackupDeleteFlags {
VIR_BACKUP_DELETE_ALL = 1, VIR_BACKUP_DELETE_ALL_CHILDREN = 2, ... }
-------------------------------------------------------------------------------- int virBackupFree(virBackupPtr backup)
Free the backup object. The backup itself is not modified. The data structure is freed and should not be used thereafter.
- backup: a backup object - returns: 0 on success or -1 on failure
-------------------------------------------------------------------------------- char* virBackupGetXMLDesc(virBackupPtr backup, unsigned int flags)
Provide an XML description of the backup
- backup: a backup object - flags: bitwise-OR of virDomainXMLFlags - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of error. The caller must free() the returned value
An example of dumped xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <creationTime>1270477159</creationTime> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- virBackupPtr virBackupLookupByUUID(const char * uuid, unsigned int flags)
Try to lookup a backup by its UUID
- uuid: a backup uuid - flags: for future use - returns: an opaque virBackupPtr on success or NULL on failure
-------------------------------------------------------------------------------- virDomainPtr virBackupGetDomain(virBackupPtr backup)
Provide the domain pointer associated with a backup. The reference counter on the domain is not increased by this call
- backup: a backup object - returns: a domain object on success or NULL on failure
-------------------------------------------------------------------------------- int virBackupFinishJob(virBackupPtr backup, unsigned int flags)
Finish async job associated with specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on if the job for the specified backup was finished or cancelled successfully or -1 otherwise
enum virBackupFinishJobFlags {
VIR_BACKUP_FINISH_JOB_CANCEL = 1, VIR_BACKUP_FINISH_JOB_CHECK = 2, ... }
Probablt want a corresponding virBackupAbortJob too.
-------------------------------------------------------------------------------- int virBackupGetParameters(virBackupPtr backup, virTypedParameterPtr params, int * nparams, unsigned int flags)
Get backup parameters. On input, @nparams gives the size of the @params array; on output, @nparams gives how many slots were filled with parameter information, which might be less but will not exceed the input value
- backup: a backup object - params: pointer to backup parameter object (return value, allocated by the caller) - nparams: pointer to number of blkio parameters; input and output - flags: for future use - returns: 0 on success or -1 on failure
-------------------------------------------------------------------------------- int virDomainSetDefaultTarget(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Set default store target for backup, snapshot, managed saved state metadata
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virDomainDeviceModifyFlags - returns: 0 on success or -1 on failure
Not sure I understand what this is doing, but probably not a big deal.
Also, virDomainUndefineFlagsValues should be expanded with VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should take into account this flag when undefining domains.
Overall I think this looks like a reasonable API design to start implementing. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
27.04.2016 15:31, Daniel P. Berrange пишет:
On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote:
Hello all,
Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome.
-------------------------------------------------------------------------------- virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Nitpick, lets call it virDomainBackupPtr rather than just virBackupPtr. Likewise use virDomainBackup as name prefix for other data types and method names
Ok
Create a new backup of a domain based on the backup xml contained in xmlDesc.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
enum virBackupCreateFlags {
to be defined }
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Start a new unmanaged backup of a domain based on the backup xml contained in xmlDesc. This function starts 'blockdev-backup' QMP command to block devices exposed as nbd servers by qemu. It is for 3d party managed backups.
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure
An example of backup xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <disks> <disk name='hda'> <target> ... nbd server parameters for 'nbd-server-start' command followed by 'blockdev-backup' and 'nbd-server-add' QMP commands OR some other suitable target </target> </disk> <disk name='hdb'> <target> ... nbd server parameters </target> </disk> </disks> </domainbackup> I'm assuming an 'unmanaged' backup would not be shown when calling virDomainBackupList later on ?
Exactly
Also with the QEMU built-in NBD server, you just have a single nbd-server-start to do, and that one server will export multiple disks as named exports. So you wouldn't repeat the NBD server parameters for all disks.
Got it.
-------------------------------------------------------------------------------- int virDomainBackupList(virDomainPtr domain, virBackupPtr ** backups, unsigned int flags)
Collect the list of domain backups for the given domain, and allocate an array to store those objects
- domain: a domain object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
enum virBackupListFlags {
VIR_BACKUP_LIST_FULL = 1, VIR_BACKUP_LIST_LATEST = 2, ...
}
-------------------------------------------------------------------------------- int virBackupListChildren(virBackupPtr backup, virBackupPtr ** backups, unsigned int flags)
Collect the list of child backups for the given backup, and allocate an array to store them
- backup: a backup object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure
-------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags) For the "managed backups" case it feels like this should be accepting a virDomainBackupPtr rather than an xmlDesc string.
Not sure because I would like to have an ability to restore not every disk a backup has but sometimes selected only.
For the "unmanaged backups", then IIUC, we would need to have a virDOmainPtr and the xmlDesc. So probably want a separate API for that. virBackupRestoreUnmanaged ?
Makes sense.
Restore a domain from a given backup.
- xmlDesc: string containing an XML description of what to restore. It can repeate complete backup xml or specify some part of disks to restore. - flags: bitwise-OR of virBackupRestoreFlags - returns: 0 on success or -1 on failure
enum virBackupRestoreFlags {
VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, ... }
An example of restore xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <disks> <disk name='vda'> <target> ... volume, file, or any possible target, (if absent, reuses current disk target) </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- int virBackupDelete(virBackupPtr backup, unsigned int flags)
Delete the specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on success or -1 on failure
enum virBackupDeleteFlags {
VIR_BACKUP_DELETE_ALL = 1, VIR_BACKUP_DELETE_ALL_CHILDREN = 2, ... }
-------------------------------------------------------------------------------- int virBackupFree(virBackupPtr backup)
Free the backup object. The backup itself is not modified. The data structure is freed and should not be used thereafter.
- backup: a backup object - returns: 0 on success or -1 on failure
-------------------------------------------------------------------------------- char* virBackupGetXMLDesc(virBackupPtr backup, unsigned int flags)
Provide an XML description of the backup
- backup: a backup object - flags: bitwise-OR of virDomainXMLFlags - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of error. The caller must free() the returned value
An example of dumped xml:
<domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <creationTime>1270477159</creationTime> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup>
-------------------------------------------------------------------------------- virBackupPtr virBackupLookupByUUID(const char * uuid, unsigned int flags)
Try to lookup a backup by its UUID
- uuid: a backup uuid - flags: for future use - returns: an opaque virBackupPtr on success or NULL on failure
-------------------------------------------------------------------------------- virDomainPtr virBackupGetDomain(virBackupPtr backup)
Provide the domain pointer associated with a backup. The reference counter on the domain is not increased by this call
- backup: a backup object - returns: a domain object on success or NULL on failure
-------------------------------------------------------------------------------- int virBackupFinishJob(virBackupPtr backup, unsigned int flags)
Finish async job associated with specified backup
- backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on if the job for the specified backup was finished or cancelled successfully or -1 otherwise
enum virBackupFinishJobFlags {
VIR_BACKUP_FINISH_JOB_CANCEL = 1, VIR_BACKUP_FINISH_JOB_CHECK = 2, ... } Probablt want a corresponding virBackupAbortJob too.
Makes sense as well
-------------------------------------------------------------------------------- int virBackupGetParameters(virBackupPtr backup, virTypedParameterPtr params, int * nparams, unsigned int flags)
Get backup parameters. On input, @nparams gives the size of the @params array; on output, @nparams gives how many slots were filled with parameter information, which might be less but will not exceed the input value
- backup: a backup object - params: pointer to backup parameter object (return value, allocated by the caller) - nparams: pointer to number of blkio parameters; input and output - flags: for future use - returns: 0 on success or -1 on failure
-------------------------------------------------------------------------------- int virDomainSetDefaultTarget(virDomainPtr domain, const char * xmlDesc, unsigned int flags)
Set default store target for backup, snapshot, managed saved state metadata
- domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virDomainDeviceModifyFlags - returns: 0 on success or -1 on failure
Not sure I understand what this is doing, but probably not a big deal.
I was trying to addess what Eric Blake said in this thread earlier regarding a default pool.
Also, virDomainUndefineFlagsValues should be expanded with VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should take into account this flag when undefining domains. Overall I think this looks like a reasonable API design to start implementing.
Thanks a lot. Will start shortly.
Regards, Daniel
On Thu, Apr 28, 2016 at 05:02:20PM +0300, Maxim Nestratov wrote:
27.04.2016 15:31, Daniel P. Berrange пишет:
On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote:
-------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags) For the "managed backups" case it feels like this should be accepting a virDomainBackupPtr rather than an xmlDesc string.
Not sure because I would like to have an ability to restore not every disk a backup has but sometimes selected only.
How about doing virDomainBackupRestore(virDomainBackupPtr backip, const char *xmlDesc, unsigned int flags); Alow xmlDesc to be NULL by default, so it uses the built-in recorded XML unless you choose to override it with custom XML. This is similar to how we let people override XML during migration. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
28.04.2016 17:50, Daniel P. Berrange пишет:
On Thu, Apr 28, 2016 at 05:02:20PM +0300, Maxim Nestratov wrote:
27.04.2016 15:31, Daniel P. Berrange пишет:
On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote:
-------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags) For the "managed backups" case it feels like this should be accepting a virDomainBackupPtr rather than an xmlDesc string. Not sure because I would like to have an ability to restore not every disk a backup has but sometimes selected only. How about doing
virDomainBackupRestore(virDomainBackupPtr backip, const char *xmlDesc, unsigned int flags);
Alow xmlDesc to be NULL by default, so it uses the built-in recorded XML unless you choose to override it with custom XML. This is similar to how we let people override XML during migration.
Cool. This fits perfectly.
Regards, Daniel
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly. Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.] For 'blockdev-backup' --------------------- -> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} } Where 'tagert' in this case means: "the name of the backup target device. (json-string)" For 'drive-backup' ----------------- -> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} } Here, 'target' means: "the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)" [...] -- /kashyap
23.03.2016 13:36, Kashyap Chamarthy пишет:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities. There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly.
Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.]
For 'blockdev-backup' ---------------------
-> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} }
Where 'tagert' in this case means:
"the name of the backup target device. (json-string)"
For 'drive-backup' -----------------
-> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} }
Here, 'target' means:
"the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)"
[...]
Looks correct. I mentioned "blockdev-backup" in my reply to Daniel's letter in this thread. The difference is how those backups are actually stored.
On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly.
Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.]
No, you're right. Blockdev-backup can backup to an arbitrary device (which can be backed by a new file), but drive-backup will only accept a new file. I don't think blockdev-backup supports incremental backups just yet, but I don't think there's any reason it can't. (Looking at it: yeah, why have I not done that yet?...)
For 'blockdev-backup' ---------------------
-> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} }
Where 'tagert' in this case means:
"the name of the backup target device. (json-string)"
For 'drive-backup' -----------------
-> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} }
Here, 'target' means:
"the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)"
[...]
--js
23.03.2016 17:27, John Snow пишет:
On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities. There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly.
Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.]
No, you're right. Blockdev-backup can backup to an arbitrary device (which can be backed by a new file), but drive-backup will only accept a new file.
I don't think blockdev-backup supports incremental backups just yet, but I don't think there's any reason it can't. (Looking at it: yeah, why have I not done that yet?...)
John, Any chances to get this implemented? Just not to use two different commands in libvirt and start using 'blockdev-backup' right away for both full and incremental backups?
For 'blockdev-backup' ---------------------
-> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} }
Where 'tagert' in this case means:
"the name of the backup target device. (json-string)"
For 'drive-backup' -----------------
-> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} }
Here, 'target' means:
"the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)"
[...]
--js
On 03/25/2016 05:34 AM, Maxim Nestratov wrote:
23.03.2016 17:27, John Snow пишет:
On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote:
On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities. There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly.
Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.]
No, you're right. Blockdev-backup can backup to an arbitrary device (which can be backed by a new file), but drive-backup will only accept a new file.
I don't think blockdev-backup supports incremental backups just yet, but I don't think there's any reason it can't. (Looking at it: yeah, why have I not done that yet?...)
John, Any chances to get this implemented? Just not to use two different commands in libvirt and start using 'blockdev-backup' right away for both full and incremental backups?
Yes, it'd be very trivial to do... but I think there is some debate currently about how to change how incrementals work, so it might not be wise for me to do this until I know what the situation is, to avoid having to retcon _two_ QMP interfaces instead of just the one. (The debate revolves around how bitmaps exist as an in-memory object and how to present them to the user -- as objects that attach to /drives/, or to /nodes/, or to both. We haven't answered this question for ourselves yet, which precludes API design.)
For 'blockdev-backup' ---------------------
-> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} }
Where 'tagert' in this case means:
"the name of the backup target device. (json-string)"
For 'drive-backup' -----------------
-> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} }
Here, 'target' means:
"the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)"
[...]
--js
On 03/21/2016 04:18 AM, Maxim Nestratov wrote:
Hi all,
It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities.
Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API.
virDomainCreateBackup - which creates a backup full/incremental of all/selected disks,
I'm also wondering if it would be better to just add a new flag to our existing virDomainBlockCopy() that specifies backup semantics (the current default creates the block copy at the point in time that the job is _ended_; whereas drive-backup creates the block copy at the oin in time where the job is _started_), rather than needing new API.
virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.
And here, I think that the existing virDomainSnapshot* API may already fit this need, if we could more closely couple the act of creating snapshots/backups with the act of disk backups. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
participants (5)
-
Daniel P. Berrange -
Eric Blake -
John Snow -
Kashyap Chamarthy -
Maxim Nestratov