On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote: > Hi all, > > It's been already quite a long time since qemu implemented QMP > "drive-backup" command to create block devices backups. Even more, since > qemu 2.4 there is a possibility to create incremental backups. Though it is > possible to backup all attached to a domain disk drives by combining them > into a single QMP transaction command, this way of creating them, not to > mention managing, remains inconvenient for an end user of libvirt. Moreover, > creating a single drive backup via QMP interface isn't handy either. That > said, it looks reasonable to introduce a *new backup API* based on QMP > "drive-backup" facilities. > > Though we can start from a single simple function, allowing to create a disk > backup by means of QMP "drive-backup" command, I'd like to discuss here the > level of management libvirt could provide for backup operations. To begin > with, here is the preliminary list of possible functions that I think make > sense for libvirt API. > > virDomainCreateBackup - which creates a backup full/incremental of > all/selected disks, > virListBackups - which lists all backups created for a particular > domain/target, > virRestoreBackup - which restores all/selected disks from a backup, > virDeleteBackup - which deletes all/selected disks from a backup. > > It looks like backup management functions, except create one, shouldn't be > or might not be bound to a particular domain and we could possibly leverage > storage pool API with some extension. Specifically, volume definition could > be extended with necessary meta data related to backups. > > The *question* is: if the whole idea about this new API as described above > or something similar makes sense? Well we certainly don't want applications going directly to the QMP monitor command to achieve this. So given demand for this kind of feature, some kind of facility will be needed in libvirt. The question is just what any API would look like. > If yes, then let's find out requirements for it (if any) and I will try to > prepare a patch set with the first RFC implementation to discuss the API in > more details. Looking forward for your opinions on the matter. Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this area ? If they have any existing models, it'd be desirable to look at them to align our APIs where appropriate.

Also how does this fit in with recent discussions on QEMU mailing list about enabling 3rd party vendor backup software by having QEMU expose an NBD server to do efficient sparse backups. That suggested a very different kind of API where libvirt would not manage the backups, but just provide an API to allow an NBD target to be enabled for the 3rd party to manage.

Regards, Daniel

On Tue, Mar 22, 2016 at 02:19:10PM +0300, Maxim Nestratov wrote: > 21.03.2016 13:36, Daniel P. Berrange пишет: >> On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote: >>> Hi all, >>> >>> It's been already quite a long time since qemu implemented QMP >>> "drive-backup" command to create block devices backups. Even more, since >>> qemu 2.4 there is a possibility to create incremental backups. Though it is >>> possible to backup all attached to a domain disk drives by combining them >>> into a single QMP transaction command, this way of creating them, not to >>> mention managing, remains inconvenient for an end user of libvirt. Moreover, >>> creating a single drive backup via QMP interface isn't handy either. That >>> said, it looks reasonable to introduce a *new backup API* based on QMP >>> "drive-backup" facilities. >>> >>> Though we can start from a single simple function, allowing to create a disk >>> backup by means of QMP "drive-backup" command, I'd like to discuss here the >>> level of management libvirt could provide for backup operations. To begin >>> with, here is the preliminary list of possible functions that I think make >>> sense for libvirt API. >>> >>> virDomainCreateBackup - which creates a backup full/incremental of >>> all/selected disks, >>> virListBackups - which lists all backups created for a particular >>> domain/target, >>> virRestoreBackup - which restores all/selected disks from a backup, >>> virDeleteBackup - which deletes all/selected disks from a backup. >>> >>> It looks like backup management functions, except create one, shouldn't be >>> or might not be bound to a particular domain and we could possibly leverage >>> storage pool API with some extension. Specifically, volume definition could >>> be extended with necessary meta data related to backups. >>> >>> The *question* is: if the whole idea about this new API as described above >>> or something similar makes sense? >> Well we certainly don't want applications going directly to the QMP >> monitor command to achieve this. So given demand for this kind of >> feature, some kind of facility will be needed in libvirt. The question >> is just what any API would look like. >> >>> If yes, then let's find out requirements for it (if any) and I will try to >>> prepare a patch set with the first RFC implementation to discuss the API in >>> more details. Looking forward for your opinions on the matter. >> Have you researched what VMWare/Hyper-V/VirtualBox APIs support in this >> area ? If they have any existing models, it'd be desirable to look at >> them to align our APIs where appropriate. > As far as I know VirtualBox doesn't have any backup API, at least there is > no reference to it at [1]. > What I know about VMWare is that they use an approach that involves > temporary snapshot creation, providing an access to quiesced disk data and > then deleting this temporary snapshot [2]. Thus, their approach is close to > what libvirt can have with 3d party managed backups, i.e. setup an NBD > server on qemu side and let 3d party backup software access it from outside. > > [1] http://download.virtualbox.org/virtualbox/SDKRef.pdf > [2] http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vddk.pg.doc/vddkBk... > >> Also how does this fit in with recent discussions on QEMU mailing list >> about enabling 3rd party vendor backup software by having QEMU expose an >> NBD server to do efficient sparse backups. That suggested a very different >> kind of API where libvirt would not manage the backups, but just provide >> an API to allow an NBD target to be enabled for the 3rd party to manage. > If you mean this thread [3] then as far as I understand, there is an > agreement that NDB protocol gets extended a bit in such a way that external > clients are able to get dirty block bitmap and this shouldn't be exposed in > libvirt API. > > For such scenarios libvirt can have the following calls: > > virDomainStartBackup, > virDomainFinishBackup > > When an external backup is started, qemu starts NBD server, allocates CBT > (changed block tracking) bitmap and returns allocated NBD port to the > caller. After that 3d party backup software is responcible for reading > appropriate portion of data according to requested CBT bitmap. > > [3] http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg03075.html Yes, that is the thread I was referring to. > Thus, we have two ways of creating backups. The first is external (3d party > managed), which is backed by QMP "blockdev-backup" command, which doesn't > need from my point of view more than two calls mentioned above. The second > is a another one, which is backed by QMP "drive-backup" command. My first > letter covered only the latter case and I believe that this backup > management API will be very useful. What is unclear still is: if there > should be disk backup facilities only or whole VM backup as well, should > libvirt be able to restore deleted VMs, should it be able to list created > backups and so on. As for me, using storage pools with some extensions of > storage pool API could perfectly solve this task. What do you think? Ok, yes, there is clearly two completely separate modes of dealing with backups, externally managed and internally (to libvirt) managed. I can understand the desire to support both modes of operating backups. I was wondering what the difference is between doing a backup of the VM vs taking a snapshot of the VM. At a high level they feel pretty much the same, just no memory is snapshot for backups. IIUC though at the disk level, they pretty much inverted, a snapshot switches the running qcow2 file for the VM to point to a new qcow2 overlay, while a backup never touchs the VM disk, always using separate qcow2 images. So I can see why we'd want explicit backup support separately from snapshot support.

At an API level it feels like the design of our snapshot APIs would map fairly naturally onto the new backups APIs, so getting consistency between the two is desirable IMHO.

In particular the snapshot API for creating a snapshot allows an XML document to be fed in which describes how each disk is to be snapshot. I think we would need the same kind of flexibility for backups, to avoid having to hardcode the fact that backups always use qcow2. ie if a VM is using RBD, we want mgmt apps to be able to indicate that the backup should use a particular RBD volume too. Of course it should also be possible for an RBD backed guest to be able to save its backups to local raw/qcow2 files. We should also be able to indicate that some disks should be skipped for pupose of backups. So backup creation clearly needs a high level of configurability in general. We don't have to implement the full support matrix but the API design should allow for it in the future. There's a question as to whether should have allow for some default backup location if none is specified. eg perhaps we should always just store backups in /var/spool/backups/libvirt/qemu by default if the user/app didn't provide an explicit list of target volumes to hold the backup. This would allow the backup API to have better ease of use in the simple case

Regards, Daniel

Hello all, Here is the first more detailed view on the list of the backup API functions that look reasonable to have. Though they are presented with parameters and short description all these are the subject to discuss and I kindly ask you to take a look and comment. Your ideas and considerations are very welcome. -------------------------------------------------------------------------------- virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Create a new backup of a domain based on the backup xml contained in xmlDesc. - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure enum virBackupCreateFlags { to be defined } An example of backup xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Start a new unmanaged backup of a domain based on the backup xml contained in xmlDesc. This function starts 'blockdev-backup' QMP command to block devices exposed as nbd servers by qemu. It is for 3d party managed backups. - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virBackupCreateFlags - returns: an opaque virBackupPtr on success or NULL on failure An example of backup xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <description>Any string</description> <disks> <disk name='hda'> <target> ... nbd server parameters for 'nbd-server-start' command followed by 'blockdev-backup' and 'nbd-server-add' QMP commands OR some other suitable target </target> </disk> <disk name='hdb'> <target> ... nbd server parameters </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- int virDomainBackupList(virDomainPtr domain, virBackupPtr ** backups, unsigned int flags) Collect the list of domain backups for the given domain, and allocate an array to store those objects - domain: a domain object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure enum virBackupListFlags { VIR_BACKUP_LIST_FULL = 1, VIR_BACKUP_LIST_LATEST = 2, ... } -------------------------------------------------------------------------------- int virBackupListChildren(virBackupPtr backup, virBackupPtr ** backups, unsigned int flags) Collect the list of child backups for the given backup, and allocate an array to store them - backup: a backup object - backups: pointer to variable to store the array containing backup objects, or NULL if the list is not required (just returns number of backups) - flags: bitwise-OR of virBackupListFlags - returns: the number of backups on success or -1 on failure -------------------------------------------------------------------------------- int virBackupRestore(const char * xmlDesc, unsigned int flags) Restore a domain from a given backup. - xmlDesc: string containing an XML description of what to restore. It can repeate complete backup xml or specify some part of disks to restore. - flags: bitwise-OR of virBackupRestoreFlags - returns: 0 on success or -1 on failure enum virBackupRestoreFlags { VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, ... } An example of restore xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <disks> <disk name='vda'> <target> ... volume, file, or any possible target, (if absent, reuses current disk target) </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- int virBackupDelete(virBackupPtr backup, unsigned int flags) Delete the specified backup - backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on success or -1 on failure enum virBackupDeleteFlags { VIR_BACKUP_DELETE_ALL = 1, VIR_BACKUP_DELETE_ALL_CHILDREN = 2, ... } -------------------------------------------------------------------------------- int virBackupFree(virBackupPtr backup) Free the backup object. The backup itself is not modified. The data structure is freed and should not be used thereafter. - backup: a backup object - returns: 0 on success or -1 on failure -------------------------------------------------------------------------------- char* virBackupGetXMLDesc(virBackupPtr backup, unsigned int flags) Provide an XML description of the backup - backup: a backup object - flags: bitwise-OR of virDomainXMLFlags - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of error. The caller must free() the returned value An example of dumped xml: <domainbackup> <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> <creationTime>1270477159</creationTime> <description>Any string</description> <parent> <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> <parent> <disks> <disk name='vda'> <target> ... volume, file, or any possible target </target> </disk> <disk name='vdb'> <target> ... </target> </disk> </disks> </domainbackup> -------------------------------------------------------------------------------- virBackupPtr virBackupLookupByUUID(const char * uuid, unsigned int flags) Try to lookup a backup by its UUID - uuid: a backup uuid - flags: for future use - returns: an opaque virBackupPtr on success or NULL on failure -------------------------------------------------------------------------------- virDomainPtr virBackupGetDomain(virBackupPtr backup) Provide the domain pointer associated with a backup. The reference counter on the domain is not increased by this call - backup: a backup object - returns: a domain object on success or NULL on failure -------------------------------------------------------------------------------- int virBackupFinishJob(virBackupPtr backup, unsigned int flags) Finish async job associated with specified backup - backup: a backup object - flags: bitwise-OR of virBackupDeleteFlags - returns: 0 on if the job for the specified backup was finished or cancelled successfully or -1 otherwise enum virBackupFinishJobFlags { VIR_BACKUP_FINISH_JOB_CANCEL = 1, VIR_BACKUP_FINISH_JOB_CHECK = 2, ... } -------------------------------------------------------------------------------- int virBackupGetParameters(virBackupPtr backup, virTypedParameterPtr params, int * nparams, unsigned int flags) Get backup parameters. On input, @nparams gives the size of the @params array; on output, @nparams gives how many slots were filled with parameter information, which might be less but will not exceed the input value - backup: a backup object - params: pointer to backup parameter object (return value, allocated by the caller) - nparams: pointer to number of blkio parameters; input and output - flags: for future use - returns: 0 on success or -1 on failure -------------------------------------------------------------------------------- int virDomainSetDefaultTarget(virDomainPtr domain, const char * xmlDesc, unsigned int flags) Set default store target for backup, snapshot, managed saved state metadata - domain: a domain object - xmlDesc: string containing an XML description of the backup - flags: bitwise-OR of virDomainDeviceModifyFlags - returns: 0 on success or -1 on failure Also, virDomainUndefineFlagsValues should be expanded with VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should take into account this flag when undefining domains. Best regards, Maxim -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list

On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote: > Hello all, > > Here is the first more detailed view on the list of the backup API functions > that > look reasonable to have. Though they are presented with parameters and short > description all these are the subject to discuss and I kindly ask you to > take a > look and comment. Your ideas and considerations are very welcome. > > > -------------------------------------------------------------------------------- > virBackupPtr virDomainBackupCreateXML(virDomainPtr domain, > const char * xmlDesc, > unsigned int flags) Nitpick, lets call it virDomainBackupPtr rather than just virBackupPtr. Likewise use virDomainBackup as name prefix for other data types and method names

> Create a new backup of a domain based on the backup xml contained in > xmlDesc. > > - domain: a domain object > - xmlDesc: string containing an XML description of the backup > - flags: bitwise-OR of virBackupCreateFlags > - returns: an opaque virBackupPtr on success or NULL on failure > > enum virBackupCreateFlags { > > to be defined > } > > > An example of backup xml: > > <domainbackup> > <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> > <description>Any string</description> > <parent> > <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> > <parent> > <disks> > <disk name='vda'> > <target> > ... volume, file, or any possible target > </target> > </disk> > <disk name='vdb'> > <target> > ... > </target> > </disk> > </disks> > </domainbackup> > > -------------------------------------------------------------------------------- > virBackupPtr virBackupStartUnmanaged(virDomainPtr domain, > const char * xmlDesc, > unsigned int flags) > > Start a new unmanaged backup of a domain based on the backup xml contained > in > xmlDesc. This function starts 'blockdev-backup' QMP command to block > devices > exposed as nbd servers by qemu. It is for 3d party managed backups. > > - domain: a domain object > - xmlDesc: string containing an XML description of the backup > - flags: bitwise-OR of virBackupCreateFlags > - returns: an opaque virBackupPtr on success or NULL on failure > > An example of backup xml: > > <domainbackup> > <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> > <description>Any string</description> > <disks> > <disk name='hda'> > <target> > ... nbd server parameters for 'nbd-server-start' command > followed by > 'blockdev-backup' and 'nbd-server-add' QMP commands > OR some other suitable target > </target> > </disk> > <disk name='hdb'> > <target> > ... nbd server parameters > </target> > </disk> > </disks> > </domainbackup> I'm assuming an 'unmanaged' backup would not be shown when calling virDomainBackupList later on ?

Also with the QEMU built-in NBD server, you just have a single nbd-server-start to do, and that one server will export multiple disks as named exports. So you wouldn't repeat the NBD server parameters for all disks.

> -------------------------------------------------------------------------------- > int virDomainBackupList(virDomainPtr domain, > virBackupPtr ** backups, > unsigned int flags) > > Collect the list of domain backups for the given domain, and allocate an > array > to store those objects > > - domain: a domain object > - backups: pointer to variable to store the array containing backup > objects, > or NULL if the list is not required (just returns number of backups) > - flags: bitwise-OR of virBackupListFlags > - returns: the number of backups on success or -1 on failure > > enum virBackupListFlags { > > VIR_BACKUP_LIST_FULL = 1, > VIR_BACKUP_LIST_LATEST = 2, > ... > > } > > -------------------------------------------------------------------------------- > int virBackupListChildren(virBackupPtr backup, > virBackupPtr ** backups, > unsigned int flags) > > Collect the list of child backups for the given backup, and allocate an > array > to store them > > - backup: a backup object > - backups: pointer to variable to store the array containing backup > objects, > or NULL if the list is not required (just returns number of backups) > - flags: bitwise-OR of virBackupListFlags > - returns: the number of backups on success or -1 on failure > > -------------------------------------------------------------------------------- > int virBackupRestore(const char * xmlDesc, > unsigned int flags) For the "managed backups" case it feels like this should be accepting a virDomainBackupPtr rather than an xmlDesc string.

For the "unmanaged backups", then IIUC, we would need to have a virDOmainPtr and the xmlDesc. So probably want a separate API for that. virBackupRestoreUnmanaged ?

> Restore a domain from a given backup. > > - xmlDesc: string containing an XML description of what to restore. It > can > repeate complete backup xml or specify some part of disks to restore. > - flags: bitwise-OR of virBackupRestoreFlags > - returns: 0 on success or -1 on failure > > enum virBackupRestoreFlags { > > VIR_BACKUP_RESTORE_NEW_DOMAIN = 1, > VIR_BACKUP_RESTORE_REWRITE_EXISTING_DISK = 2, > ... > } > > An example of restore xml: > > <domainbackup> > <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> > <disks> > <disk name='vda'> > <target> > ... volume, file, or any possible target, (if absent, reuses > current > disk target) > </target> > </disk> > <disk name='vdb'> > <target> > ... > </target> > </disk> > </disks> > </domainbackup> > > -------------------------------------------------------------------------------- > int virBackupDelete(virBackupPtr backup, > unsigned int flags) > > Delete the specified backup > > - backup: a backup object > - flags: bitwise-OR of virBackupDeleteFlags > - returns: 0 on success or -1 on failure > > enum virBackupDeleteFlags { > > VIR_BACKUP_DELETE_ALL = 1, > VIR_BACKUP_DELETE_ALL_CHILDREN = 2, > ... > } > > -------------------------------------------------------------------------------- > int virBackupFree(virBackupPtr backup) > > Free the backup object. The backup itself is not modified. The data > structure > is freed and should not be used thereafter. > > - backup: a backup object > - returns: 0 on success or -1 on failure > > -------------------------------------------------------------------------------- > char* virBackupGetXMLDesc(virBackupPtr backup, > unsigned int flags) > > Provide an XML description of the backup > > - backup: a backup object > - flags: bitwise-OR of virDomainXMLFlags > - returns: a 0 terminated UTF-8 encoded XML instance, or NULL in case of > error. The caller must free() the returned value > > An example of dumped xml: > > <domainbackup> > <uuid>54c70d40-26bb-48cd-9639-b3a1e9adf650</uuid> > <creationTime>1270477159</creationTime> > <description>Any string</description> > <parent> > <uuid>d2ee2566-bef2-408f-ac7a-eb8004186074</uuid> > <parent> > <disks> > <disk name='vda'> > <target> > ... volume, file, or any possible target > </target> > </disk> > <disk name='vdb'> > <target> > ... > </target> > </disk> > </disks> > </domainbackup> > > -------------------------------------------------------------------------------- > virBackupPtr virBackupLookupByUUID(const char * uuid, > unsigned int flags) > > Try to lookup a backup by its UUID > > - uuid: a backup uuid > - flags: for future use > - returns: an opaque virBackupPtr on success or NULL on failure > > -------------------------------------------------------------------------------- > virDomainPtr virBackupGetDomain(virBackupPtr backup) > > Provide the domain pointer associated with a backup. The reference counter > on > the domain is not increased by this call > > - backup: a backup object > - returns: a domain object on success or NULL on failure > > -------------------------------------------------------------------------------- > int virBackupFinishJob(virBackupPtr backup, > unsigned int flags) > > Finish async job associated with specified backup > > - backup: a backup object > - flags: bitwise-OR of virBackupDeleteFlags > - returns: 0 on if the job for the specified backup was finished or > cancelled > successfully or -1 otherwise > > enum virBackupFinishJobFlags { > > VIR_BACKUP_FINISH_JOB_CANCEL = 1, > VIR_BACKUP_FINISH_JOB_CHECK = 2, > ... > } Probablt want a corresponding virBackupAbortJob too.

> > -------------------------------------------------------------------------------- > int virBackupGetParameters(virBackupPtr backup, > virTypedParameterPtr params, > int * nparams, > unsigned int flags) > > Get backup parameters. On input, @nparams gives the size of the @params > array; > on output, @nparams gives how many slots were filled with parameter > information, which might be less but will not exceed the input value > > - backup: a backup object > - params: pointer to backup parameter object (return value, allocated by > the > caller) > - nparams: pointer to number of blkio parameters; input and output > - flags: for future use > - returns: 0 on success or -1 on failure > > -------------------------------------------------------------------------------- > int virDomainSetDefaultTarget(virDomainPtr domain, > const char * xmlDesc, > unsigned int flags) > > Set default store target for backup, snapshot, managed saved state metadata > > - domain: a domain object > - xmlDesc: string containing an XML description of the backup > - flags: bitwise-OR of virDomainDeviceModifyFlags > - returns: 0 on success or -1 on failure Not sure I understand what this is doing, but probably not a big deal.

> Also, virDomainUndefineFlagsValues should be expanded with > VIR_DOMAIN_UNDEFINE_BACKUP and virDomainUndefineFlags implementations should > take into account this flag when undefining domains. Overall I think this looks like a reasonable API design to start implementing.

Regards, Daniel

On Thu, Apr 28, 2016 at 05:02:20PM +0300, Maxim Nestratov wrote: > 27.04.2016 15:31, Daniel P. Berrange пишет: > >> On Fri, Apr 08, 2016 at 11:47:29PM +0300, Maxim Nestratov wrote: >>> -------------------------------------------------------------------------------- >>> int virBackupRestore(const char * xmlDesc, >>> unsigned int flags) >> For the "managed backups" case it feels like this should be accepting >> a virDomainBackupPtr rather than an xmlDesc string. > Not sure because I would like to have an ability to restore not every disk a > backup has but sometimes selected only. How about doing virDomainBackupRestore(virDomainBackupPtr backip, const char *xmlDesc, unsigned int flags); Alow xmlDesc to be NULL by default, so it uses the built-in recorded XML unless you choose to override it with custom XML. This is similar to how we let people override XML during migration.

Regards, Daniel

On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote: > Hi all, > > It's been already quite a long time since qemu implemented QMP > "drive-backup" command to create block devices backups. Even more, > since qemu 2.4 there is a possibility to create incremental backups. > Though it is possible to backup all attached to a domain disk drives > by combining them into a single QMP transaction command, this way of > creating them, not to mention managing, remains inconvenient for an > end user of libvirt. Moreover, creating a single drive backup via QMP > interface isn't handy either. That said, it looks reasonable to > introduce a *new backup API* based on QMP "drive-backup" facilities. There's also the 'blockdev-backup' command, which seems similar in operation to 'drive-backup', but differs subtly. Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts target ID; while 'drive-backup' accept target drive name, otherwise, their operation look almost identical, and both commands use backup_start() (from qemu/blockdev.c). [Added John Snow in CC to correct me if I'm wrong.] For 'blockdev-backup' --------------------- -> { "execute": "blockdev-backup", "arguments": { "device": "src-id", "sync": "full", "target": "tgt-id" } } <- { "return": {} } Where 'tagert' in this case means: "the name of the backup target device. (json-string)" For 'drive-backup' ----------------- -> { "execute": "drive-backup", "arguments": { "device": "drive0", "sync": "full", "target": "backup.img" } } <- { "return": {} } Here, 'target' means: "the target of the new image. If the file exists, or if it is a device, the existing file/device will be used as the new destination. If it does not exist, a new file will be created. (json-string)" [...]

On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote: > On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote: >> Hi all, >> >> It's been already quite a long time since qemu implemented QMP >> "drive-backup" command to create block devices backups. Even more, >> since qemu 2.4 there is a possibility to create incremental backups. >> Though it is possible to backup all attached to a domain disk drives >> by combining them into a single QMP transaction command, this way of >> creating them, not to mention managing, remains inconvenient for an >> end user of libvirt. Moreover, creating a single drive backup via QMP >> interface isn't handy either. That said, it looks reasonable to >> introduce a *new backup API* based on QMP "drive-backup" facilities. > There's also the 'blockdev-backup' command, which seems similar in > operation to 'drive-backup', but differs subtly. > > Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts > target ID; while 'drive-backup' accept target drive name, otherwise, > their operation look almost identical, and both commands use > backup_start() (from qemu/blockdev.c). [Added John Snow in CC to > correct me if I'm wrong.] > No, you're right. Blockdev-backup can backup to an arbitrary device (which can be backed by a new file), but drive-backup will only accept a new file. I don't think blockdev-backup supports incremental backups just yet, but I don't think there's any reason it can't. (Looking at it: yeah, why have I not done that yet?...)

> For 'blockdev-backup' > --------------------- > > -> { "execute": "blockdev-backup", "arguments": { "device": "src-id", > "sync": "full", > "target": "tgt-id" } } > <- { "return": {} } > > > Where 'tagert' in this case means: > > "the name of the backup target device. (json-string)" > > > For 'drive-backup' > ----------------- > > -> { "execute": "drive-backup", "arguments": { "device": "drive0", > "sync": "full", > "target": "backup.img" } } > <- { "return": {} } > > Here, 'target' means: > > "the target of the new image. If the file exists, or if it is a > device, the existing file/device will be used as the new > destination. If it does not exist, a new file will be created. > (json-string)" > > [...] > > --js

Hi all, It's been already quite a long time since qemu implemented QMP "drive-backup" command to create block devices backups. Even more, since qemu 2.4 there is a possibility to create incremental backups. Though it is possible to backup all attached to a domain disk drives by combining them into a single QMP transaction command, this way of creating them, not to mention managing, remains inconvenient for an end user of libvirt. Moreover, creating a single drive backup via QMP interface isn't handy either. That said, it looks reasonable to introduce a *new backup API* based on QMP "drive-backup" facilities. Though we can start from a single simple function, allowing to create a disk backup by means of QMP "drive-backup" command, I'd like to discuss here the level of management libvirt could provide for backup operations. To begin with, here is the preliminary list of possible functions that I think make sense for libvirt API. virDomainCreateBackup - which creates a backup full/incremental of all/selected disks,

virListBackups - which lists all backups created for a particular domain/target, virRestoreBackup - which restores all/selected disks from a backup, virDeleteBackup - which deletes all/selected disks from a backup.