On Wed, Jun 22, 2016 at 08:59:24AM +0200, Markus Ellinger wrote:
Hi everyone,
we are suddenly having a problem with executing our backup jobs. For a long time, we have used a shell script which contains the following code to backup all our virtual machines:
for domain in Testserver Faktura Fileserver Gitolite Jenkins Nexus SimpleHelp VpnGateway Wiki; do echo -n "$(date +"%Y-%m-%d %H:%M:%S") starting backup for vm ${domain} ... " >> ${vmlog} virsh dumpxml --security-info ${domain} > ${vmdir}/${domain}.xml virsh undefine ${domain} >> ${vmlog} virsh blockcopy ${domain} /var/lib/libvirt/images/${domain}.img ${vmdir}/${domain}.img --wait --finish
${vmlog} virsh define ${vmdir}/${domain}.xml >> ${vmlog} done
[...]
root@gfii-host:~# virsh undefine Gitolite error: Failed to undefine domain Gitolite error: Requested operation is not valid: cannot undefine transient domain
root@gfii-host:~# virsh blockcopy Gitolite /var/lib/libvirt/images/Gitolite.img /tmp/test-blockcopy-gitolite.img --wait --finish error: block copy still active: disk 'vda' already in active block job
root@gfii-host:~# virsh blockjob Gitolite /var/lib/libvirt/images/Gitolite.img No current block job for /var/lib/libvirt/images/Gitolite.img
I was going to suggest that in the even if you do see an active block job here, then you could've aborted it via: $ virsh blockjob Gitolite /path/to/disk/ --abort But you say there's no active block operation. (I doubt it will help, but you might want to supply "--info" to that `virsh blockjob` query. From the manual: "In --info mode, the active job information on the specified disk will be printed.")
root@gfii-host:~# virsh define /var/local/backup/vms/2016-06-22T013001/Gitolite.xml error: Failed to define domain from /var/local/backup/vms/2016-06-22T013001/Gitolite.xml error: block copy still active: domain has active block job
Although I've seen some bug reports (and were fixed) in the past of this error during different block operations, but I can't pin-point the exact reason here why you're seeing it. You might want to enable libvirt logging filters (in /etc/libvirt/libvirtd.conf) to get some more useful details: log_filters="1:libvirt 1:qemu 1:conf 1:security 3:event 3:json 3:file 3:object 1:util" log_outputs="1:file:/var/log/libvirt/libvirtd.log" (Don't forget to restart libvirt daemon before performing your test.) While we're at it, here's another another way to perform live backups (which is slightly more efficient), if it is of any help: http://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit
Of course we tried to start/stop the virtual machines, rebooted the whole host multiple times etc., but the problem comes back every night.
The machine is a Debian Wheezy machine with current updates. We are using the qemu-kvm package from wheezy-backports to enable blockcopy support.
Specifying explicit versions will be convenient. -- /kashyap