3:03 a.m.
Hi Everybody:
I am curious about how to re-create a disk image file from a running
guest when the original disk image file is accidentally deleted. This
came up recently during a test scenario but I couldn't figure out how to
do it.
Thanks,
Joe
3:53 a.m.
On Mon, Nov 05, 2012 at 11:03:11AM -0800, Joe Linoff wrote:
I am curious about how to re-create a disk image file from a running
guest when the original disk image file is accidentally deleted. This
came up recently during a test scenario but I couldn't figure out how to
do it.
If you have unlinked the image, but qemu-kvm (or whatever hypervisor
you are running) still has it open, you should be able to to recover
it by going to /proc/PID/fd and looking for a file descriptor number
that is a symlink to your deleted image. Just copy it to a safe
place. I would first run sync inside the guest and maybe suspend
it. It would be nice if you could switch it to single user mode with
all filesystems mounted read-only, but whatever you do, avoid
destroying the domain.
Or why not just attach a new disk to the guest and copy stuff inside
the guest?
3:54 a.m.
Hi Henrik:
Thank you. Both suggestions are excellent.
Have you noticed spam messages from this email list? I have gotten
several in the past couple of days.
Regards,
Joe
-----Original Message-----
From: Henrik Ahlgren [mailto:pablo@seestieto.com]
Sent: Monday, November 05, 2012 11:53 AM
To: Joe Linoff
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] re-create disk image file
On Mon, Nov 05, 2012 at 11:03:11AM -0800, Joe Linoff wrote:
I am curious about how to re-create a disk image file from a running
guest when the original disk image file is accidentally deleted. This
came up recently during a test scenario but I couldn't figure out how
to do it.
If you have unlinked the image, but qemu-kvm (or whatever hypervisor you
are running) still has it open, you should be able to to recover it by
going to /proc/PID/fd and looking for a file descriptor number that is a
symlink to your deleted image. Just copy it to a safe place. I would
first run sync inside the guest and maybe suspend it. It would be nice
if you could switch it to single user mode with all filesystems mounted
read-only, but whatever you do, avoid destroying the domain.
Or why not just attach a new disk to the guest and copy stuff inside the
guest?
4:25 a.m.
Hi Henrik:
When I tried to copy the file descriptor file a "Stale NFS file handle"
error was generated:
$ cp -f /proc/13196/fd/9 /tmp/guest.img
cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
I suppose that makes sense because the file was accidently deleted but
from the NFS server perspective it really isn't there. Do you know of
any way to work around it?
Or why not just attach a new disk to the guest and copy stuff inside
the guest?
I understand how that would work to recover individual files/directories
but I don't understand how I would be able to recover the entire image.
Can you explain?
Thanks,
Joe
-----Original Message-----
From: libvirt-users-bounces(a)redhat.com
[mailto:libvirt-users-bounces@redhat.com] On Behalf Of Joe Linoff
Sent: Monday, November 05, 2012 11:55 AM
To: Henrik Ahlgren
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] re-create disk image file
Hi Henrik:
Thank you. Both suggestions are excellent.
Have you noticed spam messages from this email list? I have gotten
several in the past couple of days.
Regards,
Joe
-----Original Message-----
From: Henrik Ahlgren [mailto:pablo@seestieto.com]
Sent: Monday, November 05, 2012 11:53 AM
To: Joe Linoff
Cc: libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] re-create disk image file
On Mon, Nov 05, 2012 at 11:03:11AM -0800, Joe Linoff wrote:
I am curious about how to re-create a disk image file from a running
guest when the original disk image file is accidentally deleted. This
came up recently during a test scenario but I couldn't figure out how
to do it.
If you have unlinked the image, but qemu-kvm (or whatever hypervisor you
are running) still has it open, you should be able to to recover it by
going to /proc/PID/fd and looking for a file descriptor number that is a
symlink to your deleted image. Just copy it to a safe place. I would
first run sync inside the guest and maybe suspend it. It would be nice
if you could switch it to single user mode with all filesystems mounted
read-only, but whatever you do, avoid destroying the domain.
Or why not just attach a new disk to the guest and copy stuff inside the
guest?
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users
4:37 a.m.
On 11/05/2012 01:25 PM, Joe Linoff wrote:
Hi Henrik:
When I tried to copy the file descriptor file a "Stale NFS file handle"
error was generated:
$ cp -f /proc/13196/fd/9 /tmp/guest.img
cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
I suppose that makes sense because the file was accidently deleted but
from the NFS server perspective it really isn't there. Do you know of
any way to work around it?
> Or why not just attach a new disk to the guest and copy stuff inside
the guest?
I understand how that would work to recover individual files/directories
but I don't understand how I would be able to recover the entire image.
Can you explain?
Wouldn't something like:
dd /dev/vda /dev/vdb
from within the guest do the trick of copying the entire image? But
that's risky if the guest is actively modifying /dev/vda at the same time.
What version of libvirt and qemu are you using? If at least 0.9.6 and
1.2 respectively, I think it should be possible to use 'virsh blockpull'
to resolve your conundrum, as follows:
# Create a new qcow2 file that wraps the deleted but still-open file
virsh snapshot-create-as $dom --no-metadata --disk-only \
--diskspec /tmp/guest.img,file=/tmp/guest.img1
# Pull the contents into the new file
virsh blockpull $dom /tmp/guest.img1 --wait --verbose
at which point, the new file /tmp/guest.img1 should contain everything
in use by the old deleted file, while no longer needing to keep the old
file around, so it will finally be safe to stop your guest.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
4:52 a.m.
Hi Eric:
Thank you for your suggestions. Here is what happened when I tried the
snapshot. I will try the dd a bit later.
VERSIONS
virsh: 0.9.10
qemu: 2.0.12
DOM: webdev
$ ls -l /proc/13196/fd/9
lrwx------. 1 qemu qemu 64 Nov 5 11:57 /proc/13196/fd/9 ->
/tools/vm/images/webdev.img (deleted)
$ cp /proc/13196/fd/9 /tools/vm/images/webdev.img
cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
$ virsh snapshot-create-as webdev \
--no-metadata \
--disk-only \
--diskspec
/tools/vm/images/webdev.img,file=/tools/vm/images/webdev.img1
error: unsupported configuration: no disk named
'/tools/vm/images/webdev.img'
I think that I must have some sort of typo but I am not familiar enough
with snapshots to see it.
-----Original Message-----
From: Eric Blake [mailto:eblake@redhat.com]
Sent: Monday, November 05, 2012 12:37 PM
To: Joe Linoff
Cc: Henrik Ahlgren; libvirt-users(a)redhat.com
Subject: Re: [libvirt-users] re-create disk image file
On 11/05/2012 01:25 PM, Joe Linoff wrote:
Hi Henrik:
When I tried to copy the file descriptor file a "Stale NFS file
handle"
error was generated:
$ cp -f /proc/13196/fd/9 /tmp/guest.img
cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
I suppose that makes sense because the file was accidently deleted but
from the NFS server perspective it really isn't there. Do you
know of
any way to work around it?
> Or why not just attach a new disk to the guest and copy stuff inside
the guest?
I understand how that would work to recover individual
files/directories but I don't understand how I would be able to
recover the
entire image.
Can you explain?
Wouldn't something like:
dd /dev/vda /dev/vdb
from within the guest do the trick of copying the entire image? But
that's risky if the guest is actively modifying /dev/vda at the same
time.
What version of libvirt and qemu are you using? If at least 0.9.6 and
1.2 respectively, I think it should be possible to use 'virsh blockpull'
to resolve your conundrum, as follows:
# Create a new qcow2 file that wraps the deleted but still-open file
virsh snapshot-create-as $dom --no-metadata --disk-only \
--diskspec /tmp/guest.img,file=/tmp/guest.img1
# Pull the contents into the new file
virsh blockpull $dom /tmp/guest.img1 --wait --verbose
at which point, the new file /tmp/guest.img1 should contain everything
in use by the old deleted file, while no longer needing to keep the old
file around, so it will finally be safe to stop your guest.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
5 a.m.
On 11/05/2012 01:52 PM, Joe Linoff wrote:
Hi Eric:
[please don't top-post on technical lists]
Thank you for your suggestions. Here is what happened when I tried the
snapshot. I will try the dd a bit later.
VERSIONS
virsh: 0.9.10
qemu: 2.0.12
Huh? The current upstream version of qemu is 1.2; I don't recognize the
2.0.12 numbering.
DOM: webdev
$ ls -l /proc/13196/fd/9
lrwx------. 1 qemu qemu 64 Nov 5 11:57 /proc/13196/fd/9 ->
/tools/vm/images/webdev.img (deleted)
$ cp /proc/13196/fd/9 /tools/vm/images/webdev.img
cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
$ virsh snapshot-create-as webdev \
--no-metadata \
--disk-only \
--diskspec
/tools/vm/images/webdev.img,file=/tools/vm/images/webdev.img1
error: unsupported configuration: no disk named
'/tools/vm/images/webdev.img'
What does this say?
virsh domblklist webdev
That is the name of the disk where you are creating the snapshot. The
--diskspec line needs two pieces of information: a designation for the
current file being snapshotted (either the absolute path you used in
your xml, or the shorthand name like 'vda' - valid names are found by
the domblklist command); and the name of the new file to create the
snapshot as.
The fact that your error message complains about "no disk named
'/tools/vm/images/webdev.img'" says that you probably didn't spell the
file the same as it was named in the XML; seeing the domblklist command
will make it clear what name you should be using instead.
# Pull the contents into the new file
virsh blockpull $dom /tmp/guest.img1 --wait --verbose
For the blockpull command, the same usage applies - you name the disk by
its path (but note that you just changed the path as a result of the
snapshot-create-as command; as another domblklist would prove) or by its
device name, such as vda.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
5:19 a.m.
[please don't top-post on technical lists]
Okay, sorry about that.
>
> Thank you for your suggestions. Here is what happened when I tried
the
> snapshot. I will try the dd a bit later.
>
> VERSIONS
> virsh: 0.9.10
> qemu: 2.0.12
Huh? The current upstream version of qemu is 1.2; I don't
recognize
the
2.0.12 numbering.
Sorry about that. It was a typo. The version seems to be 0.12.1.2. from
running
$ /usr/libexec/qemu-kvm --version
QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c)
2003-2008 Fabrice Bellard
>
> DOM: webdev
>
> $ ls -l /proc/13196/fd/9
> lrwx------. 1 qemu qemu 64 Nov 5 11:57 /proc/13196/fd/9 ->
> /tools/vm/images/webdev.img (deleted)
>
> $ cp /proc/13196/fd/9 /tools/vm/images/webdev.img
> cp: cannot stat `/proc/13196/fd/9': Stale NFS file handle
>
> $ virsh snapshot-create-as webdev \
> --no-metadata \
> --disk-only \
> --diskspec
> /tools/vm/images/webdev.img,file=/tools/vm/images/webdev.img1
> error: unsupported configuration: no disk named
> '/tools/vm/images/webdev.img'
What does this say?
virsh domblklist webdev
Target Source
------------------------------------------------
vda /var/lib/libvirt/images/webdev.img
hdc -
That is the name of the disk where you are creating the snapshot.
The --diskspec line needs two pieces of information: a designation
for the current file being snapshotted (either the absolute path
you used in your xml, or the shorthand name like 'vda' - valid
names are found by the domblklist command); and the name of the
new file to create the snapshot as.
The fact that your error message complains about "no disk named
'/tools/vm/images/webdev.img'" says that you probably didn't spell
the file the same as it was named in the XML; seeing the domblklist
command will make it clear what name you should be using instead.
I see. Thank you for the clear explanation. It appears that the file
name in the XML and the file name referenced by the fd are different:
/var/lib/libvirt/images/webdev.img (XML, link, domblklist)
/tools/vm/images/webdev.img (filer, fd reference)
I modified the command to use /var/lib/libvirt/images/webdev.img but it
had a different problem:
# virsh snapshot-create-as webdev \
--no-metadata \
--disk-only \
--diskspec
/var/lib/libvirt/images/webdev.img,file=/tools/vm/images/webdev.img1
error: operation failed: Failed to take snapshot: unknown command:
'snapshot_blkdev'
I don't understand what the message is trying to tell me. Why would
snapshot_blkdev not be available? Perhaps the version I have is too old?
>
> # Pull the contents into the new file
> virsh blockpull $dom /tmp/guest.img1 --wait --verbose
For the blockpull command, the same usage applies - you name the disk
by its path (but
note that you just changed the path as a result of the
snapshot-create-as command; as
another domblklist would prove) or by its > device name, such as
vda.
I see.
BTW, I tried the dd option with another guest and it seemed to work.
Thank you,
Joe
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
5:39 a.m.
On 11/05/2012 02:19 PM, Joe Linoff wrote:
> Huh? The current upstream version of qemu is 1.2; I don't
recognize
the
> 2.0.12 numbering.
Sorry about that. It was a typo. The version seems to be 0.12.1.2. from
running
$ /usr/libexec/qemu-kvm --version
QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright (c)
2003-2008 Fabrice Bellard
That sounds like RHEL 6 (or CentOS).
I see. Thank you for the clear explanation. It appears that the file
name in the XML and the file name referenced by the fd are different:
/var/lib/libvirt/images/webdev.img (XML, link, domblklist)
/tools/vm/images/webdev.img (filer, fd reference)
Sounds like one resolved symlinks while the other did not. Maybe it's
worth enhancing libvirt to allow all names that resolve to the same
file, rather than just the exact spelling used? At any rate, we solved
this.
I modified the command to use /var/lib/libvirt/images/webdev.img but it
had a different problem:
# virsh snapshot-create-as webdev \
--no-metadata \
--disk-only \
--diskspec
/var/lib/libvirt/images/webdev.img,file=/tools/vm/images/webdev.img1
error: operation failed: Failed to take snapshot: unknown command:
'snapshot_blkdev'
I don't understand what the message is trying to tell me. Why would
snapshot_blkdev not be available? Perhaps the version I have is too old?
Alas, your qemu is too old. Red Had did not ship live disk snapshots in
RHEL 6.3; to get that functionality, you need RHEV 3.0. Wish I had
better news, but at this point, an in-guest dd to a hot-plugged
alternative disk is the best I can think of, since there's nothing you
can do from the host side without qemu support.
BTW, I tried the dd option with another guest and it seemed to work.
Hope it works for your guest in question, then. Good luck.
--
Eric Blake eblake(a)redhat.com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
9:08 a.m.
That sounds like RHEL 6 (or CentOS).
Yup, CentOS 6.3. I should have mentioned that.
Sounds like one resolved symlinks while the other did not. Maybe
it's
worth enhancing libvirt to allow all names that resolve to the same
file,
rather than just the exact spelling used? At any rate, we solved
this.
Maybe but I don't think it is a high priority. Libvirt is pretty awesome
the way it is.
Yup, it is solved, thank you again.
Alas, your qemu is too old. Red Had did not ship live disk snapshots
in RHEL 6.3;
to get that functionality, you need RHEV 3.0. Wish I had better
news,
but at this
point, an in-guest dd to a hot-plugged alternative disk is the best I
can think of,
since there's nothing you can do from the host side without qemu
support.
That's unfortunate but at least I understand why.
Hope it works for your guest in question, then. Good luck.
Thank you. This mailing list has been great.
Regards,
Joe
4323
days inactive
4324
days old
9 comments
3 participants
participants (3)
-
Eric Blake -
Henrik Ahlgren -
Joe Linoff