9:07 a.m.
During a recent managed save operation I received the following error
message:
error: operation failed: domain save job: unexpectedly failed.
It turns out that I had run out of disk space. After a brief investigation I
discovered that libvirt_iohelper is exec'ed and is used to handle all
I/O during
a (Qemu) managed save operation. While iohelper appears to be set up to log
error conditions when they occur, for some reason the logging is getting
lost.
I'm hoping someone can help figure out why these errors are getting lost. It
would be nice to present a useful error message to the user when a
managed save
fails because of an I/O error.
I was able to work around this problem with a patch that bypasses iohelper
entirely but I doubt that is really the best thing to do. Does anyone
know why
these error messages are getting suppressed?
2:21 p.m.
On 06/24/2014 08:07 AM, Jason J. Herne wrote:
During a recent managed save operation I received the following
error
message:
error: operation failed: domain save job: unexpectedly failed.
It turns out that I had run out of disk space. After a brief
investigation I
discovered that libvirt_iohelper is exec'ed and is used to handle all
I/O during
a (Qemu) managed save operation. While iohelper appears to be set up to log
error conditions when they occur, for some reason the logging is getting
lost.
I'm hoping someone can help figure out why these errors are getting
lost.
What version of libvirt are you using, in case it is something that has
been improved in the meantime?
I think it's probably a fairly short patch, although maybe not simple to
write, to make sure that the iohelper logging setup is forced to write
to stderr (looks like that already happens), and then the caller
harvests stderr (this is probably the missing part).
Off-hand, it looks like src/fdstream.c is trying to catch stderr (if the
iohelper exits with non-zero status, then virFDStreamCloseInt replays
the buf read from errfd as part of its error message), while
src/util/virfile.c is listing the output as a warning but not feeding it
into an actual error report (and thus losing it from the logs if you
don't have logging turned on high enough to see warnings), in
virFileWrapperFdClose.
It
would be nice to present a useful error message to the user when a
managed save
fails because of an I/O error.
I was able to work around this problem with a patch that bypasses iohelper
entirely but I doubt that is really the best thing to do. Does anyone
know why
these error messages are getting suppressed?
At this point, the only reason it hasn't been fixed is because no one
has put forth the time to propose the fix. Patches are welcome.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
3:10 p.m.
On 06/24/2014 03:21 PM, Eric Blake wrote:
On 06/24/2014 08:07 AM, Jason J. Herne wrote:
> During a recent managed save operation I received the following error
> message:
>
> error: operation failed: domain save job: unexpectedly failed.
>
> It turns out that I had run out of disk space. After a brief
> investigation I
> discovered that libvirt_iohelper is exec'ed and is used to handle all
> I/O during
> a (Qemu) managed save operation. While iohelper appears to be set up to log
> error conditions when they occur, for some reason the logging is getting
> lost.
> I'm hoping someone can help figure out why these errors are getting
> lost.
What version of libvirt are you using, in case it is something that has
been improved in the meantime?
I tried this again today with a recent version pulled from git. The
following is the top commit.
commit b02fca79e858dc6e6d3209a44fe967f038ac6291
Date: Wed Jul 9 10:57:33 2014 +0200
I'll take another look at the code. Hopefully I can figure out why the
messages are getting lost.
Sorry for the delayed response.
--
-- Jason J. Herne (jjherne(a)linux.vnet.ibm.com)
3782
days inactive
3798
days old
2 comments
2 participants
participants (2)
-
Eric Blake -
Jason J. Herne