Hello Daniel,
Thank you for the response. Please also let me know why libvirtd says "No such file
or directory" even though the block images exist and are healthy.
Thanks,
Jayanth
________________________________
From: Daniel P. Berrangé <berrange(a)redhat.com>
Sent: Monday, November 20, 2023 2:36:21 PM
To: Jayanth Reddy <jayanthreddy5666(a)gmail.com>
Cc: users(a)lists.libvirt.org <users(a)lists.libvirt.org>
Subject: Re: Libvirt and Ceph: libvirtd tries to open random RBD images
On Mon, Nov 20, 2023 at 12:41:58PM +0530, Jayanth Reddy wrote:
Hello Users,
We're using libvirt with KVM and the orchestrator is Cloudstack. I raised
the issue already at Cloudstack at
https://github.com/apache/cloudstack/issues/8211 but appears to be at
libvirtd. Below is our environment & issue description:
It appears that one of our Cloudstack KVM clusters having 8 hosts is having
the issue. We have HCI on these 8 hosts and there are approximately 700+
VMs running. But strange enough, there are these logs like below on hosts.
Oct 25 13:38:11 hv-01 libvirtd[9464]: failed to open the RBD image
'087bb114-448a-41d2-9f5d-6865b62eed15': No such file or directory
Oct 25 20:35:22 hv-01 libvirtd[9464]: failed to open the RBD image
'ccc1168a-5ffa-4b6d-a953-8e0ac788ebc5': No such file or directory
Oct 26 09:48:33 hv-01 libvirtd[9464]: failed to open the RBD image
'a3fe82f8-afc9-4604-b55e-91b676514a18': No such file or directory
Oct 26 10:38:17 hv-01 libvirtd[9464]: End of file while reading data:
Input/output error
We've got DNS servers on which there is an`A` record resolving to all the
IPv4 Addresses of 8 monitors and there have not been any issues with the
DNS resolution. But the issue of "failed to open the RBD image
'ccc1168a-5ffa-4b6d-a953-8e0ac788ebc5': No such file or directory" gets
more weird because the VM that is making use of that RBD image lets say
"087bb114-448a-41d2-9f5d-6865b62eed15" is running on altogether different
host like "hv-06". On further inspection of that specific Virtual Machine,
it has been running on that host "hv-06" for more than 4 months or so
(looked at "Last updated" field). Fortunately, the Virtual Machine has no
issues and has been running since then.
This is an indication that you have a storage pool defined with the
RBD backend. Libvirt tries to enumerate all the images in the pool,
and has to open the RBD image to query its size. I won't be trying
to read/write the payload of the image.
With regards,
Daniel
--
|:
https://berrange.com -o-
https://www.flickr.com/photos/dberrange :|
|:
https://libvirt.org -o-
https://fstop138.berrange.com :|
|:
https://entangle-photo.org -o-
https://www.instagram.com/dberrange :|