On Tue, Feb 22, 2022 at 06:23:13PM +0300, Дмитрий wrote:

Updated libvirt with:

# dnf install @virt

Not its 2.module_el8.6.0+1087+b42c8331

Testing:

# virt-df -d kvm3551

But still, issue remains. Here how it looks like in /proc (all these socket/pipe symlinks are broken):

# ls -lah /proc/$(pgrep -f libvirtd)/fd/*

lrwx------ 1 root root 64 фев 22 18:17 /proc/3600693/fd/49 -> 'socket:[99301766]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/5 -> 'socket:[99234506]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/51 -> 'socket:[99295594]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/52 -> 'socket:[99288645]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/53 -> 'socket:[99296279]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/54 -> 'socket:[99289772]' lrwx------ 1 root root 64 фев 22 18:17 /proc/3600693/fd/55 -> 'socket:[99300744]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/56 -> 'socket:[99284626]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/57 -> 'socket:[99294777]' lrwx------ 1 root root 64 фев 22 18:17 /proc/3600693/fd/58 -> 'socket:[99297768]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/59 -> 'socket:[99279867]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/6 -> 'socket:[99282298]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/60 -> 'socket:[99286467]' l-wx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/7 -> /run/libvirtd.pid lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/70 -> 'socket:[99286794]' lr-x------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/8 -> 'pipe:[99282299]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/81 -> 'socket:[99281902]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/82 -> 'socket:[99288059]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/84 -> 'socket:[99285702]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/85 -> 'socket:[99300385]' lrwx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/88 -> 'socket:[99291046]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/89 -> 'socket:[99299347]' l-wx------ 1 root root 64 фев 22 18:13 /proc/3600693/fd/9 -> 'pipe:[99282299]' lrwx------ 1 root root 64 фев 22 18:15 /proc/3600693/fd/92 -> 'socket:[99284744]'

It is generally more useful to use 'lsof -p $LIBVIRT_PID', so you get readable names for sockets. Also compare before & after so you can identify which is the new "leaked" file. Also is that a one time leak, or is a new FD leaked for every virt-df command that is run ? Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On 2/22/22 16:36, Дмитрий wrote:

# lsof -p $(cat /run/libvirtd.pid) > run1 # virt-df -d kvm3586 # lsof -p $(cat /run/libvirtd.pid) > run2 # diff run1 run2 196a197

...

libvirtd 3618992 root 54u unix 0xffff902dc0c58440 0t0 99330533 type=STREAM

I wonder whether this is the glib event loop issue I've fixed a while ago. Symptoms look similar. https://gitlab.com/libvirt/libvirt/-/commit/5de203f8795d96229d2663e9ea1a24fb... Let me check if that's backported into CentOS and what versions. Michal

On 3/1/22 12:40, Дмитрий wrote:

Any update on this issue? Does anyone have a success on reproducing that? Should I upgrade/downgrade/sidegrade?

Additional info:

I am using CentOS 8 Stream, but with kernel

5.14.15-1.el8.elrepo.x86_64 #1 SMP Tue Oct 26 11:45:20 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux

I believe this was fixed in RHEL-8, by a RHEL-only patch, because in there glib backported a commit that broke our upstream workaround. And since CentOS 8 Stream is behind RHEL you will get the fix eventually (sorry I don't know the schedule). However, I don't think that the RHEL-only patch can be merged upstream, because then it would break distros with older glib that did not mangle it. I'm sorry, I don't have a better answer. Michal