On 1/27/21 12:58 AM, Christian Ehrhardt wrote:
On Wed, Nov 4, 2020 at 7:47 AM Neal Gompa <ngompa13@gmail.com> wrote:
On Tue, Nov 3, 2020 at 9:26 PM Jim Fehlig <jfehlig@suse.com> wrote:
When restarting libvirt services and sockets *and* libvirt-guests.service is running, the latter will sometimes hang when trying to connect to libvirtd. Even though libvirt-guests has 'Wants=libvirtd.service' and 'After=libvirtd.service', we can see via journalctl that it is not shutdown before libvirtd when executing something like
systemctl try-restart libvirtd.service libvirtd.socket \ libvirtd-ro.socket virtlockd.service virtlockd.socket \ virtlogd.service virtlogd.socket virt-guest-shutdown.target
...
Adding 'Requires=libvirtd.service' to virt-guest-shutdown.target results in expected behavior
Hi Jim (and all others), This change is accepted since 6.10 [1] - while testing 7.0 for Debian I've found that this change regresses us bringing back bug 955216 [2] for us. Based on how the require statement works I think it is bringing the same issue to everyone (not just Debian/Ubuntu).
Nod. I verified the same behavior on SUSE :-(.
That issue is - once you restart libvirtd.service (common on upgrades) it will now also always cycle libvirt-guests.service through a restart. But cycling libvirt-guests.service means shutting down and starting all guests which suddenly makes upgrading libvirt very disruptive.
Agreed!
For the time being I've suggested we revert this change in Debian/Ubuntu, but I wonder if there could be something that ensures the ordering Jim wanted to achieve without hard-linking (via ) a libvirtd.service restart to also restart libvirt-guests.service?
Suggestions and/or comments very welcomed! I don't have other ideas ATM. In the meantime I've sent a patch to revert the commit https://www.redhat.com/archives/libvir-list/2021-January/msg01198.html Regards, Jim
--- log showing that libvirtd.service now restarts libvirt-guests as well ---
root@h-libvirt-netcheck-up:~# systemctl status libvirt-guests.service ● libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/lib/systemd/system/libvirt-guests.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2021-01-27 06:33:41 UTC; 1h 19min ago Docs: man:libvirtd(8) https://libvirt.org Main PID: 56753 (code=exited, status=0/SUCCESS) Tasks: 0 (limit: 38269) Memory: 0B CGroup: /system.slice/libvirt-guests.service
Jan 27 06:33:41 h-libvirt-netcheck-up systemd[1]: Starting Suspend/Resume Running libvirt Guests... Jan 27 06:33:41 h-libvirt-netcheck-up systemd[1]: Finished Suspend/Resume Running libvirt Guests.
root@h-libvirt-netcheck-up:~# systemctl restart libvirtd
root@h-libvirt-netcheck-up:~# systemctl status libvirt-guests.service ● libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/lib/systemd/system/libvirt-guests.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2021-01-27 07:52:57 UTC; 4s ago Docs: man:libvirtd(8) https://libvirt.org Process: 57626 ExecStart=/usr/lib/libvirt/libvirt-guests.sh start (code=exited, status=0/SUCCESS) Main PID: 57626 (code=exited, status=0/SUCCESS)
Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Starting Suspend/Resume Running libvirt Guests... Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Finished Suspend/Resume Running libvirt Guests.
You see new PID, and entries that it stopped/started all guests.
--- journal while the above restarts happened ---
Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopping Suspend/Resume Running libvirt Guests... Jan 27 07:52:57 h-libvirt-netcheck-up libvirt-guests.sh[57593]: Running guests on default URI: Jan 27 07:52:57 h-libvirt-netcheck-up libvirt-guests.sh[57602]: no running guests. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirt-guests.service: Succeeded. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopped Suspend/Resume Running libvirt Guests. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopped target Libvirt guests shutdown. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopping Libvirt guests shutdown. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopping Virtualization daemon... Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirtd.service: Succeeded. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirtd.service: Unit process 343 (dnsmasq) remains running after unit stopped. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirtd.service: Unit process 344 (dnsmasq) remains running after unit stopped. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Stopped Virtualization daemon. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirtd.service: Found left-over process 343 (dnsmasq) in control group while starting unit. Ignoring. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: libvirtd.service: Found left-over process 344 (dnsmasq) in control group while starting unit. Ignoring. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Starting Virtualization daemon... Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Started Virtualization daemon. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Reached target Libvirt guests shutdown. Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Starting Suspend/Resume Running libvirt Guests... Jan 27 07:52:57 h-libvirt-netcheck-up systemd[1]: Finished Suspend/Resume Running libvirt Guests.
[1]: https://gitlab.com/libvirt/libvirt/-/commit/f035f53baa2e5dc00b8e866e594672a9... [2]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955216