Re: [libvirt-users] systemctl libvirt-guests.service fails to start during host boot

Hi Andrea, Yep exactly, all software / kernel are in latest version I have tried and as you expected it didn't changed anything, same issue. I made more test and it is very reproductible, let me give you my steps to reproduce it : -I shutdown -r now my host. -when it restarted, first thing to check is : systemctl status libvirt-guests.service, I got : libvirt-guests.service - Suspend Active Libvirt Guests Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2016-09-15 17:40:07 CEST; 1min 47s ago Docs: man:libvirtd(8) http://libvirt.org Main PID: 329 (code=exited, status=1/FAILURE) Sep 15 17:40:07 libvirt-guests.sh[329]: Resuming guest : error: failed to connect to the hypervisor Sep 15 17:40:07 libvirt-guests.sh[329]: error: no valid connection Sep 15 17:40:07 libvirt-guests.sh[329]: error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory Sep 15 17:40:07 libvirt-guests.sh[329]: Resuming guest : error: failed to connect to the hypervisor Sep 15 17:40:07 libvirt-guests.sh[329]: error: no valid connection Sep 15 17:40:07 libvirt-guests.sh[329]: error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Main process exited, code=exited, status=1/FAILURE Sep 15 17:40:07 systemd[1]: Failed to start Suspend Active Libvirt Guests. Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Unit entered failed state. Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Failed with result 'exit-code'. Even if the service is in FAILED mode, my guests are started correctly. If I do a systemctl restart libvirt-guests.service, everything is good : libvirt-guests.service - Suspend Active Libvirt Guests Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; enabled; vendor preset: disabled) Active: active (exited) since Thu 2016-09-15 17:42:01 CEST; 9min ago Docs: man:libvirtd(8) http://libvirt.org Main PID: 605 (code=exited, status=0/SUCCESS) Sep 15 17:42:01 systemd[1]: Starting Suspend Active Libvirt Guests... Sep 15 17:42:01 systemd[1]: Started Suspend Active Libvirt Guests. Now if I do a shutdown -r now again, I can see both in my libvirt logs + in my guests log that an ACPI commands has been sent from libvirt and my guests shutdown gently, so everything is ok. When I start again, and try to do a systemctl status libvirt-guests.service, I got the same FAILED status as before.... I made a research on my logs : Sep 15 17:39:29 libvirt-guests.sh[1164]: Shutting down guests on default URI... Sep 15 17:39:29 libvirt-guests.sh[1164]: Starting shutdown on guest: ape Sep 15 17:39:30 libvirt-guests.sh[1164]: Waiting for guest ape to shut down, 30 seconds left Sep 15 17:39:35 libvirt-guests.sh[1164]: Waiting for guest ape to shut down, 25 seconds left Sep 15 17:39:36 libvirtd[326]: internal error: End of file from monitor Sep 15 17:39:36 libvirtd[326]: Cannot write data: Broken pipe Sep 15 17:39:36 libvirt-guests.sh[1164]: Shutdown of guest ape complete. Sep 15 17:39:36 libvirt-guests.sh[1164]: Starting shutdown on guest: om Sep 15 17:39:37 libvirt-guests.sh[1164]: Waiting for guest om to shut down, 30 seconds left Sep 15 17:39:40 libvirtd[326]: internal error: End of file from monitor Sep 15 17:39:40 libvirtd[326]: Cannot write data: Broken pipe Sep 15 17:39:40 libvirt-guests.sh[1164]: Shutdown of guest om complete. Sep 15 17:40:07 libvirt-guests.sh[329]: Resuming guests on default URI... Sep 15 17:40:07 libvirt-guests.sh[329]: Resuming guest : error: failed to connect to the hypervisor Sep 15 17:40:07 libvirt-guests.sh[329]: error: no valid connection Sep 15 17:40:07 libvirt-guests.sh[329]: error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory Sep 15 17:40:07 libvirt-guests.sh[329]: Resuming guest : error: failed to connect to the hypervisor Sep 15 17:40:07 libvirt-guests.sh[329]: error: no valid connection Sep 15 17:40:07 libvirt-guests.sh[329]: error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Main process exited, code=exited, status=1/FAILURE Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Unit entered failed state. Sep 15 17:40:07 systemd[1]: libvirt-guests.service: Failed with result 'exit-code'. Sep 15 17:40:07 libvirtd[325]: libvirt version: 2.1.0 Sep 15 17:40:07 libvirtd[325]: hostname: oo.localdomain Sep 15 17:40:07 libvirtd[325]: Cannot check dnsmasq binary /sbin/dnsmasq: No such file or directory Sep 15 17:40:07 libvirtd[325]: direct firewall backend requested, but /sbin/ebtables is not available: No such file or directory Sep 15 17:40:07 libvirtd[325]: internal error: Failed to initialize a valid firewall backend Sep 15 17:40:07 libvirtd[325]: internal error: Failed to find path for dmidecode binary Sep 15 17:40:07 libvirtd[325]: internal error: Failed to find path for dmidecode binary Sep 15 17:40:08 virtlogd[421]: libvirt version: 2.1.0 We see the same error as systemctl status plus some error like " internal error: End of file from monitor" and others type. Don't know if I have anything to take a look there ? Finally, I also notice some very strange thing: If I stop the service with systemctl stop libvirt-guests.service it tooks a while to execute the stop command and fails. Sep 15 18:00:44 libvirt-guests.sh[978]: Starting shutdown on guest: ape Sep 15 18:00:45 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 30 seconds left Sep 15 18:00:50 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 25 seconds left Sep 15 18:00:55 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 20 seconds left Sep 15 18:01:00 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 15 seconds left Sep 15 18:01:06 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 10 seconds left Sep 15 18:01:11 libvirt-guests.sh[978]: Waiting for guest ape to shut down, 5 seconds left Sep 15 18:01:15 libvirt-guests.sh[978]: Shutdown of guest ape failed to complete in time. So my guests are not able to shutdown properly but when I shutdown -r now my host I can see that my guests have been shutdown gently, I guess there is something related on my issue.... Many thanks for your help again