I have a Fedora36 laptop which hosts VMs with RHEL7 using libvirt. One of the RHEL7 VMs, runs remote commands (as root) to 'start' another VM by way of my laptop. In other words, the following command is run: virsh --connect 'qemu+ssh://192.168.120.1/system' start beaker-test-vm1.beaker If I run non-remote version of the command on the laptop, it is successful. For example, virsh --connect qemu:///system start beaker-test-vm1.beaker <-- Successful on laptop. If I do a query like the following *(notice socket use)*, it is successful. virsh -d0 --connect 'qemu+ssh://192.168.120.1/system?*socket*=/var/run/libvirt/libvirt-sock-ro' domstate beaker-test-vm1.beaker Without socket, I get the following error: *error: failed to connect to the hypervisor* *error: End of file while reading data: Ncat: No such file or directory.: Input/output error* This does not work for 'start' because I believe this is a read-only socket since I see the error: error: Failed to start domain beaker-test-vm1.beaker error: operation forbidden: read only access prevents virDomainCreate When I look at my laptop, there is no /var/run/libvirt/libvirt-sock. So.....I've been wondering whether RHEL7 virsh/libvirt is compatible with Fedora36. Is there a work-around? I can't change the distros on my laptop or VMs. Carol

On Wed, Jul 27, 2022 at 01:18:00PM -0400, Carol Bouchard wrote: > I have a Fedora36 laptop which hosts VMs with RHEL7 using libvirt. One of > the RHEL7 VMs, runs remote commands (as root) to 'start' another VM by > way of my laptop. In other words, the following command is run: > virsh --connect 'qemu+ssh://192.168.120.1/system' start > beaker-test-vm1.beaker > If I run non-remote version of the command on the laptop, it is successful. > For example, > virsh --connect qemu:///system start beaker-test-vm1.beaker <-- Successful > on laptop. > > If I do a query like the following *(notice socket use)*, it is successful. > virsh -d0 --connect > 'qemu+ssh://192.168.120.1/system?*socket*=/var/run/libvirt/libvirt-sock-ro' > domstate beaker-test-vm1.beaker > > Without socket, I get the following error: > > *error: failed to connect to the hypervisor* > > *error: End of file while reading data: Ncat: No such file or directory.: > Input/output error* > > This does not work for 'start' because I believe this is a read-only socket > since I see the error: > error: Failed to start domain beaker-test-vm1.beaker > error: operation forbidden: read only access prevents virDomainCreate > > When I look at my laptop, there is no /var/run/libvirt/libvirt-sock. > So.....I've been wondering > whether RHEL7 virsh/libvirt is compatible with Fedora36. Is there a > work-around? I can't > change the distros on my laptop or VMs. Hello, since Fedora 35 libvirt has used the modular daemons architecture. What this means for you (and appears very confusing) is that when you try starting a VM locally on your F36 laptop, your virsh client doesn't connect to libvirt-sock anymore, there's a dedicated connection socket for each of the daemons now and instead will connect to virtqemud-sock. Now, old virsh clients like the one you have on your RHEL7 don't know about this and expect to connect to libvirt-sock instead. In order to create that socket and restore functionality for old clients you need to start and enable the virtproxyd.socket systemd unit which proxies old client connections to the new sockets we have. Why the virtproxyd socket isn't running by default unless you disabled it beats me, since: $ systemctl status virtproxyd Loaded: loaded (/usr/lib/systemd/system/virtproxyd.socket; enabled; vendor preset: enabled) ^^^here^^^ is set correctly after installation. Anyhow, just do: $ sudo systemctl enable --now virtproxyd.socket on your laptop and you're good to go Regards, Erik So, if you don't have the libvirt-sock created that means the virtproxyd.socket systemd unit isn't active. Just enable the socket and try again. Here, I simulated it for you with my VMs: VM1: $ cat /etc/os-release NAME="Fedora Linux" VERSION="36 (Thirty Six)" ...

On Wed, Jul 27, 2022 at 01:18:00PM -0400, Carol Bouchard wrote: > I have a Fedora36 laptop which hosts VMs with RHEL7 using libvirt. One of > the RHEL7 VMs, runs remote commands (as root) to 'start' another VM by > way of my laptop. In other words, the following command is run: > virsh --connect 'qemu+ssh://192.168.120.1/system' start > beaker-test-vm1.beaker > If I run non-remote version of the command on the laptop, it is successful. > For example, > virsh --connect qemu:///system start beaker-test-vm1.beaker <-- Successful > on laptop. > > If I do a query like the following *(notice socket use)*, it is successful. > virsh -d0 --connect > 'qemu+ssh:// 192.168.120.1/system?*socket*=/var/run/libvirt/libvirt-sock-ro' > domstate beaker-test-vm1.beaker > > Without socket, I get the following error: > > *error: failed to connect to the hypervisor* > > *error: End of file while reading data: Ncat: No such file or directory.: > Input/output error* This is peculiar, it suggests that /var/run/libvirt/libvirt-sock does not exist, while /var/run/libvirt/libvirt-sock-ro does exist. This ought to be an impossible situation in general. As Erik says, In Fedora 36 we have the moduler daemons, so these two sockets are provided by 'virtproxyd.socket' and 'virtproxyd-ro.socket' unit files, so make sure both of those are running. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Fri, Jul 29, 2022 at 07:49:16AM -0400, Carol Bouchard wrote: > TY VM!!! > virtproxyd was disabled and I can assure you I didn't disable it. > > /run/libvirt/libvirt-sock now exists AND > the remote virsh actions were successful. > > Background on my fedora36 install. I did not do an upgrade. This > was a fresh install on a new laptop. I could see libvirt was > running so I assumed it was intact. I had enabled/disabled > libvirtd only because my remote virsh commands were not working. Enabling/disabling libvirtd probably interfered with virtproxyd, as they both want the same sockets. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Fri, Jul 29, 2022 at 08:19:04AM -0400, Carol Bouchard wrote: > The only reason I toggled libvirtd was because the remote virsh commands > failed and > I could see the socket didn't exist. That suggests to me that virtproxyd > wasn't up AND > it was configured at disabled. Note, that it is important to distinguish the service from the sockets. In Fedora 36, not only are we using modular daemons by default, but we also use socket activation. In a fresh install, you should only see the following services enabled virtqemud.service virtxend.service virtlxcd.service Those are needed becuase of requiremenmt todo VM auto-start on host bootup. For everything else we only expect sockets enabled: virtproxyd.socket virtinterfaced.socket virtnetworkd.socket virtnodedevd.socket virtnwfilterd.socket virtsecretd.socket virtstoraged.socket IOW, virtproxyd.service should not be enabled - it is autostarted when someone connects to virtproxyd.socket. And nothing related to libvirtd.service or libvirtd.socket should be enabled With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Aug 01, 2022 at 07:42:51AM -0400, Carol Bouchard wrote: > These services do not exist > > [cbouchar@ibm-p8-rhevm-17-fsp cbouchar-devel]$ systemctl status > virtxend.service > Unit virtxend.service could not be found. > [cbouchar@ibm-p8-rhevm-17-fsp cbouchar-devel]$ systemctl status > virtxcd.service > Unit virtxcd.service could not be found. That's fine if you've not installed that piece of functionality. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

On Mon, Aug 01, 2022 at 08:41:02AM -0400, Carol Bouchard wrote: > I believe I've found what has messed things up. I've run beaker-in-a-box > ansible scripts > which includes the following role: > https://github.com/beaker-project/beaker-in-a-box/blob/master/roles/libvi... > as well as a few other changes. That needs to be changed to not touch the .services at all, and instead deal with .sockets. And also not libvirtd, for RHEL >= 9 or Fedora >= 35 > > So I found these instructions for installing libvirt but it seems obsolete > for fedora36 > https://computingforgeeks.com/how-to-install-kvm-on-fedora/ > Do you have a list of install steps for the various fedoras? The install steps haven't changed. All that's different is what systemd units are active by default. For a fresh install the right thing should "just work" out of the box, due to systemd presets. The problem is this ansible script is trying to manually start systemd services, which ignores the fact they shouldn't be running, as they are socket activated, and also doesnt' take into account libvirtd is no longer used on new Fedora/RHEL > I do have things working with info you and Martin Kletzander provided me > but would like to see > a formal list in case there is something else that comes up. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|