- Users - libvirt List Archives

TLS live migrations and cgroups (cpuset)
by Giovanni Tirloni 30 May '24

30 May '24

Hello, I have got a situation where a live migration (TLS) is failing because the virtual machine has the following configuration: <vcpu placement='static' cpuset='0-1,3,5,7-47,49,51,53,55-95'>2</vcpu> This is in place because the host is shared with some other application with strict requirements and so the `machine.slice` was limited to these CPUs. During a live migration, libvirtd (running as root) seems to spawn a new rpc-libvirtd process with the same uid/gid defined in qemu.conf (non-root). It then fails to write to `cpuset.cpus`: virNetClientProgramDispatchError:172 : Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d80\x2dinstance\x2d0000007c.scope/libvirt/emulator/cpuset.cpus': Permission denied Here are the running processes during the live migration: UID PID GID PPID CMD root 2639801 0 2639361 /usr/sbin/libvirtd --listen 12345 3033179 109 2638878 /usr/sbin/libvirtd --listen 12345 3033179 109 2638878 [rpc-libvirtd] (processes with uid=12345 are spawned during the migration. gid=109 is `kvm`) Changing group membership and file permissions has no effect because the file/directory permissions in machine.slice are not inherited inside the machine scope (a workaround was setting regular files as 664 and making the non-root UID of rpc-libvirtd be a member of the root group). I did not try to change the user/group in qemu.conf for fear of breaking other parts of the system or sabotaging security (e.g. by setting it to root/root for qemu-system-x86_64). The systemd scope for the machine has "Delegate=yes" so systemd expects the manager process to fully own the cgroups configuration. So looking at libvirtd, I cannot find a configuration setting that would either change the UID/GID of that rpc-libvirtd process or how to tweak the cgroups file permissions. Any insights? Environment: * Ubuntu 22.04 x86_64 * systemd 249.11 * libvirt 8.0.0

1 0

How can I create a VM in virt-manager that can access the internet through the host but can't even see the host or other devices on the LAN?
by imthenachoman＠gmail.com 22 May '24

22 May '24

I want to be able to create multiple VMs for testing purposes -- questionable websites, files that could be infected with a virus, etc. I want to make sure these VMs should be able to access the internet (through the host) but they should not be able to access the host or any other device on the LAN. I tried `NAT` mode but that let's me access the host and other devices on the LAN. I tried `Isolated` mode but that doesn't give me internet access. I don't want to run a second VM and route traffic through it. I want to avoid creating FW rules on the host or VMs because I'll be creating and destroying VMs all the time. And I think I need to use some kind of NAT because I don't want my router to know/see the VMs as clients. What do I need to do?

7 10

How to live migrate with copy-storage-inc and zfs snapshot replication?
by Dan Vatca 21 May '24

21 May '24

Hello everybody, I am trying to understand how I can create a setup with two libvirt + kvm + zfs systems, in which I can: 1. Create a domain on sys-a, with storage space on a zfs filesystem. This filesystem stores the qcow2 image, any qcow2 disk snapshots, memory dumps and domain XML definition. 2. Take consistent ZFS snapshots every 15 minutes, and sync them to sys-b. The snapshots will be replicated using syncoid. These will be using scripts to quiesce the VM. 3. Given that most of the disk is synced (at most 15 minutes ago), what are the things I need to ensure (libvirt snapshots, memory snapshot, etc), so that I can do a live migration while copying just the incremental changes since the last snapshot I already transferred? I was trying to use virsh migrate —copy-storage-inc, however I could not find any working example that would copy just the incremental changes. I could find the theory of the snapshot backing chain, however —copy-storage-inc complains that it cannot replicate while there are snapshots present. (error: Requested operation is not valid: cannot migrate domain with 1 snapshots). I this scenario possible? Dan Vatca

2 1

Guests: Unable to get IP address
by Arun Mani J 21 May '24

21 May '24

Hey all. I'm exploring virt-manager, libvirt etc. I downloaded Debian 12 KDE ISO and when launched using virt-manager, it does not seem to connect to network. nmcli says it has failed to get an IP address. However, if I run the same ISO inside GNOME Boxes, then it is able to connect to network. I checked the XML configuration of both and found this: ```xml <interface type="user"> <mac address="52:54:00:ea:f9:69"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface> ``` However, the one created via virt-manager has this: ```xml <interface type="network"> <mac address="52:54:00:5d:c7:9e"/> <source network="default"/> <model type="virtio"/> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface> ``` What I'm not able to understand is, why does the user mode networking work fine but not network mode? My ultimate goal is to set up routed configuration, so one can directly SSH into the guests. Packages: - virt-manager - 4.1.0 (Debian) - libvirt0 - 10.3.0-3 (Debian) - GNOME Boxes - 46.0 (Flatpak) Please let me know if you need any other information. It is a pretty much clean installation with no extra configuration, so I don't think I messed up anything. Thank you! Arun Mani J

2 1

stats : active memory / disk latency
by Peter Quiring 21 May '24

21 May '24

Two stats that VMWare offers that I can't seem to find in libvirt is "active" memory and disk latency. Any idea how I could extract these stats? libvirt offers memory stats such as maximum, current, unused, usable, etc. but these are all 100% or 0% of what the VM has available. Looking for an "active" value. /proc/memstat has similar stats within a Linux VM but need to get from libvirt. Disk latency is also sometimes critical information. Thanks, -- Peter Quiring

2 1

qemu guest agent with openbsd
by Christian Haumesser 13 May '24

13 May '24

Hello, I have an OpenBSD 7.5 guest running on Debian bookworm with libvirt (9.0) and qemu (7.2). I'd like to use the qemu guest agent in this guest, but I can't seem to figure out how to craft the libvirt xml to expose the serial port in a way that OpenBSD and libvirt can use it together (or if it’s even possible). Per the libvirt docs <https://wiki.libvirt.org/Qemu_guest_agent.html>, I’m currently using: > <channel type='unix'> > <source mode='bind'/> > <target type='virtio' name='org.qemu.guest_agent.0'/> > </channel> But OpenBSD doesn’t directly support the virtio console driver. Consistent with this 2020 thread <https://undeadly.org/cgi?action=article;sid=20200514073852> from the OpenBSD ports making list, I see the following message in my guest’s dmesg output: > virtio5 at pci0 dev 10 function 0 "Qumranet Virtio Console" rev 0x00 > virtio5: no matching child driver; not configured Thanks to this libvir-list thread <https://listman.redhat.com/archives/libvir-list/2015-October/120250.html> from 2015, I’ve realized that I can manually expose an ISA serial console that will allow me to connect to the guest agent using native qemu tooling; but there doesn’t seem to be a way to create a channel that libvirt can communicate over without virtio console support in the guest. The thread discusses a couple of approaches to resolving this issue with code changes, but it seems like the discussion stalled out before anything happened. Am I missing something or is it still impossible? Thanks!

2 2

jfKVM - new app using libvirt
by Peter Quiring 06 May '24

06 May '24

jfKVM is a new web-based VM management system based on libvirt. jfkvm.sf.net https://pquiring.github.io/javaforce/projects/jfkvm/docs/help.html Thanks, -- Peter Quiring

1 1

What's wrong with internal VM snapshots?
by Marius Vollmer 06 May '24

06 May '24

Hi! I am trying to improve the support for VM snapshots in the Cockpit web console, and I am afraid I have questions... We have been asked to prefer the "external" over the "internal" snapshot format, at least on RHEL. I haven't yet figured out why, and consequently I am struggling with deciding how hard the Cockpit UI should push people towards external snapshots. So, what's wrong with internal snapshots? I heard they are "unreliable", but how so in detail? Does the data structure inside the qcow2 files get corrupted easily? Do they behave poorly when the snapshot process runs out of disk in the middle? That sort of thing would help me a lot to figure out what Cockpit should be doing on platforms other than RHEL. And how well (or how soon) can external snapshots be expected to work? I have severely messed up my libvirt state a couple of times while playing around with them, and my confidence in them right now isn't great. :-) Are you surprised by this? Or are external snapshots not yet considered ready? (Most recent example from my experiments: Deleting a full system snapshot of a paused machine fails with "internal error: unable to execute QEMU command 'block-commit': Block node is read-only". Reverting to it works and after that it can also be deleted, all while the VM is paused.) Thanks!

2 5

Error Cannot acquire state change lock from remoteDispatchDomainMigratePrepare3Params during live migration of domains
by Christian Rohmann 25 Apr '24

25 Apr '24

Hallo libvirt-users! we observe lock-ups / timeouts with in prometheus-libvirt-exporter (https://github.com/inovex/prometheus-libvirt-exporter) when libvirt is live-migrating domains: > Timed out during operation: cannot acquire state change lock (held by > monitor=remoteDispatchDomainMigratePrepare3Params) All of the source code can be found at: https://github.com/inovex/prometheus-libvirt-exporter/blob/master/pkg/expor…. Basically the error happens when DomainMemoryStats or other operational domain info is queried via the libvirt socket. 1) We are actually using the read-only socket at '/var/run/libvirt/libvirt-sock-ro', so there should not be any locking required. Is there any way to not run into lock contention, like running a request with some "nolock" indication? 2) This being reported as timeout waiting for the lock, what is the timeout and would waiting a bit longer help? Or is the lock active during the whole time a domain live migration is running? 3) Is this in any way related to the type of migration? Tunneled vs. native (https://libvirt.org/migration.html)? 4) Is there any indication that we could use to skip those domains (or certain queries)? The same issue was actually previously reported for another implementation of a Prometheus exporter (https://github.com/kumina/libvirt_exporter/issues/33) Currently the exporter locks up or throws the mentioned timeout errors during the the migration of 200 domains, 5 at a time. It would be awesome to find a way to make this work as smooth as possible, even during live migrations! I am thankful for any insights into how the libvirt socket, the various calls, the locking mechanisms or live migration modes work! Regards Christian

5 10

How to configure the default connection for virsh
by Nir Soffer 18 Apr '24

18 Apr '24

We are using minikube vms, which require adding the user to to libvirt group[1]. We use `virsh -c qemu:///system` for debugging these vms or related libvirt networks. Using virsh without specifying the '-c' argument is a common mistake that leads to trouble, for example creating the libvirt default network incorrectly. I wonder if it is possible to configure virsh so it uses -c qemu:///system by default? We know that this is not great, but we don't know a better way to use minikube with the kvm2 driver. The minikube vms are started for creating a development environment and during CI, and they cannot require a password or any complicated setup. [1] https://github.com/kubernetes/minikube/issues/3467 Nir

2 2