cpu vmx migration issue
by d tbsky
Hi:
I update our RHEL9 system to RHEL 9.4, which brings libvirt 10.0.
I try to calculate the cpu baseline for our two-node cluster with
command "virsh domcapabilities" then "virsh hypervisor-cpu-baseline
--migratable". the result has many cpu features begin with "vmx".
the test cluster has cpu "intel E3-1280 V3" and "intel I3-9100F".
when I try live migrate vm, it failed and told me "guest CPU doesn't
match specification: missing features:
vmx-apicv-register,vmx-apicv-vid,vmx-posted-intr".
at another cluster with cpu " Intel i5-2520M" and "Intel
i7-9750H" the migration works fine for the calculated cpu result.
although there are still many "vmx" cpu features in the result.
if I delete all these vmx features, then migration works fine for
both cluster, like old days.
I wonder what's the benefit to expose these vmx features to guest
if I don't do any nested virtualization.
is it ok the drop all these vmx cpu features for a guest?
thanks a lot for help!
3 months, 2 weeks
Re: Guests: Unable to get IP address
by Michal Prívozník
Please keep the list on CC so that others can benefit from the
conversation too.
On 5/21/24 14:34, Arun Mani J wrote:
> Thanks for the reply!
>
> I recreated the NAT network using the guide from https://wiki.libvirt.org/Networking.html.
>
> The configuration now reads as:
> <network>
> <name>default</name>
> <uuid>33376669-6292-44c9-b26b-9d2ee69826a7</uuid>
> <forward mode="nat">
> <nat>
> <port start="1024" end="65535"/>
> </nat>
> </forward>
> <bridge name="virbr0" stp="on" delay="0"/>
> <mac address="52:54:00:78:76:0f"/>
> <ip address="192.168.122.1" netmask="255.255.255.0">
> <dhcp>
> <range start="192.168.122.2" end="192.168.122.254"/>
> </dhcp>
> </ip>
> </network>
>
> There is a dhcp section but still I'm not able to get my guest working.
>
> I have dnsmasq-base package installed in my host. However I'm not sure how to see if that is running. Like I opened Systems Monitor and can't find any process with that name.
>
> So may be that's the problem?
Well, have you restarted the network? You need the 'virsh net-destroy
default && virsh net-start default' combo to restart it. After that you
should see dnsmasq process, e.g. via 'ps axf | grep dnsmasq'.
Michal
4 months
TLS live migrations and cgroups (cpuset)
by Giovanni Tirloni
Hello,
I have got a situation where a live migration (TLS) is failing because
the virtual machine has the following configuration:
<vcpu placement='static' cpuset='0-1,3,5,7-47,49,51,53,55-95'>2</vcpu>
This is in place because the host is shared with some other
application with strict requirements and so the `machine.slice` was
limited to these CPUs.
During a live migration, libvirtd (running as root) seems to spawn a
new rpc-libvirtd process with the same uid/gid defined in qemu.conf
(non-root). It then fails to write to `cpuset.cpus`:
virNetClientProgramDispatchError:172 : Unable to write to
'/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d80\x2dinstance\x2d0000007c.scope/libvirt/emulator/cpuset.cpus':
Permission denied
Here are the running processes during the live migration:
UID PID GID PPID CMD
root 2639801 0 2639361 /usr/sbin/libvirtd --listen
12345 3033179 109 2638878 /usr/sbin/libvirtd --listen
12345 3033179 109 2638878 [rpc-libvirtd]
(processes with uid=12345 are spawned during the migration. gid=109 is `kvm`)
Changing group membership and file permissions has no effect because
the file/directory permissions in machine.slice are not inherited
inside the machine scope (a workaround was setting regular files as
664 and making the non-root UID of rpc-libvirtd be a member of the
root group).
I did not try to change the user/group in qemu.conf for fear of
breaking other parts of the system or sabotaging security (e.g. by
setting it to root/root for qemu-system-x86_64).
The systemd scope for the machine has "Delegate=yes" so systemd
expects the manager process to fully own the cgroups configuration. So
looking at libvirtd, I cannot find a configuration setting that would
either change the UID/GID of that rpc-libvirtd process or how to tweak
the cgroups file permissions.
Any insights?
Environment:
* Ubuntu 22.04 x86_64
* systemd 249.11
* libvirt 8.0.0
4 months, 2 weeks
How can I create a VM in virt-manager that can access the internet
through
the host but can't even see the host or other devices on the LAN?
by imthenachoman＠gmail.com
I want to be able to create multiple VMs for testing purposes -- questionable websites, files that could be infected with a virus, etc.
I want to make sure these VMs should be able to access the internet (through the host) but they should not be able to access the host or any other device on the LAN.
I tried `NAT` mode but that let's me access the host and other devices on the LAN.
I tried `Isolated` mode but that doesn't give me internet access.
I don't want to run a second VM and route traffic through it.
I want to avoid creating FW rules on the host or VMs because I'll be creating and destroying VMs all the time.
And I think I need to use some kind of NAT because I don't want my router to know/see the VMs as clients.
What do I need to do?
4 months, 3 weeks
How to live migrate with copy-storage-inc and zfs snapshot replication?
by Dan Vatca
Hello everybody,
I am trying to understand how I can create a setup with two libvirt + kvm +
zfs systems, in which I can:
1. Create a domain on sys-a, with storage space on a zfs filesystem. This
filesystem stores the qcow2 image, any qcow2 disk snapshots, memory dumps
and domain XML definition.
2. Take consistent ZFS snapshots every 15 minutes, and sync them to sys-b.
The snapshots will be replicated using syncoid. These will be using scripts
to quiesce the VM.
3. Given that most of the disk is synced (at most 15 minutes ago), what are
the things I need to ensure (libvirt snapshots, memory snapshot, etc), so
that I can do a live migration while copying just the incremental changes
since the last snapshot I already transferred?
I was trying to use virsh migrate —copy-storage-inc, however I could not
find any working example that would copy just the incremental changes. I
could find the theory of the snapshot backing chain, however
—copy-storage-inc complains that it cannot replicate while there are
snapshots present. (error: Requested operation is not valid: cannot migrate
domain with 1 snapshots).
I this scenario possible?
Dan Vatca
4 months, 3 weeks
Guests: Unable to get IP address
by Arun Mani J
Hey all.
I'm exploring virt-manager, libvirt etc. I downloaded Debian 12 KDE ISO and when launched using virt-manager, it does not seem to connect to network. nmcli says it has failed to get an IP address.
However, if I run the same ISO inside GNOME Boxes, then it is able to connect to network.
I checked the XML configuration of both and found this:
```xml
<interface type="user">
<mac address="52:54:00:ea:f9:69"/>
<model type="virtio"/>
<address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface>
```
However, the one created via virt-manager has this:
```xml
<interface type="network">
<mac address="52:54:00:5d:c7:9e"/>
<source network="default"/>
<model type="virtio"/>
<address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>
```
What I'm not able to understand is, why does the user mode networking work fine but not network mode?
My ultimate goal is to set up routed configuration, so one can directly SSH into the guests.
Packages:
- virt-manager - 4.1.0 (Debian)
- libvirt0 - 10.3.0-3 (Debian)
- GNOME Boxes - 46.0 (Flatpak)
Please let me know if you need any other information. It is a pretty much clean installation with no extra configuration, so I don't think I messed up anything.
Thank you!
Arun Mani J
4 months, 3 weeks
stats : active memory / disk latency
by Peter Quiring
Two stats that VMWare offers that I can't seem to find in libvirt is
"active" memory and disk latency.
Any idea how I could extract these stats?
libvirt offers memory stats such as maximum, current, unused, usable,
etc. but these are all 100% or 0% of what the VM has available.
Looking for an "active" value. /proc/memstat has similar stats within
a Linux VM but need to get from libvirt.
Disk latency is also sometimes critical information.
Thanks,
--
Peter Quiring
4 months, 3 weeks
qemu guest agent with openbsd
by Christian Haumesser
Hello,
I have an OpenBSD 7.5 guest running on Debian bookworm with libvirt (9.0) and qemu (7.2).
I'd like to use the qemu guest agent in this guest, but I can't seem to figure out how to craft the libvirt xml to expose the serial port in a way that OpenBSD and libvirt can use it together (or if it’s even possible).
Per the libvirt docs <https://wiki.libvirt.org/Qemu_guest_agent.html>, I’m currently using:
> <channel type='unix'>
> <source mode='bind'/>
> <target type='virtio' name='org.qemu.guest_agent.0'/>
> </channel>
But OpenBSD doesn’t directly support the virtio console driver. Consistent with this 2020 thread <https://undeadly.org/cgi?action=article;sid=20200514073852> from the OpenBSD ports making list, I see the following message in my guest’s dmesg output:
> virtio5 at pci0 dev 10 function 0 "Qumranet Virtio Console" rev 0x00
> virtio5: no matching child driver; not configured
Thanks to this libvir-list thread <https://listman.redhat.com/archives/libvir-list/2015-October/120250.html> from 2015, I’ve realized that I can manually expose an ISA serial console that will allow me to connect to the guest agent using native qemu tooling; but there doesn’t seem to be a way to create a channel that libvirt can communicate over without virtio console support in the guest. The thread discusses a couple of approaches to resolving this issue with code changes, but it seems like the discussion stalled out before anything happened.
Am I missing something or is it still impossible?
Thanks!
5 months
What's wrong with internal VM snapshots?
by Marius Vollmer
Hi!
I am trying to improve the support for VM snapshots in the Cockpit web
console, and I am afraid I have questions...
We have been asked to prefer the "external" over the "internal" snapshot
format, at least on RHEL. I haven't yet figured out why, and
consequently I am struggling with deciding how hard the Cockpit UI
should push people towards external snapshots.
So, what's wrong with internal snapshots?
I heard they are "unreliable", but how so in detail? Does the data
structure inside the qcow2 files get corrupted easily? Do they behave
poorly when the snapshot process runs out of disk in the middle? That
sort of thing would help me a lot to figure out what Cockpit should be
doing on platforms other than RHEL.
And how well (or how soon) can external snapshots be expected to work?
I have severely messed up my libvirt state a couple of times while
playing around with them, and my confidence in them right now isn't
great. :-) Are you surprised by this? Or are external snapshots not yet
considered ready?
(Most recent example from my experiments: Deleting a full system
snapshot of a paused machine fails with "internal error: unable to
execute QEMU command 'block-commit': Block node is read-only". Reverting
to it works and after that it can also be deleted, all while the VM is
paused.)
Thanks!
5 months, 1 week
