- Users - libvirt List Archives

Host network conflicting with guests network
by bd730c5053df9efb 19 Sep '24

19 Sep '24

Hi all! I have a question about networking and haven't been able to find it online. I need to replicate a server installation in a host to be able to test a procedure but the host is on the same network as the servers so I am not being able to create an isolated network on the host because there is an interface in the same range of ip adresses. Is there any way to create this isolated virtual network without it conflicting with the hosts network? Thanks in advance! Best regards, Dave. Sent with Proton Mail secure email.

3 8

libvirtError with pipewire audio backend
by Karl Frederick Roldan 17 Sep '24

17 Sep '24

Hi everyone, I'm having trouble regarding using the pipewire backend for libvirt. My OS is GNU Guix with Wayland and pipewire running with wireplumber on a user session. I have the following versions: libvirt-9.10.0 (with pipewire enabled) qemu-8.2.2 (with pipewire enabled) libpipewire-1.0.3 I only modified the =<audio>= configuration in the xml file using =sudo virsh edit win10= as follows: <audio id='1' type='pipewire' runtimeDir='/run/user/1000'/> I can confirm that pipewire is running and that I have pipewire sockets in the directory specified. But when running the VM using virt-manager, I get the following error: Error starting domain: internal error: process exited while connecting to monitor: 2024-09-17T05:54:03.785811Z qemu-system-x86_64: Failed to initialize PW context Traceback (most recent call last): File "/gnu/store/nxpr6npj618pqmlanqqh7ks9a92jgm98-virt-manager-4.1.0/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/gnu/store/nxpr6npj618pqmlanqqh7ks9a92jgm98-virt-manager-4.1.0/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb callback(*args, **kwargs) File "/gnu/store/nxpr6npj618pqmlanqqh7ks9a92jgm98-virt-manager-4.1.0/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn ret = fn(self, *args, **kwargs) File "/gnu/store/nxpr6npj618pqmlanqqh7ks9a92jgm98-virt-manager-4.1.0/share/virt-manager/virtManager/object/domain.py", line 1402, in startup self._backend.create() File "/gnu/store/yaqxaja0ckyf6fm7gnd9iqlrijz6qcqy-python-libvirt-10.6.0/lib/python3.10/site-packages/libvirt.py", line 1362, in create raise libvirtError('virDomainCreate() failed') libvirt.libvirtError: internal error: process exited while connecting to monitor: 2024-09-17T05:54:03.785811Z qemu-system-x86_64: Failed to initialize PW context If I switch the audio backend back to =spice=, the VM will run but there's no sound. I am not sure if this is a virt-manager problem or I somehow misconfigured libvirt. How should I proceed about this problem?

2 2

About virsh(1) and Postcopy migration
by Prasad Pandit 04 Sep '24

04 Sep '24

Hello, * virsh(1) offers multiple options to initiate Postcopy migration: 1) virsh migrate --postcopy --postcopy-after-precopy 2) virsh migrate --postcopy + virsh migrate-postcopy 3) virsh migrate --postcopy --timeout <N> --timeout-postcopy When Postcopy migration is invoked via options (2) or (3) above, the migrated guest on the destination host hangs sometimes. But such a hang is not reproducible with option (1) above. * When using option (1) above, libvirtd(8) waits for the first pass of pre-copy to finish before enabling postcopy migration. * Does the same waiting happen when using options (2) and (3) above? === 2024-07-24 14:16:27.448+0000: msg={"execute":"migrate" 2024-07-24 14:16:29.318+0000: msg={"execute":"migrate-start-postcopy" 2024-07-24 14:28:39.737+0000: msg={"execute":"migrate" 2024-07-24 14:28:41.119+0000: msg={"execute":"migrate-start-postcopy" 2024-07-24 14:44:11.684+0000: msg={"execute":"migrate" 2024-07-24 14:44:12.835+0000: msg={"execute":"migrate-start-postcopy" 2024-07-24 14:48:00.675+0000: msg={"execute":"migrate" 2024-07-24 14:48:02.319+0000: msg={"execute":"migrate-start-postcopy" 2024-07-24 15:03:36.110+0000: msg={"execute":"migrate" 2024-07-24 15:03:37.341+0000: msg={"execute":"migrate-start-postcopy" 2024-07-24 16:05:25.602+0000: msg={"execute":"migrate" 2024-07-24 16:05:26.756+0000: msg={"execute":"migrate-start-postcopy" === * While running migration tests with options (2) and (3) above, switch to postcopy appears to happen within 2 seconds of starting migration. - Is that reasonable time to switch from pre-copy to postcopy? - Is there an ideal time to wait before switching to postcopy? * The feature page below suggests to wait until one cycle of RAM migration has completed -> https://wiki.qemu.org/Features/PostCopyLiveMigration * I'd much appreciate any clarification/confirmation about this. Thank you. --- -Prasad

2 3

Possible issue with shared folder from host to virtual machine
by Mulloch94＠protonmail.com 03 Sep '24

03 Sep '24

I ran into an issue when trying to connect a shared folder from my host to my virtual OS in KVM. The folder actually does appear in the virtual OS, but it's empty, unlike the folder in the host which has files in it. I'm not ruling out the possibility that I configured something wrong, but the instructions on setting it up seemed to be pretty straightforward. Basically I did the following procedures: mkdir ~/shared chmod 777 ~/shared Went to VirtManager enabled shared memory, added FileSystem with driver virtiofs. My source path to find the folder was /home/username/shared and the target path was /mnt/shared. Upon booting the virtual machine everything seemed to work. I found the folder where it should be, only there was nothing in it. And on the host there was files I had already placed in the folder. So this is where I'm at now.

2 1

dnsmasq DHCP server with nwfilter
by Kai 30 Aug '24

30 Aug '24

Hello, I'm trying to set up a nwfilter ruleset, where the client only should be able to answer to incoming requests and pings. The outbound traffic (LAN and Internet) shouldn't be working. I've gut the rules as mentioned below (I moved all filterref inside for debugging): <filter name='fwrule-test0' chain='root' priority='-700'> <uuid>89daa6f3-0300-439d-bbba-4d298b4420f2</uuid> <rule action='accept' direction='out' priority='100'> <ip protocol='udp' srcportstart='68' dstportstart='67'/> </rule> <rule action='accept' direction='in' priority='101'> <ip protocol='udp' srcportstart='67' dstportstart='68'/> </rule> <rule action='accept' direction='out' priority='200'> <ip dstipaddr='10.16.136.6'/> </rule> <rule action='accept' direction='out' priority='200'> <ip dstipaddr='10.16.136.9'/> </rule> <rule action='accept' direction='in' priority='250'> <all/> </rule> <rule action='accept' direction='inout' priority='300'> <all state='ESTABLISHED,RELATED'/> </rule> <rule action='accept' direction='inout' priority='301'> <icmp/> </rule> <rule action='accept' direction='out' priority='400'> <udp dstportstart='53'/> </rule> <rule action='accept' direction='inout' priority='400'> <mac protocolid='arp'/> </rule> <rule action='drop' direction='out' priority='800'> <all/> </rule> </filter> When the guest already has a proper IP address, this seems to work on first sight, the client can't talk to the internet anymore, but is reachable for TCP and UDP requests. However, I can't get DHCP working. I'm using the integrated dnsmasq service for DHCP. It works again, when I remove the last DROP rule taking care of the rest. I looked inside tcpdump / Wireshark for the corresponding interface (virbr4). With the enabled DHCP port rules I can see that DHCP requests go out to 255.255.255.255. I also activated dnsmasq logging for the virbr4 instance. Here, I don't get any DHCP logs. Without the last DROP rule, I can see clients getting an IP address. I currently have no idea where to look "in between" as the dnsmasq is listening von virbr4. My expectation for DHCP was ports 67 <-> 68 to be open as in the nwfilter 'allow-dhcp'. Am I missing here something? Thank you! Kai

2 2

any update
by dh8827.kim＠samsung.com 30 Aug '24

30 Aug '24

Has there been any update? Has anyone solved this issue? I am experiencing the same problem. To resolve this, try downgrading the libvirtexport version to tinkoff. ( https://github.com/Tinkoff/libvirt-exporter/blob/master/libvirt_exporter.go )

1 0

autostart sessiioned vms
by daggs 28 Aug '24

28 Aug '24

Greetings, I'm running sessioned vms which I want to start them up at boot. I've marked a vm inside a use as autostart, added libvirtd to the boot order and rebooted but it didn't started the vm. I tried adding libvirt-guests to bott services but my sessioned vm is still not autostarting. what is the proper way to do so? Thanks, Dagg

2 6

KVM image fails to resume
by libvirt＠eyal.emu.id.au 27 Aug '24

27 Aug '24

I upgraded f38->f40 but left the vm saved rather than shutdown (my bad but here we are) Attempting to restore the vm with 'virsh restore /data1/VMs/libvirt/images/e4.saved' I get: error: Failed to restore domain from /data1/VMs/libvirt/images/e4.saved error: operation failed: guest CPU doesn't match specification: extra features: vmx-ins-outs,vmx-true-ctls,vmx-store-lma,vmx-activity-hlt,vmx-vmwrite-vmexit-fields,vmx-apicv-xapic,vmx-ept,vmx-desc-exit,vmx-rdtscp-exit,vmx-vpid,vmx-wbinvd-exit,vmx-unrestricted-guest,vmx-rdrand-exit,vmx-invpcid-exit,vmx-vmfunc,vmx-shadow-vmcs,vmx-rdseed-exit,vmx-pml,vmx-ept-execonly,vmx-page-walk-4,vmx-ept-2mb,vmx-ept-1gb,vmx-invept,vmx-eptad,vmx-invept-single-context,vmx-invept-all-context,vmx-invvpid,vmx-invvpid-single-addr,vmx-invvpid-all-context,vmx-intr-exit,vmx-nmi-exit,vmx-vnmi,vmx-preemption-timer,vmx-vintr-pending,vmx-tsc-offset,vmx-hlt-exit,vmx-invlpg-exit,vmx-mwait-exit,vmx-rdpmc-exit,vmx-rdtsc-exit,vmx-cr3-load-noexit,vmx-cr3-store-noexit,vmx-cr8-load-exit,vmx-cr8-store-exit,vmx-flexpriority,vmx-vnmi-pending,vmx-movdr-exit,vmx-io-exit,vmx-io-bitmap,vmx-mtf,vmx-msr-bitmap,vmx-monitor-exit,vmx-pause-exit,vmx-secondary-ctls,vmx-exit-nosave-debugctl,vmx-exit-load-perf-global-ctrl,vmx-exit-ack-i ntr,vmx-exit-save-pat,vmx-exit-load-pat,vmx-exit-save-efer,vmx-exit-load-efer,vmx-exit-save-preemption-timer,vmx-entry-noload-debugctl,vmx-entry-ia32e-mode,vmx-entry-load-perf-global-ctrl,vmx-entry-load-pat,vmx-entry-load-efer,vmx-eptp-switching Then, trying to resume from the "virtual Machine Manager" UI gets the message: ===== Error unpausing domain: Requested operation is not valid: domain is not running Error unpausing domain: Requested operation is not valid: domain is not running Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn ret = fn(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/share/virt-manager/virtManager/object/domain.py", line 1437, in resume self._backend.resume() File "/usr/lib64/python3.12/site-packages/libvirt.py", line 2425, in resume raise libvirtError('virDomainResume() failed') libvirt.libvirtError: Requested operation is not valid: domain is not running I searched for a solution and most say to fiddle with some settings and reboot. I cannot reboot, I want to resume (unpause). How can I restore the vm without crashing it (throwing away the saved memory), if it even boots this way? TIA

2 4

autostart fails on host boot. NAT-bridge doesn't come up fast enough
by Fred Clift 26 Aug '24

26 Aug '24

I have 18 VMs that are all supposed to attach to a NAT-bridge. The bridge definition seems to be ok - it used to work when I didn't have any guests defined. It worked when I had only a couple guests defined. Now that I have 18 of them, it seems to take more than a minute for virbr0 to come up, and the guests all fail to autostart because the bridge doesn't exist yet. The host system (AlmaLinux 9.4) has dual Xeon(R) CPU E5-2680v4 with plenty of cores and ram. Is there some easy way to delay the start of the guests to wait for virbr0 to be up? Or can I specify a boot order? I assume that libvirtd autostarts networks before guests but I really have no idea... Can I specify an autostart dependency? As a stopgap measure I turned 'autostart' off on these and made a systemd-run bash script that just starts the network, and then starts the guests all serially. It has worked 10 or so test boots in a row. I'd prefer not to have to use an external script if possible. Fred Clift

2 3

splitting a bt wifi combo between two vms
by daggs 26 Aug '24

26 Aug '24

Greetings, I have the following combo device: dagg@NCC-5001D ~ $ lsusb -d8087:0029 Bus 001 Device 002: ID 8087:0029 Intel Corp. AX200 Bluetooth dagg@NCC-5001D ~ $ lspci -vv -s 03:00.0 03:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a) Subsystem: Intel Corporation Wi-Fi 6 AX200NGW Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 34 IOMMU group: 13 Region 0: Memory at fc600000 (64-bit, non-prefetchable) [size=16K] Capabilities: <access denied> Kernel modules: iwlwifi I want to pass each of them to a different vm. should I expect problems? Thanks, Dagg

3 3

Updating authentication for a Ceph (RBD) disk in a live domain
by will.gorman＠joyent.com 21 Aug '24

21 Aug '24

Is it possible to update and change the <auth/> for an RBD network disk while the domain the disk is attached to is running and without detaching/reattaching the disk? For example if I have a disk attached like the following: <disk type='network' device='disk'> <driver name='qemu' type='raw'/> <auth username='someuser'> <secret type='ceph' usage='someuser key'/> </auth> <source protocol='rbd' name='somepool/someimage'> <host name='127.0.0.1' port='3300'/> </source> <target dev='sdd' bus='scsi'/> <alias name='scsi0-0-0-3'/> <address type='drive' controller='0' bus='0' target='0' unit='3'/> </disk> If I want to change the auth to <auth username='someotheruser'> <secret type='ceph' usage='someotheruser key'/> </auth> can I do that without either attaching/detaching the disk or stopping/restarting the domain? I've tried `virsh update-device domain disk.xml --live --persistent` using xml identical to the current disk except for the auth and it says "Device updated successfully" but when I check the domain with `dumpxml` I can still see the original auth settings for the disk.

2 1

cap seach for session libvirt
by daggs 09 Aug '24

09 Aug '24

Greetings, I'm working on allowing a session vm to create a tap iface. the vm has this defintion: <interface type='ethernet'> <mac address='52:54:00:a7:79:6b'/> <target dev='veth0'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </interface> when I try to start the vm, I get this error: Unable to create tap device veth0: Operation not permitted searching the code led me to this line: https://github.com/libvirt/libvirt/blob/0caacf47d7b423db9126660fb0382ed56cd… I've looked on line and found out I need the net_admin_cap set. so I took the relevant code in to a dedicated test file and using pam_cap I've defined such cap for the test file, all went well. so I took it back to virsh and defined that cap to virsh but I'm still getting the same issue, see: https://ibb.co/zHggRQZ the os is debian 12 any ideas why I'm still getting this error? Thanks, Dagg

2 4

luks devices and libvirt
by Marc Haber 05 Aug '24

05 Aug '24

Hi, this is an ongoing issue. I don't know whether I ever have addresses this here, but it's still annoying. I am using debian unstable, libvirt 10.5.0, virt-manager 4.1.0, qemu 9.0.2. I work through virt-manager, rarely I use virsh. I regularly configure virtual disks that are located on a luks-encrypted LVM volume. when unlocked, the block devices appears as /dev/mapper/foo and is a symlink to a ../dm-xx node with xx being a random number, ../dm-xx being a regular block device. To facilitate this, I have defined a storage pool with this XML: <pool type="dir"> <name>mapper</name> <uuid></uuid> <capacity unit="bytes">24598757376</capacity> <allocation unit="bytes">0</allocation> <available unit="bytes">24598757376</available> <source> </source> <target> <path>/dev/mapper</path> <permissions> <mode>0755</mode> <owner>0</owner> <group>0</group> </permissions> </target> </pool> This is necessary as the storage type "LVM volume group" now insists on a volume group name, and the DM mappings created by LUKS dont have a volume group name. When I add a disk to a VM from this storage pool, it generates the XML: <disk type="file" device="disk"> <driver name="qemu" type="raw"/> <source file="/dev/mapper/wintest"/> <target dev="vda" bus="virtio"/> <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/> </disk> qemu won't start with this settings: error: Failed to start domain 'win11test' error: internal error: QEMU unexpectedly closed the monitor (vm='win11test'): 2024-07-28T15:20:25.250387Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/dev/mapper/wintest","node-name":"libvirt-1-storage","read-only":false}: 'file' driver requires '/dev/mapper/wintest' to be a regular file Changing the XML to <disk type="block" device="disk"> <driver name="qemu" type="raw"/> <source dev="/dev/mapper/wintest"/> <target dev="vda" bus="virtio"/> <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/> </disk> (note type="block" and "source dev") makes the VM work. Can virt-manager somehow be coaxed into generating XML that works here? If not, is this a virt-manager issue or should qemu just accept type="file" and "source file")? Greetings Marc -- ----------------------------------------------------------------------------- Marc Haber | "I don't trust Computers. They | Mailadresse im Header Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402 Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421

2 1

setting bridge for VMs IP assignment by router's DHCP server
by Germano Massullo 02 Aug '24

02 Aug '24

I am running a libvirt hostmachine (Fedora 40) which has 192.168.1.6 IP address, assigned by router's DHCP server. I want the libvirt VMs IPs to be assigned by router's DHCP server, so I tried to setup a bridge via # virsh net-define foo.xml and trying the following files as xml file, but they all failed to achieve the task. Here I list the two XML variants I tried to use 1) <network> <name>bridge-no-nat</name> <bridge name='virbr1_no-nat' stp='on' delay='0'/> <forward mode='open'/> </network> RETURNS: open forwarding requested, but no IP address provided for network 'bridge-no-nat' 2) I created in nmtui the bridge virbr1_no_nat then I used following XML for virsh net-define <network> <name>br1_no_nat</name> <forward mode='bridge'/> <bridge name='virbr1_no_nat'/> <virtualport type='openvswitch'/> <portgroup name='default'/> </network> then I configure the VM network to use but when I start the VM I get error: internal error: Unable to add port vnet1 to OVS bridge virbr1_no_nat: <null> Do you know how I can solve this? Thank you

3 2

KVM static internal networking without host bridge interface (virbr)
by Daniel 01 Aug '24

01 Aug '24

How to set up an internal network between two KVM network interfaces while using static networking (avoiding dnsmasq) and while avoiding a host bridge interface (virbr)? Currently I am using this for the network. <network> <name>Internal</name> <bridge name='virbr2' stp='on' delay='0'/> </network> And then for the VM. <interface type='network'> <source network='Internal'/> <model type='virtio'/> <driver name='qemu'/> </interface> * I would like to avoid the host `virbr2` interface. This is because ideally package sniffers on the host such as tshark / wireshark would be unable to see these packages following between an internal network between two VMs. * SLIRP should be avoided due to past security issues. [1] * dnsmasq on the host operating system or inside the VMs should also be avoided in favor of static IP addresses. By comparison, this is possible in VirtualBox. [2] Is that possible with KVM too? Could you please show an example configuration file on how to accomplish that? [1] CVE-2019-6778 [2] VirtualBox has this capability. VirtualBox can have an internal network using static networking. No vibr bridge interfaces can be seen on the host operating system. And VM to VM internal traffic is not visible to package analyzers on the host operating system either. Regards, Daniel -- Daniel Winzen Steinkaulstr. 47 52070 Aachen Germany Web: https://danwin1210.de/ E-Mail: daniel(a)danwin1210.de Phone: +49 176 98819809 PGP-Key: https://danwin1210.de/pgp.txt

2 3

Certificate verification error for qemu while migrating
by jdeberles＠gmail.com 29 Jul '24

29 Jul '24

Hello, I'm running ovirt 4.4.10 (using libvirt 7.10.0-1.module_el8.6.0+1046+bd8eec5e) and I have the following qemu error while I launch a VM migration Jul 3 12:37:07myhostname1 journal[958949]: Certificate [session] owner does not match the hostname myhostname2 Jul 3 12:37:07 myhostname1journal[958949]: Certificate check failed Certificate [session] owner does not match the hostname myhostname2 Jul 3 12:37:07 myhostname1 journal[958949]: authentication failed: Failed to verify peer's certificate Jul 3 12:37:07myhostname1 journal[958949]: operation failed: Failed to connect to remote libvirt URI qemu+tls://myhostname3/system: authentication failed: Failed to verify peer's certificate To avoid this error I set the following paramaters inside the /etc/libvirt/qemu.conf and restard vdsmd and libvirtd daemons. migrate_tls_x509_verify = 0 default_tls_x509_verify = 0 But I still have the same error. Can you help me to understand why this set of parameters are not working as expected ? kind regards, Julien

3 4

can this network setup works?
by daggs 13 Jul '24

13 Jul '24

Greetings, I have this setup. host and two vms, a and b. vm a servers as router, it has one physical nic passed directly into it. I want to connect vm a to vm b and the host to provide internet access to both. so I've created a virtual switch like this: <network> <name>default</name> <uuid>f90b3044-81c1-4c22-98df-8bbca3153f21</uuid> <bridge name='virsw0' stp='on' delay='0'/> <mac address='52:54:00:6b:1b:92'/> </network> with this xml entry: <interface type='bridge'> <mac address='xx:xx:xx:xx:xx:xx'/> <source bridge='virsw0'/> <model type='virtio'/> <address type='pci' domain='x' bus='x' slot='x' function='x'/> </interface> in vm a and vm b should connect the both to the virtual switch thus allowing vm b internet access. this however doesn't solves the host connection to the internet. reading docs points to the fact I can add new tun devs outside of libvirt and add it to the virtual switch. so I did this: $ ip tuntap add QemuTap0 mode tap user root $ ip link set QemuTap0 master virsw0 and brctl show virsw0 returns this: $ brctl show virsw0 bridge name bridge id STP enabled interfaces virsw0 8000.5254006b1b92 yes QemuTap0 so theoretically speaking, when vm a is started and libvirt creates a device and connects it to the virtual switch and run dhcpd, vm a will provide ip. am I correct? Thanks, Dagg

1 0

per user vm isolation with shared network
by daggs 02 Jul '24

02 Jul '24

Greetings, I have two vm which I want to isolate per user, if I'm not mistaken, I can to that with per session uri. but I want to setup a virtual bridge so they will get connected with each other. looks like that if I define the network as system, it isn't visible in the session. is there a way to do that? if I define the same network in both sessions, will it work? Thanks, Dagg

2 8

Re: dm-crypt performance regression due to workqueue changes
by Mikulas Patocka 01 Jul '24

01 Jul '24

On Sun, 30 Jun 2024, Tejun Heo wrote: > Hello, > > On Sat, Jun 29, 2024 at 08:15:56PM +0200, Mikulas Patocka wrote: > > > With 6.5, we get 3600MiB/s; with 6.6 we get 1400MiB/s. > > > > The reason is that virt-manager by default sets up a topology where we > > have 16 sockets, 1 core per socket, 1 thread per core. And that workqueue > > patch avoids moving work items across sockets, so it processes all > > encryption work only on one virtual CPU. > > > > The performance degradation may be fixed with "echo 'system' > > >/sys/module/workqueue/parameters/default_affinity_scope" - but it is > > regression anyway, as many users don't know about this option. > > > > How should we fix it? There are several options: > > 1. revert back to 'numa' affinity > > 2. revert to 'numa' affinity only if we are in a virtual machine > > 3. hack dm-crypt to set the 'numa' affinity for the affected workqueues > > 4. any other solution? > > Do you happen to know why libvirt is doing that? There are many other > implications to configuring the system that way and I don't think we want to > design kernel behaviors to suit topology information fed to VMs which can be > arbitrary. > > Thanks. I don't know why. I added users(a)lists.libvirt.org to the CC. How should libvirt properly advertise "we have 16 threads that are dynamically scheduled by the host kernel, so the latencies between them are changing and unpredictable"? Mikulas

4 7

Command line equivalent for cpu pinning
by Gianluca Cecchi 28 Jun '24

28 Jun '24

Hello, on my Fedora 39 with libvirt 9.7.0-3.fc39 qemu-kvm 8.1.3-5.fc39 kernel 6.8.11-200.fc39.x86_64 I'm testing cpu pinning The hw is a Nuc with 13th Gen Intel(R) Core(TM) i7-1360P If I pass from this in my guest xml: <vcpu placement='static'>4</vcpu> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='1'/> </cpu> to this: <vcpu placement='static'>4</vcpu> <cputune> <vcpupin vcpu='0' cpuset='0'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='4'/> <vcpupin vcpu='3' cpuset='6'/> </cputune> <cpu mode='host-passthrough' check='none' migratable='on'> <topology sockets='1' dies='1' cores='4' threads='1'/> </cpu> It seems to me that the generated command line of the qemu-system-x88_64 process doesn't change. As if the cputune options were not considered What should I see as different? Actually it seems it is indeed honored, because if I run stress-ng in the VM in the second scenario and top command in the host, I see only pcpu 0,2,4,6 going up with the load. Instead the first scenario keeps several different cpus alternating in the load. The real question could be: if I want to reproduce from the command line the cputune options, how can I do it? Is it only a cpuset wrapper used for the qemu-system-x86_64 process to place it in a cpuset control group? I see for the pid of the process $ sudo cat /proc/340215/cgroup 0::/machine.slice/machine-qemu\x2d8\x2dc7anstgt.scope/libvirt/emulator and $ sudo systemd-cgls /machine.slice CGroup /machine.slice: └─machine-qemu\x2d8\x2dc7anstgt.scope … └─libvirt ├─340215 /usr/bin/qemu-system-x86_64.... ├─vcpu1 ├─vcpu2 ├─vcpu0 ├─emulator └─vcpu3 What could be an easy command to replicate from the command line what virsh does? Thanks in advance Gianluca

3 3

cpu vmx migration issue
by d tbsky 25 Jun '24

25 Jun '24

Hi: I update our RHEL9 system to RHEL 9.4, which brings libvirt 10.0. I try to calculate the cpu baseline for our two-node cluster with command "virsh domcapabilities" then "virsh hypervisor-cpu-baseline --migratable". the result has many cpu features begin with "vmx". the test cluster has cpu "intel E3-1280 V3" and "intel I3-9100F". when I try live migrate vm, it failed and told me "guest CPU doesn't match specification: missing features: vmx-apicv-register,vmx-apicv-vid,vmx-posted-intr". at another cluster with cpu " Intel i5-2520M" and "Intel i7-9750H" the migration works fine for the calculated cpu result. although there are still many "vmx" cpu features in the result. if I delete all these vmx features, then migration works fine for both cluster, like old days. I wonder what's the benefit to expose these vmx features to guest if I don't do any nested virtualization. is it ok the drop all these vmx cpu features for a guest? thanks a lot for help!

2 9

unable to start a nested vm due to iommu_group issue
by daggs 22 Jun '24

22 Jun '24

Greetings, I'm working on a new os for my server which runs 2 vms. I'm using nested vms to work on it so I won't take down the server. the new os is alpine 3.20, qemu is 9.0.1, libvirt is 10.3.0. I have a device I want to pass to one of the guests, here is the iommu group layout: IOMMU Group 18 00:1f.0 ISA bridge [0601]: Intel Corporation 82801IB (ICH9) LPC Interface Controller [8086:2918] (rev 02) IOMMU Group 18 00:1f.2 SATA controller [0106]: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02) IOMMU Group 18 00:1f.3 SMBus [0c05]: Intel Corporation 82801I (ICH9 Family) SMBus Controller [8086:2930] (rev 02) IOMMU Group 18 00:1f.4 Audio device [0403]: Intel Corporation 82801I (ICH9 Family) HD Audio Controller [8086:293e] (rev 03) the device in question is 00:1f.4: utils-server:/home/igor# lspci -s 00:1f.4 -nnv 00:1f.4 Audio device [0403]: Intel Corporation 82801I (ICH9 Family) HD Audio Controller [8086:293e] (rev 03) Subsystem: Red Hat, Inc. QEMU Virtual Machine [1af4:1100] Flags: bus master, fast devsel, latency 0, IRQ 10, IOMMU group 18 Memory at e1080000 (32-bit, non-prefetchable) [size=16K] Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+ Kernel driver in use: vfio-pci to achieve that, I've compiled the kernel with acs patch and defined it in the kernel cmdline: BOOT_IMAGE=/boot/vmlinuz-lts root=UUID=44299ace-27c6-4047-8cbd-bbffcc0a65f0 ro modules=sd-mod,usb-storage,ext4 quiet rootfstype=ext4 iommu=pt intel_iommu=on pcie_acs_override=id:8086:293e,8086:10c9 I see in dmesg, acs is enabled, see: # dmesg | grep "ACS overrides" [ 0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA the xml file can be found at https://bpa.st/FARQ. when I try to start the vm, I get this: # virsh start streamer error: Failed to start domain 'streamer' error: internal error: QEMU unexpectedly closed the monitor (vm='streamer'): 2024-06-21T08:39:17.959476Z qemu-system-x86_64: -device {"driver":"vfio-pci","host":"0000:00:1f.4","id":"hostdev0","bus":"pcie.0","addr":"0x1f.0x5"}: vfio 0000:00:1f.4: group 18 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver any ideas what am I missing? is it possible this cannot work within a vm? Thanks, Dagg

1 1

librbd encryption and guest XML
by melanie witt 17 Jun '24

17 Jun '24

Hi, I have been trying to use the librbd engine to run a guest from an encrypted RBD image and am running into some problems. What I would like to do is: 1. Start from an unencrypted raw image with an OS 2. Make an encrypted clone of that image 3. Boot a guest from the encrypted clone image What I have tried so far (simplified): 1. Make a clone of the unencrypted image rbd clone images/unencrypted@snap images/encryptedclone 2. Format the clone image with encryption rbd encryption format images/encryptedclone luks1 passphrase.bin 3. Create guest XML with the encrypted clone [...] <disk type="network" device="disk"> <driver type="raw" cache="writeback"/> <source protocol="rbd" name="images/encryptedclone"> <host name="127.0.0.1" port="6789"/> <encryption format="luks" engine="librbd"> <secret type="passphrase" uuid="secretuuid"/> </encryption> </source> <auth username="cinder"> <secret type="ceph" uuid="othersecretuuid"/> </auth> <target dev="vda" bus="virtio"/> </disk> [...] and virDomainCreateWithFlags() with the XML. I don't get any errors from libvirt (no errors about loading encryption) but this configuration does not seem to work, the guest won't boot. If anyone can give me a hint what I'm doing wrong, I would appreciate it. Cheers, -melwitt

2 3

brew services start libvirt failed to start libvirt on MacOS Sonoma
by absiplant＠gmail.com 17 Jun '24

17 Jun '24

Hello, I am hoping someone can point me in the right direction regarding issues I have starting up libvirt services on MacOS Sonoma. I am running: macOS Sonoma 14.5 llibvirt: stable 10.4.0 (bottled) qemu: stable 9.0.1 (bottled) I have the following entries in libvirt.conf: listen_tls = 0 listen_tcp = 1 listen_addr = "0.0.0.0" unix_sock_group = "libvirt" unix_sock_ro_perms = "0777" unix_sock_rw_perms = "0770" unix_sock_admin_perms = "0700" unix_sock_dir = "/opt/homebrew/var/run/libvirt" auth_unix_ro = "none" auth_unix_rw = "none" auth_tls = "none" tls_no_sanity_certificate = 1 tls_no_verify_certificate = 1 log_filters="1:qemu 1:libvirt 4:object 4:json 2:event 1:util" log_outputs="1:file:/opt/homebrew/var/log/libvirt/libvirtd.log" I added ```firewalld = 0``` and ```irewall_backend = "none"```. These made no difference, I have since removed both. Snippets of the log in debug mode indicate a usable firewall backend cannot be found, is there a way to resolve this on macOS? 8617315328: debug : virSystemdNotifyStartup:648 : Skipping systemd notify, not requested 6105182208: debug : virThreadJobSet:97 : Thread 6105182208 is now running job daemon-init 6105182208: warning : virProcessGetStartTime:1205 : Process start time of pid 10814 not available on this platform 6105182208: debug : virGDBusIsServiceEnabled:386 : Service org.freedesktop.login1 is unavailable 6105182208: debug : virStateInitialize:663 : Running global init for Remote state driver 6105182208: debug : virStateInitialize:670 : State init result 1 (mandatory=0) 6105182208: debug : virStateInitialize:663 : Running global init for bridge state driver 6105182208: error : virNetworkLoadDriverConfig:147 : internal error: could not find a usable firewall backend 6105182208: debug : virStateInitialize:670 : State init result -1 (mandatory=0) 6105182208: error : virStateInitialize:674 : Initialisation of bridge state driver failed: internal error: could not find a usable firewall backend 6105182208: error : daemonRunStateInit:617 : Driver state initialisation failed 6105182208: debug : virThreadJobClear:122 : Thread 6105182208 finished job daemon-init with ret=0 Thanks.

3 2

vfio usage in a vm
by daggs 14 Jun '24

14 Jun '24

Greetings, I wanted to know if it is possible to attach a virtual nic (shows up as 02:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (rev 20)) to the vfio module, when I modprobe it I get this error: [ 854.624668] vfio-pci: probe of 0000:02:01.0 failed with error -22 Thanks, Dagg

2 4