- Users - libvirt List Archives

Hanging migration leaving domain in paused state
by Yangchen Ye 21 Jul '25

21 Jul '25

Hi, We are running libvirt 8.0.0, and sometimes live migration could not finish (because the guest is dirtying the memory too fast). We implemented a monitor that increases max downtime when it observed that "Data Remaining" bumps up. But we found a strange sequence of events from the monitor, which leads to a paused domain on the destination hypervisor: The monitor sees Data Remaining bumping up and increases max downtime up to 20 seconds, but weird thing is that after a period of time, it started reporting "Data Remaining" and "Data Total" is both 0, but the migration job is still unfinished: "Migration in progress - DataTotal: 85 904728064, DataRemaining: 22201458688, TimeElapsed: 20005, MaxDowntime: 500, DirtyRate: 0" "Migration in progress - DataTotal: 85904728064, DataRemaining: 43801825280, TimeElapsed: 10005, MaxDowntime: 500, DirtyRate: 0" "Migration in progress - DataTotal: 85904728064, DataRemaining: 52382912512, TimeElapsed: 10004, MaxDowntime: 500, DirtyRate: 0" (DataRemaining bumps up, we start increasing max downtime) "Migration in progress - DataTotal: 85904728064, DataRemaining: 4219596800, TimeElapsed: 40004, MaxDowntime: 1500, DirtyRate: 0" (Last poll where we see the job info) After which monitor logs "Migration in progress - DataTotal: 0, DataRemaining: 0, TimeElapsed: 40004, MaxDowntime: 13500, DirtyRate: 0" The domain is always running on the source hypervisor but there is a paused domain on the destination hypervisor which is paused at start up. Trying to understand what might have happened: - Is this a known issue for live migrating high memory activity guests and the way we interact with libvirt? - What is the recommended way to ensure that a started live migration always run to completion if we don't care about downtime? Appreciate any help here Yangchen Ye

1 1

How to disable sysctl calls performed by libvirt?
by Chimmie Firefly 11 Jul '25

11 Jul '25

Hello, I have an issue whenever using `/proc/sys/ipv6/conf` as read-only mountpoint. While the libvirt by itself is running fine, I'm unable to create NAT network interfaces because of the following error: ``` 2025-07-05 19:33:32.821+0000: 100: error : networkSetIPv6Sysctl:1847 : cannot write to '/proc/sys/net/ipv6/conf/virbr0/disable_ipv6' on bridge 'virbr0': Read-only file system 2025-07-05 19:33:32.841+0000: 100: error : virNetDevSetIFFlag:601 : Cannot get interface flags on 'virbr0': No such device 2025-07-05 19:33:32.947+0000: 100: error : virNetlinkDelLink:688 : error destroying network device virbr0: No such device ``` While using iproute2 and bridge utils, I'm able to create and configure new bridge device, along with attaching the VM to the interface. I've used the following commands: ``` sysctl -w net.ipv4.ip_forward=1 brctl addbr br0 ip link set br0 up ip addr add 192.168.100.1/24 dev br0 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE iptables -A FORWARD -i br0 -o eth0 -j ACCEPT iptables -A FORWARD -i eth0 -o br0 -m state --state RELATED,ESTABLISHED -j ACCEPT ``` Is libvirt allowing to disable sysctl calls in the configuration files? If so, what should I set in the config?

2 1

fsck unable to fix fs issue
by ola . 08 Jul '25

08 Jul '25

I am able to boot VMs by using rbd as the root disk. When I restart and stop the VM everything works fine however, anytime the host goes down say due to a power outage, when next I try to boot the VM, I run into a situation where the root disk gets corrupted and get stuck at "initramfs". I have tried to fix this but to no avail. Here are the errors I get when I try yo use fix the fs issue with fsck manually. done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... [ 7.760625] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes Scanning for Btrfs filesystems done. Begin: Will now check root file system ... fsck from util-linux 2.37.2 [/usr/sbin/fsck.ext4 (1) -- /dev/vda1] fsck.ext4 -a -C0 /dev/vda1 [ 7.866954] blk_update_request: I/O error, dev vda, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0 cloudimg-rootfs: recovering journal [ 8.164279] blk_update_request: I/O error, dev vda, sector 227328 op 0x1:(WRITE) flags 0x800 phys_seg 24 prio class 0 [ 8.168272] Buffer I/O error on dev vda1, logical block 0, lost async page write [ 8.170413] Buffer I/O error on dev vda1, logical block 1, lost async page write [ 8.172545] Buffer I/O error on dev vda1, logical block 2, lost async page write [ 8.174601] Buffer I/O error on dev vda1, logical block 3, lost async page write [ 8.176651] Buffer I/O error on dev vda1, logical block 4, lost async page write [ 8.178694] Buffer I/O error on dev vda1, logical block 5, lost async page write [ 8.180601] Buffer I/O error on dev vda1, logical block 6, lost async page write [ 8.182641] Buffer I/O error on dev vda1, logical block 7, lost async page write [ 8.184710] Buffer I/O error on dev vda1, logical block 8, lost async page write [ 8.186744] Buffer I/O error on dev vda1, logical block 9, lost async page write [ 8.188748] blk_update_request: I/O error, dev vda, sector 229392 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 8.191433] blk_update_request: I/O error, dev vda, sector 229440 op 0x1:(WRITE) flags 0x800 phys_seg 32 prio class 0 [ 8.194204] blk_update_request: I/O error, dev vda, sector 229480 op 0x1:(WRITE) flags 0x800 phys_seg 16 prio class 0 [ 8.196976] blk_update_request: I/O error, dev vda, sector 229512 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 8.243612] blk_update_request: I/O error, dev vda, sector 229544 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 8.246068] blk_update_request: I/O error, dev vda, sector 229640 op 0x1:(WRITE) flags 0x800 phys_seg 32 prio class 0 [ 8.248668] blk_update_request: I/O error, dev vda, sector 229688 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 8.251174] blk_update_request: I/O error, dev vda, sector 229704 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 fsck.ext4: Input/output error while recovering journal of cloudimg-rootfs fsck.ext4: unable to set superblock flags on cloudimg-rootfs cloudimg-rootfs: ********** WARNING: Filesystem still has errors ********** fsck exited with status code 12 done. Failure: File system check of the root filesystem failed The root filesystem on /dev/vda1 requires a manual fsck BusyBox v1.30.1 (Ubuntu 1:1.30.1-7ubuntu3.1) built-in shell (ash) Enter 'help' for a list of built-in commands. (initramfs) fsck.ext4 -f -y /dev/vda1 e2fsck 1.46.5 (30-Dec-2021) [ 24.286341] print_req_error: 174 callbacks suppressed [ 24.286358] blk_update_request: I/O error, dev vda, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0 cloudimg-rootfs: recovering journal [ 24.552343] blk_update_request: I/O error, dev vda, sector 227328 op 0x1:(WRITE) flags 0x800 phys_seg 24 prio class 0 [ 24.556674] buffer_io_error: 5222 callbacks suppressed [ 24.558925] Buffer I/O error on dev vda1, logical block 0, lost async page write [ 24.562116] Buffer I/O error on dev vda1, logical block 1, lost async page write [ 24.565161] Buffer I/O error on dev vda1, logical block 2, lost async page write [ 24.567872] Buffer I/O error on dev vda1, logical block 3, lost async page write [ 24.570586] Buffer I/O error on dev vda1, logical block 4, lost async page write [ 24.573418] Buffer I/O error on dev vda1, logical block 5, lost async page write [ 24.575940] Buffer I/O error on dev vda1, logical block 6, lost async page write [ 24.578622] Buffer I/O error on dev vda1, logical block 7, lost async page write [ 24.581386] Buffer I/O error on dev vda1, logical block 8, lost async page write [ 24.583873] Buffer I/O error on dev vda1, logical block 9, lost async page write [ 24.586410] blk_update_request: I/O error, dev vda, sector 229392 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 24.589821] blk_update_request: I/O error, dev vda, sector 229440 op 0x1:(WRITE) flags 0x800 phys_seg 32 prio class 0 [ 24.593380] blk_update_request: I/O error, dev vda, sector 229480 op 0x1:(WRITE) flags 0x800 phys_seg 16 prio class 0 [ 24.596615] blk_update_request: I/O error, dev vda, sector 229512 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 24.643829] blk_update_request: I/O error, dev vda, sector 229544 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 24.646924] blk_update_request: I/O error, dev vda, sector 229640 op 0x1:(WRITE) flags 0x800 phys_seg 32 prio class 0 [ 24.650051] blk_update_request: I/O error, dev vda, sector 229688 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 [ 24.653128] blk_update_request: I/O error, dev vda, sector 229704 op 0x1:(WRITE) flags 0x800 phys_seg 8 prio class 0 fsck.ext4: Input/output error while recovering journal of cloudimg-rootfs fsck.ext4: unable to set superblock flags on cloudimg-rootfs cloudimg-rootfs: ********** WARNING: Filesystem still has errors ********** This is what my rbd disk looks like <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <auth username='dove'> <secret type='ceph' uuid='b608caae-5eb4-45cc-bfd4-0b4ac11c7613'/> </auth> <source protocol='rbd' name='vms/wing-64700f1d-8c469a54-3f50-4d1e-9db2-2b6ea5f3d14a'> <host name='x.168.1.x' port='6789'/> </source> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> So, my questions are; - How do I prevent this from happening as i have tried different options like changing the "cache" value for the disk template? - How can this be fixed? Thanks

2 1

RBD Namespace
by Rodrigo Prieto 23 Jun '25

23 Jun '25

Hello, I want to connect to an RBD pool specifying the namespace, but it doesn’t work. If I connect directly to the pool, it works. For example: <source protocol="rbd" name="rbd-pool/vm"> But when I try to specify the namespace like this, I cannot connect: <source protocol="rbd" name="rbd-pool/vm" namespace="vms"> <source protocol="rbd" name="rbd-pool/vms/vm"> Error al iniciar dominio: error interno: process exited while connecting to monitor: 2025-06-20T22:54:55.076193Z qemu-system-x86_64: -blockdev {"driver":"rbd","pool":"rbd-pool","image":"loli/mari","server":[{"host":"ceph-admin","port":"6789"}],"user":"admin","auth-client-required":["cephx","none"],"key-secret":"libvirt-2-storage-auth-secret0","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: error reading header from loli/mari: No such file or directory Using the rbd command from the terminal, I can connect both ways; the error occurs only with libvirt. I’m using the following versions on Debian 12: libvirtd (libvirt) 9.0.0 QEMU emulator version 7.2.17 (Debian 1:7.2+dfsg-7+deb12u13) Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers Thanks.

2 2

Re: Question about managed/unmanaged persistent reservation disks
by Michal Prívozník 17 Jun '25

17 Jun '25

On 6/10/25 19:15, Simon Coter wrote: > Adding users DL to possibly reach out a wider audience. > > Simon > Dropping devel list as this is users list material. >> On Jun 9, 2025, at 7:28 PM, Annie Li <annie.li(a)oracle.com> wrote: >> >> Hello, >> >> I've been looking at source code related to persistent reservation and >> got confused a little bit about managed persistent reservation disks. >> For disk configured with 'managed=yes' as the following, >> >> <reservations managed='yes'> >> <source type='unix' path='/var/lib/libvirt/qemu/domain-7- >> brml10g19-iscsi-rese/pr-helper0.sock' mode='client'/> >> </reservations> >> >> libvirt is responsible for starting a pr-helper program with a >> specific associated socket file. The following source code shows that >> there is only one pr-helper and socket file associated with the >> managed disks for one VM. >> >> const char * >> qemuDomainGetManagedPRAlias(void) >> { >> return "pr-helper0"; >> } >> char * >> qemuDomainGetManagedPRSocketPath(qemuDomainObjPrivate *priv) >> { >> return g_strdup_printf("%s/%s.sock", priv->libDir, >> qemuDomainGetManagedPRAlias()); >> } >> >> So if the VM is booted with multiple disks configured with >> 'managed=yes' for reservation, I suppose these multiple disks share >> the this managed pr-helper and socket file. However, per the qemu >> document, https://www.qemu.org/docs/master/interop/pr-helper.html >> <https://www.qemu.org/docs/master/interop/pr-helper.html> >> "It is invalid to send multiple commands concurrently on the same >> socket. It is however possible to connect multiple sockets to the >> helper and send multiple commands to the helper for one or more file >> descriptors." >> This certainly did not use to be the case. IIRC this was discussed in this very old thread: https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/UUL3… Maybe QEMU has some internal lock that does the right thing and serializes requests? Michal >> Due to this limitation above, only one persistent reservation disk is >> allowed as managed in theory. However, libvirt doesn't throw out any >> error or warning when the VM is booted up with multiple managed >> persistent reservation disks. I am wondering if I've missed something >> here? >> >> For unmanaged persistent reservation disks, libvirt doesn't start the >> pr-helper program for them. It is user's responsibility to start this >> program with customized socket file per disk, but the complexity >> increases with numbers of persistent reservation disks, especially in >> the case of hotplug/hotunplog. Is there any plan to support multiple >> managed persistent reservation disks with separate pr-helper/socket file? >> >> Any suggestions/clarifications are greatly appreciated. >> >> Thanks >> >> Annie >> >

3 7

significant delay live migrating a vm with an sr-iov interface
by Paul B. Henson 10 Jun '25

10 Jun '25

I'm using libvirt under Debian 12 (9.0.0-4+deb12u2 w/qemu 7.2+dfsg-7+deb12u12). I have a vm using sr-iov, and configured it with a failover macvtap interface so I could live migrate it. However, there is a significant delay at the end of the migration resulting in a lot of lost traffic. If I only have the macvtap interface, migration completes immediately at the end of the transfer of memory with no loss of traffic. I enabled debug logging, and found the following. On the source system, it logs that the system is paused for the cutover: 2025-04-30 01:08:12.526+0000: 1696180: debug : qemuMigrationAnyCompleted:1957 : Migration paused before switchover at that point, for almost a minute, the source system just keeps printing the same statistics: 2025-04-30 01:08:12.923+0000: 1696272: info : qemuMonitorJSONIOProcessLine:208 : QEMU_MONITOR_RECV_REPLY: mon=0x7f8fdc0ad2f0 reply={"return": {"expected-downtime": 300, "status": "device", "setup-time": 297, "total-time": 26107, "ram": {"total": 137452265472, "postcopy-requests": 0, "dirty-sync-count": 3, "multifd-bytes": 2821784576, "pages-per-second": 297855, "downtime-bytes": 13208, "page-size": 4096, "remaining": 0, "postcopy-bytes": 0, "mbps": 9786.9158461538464, "transferred": 3117658825, "dirty-sync-missed-zero-copy": 0, "precopy-bytes": 295861041, "duplicate": 32874480, "dirty-pages-rate": 56, "skipped": 0, "normal-bytes": 2804301824, "normal": 684644}}, "id": "libvirt-577"} [...] 2025-04-30 01:09:06.290+0000: 1696272: info : qemuMonitorJSONIOProcessLine:208 : QEMU_MONITOR_RECV_REPLY: mon=0x7f8fdc0ad2f0 reply={"return": {"expected-downtime": 300, "status": "device", "setup-time": 297, "total-time" : 79474, "ram": {"total": 137452265472, "postcopy-requests": 0, "dirty-sync-count": 3, "multifd-bytes": 2821784576, "pages-per-second": 297855, "downtime-bytes": 13208, "page-size": 4096, "remaining": 0, "postcopy-bytes" : 0, "mbps": 9786.9158461538464, "transferred": 3117658825, "dirty-sync-missed-zero-copy": 0, "precopy-bytes": 295861041, "duplicate": 32874480, "dirty-pages-rate": 56, "skipped": 0, "normal-bytes": 2804301824, "normal": 684644}}, "id": "libvirt-629"} until finally it completes: 2025-04-30 01:09:06.327+0000: 1696272: info : qemuMonitorJSONIOProcessLine:203 : QEMU_MONITOR_RECV_EVENT: mon=0x7f8fdc0ad2f0 event={"timestamp": {"seconds": 1745975346, "microseconds": 327382}, "event": "MIGRATION", "dat a": {"status": "completed"}} On the destination side, it says something about negotiating failover for the network link: 2025-04-30 01:08:12.923+0000: 1384503: info : qemuMonitorJSONIOProcessLine:203 : QEMU_MONITOR_RECV_EVENT: mon= 0x7fc7900ab2f0 event={"timestamp": {"seconds": 1745975292, "microseconds": 922783}, "event": "FAILOVER_NEGOTIA TED", "data": {"device-id": "ua-sr-iov-backup"}} Then nothing happens for about a minute until it says it is done: 2025-04-30 01:09:06.328+0000: 1384503: debug : qemuMonitorJSONIOProcessLine:189 : Line [{"timestamp": {"second s": 1745975346, "microseconds": 327991}, "event": "MIGRATION", "data": {"status": "completed"}}] Any thoughts on what is going on here to cause this delay? It's clearly somehow related to the sv-iov component of the migration. Thanks much…

2 1

virt-install iscsi direct - Target not found
by kgore4 une 10 Jun '25

10 Jun '25

When I try to use an iscsi direct pool in a "--disk" clause for virt-install, I get error "iSCSI: Failed to connect to LUN : Failed to log in to target. Status: Target not found(515)". I've seen that sort of error before when the initiator name isn't used. The SAN returns different LUNS depending on the initiator. I've run out of ideas on what to try next. Any advice welcome. I've included what I thought was relevent below. klint. The disk parameter to virt-install is (it's part of a script but the variables are correct when executed) [code] --disk vol=${poolName}/unit:0:0:${vLun1},xpath.set="./source/initiator/iqn/@name='iqn.2024-11.localdomain.agbu.agbuvh1:${vName}'" \ [/code] I added the xpath.set as I noticed that the initiator wasn't in the debug output of the disk definition and it didn't work without it either. The iscsi direct pool is defined and it appears to work - it's active and vol-list shows the correct luns. Using --debug on virt install, I can see the drive is detected early in the process as it's got the size of the drive. [code] [Wed, 19 Feb 2025 16:43:33 virt-install 2174805] DEBUG (cli:3554) Parsed --disk volume as: pool=agbu-ldap1 vol=unit:0:0:3 [Wed, 19 Feb 2025 16:43:33 virt-install 2174805] DEBUG (disk:648) disk.set_vol_object: volxml= <volume type='network'> <name>unit:0:0:3</name> <key>ip-10.1.4.3:3260-iscsi-iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3</key> <capacity unit='bytes'>49996103168</capacity> <allocation unit='bytes'>49996103168</allocation> <target> <path>ip-10.1.4.3:3260-iscsi-iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3</path> </target> </volume> [Wed, 19 Feb 2025 16:43:33 virt-install 2174805] DEBUG (disk:650) disk.set_vol_object: poolxml= <pool type='iscsi-direct'> <name>agbu-ldap1</name> <uuid>1c4ae810-9bae-433c-a92f-7d3501b6ba80</uuid> <capacity unit='bytes'>49996103168</capacity> <allocation unit='bytes'>49996103168</allocation> <available unit='bytes'>0</available> <source> <host name='10.1.4.3'/> <device path='iqn.1992-09.com.seagate:01.array.00c0fff6c846'/> <initiator> <iqn name='iqn.2024-11.localdomain.agbu.agbuvh1:agbu-ldap'/> </initiator> </source> </pool> [/code] The generated initial_xml for the disk looks like [code] <disk type="network" device="disk"> <driver name="qemu" type="raw"/> <source protocol="iscsi" name="iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3"> <host name="10.1.4.3"/> <initiator> <iqn name="iqn.2024-11.localdomain.agbu.agbuvh1:agbu-ldap"/> </initiator> </source> <target dev="vda" bus="virtio"/> </disk> [/code] The generated final_xml looks like [code] <disk type="network" device="disk"> <driver name="qemu" type="raw"/> <source protocol="iscsi" name="iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3"> <host name="10.1.4.3"/> <initiator> <iqn name="iqn.2024-11.localdomain.agbu.agbuvh1:agbu-ldap"/> </initiator> </source> <target dev="vda" bus="virtio"/> </disk> [/code] The full error is [code] [Wed, 19 Feb 2025 16:43:35 virt-install 2174805] DEBUG (cli:256) File "/usr/bin/virt-install", line 8, in <module> virtinstall.runcli() File "/usr/share/virt-manager/virtinst/virtinstall.py", line 1233, in runcli sys.exit(main()) File "/usr/share/virt-manager/virtinst/virtinstall.py", line 1226, in main start_install(guest, installer, options) File "/usr/share/virt-manager/virtinst/virtinstall.py", line 974, in start_install fail(e, do_exit=False) File "/usr/share/virt-manager/virtinst/cli.py", line 256, in fail log.debug("".join(traceback.format_stack())) [Wed, 19 Feb 2025 16:43:35 virt-install 2174805] ERROR (cli:257) internal error: process exited while connecting to monitor: 2025-02-19T05:43:35.075695Z qemu-system-x86_64: -blockdev {"driver":"iscsi","portal":"10.1.4.3:3260","target":"iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3","lun":0,"transport":"tcp","initiator-name":"iqn.2024-11.localdomain.agbu.agbuvh1:agbu-ldap","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: iSCSI: Failed to connect to LUN : Failed to log in to target. Status: Target not found(515) [Wed, 19 Feb 2025 16:43:35 virt-install 2174805] DEBUG (cli:259) Traceback (most recent call last): File "/usr/share/virt-manager/virtinst/virtinstall.py", line 954, in start_install domain = installer.start_install( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/share/virt-manager/virtinst/install/installer.py", line 695, in start_install domain = self._create_guest( ^^^^^^^^^^^^^^^^^^^ File "/usr/share/virt-manager/virtinst/install/installer.py", line 637, in _create_guest domain = self.conn.createXML(initial_xml or final_xml, 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/libvirt.py", line 4481, in createXML raise libvirtError('virDomainCreateXML() failed') libvirt.libvirtError: internal error: process exited while connecting to monitor: 2025-02-19T05:43:35.075695Z qemu-system-x86_64: -blockdev {"driver":"iscsi","portal":"10.1.4.3:3260","target":"iqn.1992-09.com.seagate:01.array.00c0fff6c846-lun-3","lun":0,"transport":"tcp","initiator-name":"iqn.2024-11.localdomain.agbu.agbuvh1:agbu-ldap","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: iSCSI: Failed to connect to LUN : Failed to log in to target. Status: Target not found(515) [/code] Things that could affect the answer * What I'm calling a SAN is a seagate exos-x iscsi unit * virtual host is debian 12 * virsh version 9.0.0 * iscsiadm version 2.1.8

2 1

Guest cannot access host HTTP service in NAT
by icefrog1950＠gmail.com 02 Jun '25

02 Jun '25

Hi, I hit a libvirt networking problem that guest cannot access host HTTP service. I debug this issue and tried some efforts. Thanks for your suggestions! Environment ------------- guest IP: 192.168.122.46 (Linux, default NAT, installed using virt-manager) host1 IP: 192.168.3.16 (Centos 8.5 running libvirt and qemu, default libvirt iptable rules) HTTP service: 192.168.3.16:70 (firewall rules have allowed this port) host2: 192.168.3.65:70 (for test only) guest network xml <interface type="network"> <mac address="52:54:00:f5:a8:9f"/> <source network="default" portid="6e8ce7e7-6517-43fa-b113-aaddb6c1bc08" bridge="virbr0"/> <target dev="vnet2"/> <model type="e1000"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/> </interface> 1. guest and host1/host2 can ping each other 2. *guest can visit host2 HTTP 3. *guest cannot visit host1 HTTP When I capture traffic in guest, Wireshark shows: 192.168.122.46 -> 192.168.3.16 // SYN ok 192.168.122.46 <- 192.168.3.16 // Destination unreachable (Port unreachable) guest route table: ------------------ Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.122.1 0.0.0.0 UG 100 0 0 eth0 192.168.122.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 host1 route table: ------------------ Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.3.1 0.0.0.0 UG 100 0 0 enp3s0 192.168.3.0 0.0.0.0 255.255.255.0 U 100 0 0 enp3s0 192.168.122.0 0.0.0.0 255.255.255.0 UG 0 0 0 virbr0 I delete the last rule and add a rule to make sure host1 visits guests will go through 192.168.122.1 ------------------ Destination Gateway Genmask Flags Metric Ref Use Iface (other rules omitted) 192.168.122.0 192.168.122.1 255.255.255.0 UG 0 0 0 virbr0 However, traceroute shows this new rule does not work (which should go to 192.168.122.1 first), and guest cannot visit host1 HTTP request. [!] guest visit http://192.168.3.16:70 does not go through 192.168.122.1 traceroute 192.168.122.46 traceroute to 192.168.122.46 (192.168.122.46), 30 hops max, 60 byte packets 1 192.168.122.46 (192.168.122.46) 0.200 ms 0.194 ms 0.194 ms # guest visit http://192.168.3.65:70 goes through 192.168.122.1 traceroute 192.168.3.65 traceroute to 192.168.3.65 (192.168.3.65), 30 hops max, 60 byte packets 1 192.168.122.1 (192.168.122.1) 0.244 ms 0.050 ms 0.120 ms 2 192.168.3.65 (192.168.3.65) 0.823 ms !X 0.802 ms !X 0.789 ms !X In sum, is there a way to force the guest go through 192.168.122.1 when visiting the hosting machine? ------------------- libvirt iptable rules: -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N LIBVIRT_INP -N LIBVIRT_OUT -N LIBVIRT_FWO -N LIBVIRT_FWI -N LIBVIRT_FWX -A INPUT -j LIBVIRT_INP -A FORWARD -j LIBVIRT_FWX -A FORWARD -j LIBVIRT_FWI -A FORWARD -j LIBVIRT_FWO -A OUTPUT -j LIBVIRT_OUT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT -A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT -A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable # delete this rule does not work -A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable # delete this rule does not work -A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT

2 1

Booting from DVD
by Vince Schielack III 20 May '25

20 May '25

I have need to pass the host optical drive through to a guest and have the guest boot from it. I can set the boot order and make sure the ROM BAR is enabled, but the VM won’t boot from the dvd unless I explicitly select it in the boot menu (3rd option). I’d like to do this to avoid a lengthy and unnecessary dd copy to an ISO. That would essentially double my system installation time by reading the DVD twice.

1 0

Windows 11 fails to install under QEMU/libvirt while VirtualBox succeeds — why?
by Paul Larochelle 20 May '25

20 May '25

Hello, I'm writing to better understand a surprising discrepancy I encountered while attempting to install Windows 11 in different virtualization environments. On my Arch Linux system, I tested two setups: 1. **VirtualBox (GUI):** Windows 11 installs successfully out of the box. 2. **QEMU/KVM with libvirt (manually crafted XML):** Windows 11 refuses to install, stating that the system doesn't meet the requirements. The libvirt domain configuration includes: - UEFI boot using OVMF (`OVMF_CODE.4m.fd` and `OVMF_VARS.ms.fd`) - TPM 2.0 emulator (`tpm-crb` with `backend type='emulator' version='2.0'`) - Secure Boot enabled (verified using Microsoft-signed vars) - 8 GiB of RAM, 4 vCPUs - VirtIO disk + virtio-win ISO attached - QXL or VirtIO video model - `<hyperv>` feature set enabled - Valid boot order (CD-ROM first, then disk) Despite this, Windows 11 either refuses installation with the "This PC can't run Windows 11" message or fails to detect a valid bootable device. In contrast, VirtualBox seems to pass all checks without exposing TPM configuration explicitly or enabling Secure Boot manually. --- **My question:** What is VirtualBox doing under the hood that makes Windows 11 accept the environment without issues? - Is it exposing a minimal TPM implicitly? - Is it modifying SMBIOS/ACPI fields in a way that satisfies Windows validation logic? - Are there known tricks or missing XML elements in libvirt domains to replicate this behavior? My goal is not to bypass Microsoft's requirements, but rather to understand the technical differences and replicate a compliant setup in QEMU/libvirt, ideally without resorting to ISO modifications. Any insight or guidance would be highly appreciated. Best regards, Paul [signature_paul.png] Paul Larochelle 819 342-5487 Envoyé avec la messagerie sécurisée [Proton Mail.](https://proton.me/mail/home)

4 3

libvirt home assistant integration
by Bram Diederik 16 May '25

16 May '25

Hello libvirt users. I had fun with chatgpt (and lots of frustration) and created an integration I missed for some time for home assistant: a libvirt integration. view your libvirt systems inside home assistant, control them. (state and snapshot control) and have the display shown as a camera. https://github.com/Bram-diederik/home-assistant-libvirt-integration/ cheers. Bram Diederik

2 2

Cannot restore internal snapshots since libvirt 11.2.0
by anonym 12 May '25

12 May '25

Hi! Since libvirt 11.2.0 attempting to restore internal snapshots fails for me with: error: Failed to revert snapshot foo1 error: operation failed: load of internal snapshot 'foo1' job failed: Device 'libvirt-1-format' is writable but does not support snapshots I reported this issue to GitLab [0] where there is more info and a reproducer, but I also wanted to ask the wider community here if anyone has experienced this and maybe even come up with a workaround? [0] https://gitlab.com/libvirt/libvirt/-/issues/771 Cheers!

2 1

vm stopped booting
by daggs 22 Apr '25

22 Apr '25

Greetings, my windows vm stoop working, I think it is related kernel version, 6.14.1 works but 6.14.2 seems not to looking at the libvirt-qemu log file shows this: 2025-04-22 17:28:52.082+0000: starting up libvirt version: 11.2.0, qemu version: 9.2.3, kernel: 6.14.3-gentoo, hostname: NCC-5001D.lan LC_ALL=C \ PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin:/usr/lib/llvm/19/bin:/etc/eselect/wine/bin:/opt/cuda/bin \ HOME=/var/lib/libvirt/qemu/domain-1-windows \ USER=root \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-windows/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-windows/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-windows/.config \ /usr/bin/qemu-system-x86_64 \ -name guest=windows,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-windows/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/edk2/OvmfX64/OVMF_CODE_4M.secboot.qcow2","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"qcow2","file":"libvirt-pflash0-storage","backing":null}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/windows_VARS.qcow2","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"qcow2","file":"libvirt-pflash1-storage","backing":null}' \ -machine pc-q35-8.1,usb=off,smm=on,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,hpet=off,acpi=on \ -accel kvm \ -cpu host,migratable=on,topoext=on,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=wateverr,kvm=off,host-cache-info=on,l3-cache=off \ -global driver=cfi.pflash01,property=secure,value=on \ -m size=31457280k \ -overcommit mem-lock=off \ -smp 16,sockets=1,dies=1,clusters=1,cores=8,threads=2 \ -object '{"qom-type":"memory-backend-memfd","id":"ram-node0","share":true,"size":32212254720}' \ -numa node,nodeid=0,cpus=0-15,memdev=ram-node0 \ -uuid e8fc4f01-5b87-4e2d-a30f-fb72318688f7 \ -display none \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=27,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \ -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \ -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \ -device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \ -device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \ -device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \ -device '{"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"}' \ -device '{"driver":"pcie-root-port","port":24,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x3"}' \ -device '{"driver":"pcie-root-port","port":25,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3.0x1"}' \ -device '{"driver":"pcie-root-port","port":26,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x2"}' \ -device '{"driver":"pcie-root-port","port":27,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x3"}' \ -device '{"driver":"pcie-root-port","port":28,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x4"}' \ -device '{"driver":"pcie-root-port","port":29,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x5"}' \ -device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}' \ -device '{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pcie.0","addr":"0xc"}' \ -blockdev '{"driver":"file","filename":"/usr/share/drivers/windows/virtio-win.iso","node-name":"libvirt-1-storage","read-only":true}' \ -device '{"driver":"ide-cd","bus":"ide.1","drive":"libvirt-1-storage","id":"sata0-0-1"}' \ -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-1-windows/fs0-fs.sock \ -device '{"driver":"vhost-user-fs-pci","id":"fs0","chardev":"chr-vu-fs0","queue-size":1024,"tag":"home@linux","bus":"pci.6","addr":"0x0"}' \ -netdev '{"type":"tap","fd":"29","vhost":true,"vhostfd":"31","id":"hostnet0"}' \ -device '{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:43:79:55","bus":"pci.1","addr":"0x0"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/1-windows-swtpm.sock \ -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \ -device '{"driver":"tpm-tis","tpmdev":"tpm-tpm0","id":"tpm0"}' \ -object '{"qom-type":"input-linux","id":"input0","evdev":"/var/lib/libvirt/helpers_state/windows/mouse-event"}' \ -object '{"qom-type":"input-linux","id":"input1","evdev":"/var/lib/libvirt/helpers_state/windows/keyboard-event","repeat":true,"grab_all":true,"grab-toggle":"ctrl-ctrl"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -global ICH9-LPC.noreboot=off \ -watchdog-action reset \ -device '{"driver":"vfio-pci","host":"0000:06:00.0","id":"hostdev0","bus":"pci.5","multifunction":true,"addr":"0x0","romfile":"/var/lib/libvirt/roms/gpu-10de:1c82-uefi.rom"}' \ -device '{"driver":"vfio-pci","host":"0000:06:00.1","id":"hostdev1","bus":"pci.5","addr":"0x0.0x1"}' \ -device '{"driver":"vfio-pci","host":"0000:04:00.1","id":"hostdev2","bus":"pci.7","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0000:08:00.4","id":"hostdev3","bus":"pci.4","addr":"0x0"}' \ -device '{"driver":"vfio-pci","host":"0000:03:00.0","id":"hostdev4","bus":"pci.9","addr":"0x0"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/003/002","id":"hostdev5","bus":"usb.0","port":"1"}' \ -device '{"driver":"usb-host","id":"hostdev6","bus":"usb.0","port":"2"}' \ -device '{"driver":"usb-host","hostdevice":"/dev/bus/usb/001/002","id":"hostdev7","bus":"usb.0","port":"3"}' \ -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on 2025-04-22 17:28:52.084+0000: 5390: debug : virProcessSetMaxMemLock:923 : Locked memory for process 5390 limited to 33285996544 bytes 2025-04-22 17:28:52.084+0000: 5390: debug : virExec:881 : Run hook 0x7fb3b82902a0 0x7fb3bb4435d0 2025-04-22 17:28:52.084+0000: 5390: debug : qemuProcessHook:3136 : Obtaining domain lock 2025-04-22 17:28:52.084+0000: 5390: debug : virDomainLockProcessStart:175 : plugin=0x7fb37016de00 dom=0x7fb3704b9bd0 paused=1 fd=0x7fb3bb4432f0 2025-04-22 17:28:52.084+0000: 5390: debug : virDomainLockManagerNew:130 : plugin=0x7fb37016de00 dom=0x7fb3704b9bd0 withResources=1 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerPluginGetDriver:275 : plugin=0x7fb37016de00 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerNew:298 : driver=0x7fb3c09e5440 type=0 nparams=5 params=0x7fb3bb4431d0 flags=0x1 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerLogParams:96 : key=uuid type=uuid value=e8fc4f01-5b87-4e2d-a30f-fb72318688f7 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerLogParams:89 : key=name type=string value=windows 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerLogParams:77 : key=id type=uint value=1 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerLogParams:77 : key=pid type=uint value=5390 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerLogParams:92 : key=uri type=cstring value=qemu:///system 2025-04-22 17:28:52.084+0000: 5390: debug : virDomainLockManagerNew:143 : Adding leases 2025-04-22 17:28:52.084+0000: 5390: debug : virDomainLockManagerNew:148 : Adding disks 2025-04-22 17:28:52.084+0000: 5390: debug : virDomainLockManagerAddImage:87 : Add disk /usr/share/drivers/windows/virtio-win.iso 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerAddResource:324 : lock=0x7fb3b0044350 type=0 name=/usr/share/drivers/windows/virtio-win.iso nparams=0 params=(nil) flags=0x1 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerAcquire:342 : lock=0x7fb3b0044350 state='<null>' flags=0x3 action=0 fd=0x7fb3bb4432f0 2025-04-22 17:28:52.084+0000: 5390: debug : virLockManagerFree:380 : lock=0x7fb3b0044350 2025-04-22 17:28:52.084+0000: 5390: info : virObjectUnref:378 : OBJECT_UNREF: obj=0x7fb370114eb0 2025-04-22 17:28:52.084+0000: 5390: debug : qemuProcessHook:3181 : Hook complete ret=0 2025-04-22 17:28:52.084+0000: 5390: debug : virExec:883 : Done hook 0 2025-04-22 17:28:52.084+0000: 5390: debug : virExecCommon:456 : Setting child uid:gid to 77:77 with caps 0 2025-04-22 17:28:52.084+0000: 5390: debug : virCommandHandshakeChild:402 : Notifying parent for handshake start on 33 2025-04-22 17:28:52.084+0000: 5390: debug : virCommandHandshakeChild:410 : Waiting on parent for handshake complete on 34 2025-04-22 17:28:52.100+0000: 5390: error : virCommandHandshakeChild:418 : libvirtd quit during handshake: Input/output error libvirt: error : libvirtd quit during handshake: Input/output error 2025-04-22 17:28:52.100+0000: shutting down, reason=failed any hints on what can cause the handshake error?

1 0

virt-manager with SSH and 2FA with TOTP?
by Andreas Haumer 15 Apr '25

15 Apr '25

Hi! (I hope, this is the right list for my question. I already posted it to the debian-user ML, but someone pointed me to this list. Alas, there is no virt-manager ML anymore) In our network we have several Debian systems working as VM host running QEMU+KVM based virtual machines. I usually use virt-manager on my workstation as GUI to connect to the VM host, manage the VMs and also to connect to the VM console if needed. To connect to the VM host I use SSH with public key authentication. On the commandline with virsh this looks like this (example): andreas@ws1:~> virsh -c qemu+ssh://root@maxwell/system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # So far, so good. Recently I decided to increase our internal network security standards and activated 2FA with time-based one-time passwords on several hosts. (The idea is to eventually have 2FA for SSH for all users on all hosts in our network) This works very well and even quite comfortable with authenticator-apps on my smartphone or KeePassXC on my workstation generating the TOTP. Example: andreas@ws1:~> ssh root@mach Enter OTP: Linux mach 6.1.0-32-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.129-1 (2025-03-06) x86_64 root@mach:~# So for a successful SSH connection I now have to enter a valid TOTP (generated by the authenticator app) and then it connects. Connecting to the host with virsh on the commandline also works in a similar way: andreas@ws1:~> virsh -c qemu+ssh://root@mach/system Enter OTP: Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # All fine. Works as designed... When I use virt-manager to connect to the VM host, the GUI opens a dialog asking for the OTP and then connects, showing the list of all configured VMs etc. I can also open the configuration of a given VM, manage and change it. All fine, too... But when I try to use virt-manager to connect to the console of a specific VM, it doesn't work as expected. virt-manager opens a new window for the console, but also endlessly keeps opening password entry dialogs. As soon as I enter the current OTP and klick "ok", another dialog is opened, again asking for another OTP. And so on... (These are one-time passwords, valid for 30 seconds, which cannot be re-used) I can connect to the VM console with a SPICE viewer like remmina using SSH port forwarding like this: andreas@ws1:~> ssh -L 5906:localhost:5906 root@mach Enter OTP: root@mach:~# (where 5906 is the SPICE port for the VM in question) And then use remmina to connect to port 5906 on localhost. This gives me the SPICE console of the VM. Of course, this is not as comfortable as using virt-manager. But with virt-manager I haven't found a way to successfully connect to the VM console with 2FA in place. So, finally, my question: Did anyone on this list manage to use virt-manager to connect to a VM console using SSH with 2FA? Thanks! - andreas -- Andreas Haumer *x Software + Systeme | mailto:andreas@xss.co.at Karmarschgasse 51/2/20 | https://www.xss.co.at/ A-1100 Vienna, Austria | Tel: +43-1-6060114

5 8

Is there any way to apply firewall rules to passt user mode networking?
by notsyncing＠163.com 12 Apr '25

12 Apr '25

Hello, I'm testing user mode network under session mode with libvirt and passt backend, and I found that the guest vm can access the whole host LAN(host is 192.168.1.2, guest ip configured to 192.168.100.2, but guest can access 192.168.1.3, which should not be allowed). Using firewalld dropping all traffic from guest ip has no effect at all. Is there any way to prevent guest accessing host LAN, while keeping guest's internet access? Thanks!

1 0

iptables rule affects vnet0?
by daggs 09 Apr '25

09 Apr '25

Greetings, I have a 2 vms on a host that can communicate with other hosts on the system. the two vms are connected by a virtsw0 bridge (vm1 and vm2) and on of the vms (vm1) has another connection (vnet0) to the host. it seems that the host can only communicate with the vm that has a direct connection to it and not the other vm (virtsw0 allows connection beteen both vms) here are the rules libvirt creates: -P INPUT ACCEPT -P FORWARD ACCEPT -P OUTPUT ACCEPT -N LIBVIRT_FWI -N LIBVIRT_FWO -N LIBVIRT_FWX -N LIBVIRT_INP -N LIBVIRT_OUT -A INPUT -j LIBVIRT_INP -A FORWARD -j LIBVIRT_FWX -A FORWARD -j LIBVIRT_FWI -A FORWARD -j LIBVIRT_FWO -A OUTPUT -j LIBVIRT_OUT -A LIBVIRT_FWI -o virsw0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWO -i virsw0 -j REJECT --reject-with icmp-port-unreachable -A LIBVIRT_FWX -i virsw0 -o virsw0 -j ACCEPT -A LIBVIRT_INP -i virsw0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virsw0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_INP -i virsw0 -p udp -m udp --dport 67 -j ACCEPT -A LIBVIRT_INP -i virsw0 -p tcp -m tcp --dport 67 -j ACCEPT -A LIBVIRT_OUT -o virsw0 -p udp -m udp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virsw0 -p tcp -m tcp --dport 53 -j ACCEPT -A LIBVIRT_OUT -o virsw0 -p udp -m udp --dport 68 -j ACCEPT -A LIBVIRT_OUT -o virsw0 -p tcp -m tcp --dport 68 -j ACCEPT and here are the stats: Chain INPUT (policy ACCEPT 17405 packets, 1677K bytes) pkts bytes target prot opt in out source destination 17434 1688K LIBVIRT_INP all -- * * 0.0.0.0/0 0.0.0.0/0 Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 LIBVIRT_FWX all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 LIBVIRT_FWI all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 LIBVIRT_FWO all -- * * 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 21058 packets, 3788K bytes) pkts bytes target prot opt in out source destination 21058 3788K LIBVIRT_OUT all -- * * 0.0.0.0/0 0.0.0.0/0 Chain LIBVIRT_FWI (1 references) pkts bytes target prot opt in out source destination 0 0 REJECT all -- * virsw0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain LIBVIRT_FWO (1 references) pkts bytes target prot opt in out source destination 0 0 REJECT all -- virsw0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain LIBVIRT_FWX (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- virsw0 virsw0 0.0.0.0/0 0.0.0.0/0 Chain LIBVIRT_INP (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- virsw0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- virsw0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 29 10608 ACCEPT udp -- virsw0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- virsw0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 Chain LIBVIRT_OUT (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- * virsw0 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- * virsw0 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 0 0 ACCEPT udp -- * virsw0 0.0.0.0/0 0.0.0.0/0 udp dpt:68 0 0 ACCEPT tcp -- * virsw0 0.0.0.0/0 0.0.0.0/0 tcp dpt:68 is it possible I cannot access vm2 from the host because of the rules above? Thanks, Dagg

1 0

How to use microVM as a machine type?
by Qu Wenruo 08 Apr '25

08 Apr '25

Hi, Recently I'm hitting a bug related to edk2 that sometimes boot up can be very slow (before loading the bootloader): https://github.com/tianocore/edk2/issues/10765 It looks like the cause is the related to flash IOs. So I want to try using microVM machine type, as it's json file describes the device mapping is "memory", which should work around the slow boot. But I'm unable to setup a machine using microVM machine type in virtual-manager. And manually editing the xml file doesn't let me go any further, as VMM creates a default xml with ACPI feature, which is not supported by microVM. So I'm wondering, what's the proper way to use microVM machine type inside libvirt? Thanks, Qu

2 1

libvirt KVM virtual NIC as ens160 for running a virtual appliance designed for Vmware
by Sascha Frey 31 Mar '25

31 Mar '25

Hi, I want to run a virtual appliance which was designed for Vmware on libvirt Linux/KVM. Unfortunately, they hardcoded the kernel NIC name to ens160 inside that image. I can create a libvirt XML which creates a NIC recognised as ensXX, but not ens160, because libvirt doesn’t allow PCI slots over 0x1f. Is there any possible way to create a virtual NIC ens160 without modifying the image? Thanks.

2 1

Re: Capture pcap for each VM
by Laine Stump 31 Mar '25

31 Mar '25

(added users(a)lists.libvirt.org back into the Cc so that anyone else asking the same question in the future will benefit from the answer) On 3/28/25 8:12 AM, James Liu wrote: > Hi Laine, > > I tried your first solution but somehow it didn't work(the network is > NAT). I edited the xml in virt-manager gui. Whenever I added following > options, and click "Apply", the virt-manager will revert(remove) the > "alias" tag; consequently, the qemu command line couldn't find the > device name('foo' in this example). Sorry, I had meant to point this out but somehow forgot - user-specified aliases must begin with "ua-", otherwise they will be ignored. So instead of using "<alias name='foo'/>", use "<alias name='ua-foo'/>" (and then in the <qemu:arg value> use "netdev=hostua-foo") Also note that your guest should be shutdown while you're making these changes. > > Appreciate your suggestions. > > <interface type="network"> > <mac address="52:54:00:3a:bf:52"/> > <source network="default"/> > <model type="e1000e"/> > <alias name="*foo*"/> # virt-manager will always remove > this tag > <address type="pci" domain="0x0000" bus="0x01" slot="0x00" > function="0x0"/> > </interface> > > <qemu:commandline> > <qemu:arg value="-object"/> > <qemu:arg value="filter-dump,id=f1,netdev=*foo*,file=/tmp/xyzzy.pcap"/> > </qemu:commandline>

1 0

Clarifying isolated network rules in nftables
by Alexey Kashavkin 27 Mar '25

27 Mar '25

Hi, I’m trying to understand how firewall filter works for isolated network in libvirt v11.1.0. When I start the network I can see following rules in nftables: table ip libvirt_network { chain forward { type filter hook forward priority filter; policy accept; counter packets 0 bytes 0 jump guest_cross counter packets 0 bytes 0 jump guest_input counter packets 0 bytes 0 jump guest_output } chain guest_output { iif "virbr3" counter packets 0 bytes 0 reject } chain guest_input { oif "virbr3" counter packets 0 bytes 0 reject } chain guest_cross { iif "virbr3" oif "virbr3" counter packets 0 bytes 0 accept } chain guest_nat { type nat hook postrouting priority srcnat; policy accept; } } But when I start ping from one VM to another on the same isolated network, I don't see an increase in counters in either chain. In the libvirt code, I found a comment in src/network/network_nftables.c: /** * nftablesAddForwardAllowCross: * * Add a rule to @fw to allow traffic to go across @iface (the virtual * network's bridge) from one port to another. This allows all traffic * between guests on the same virtual network. */ But it seems that these rules don't work and are not needed. If I delete this table or some chains, nothing happens. VMs have connectivity with each other on this network. What are these rules for?

2 1

Capture pcap for each VM
by icefrog1950＠gmail.com 26 Mar '25

26 Mar '25

Hi, Is possible to capture pcaps for each VM individually? QEMU supports command line '-object filter-dump, file=test.pcap'. I'm not sure if Libvirt supports this features, or there are better ways to solve this. Many thanks.

2 1

Running DHCP server inside of the Guest VM instead of the host.
by gameplayer2019pl＠tutamail.com 26 Mar '25

26 Mar '25

Hello, I've recently tried to run KEA DHCP server inside of a Debian VM and the following host interface configuration, attached to that VM: ``` <network> <name>netasn-dhcpv6</name> <bridge name="netasn-dhcpv6" stp="on" delay="0"/> <mtu size="1500"/> <mac address="XX:XX:XX:XX:XX:XX"/> <dns enable="no"/> <ip family="ipv6" address="2a14:7581:feb::1" prefix="64"> </ip> </network> ``` But whenever I'm trying to run dhclient on another Debian VM attached to the same network I couldn't obtain the IPv6 lease from KEA DHCP. The host is as well running Debian, 12 version. Is there anyway to make the DHCP work from Guest VM instead of using built-in libvirt DHCP?

2 2

Integrating Open Virtual Network with Libvert Provisioned VMs
by odukoyaonline＠gmail.com 26 Mar '25

26 Mar '25

I am having a little confusion trying to integrate OVN defined networks(I tried with Ovs and it worked fine) with the VMs I have provisioned with Libvirt. I want to ask if there are resource around this and if not, will appreciate input from anyone who has a prior experience with it no matter how little.

1 0

General use of zstd instead of zlib compression
by Michael Niehren 17 Mar '25

17 Mar '25

Hi, currently i use on all VM's qcow2-Images with zlib compression. If i do an Backup, the Backup-Image will be compressed with zstd Level 3 to shrink the image for transfering it over not so fast internet connections. So, why not directly using zstd compression on the images. Are there any reason's not to do that ? As i always use virt-manager for administration, i patched qemu (V9.2.2) to create on default zstd compressed images (only 1 change in line 3525). So newly created images do have compression type zstd, which work's (qemi-img info). I see one unusual thing. If i do an qemi-img convert with compression_type=zstd the size of the converted image is much smaller than the original file while "qemu-img info" shows on both as compression type zstd. Do they use different compression levels, maybe ? If i now do an virsh backup-begin <domain>, the backup-Image does also has a bigger size than the original, while showing zstd as compression type (qemu-img info). If i convert it with the similar command as above, both converted images has nearly the same size. Even if i copy the smaller converted image to the original and boot the vm from the smaller image, the backup-image (after backup-begin) is bigger. So, i am confused. Are there any explanations about the different image sizes or what's going on here ? best regards Michael

3 10

live migration of SR-IOV vm
by Paul B. Henson 17 Mar '25

17 Mar '25

I have a vm using an sr-iov NIC that I'm testing live migration on (Debian 12, OS packages). Per the documentation, I have the sr-iov link set as transient with a pointer to the persistent virtio link: <interface type='network'> <mac address='52:54:00:a1:e0:38'/> <source network='sr-iov-intel-10G-1'/> <vlan> <tag id='400'/> </vlan> <model type='virtio'/> <teaming type='transient' persistent='ua-sr-iov-backup'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> and a persistent virto link down by default: <interface type='direct'> <mac address='52:54:00:a1:e0:38'/> <source dev='eno5np0.400' mode='bridge'/> <model type='virtio'/> <teaming type='persistent'/> <link state='down'/> <alias name='ua-sr-iov-backup'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </interface> The failover driver finds this in the vm: 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 100 0 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master eth0 state DOWN mode DEFAULT group default qlen 1000 10: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000 and the network works fine. However, during migration, the sr-iov interface is removed, but the link on the virtio interface is *not* brought up: 2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master eth0 state DOWN mode DEFAULT group default qlen 1000 resulting in no network for part of the migration. Once the box finished migrating, the replacement sv-iov link is plugged back in, and all is well once again: 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master eth0 state DOWN mode DEFAULT group default qlen 1000 11: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master eth0 state UP mode DEFAULT group default qlen 1000 My understanding was that virsh migrate was supposed to automatically bring up the virtio link when the sr-iov link is removed? Or do I need to explicitly bring it up myself before the migration and take it down after? If I bring up the link manually before the migration, there are a few packets lost at that time, but then none are lost during the migration when the sr-iov link is pulled, or after the migration when I shut that link down again. Ideally no packets would be lost :), but I realize that's unlikely in practice... Thanks...

1 1