Running libvirt without dnsmasq
by procmem＠riseup.net
Hi, we are trying to document a way for our users to run libvirt without dnsmasq to reduce attack surface on the host. We are aware that the default network uses it but plan to disable that and use our own custom configured networks instead. Uninstalling dnsmasq causes libvirt to refuse to start even if the default network is no longer running. Is this possible or is this something that needs code changes upstream?
3 months, 2 weeks
trustGuestRxFilters broken after upgrade to Debian 12
by Paul B. Henson
We've been running Debian 11 for a while, using sr-iov:
<network>
<name>sr-iov-intel-10G-1</name>
<uuid>6bdaa4c8-e720-4ea0-9a50-91cb7f2c83b1</uuid>
<forward mode='hostdev' managed='yes'>
<pf dev='eth2'/>
</forward>
</network>
and allocating vf's from the pool:
<interface type='network' trustGuestRxFilters='yes'>
<mac address='52:54:00:08:da:5b'/>
<source network='sr-iov-intel-10G-1'/>
<vlan>
<tag id='50'/>
</vlan>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
After upgrading to Debian 12, when I try to start any vm which uses the
trustGuestRxFilters option, it fails to start with the message:
error: internal error: unable to execute QEMU command 'query-rx-filter':
invalid net client name: hostdev0
If I remove the option, it starts fine (but of course is broken
functionality wise as the option wasn't there just for fun :) ).
Any thoughts on what's going on here? The Debian 12 versions are:
libvirt-daemon/stable,now 9.0.0-4
qemu-system-x86/stable,now 1:7.2+dfsg-7+deb12u3
I see Debian 12 backports has version 8.1.2+ds-1~bpo12+1 of qemu, but no
newer versions of libvirt. I haven't tried the backports version to
see if that resolves the problem.
Thanks much...
4 months, 3 weeks
KVM Fails 3D Acceleration on Debian (failed to validate against
schema)
by bo0od＠whonix.org
Full VM xml:
```
<domain type='kvm'>
<name>Whonix-Gateway</name>
<uuid>c011d3d3-8383-47a8-b2cd-d54f883d8af1</uuid>
<genid>bc5b233c-46ea-4bfb-ad99-00f1165a09e6</genid>
<description>Do not change any settings if you do not understand the consequences! Learn more: https://www.whonix.org/wiki/KVM#XML_Settings</description>
<memory dumpCore='off' unit='KiB'>1250000</memory>
<currentMemory unit='KiB'>1250000</currentMemory>
<blkiotune>
<weight>250</weight>
</blkiotune>
<memoryBacking>
<nosharepages/>
<allocation mode='ondemand'/>
<discard/>
</memoryBacking>
<vcpu placement='static' cpuset='0'>1</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-9.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<hap state='on'/>
<pvspinlock state='on'/>
<pmu state='off'/>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough' check='none' migratable='on'/>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup' track='guest'/>
<timer name='kvmclock' present='yes'/>
<timer name='pit' present='no'/>
<timer name='hpet' present='no'/>
<timer name='hypervclock' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/Whonix-Gateway.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</disk>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<controller type='usb' index='0' model='qemu-xhci'>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</controller>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x11'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x12'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x13'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0x14'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0x15'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
</controller>
<controller type='pci' index='7' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0x16'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
</controller>
<interface type='network'>
<mac address='52:54:00:c1:95:49'/>
<source network='Whonix-External'/>
<model type='virtio'/>
<driver name='qemu'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
<interface type='network'>
<mac address='52:54:00:bf:80:2e'/>
<source network='Whonix-Internal'/>
<model type='virtio'/>
<driver name='qemu'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</interface>
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' autoport='yes'>
<listen type='address'/>
<clipboard copypaste='no'/>
<filetransfer enable='no'/>
<gl enable='yes'/>
</graphics>
<audio id='1' type='spice'/>
<video>
<model type='virtio' heads='1' primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
<acceleration accel3d='yes' accel2d='yes'/>
<gl enable='yes'/>
</video>
<watchdog model='itco' action='reset'/>
<memballoon model='none'/>
<rng model='virtio'>
<rate bytes='1024' period='1000'/>
<backend model='random'>/dev/random</backend>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</rng>
</devices>
</domain>
```
But it will always show:
```
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content
```
Any idea how to resolve this?
6 months
About virsh(1) and Postcopy migration
by Prasad Pandit
Hello,
* virsh(1) offers multiple options to initiate Postcopy migration:
1) virsh migrate --postcopy --postcopy-after-precopy
2) virsh migrate --postcopy + virsh migrate-postcopy
3) virsh migrate --postcopy --timeout <N> --timeout-postcopy
When Postcopy migration is invoked via options (2) or (3) above, the migrated guest on the destination host hangs sometimes. But such a hang is not reproducible with option (1) above.
* When using option (1) above, libvirtd(8) waits for the first pass of pre-copy to finish before enabling postcopy migration.
* Does the same waiting happen when using options (2) and (3) above?
===
2024-07-24 14:16:27.448+0000: msg={"execute":"migrate"
2024-07-24 14:16:29.318+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:28:39.737+0000: msg={"execute":"migrate"
2024-07-24 14:28:41.119+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:44:11.684+0000: msg={"execute":"migrate"
2024-07-24 14:44:12.835+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:48:00.675+0000: msg={"execute":"migrate"
2024-07-24 14:48:02.319+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 15:03:36.110+0000: msg={"execute":"migrate"
2024-07-24 15:03:37.341+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 16:05:25.602+0000: msg={"execute":"migrate"
2024-07-24 16:05:26.756+0000: msg={"execute":"migrate-start-postcopy"
===
* While running migration tests with options (2) and (3) above, switch to postcopy appears to happen within 2 seconds of starting migration.
- Is that reasonable time to switch from pre-copy to postcopy?
- Is there an ideal time to wait before switching to postcopy?
* The feature page below suggests to wait until one cycle of RAM migration has completed
-> https://wiki.qemu.org/Features/PostCopyLiveMigration
* I'd much appreciate any clarification/confirmation about this.
Thank you.
---
-Prasad
6 months, 3 weeks
dnsmasq DHCP server with nwfilter
by Kai
Hello,
I'm trying to set up a nwfilter ruleset, where the client only should be
able to answer to incoming requests and pings. The outbound traffic (LAN
and Internet) shouldn't be working.
I've gut the rules as mentioned below (I moved all filterref inside for
debugging):
<filter name='fwrule-test0' chain='root' priority='-700'>
<uuid>89daa6f3-0300-439d-bbba-4d298b4420f2</uuid>
<rule action='accept' direction='out' priority='100'>
<ip protocol='udp' srcportstart='68' dstportstart='67'/>
</rule>
<rule action='accept' direction='in' priority='101'>
<ip protocol='udp' srcportstart='67' dstportstart='68'/>
</rule>
<rule action='accept' direction='out' priority='200'>
<ip dstipaddr='10.16.136.6'/>
</rule>
<rule action='accept' direction='out' priority='200'>
<ip dstipaddr='10.16.136.9'/>
</rule>
<rule action='accept' direction='in' priority='250'>
<all/>
</rule>
<rule action='accept' direction='inout' priority='300'>
<all state='ESTABLISHED,RELATED'/>
</rule>
<rule action='accept' direction='inout' priority='301'>
<icmp/>
</rule>
<rule action='accept' direction='out' priority='400'>
<udp dstportstart='53'/>
</rule>
<rule action='accept' direction='inout' priority='400'>
<mac protocolid='arp'/>
</rule>
<rule action='drop' direction='out' priority='800'>
<all/>
</rule>
</filter>
When the guest already has a proper IP address, this seems to work on
first sight, the client can't talk to the internet anymore, but is
reachable for TCP and UDP requests.
However, I can't get DHCP working. I'm using the integrated dnsmasq
service for DHCP.
It works again, when I remove the last DROP rule taking care of the rest.
I looked inside tcpdump / Wireshark for the corresponding interface
(virbr4). With the enabled DHCP port rules I can see that DHCP requests
go out to 255.255.255.255.
I also activated dnsmasq logging for the virbr4 instance. Here, I don't
get any DHCP logs.
Without the last DROP rule, I can see clients getting an IP address. I
currently have no idea where to look "in between" as the dnsmasq is
listening von virbr4.
My expectation for DHCP was ports 67 <-> 68 to be open as in the
nwfilter 'allow-dhcp'.
Am I missing here something?
Thank you!
Kai
6 months, 4 weeks
autostart sessiioned vms
by daggs
Greetings,
I'm running sessioned vms which I want to start them up at boot.
I've marked a vm inside a use as autostart, added libvirtd to the boot order and rebooted but it didn't started the vm.
I tried adding libvirt-guests to bott services but my sessioned vm is still not autostarting.
what is the proper way to do so?
Thanks,
Dagg
7 months
KVM image fails to resume
by libvirt＠eyal.emu.id.au
I upgraded f38->f40 but left the vm saved rather than shutdown (my bad but here we are)
Attempting to restore the vm with 'virsh restore /data1/VMs/libvirt/images/e4.saved' I get:
error: Failed to restore domain from /data1/VMs/libvirt/images/e4.saved
error: operation failed: guest CPU doesn't match specification: extra features: vmx-ins-outs,vmx-true-ctls,vmx-store-lma,vmx-activity-hlt,vmx-vmwrite-vmexit-fields,vmx-apicv-xapic,vmx-ept,vmx-desc-exit,vmx-rdtscp-exit,vmx-vpid,vmx-wbinvd-exit,vmx-unrestricted-guest,vmx-rdrand-exit,vmx-invpcid-exit,vmx-vmfunc,vmx-shadow-vmcs,vmx-rdseed-exit,vmx-pml,vmx-ept-execonly,vmx-page-walk-4,vmx-ept-2mb,vmx-ept-1gb,vmx-invept,vmx-eptad,vmx-invept-single-context,vmx-invept-all-context,vmx-invvpid,vmx-invvpid-single-addr,vmx-invvpid-all-context,vmx-intr-exit,vmx-nmi-exit,vmx-vnmi,vmx-preemption-timer,vmx-vintr-pending,vmx-tsc-offset,vmx-hlt-exit,vmx-invlpg-exit,vmx-mwait-exit,vmx-rdpmc-exit,vmx-rdtsc-exit,vmx-cr3-load-noexit,vmx-cr3-store-noexit,vmx-cr8-load-exit,vmx-cr8-store-exit,vmx-flexpriority,vmx-vnmi-pending,vmx-movdr-exit,vmx-io-exit,vmx-io-bitmap,vmx-mtf,vmx-msr-bitmap,vmx-monitor-exit,vmx-pause-exit,vmx-secondary-ctls,vmx-exit-nosave-debugctl,vmx-exit-load-perf-global-ctrl,vmx-exit-ack-i
ntr,vmx-exit-save-pat,vmx-exit-load-pat,vmx-exit-save-efer,vmx-exit-load-efer,vmx-exit-save-preemption-timer,vmx-entry-noload-debugctl,vmx-entry-ia32e-mode,vmx-entry-load-perf-global-ctrl,vmx-entry-load-pat,vmx-entry-load-efer,vmx-eptp-switching
Then, trying to resume from the "virtual Machine Manager" UI gets the message:
=====
Error unpausing domain: Requested
operation is not valid: domain is not
running
Error unpausing domain: Requested operation is not valid: domain is not running
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
ret = fn(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/virt-manager/virtManager/object/domain.py", line 1437, in resume
self._backend.resume()
File "/usr/lib64/python3.12/site-packages/libvirt.py", line 2425, in resume
raise libvirtError('virDomainResume() failed')
libvirt.libvirtError: Requested operation is not valid: domain is not running
I searched for a solution and most say to fiddle with some settings and reboot. I cannot reboot, I want to resume (unpause).
How can I restore the vm without crashing it (throwing away the saved memory), if it even boots this way?
TIA
7 months
autostart fails on host boot. NAT-bridge doesn't come up fast enough
by Fred Clift
I have 18 VMs that are all supposed to attach to a NAT-bridge.
The bridge definition seems to be ok - it used to work when I didn't have
any guests defined. It worked when I had only a couple guests defined.
Now that I have 18 of them, it seems to take more than a minute for virbr0
to come up, and the guests all fail to autostart because the bridge doesn't
exist yet.
The host system (AlmaLinux 9.4) has dual Xeon(R) CPU E5-2680v4 with plenty
of cores and ram.
Is there some easy way to delay the start of the guests to wait for virbr0
to be up? Or can I specify a boot order? I assume that libvirtd
autostarts networks before guests but I really have no idea... Can I
specify an autostart dependency?
As a stopgap measure I turned 'autostart' off on these and made a
systemd-run bash script that just starts the network, and then starts the
guests all serially. It has worked 10 or so test boots in a row. I'd
prefer not to have to use an external script if possible.
Fred Clift
7 months
splitting a bt wifi combo between two vms
by daggs
Greetings,
I have the following combo device:
dagg@NCC-5001D ~ $ lsusb -d8087:0029
Bus 001 Device 002: ID 8087:0029 Intel Corp. AX200 Bluetooth
dagg@NCC-5001D ~ $ lspci -vv -s 03:00.0
03:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 34
IOMMU group: 13
Region 0: Memory at fc600000 (64-bit, non-prefetchable) [size=16K]
Capabilities: <access denied>
Kernel modules: iwlwifi
I want to pass each of them to a different vm.
should I expect problems?
Thanks,
Dagg
7 months
