About virsh(1) and Postcopy migration
by Prasad Pandit
Hello,
* virsh(1) offers multiple options to initiate Postcopy migration:
1) virsh migrate --postcopy --postcopy-after-precopy
2) virsh migrate --postcopy + virsh migrate-postcopy
3) virsh migrate --postcopy --timeout <N> --timeout-postcopy
When Postcopy migration is invoked via options (2) or (3) above, the migrated guest on the destination host hangs sometimes. But such a hang is not reproducible with option (1) above.
* When using option (1) above, libvirtd(8) waits for the first pass of pre-copy to finish before enabling postcopy migration.
* Does the same waiting happen when using options (2) and (3) above?
===
2024-07-24 14:16:27.448+0000: msg={"execute":"migrate"
2024-07-24 14:16:29.318+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:28:39.737+0000: msg={"execute":"migrate"
2024-07-24 14:28:41.119+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:44:11.684+0000: msg={"execute":"migrate"
2024-07-24 14:44:12.835+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 14:48:00.675+0000: msg={"execute":"migrate"
2024-07-24 14:48:02.319+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 15:03:36.110+0000: msg={"execute":"migrate"
2024-07-24 15:03:37.341+0000: msg={"execute":"migrate-start-postcopy"
2024-07-24 16:05:25.602+0000: msg={"execute":"migrate"
2024-07-24 16:05:26.756+0000: msg={"execute":"migrate-start-postcopy"
===
* While running migration tests with options (2) and (3) above, switch to postcopy appears to happen within 2 seconds of starting migration.
- Is that reasonable time to switch from pre-copy to postcopy?
- Is there an ideal time to wait before switching to postcopy?
* The feature page below suggests to wait until one cycle of RAM migration has completed
-> https://wiki.qemu.org/Features/PostCopyLiveMigration
* I'd much appreciate any clarification/confirmation about this.
Thank you.
---
-Prasad
1 week, 5 days
dnsmasq DHCP server with nwfilter
by Kai
Hello,
I'm trying to set up a nwfilter ruleset, where the client only should be
able to answer to incoming requests and pings. The outbound traffic (LAN
and Internet) shouldn't be working.
I've gut the rules as mentioned below (I moved all filterref inside for
debugging):
<filter name='fwrule-test0' chain='root' priority='-700'>
<uuid>89daa6f3-0300-439d-bbba-4d298b4420f2</uuid>
<rule action='accept' direction='out' priority='100'>
<ip protocol='udp' srcportstart='68' dstportstart='67'/>
</rule>
<rule action='accept' direction='in' priority='101'>
<ip protocol='udp' srcportstart='67' dstportstart='68'/>
</rule>
<rule action='accept' direction='out' priority='200'>
<ip dstipaddr='10.16.136.6'/>
</rule>
<rule action='accept' direction='out' priority='200'>
<ip dstipaddr='10.16.136.9'/>
</rule>
<rule action='accept' direction='in' priority='250'>
<all/>
</rule>
<rule action='accept' direction='inout' priority='300'>
<all state='ESTABLISHED,RELATED'/>
</rule>
<rule action='accept' direction='inout' priority='301'>
<icmp/>
</rule>
<rule action='accept' direction='out' priority='400'>
<udp dstportstart='53'/>
</rule>
<rule action='accept' direction='inout' priority='400'>
<mac protocolid='arp'/>
</rule>
<rule action='drop' direction='out' priority='800'>
<all/>
</rule>
</filter>
When the guest already has a proper IP address, this seems to work on
first sight, the client can't talk to the internet anymore, but is
reachable for TCP and UDP requests.
However, I can't get DHCP working. I'm using the integrated dnsmasq
service for DHCP.
It works again, when I remove the last DROP rule taking care of the rest.
I looked inside tcpdump / Wireshark for the corresponding interface
(virbr4). With the enabled DHCP port rules I can see that DHCP requests
go out to 255.255.255.255.
I also activated dnsmasq logging for the virbr4 instance. Here, I don't
get any DHCP logs.
Without the last DROP rule, I can see clients getting an IP address. I
currently have no idea where to look "in between" as the dnsmasq is
listening von virbr4.
My expectation for DHCP was ports 67 <-> 68 to be open as in the
nwfilter 'allow-dhcp'.
Am I missing here something?
Thank you!
Kai
2 weeks, 2 days
autostart sessiioned vms
by daggs
Greetings,
I'm running sessioned vms which I want to start them up at boot.
I've marked a vm inside a use as autostart, added libvirtd to the boot order and rebooted but it didn't started the vm.
I tried adding libvirt-guests to bott services but my sessioned vm is still not autostarting.
what is the proper way to do so?
Thanks,
Dagg
2 weeks, 4 days
KVM Fails 3D Acceleration on Debian (failed to validate against
schema)
by bo0od＠whonix.org
Full VM xml:
```
<domain type='kvm'>
<name>Whonix-Gateway</name>
<uuid>c011d3d3-8383-47a8-b2cd-d54f883d8af1</uuid>
<genid>bc5b233c-46ea-4bfb-ad99-00f1165a09e6</genid>
<description>Do not change any settings if you do not understand the consequences! Learn more: https://www.whonix.org/wiki/KVM#XML_Settings</description>
<memory dumpCore='off' unit='KiB'>1250000</memory>
<currentMemory unit='KiB'>1250000</currentMemory>
<blkiotune>
<weight>250</weight>
</blkiotune>
<memoryBacking>
<nosharepages/>
<allocation mode='ondemand'/>
<discard/>
</memoryBacking>
<vcpu placement='static' cpuset='0'>1</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-9.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<hap state='on'/>
<pvspinlock state='on'/>
<pmu state='off'/>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough' check='none' migratable='on'/>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup' track='guest'/>
<timer name='kvmclock' present='yes'/>
<timer name='pit' present='no'/>
<timer name='hpet' present='no'/>
<timer name='hypervclock' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/Whonix-Gateway.qcow2'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</disk>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<controller type='usb' index='0' model='qemu-xhci'>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</controller>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pcie-root'/>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x11'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0x12'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0x13'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0x14'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0x15'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
</controller>
<controller type='pci' index='7' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0x16'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
</controller>
<interface type='network'>
<mac address='52:54:00:c1:95:49'/>
<source network='Whonix-External'/>
<model type='virtio'/>
<driver name='qemu'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>
<interface type='network'>
<mac address='52:54:00:bf:80:2e'/>
<source network='Whonix-Internal'/>
<model type='virtio'/>
<driver name='qemu'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</interface>
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' autoport='yes'>
<listen type='address'/>
<clipboard copypaste='no'/>
<filetransfer enable='no'/>
<gl enable='yes'/>
</graphics>
<audio id='1' type='spice'/>
<video>
<model type='virtio' heads='1' primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
<acceleration accel3d='yes' accel2d='yes'/>
<gl enable='yes'/>
</video>
<watchdog model='itco' action='reset'/>
<memballoon model='none'/>
<rng model='virtio'>
<rate bytes='1024' period='1000'/>
<backend model='random'>/dev/random</backend>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</rng>
</devices>
</domain>
```
But it will always show:
```
error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng
Extra element devices in interleave
Element domain failed to validate content
```
Any idea how to resolve this?
2 weeks, 5 days
KVM image fails to resume
by libvirt＠eyal.emu.id.au
I upgraded f38->f40 but left the vm saved rather than shutdown (my bad but here we are)
Attempting to restore the vm with 'virsh restore /data1/VMs/libvirt/images/e4.saved' I get:
error: Failed to restore domain from /data1/VMs/libvirt/images/e4.saved
error: operation failed: guest CPU doesn't match specification: extra features: vmx-ins-outs,vmx-true-ctls,vmx-store-lma,vmx-activity-hlt,vmx-vmwrite-vmexit-fields,vmx-apicv-xapic,vmx-ept,vmx-desc-exit,vmx-rdtscp-exit,vmx-vpid,vmx-wbinvd-exit,vmx-unrestricted-guest,vmx-rdrand-exit,vmx-invpcid-exit,vmx-vmfunc,vmx-shadow-vmcs,vmx-rdseed-exit,vmx-pml,vmx-ept-execonly,vmx-page-walk-4,vmx-ept-2mb,vmx-ept-1gb,vmx-invept,vmx-eptad,vmx-invept-single-context,vmx-invept-all-context,vmx-invvpid,vmx-invvpid-single-addr,vmx-invvpid-all-context,vmx-intr-exit,vmx-nmi-exit,vmx-vnmi,vmx-preemption-timer,vmx-vintr-pending,vmx-tsc-offset,vmx-hlt-exit,vmx-invlpg-exit,vmx-mwait-exit,vmx-rdpmc-exit,vmx-rdtsc-exit,vmx-cr3-load-noexit,vmx-cr3-store-noexit,vmx-cr8-load-exit,vmx-cr8-store-exit,vmx-flexpriority,vmx-vnmi-pending,vmx-movdr-exit,vmx-io-exit,vmx-io-bitmap,vmx-mtf,vmx-msr-bitmap,vmx-monitor-exit,vmx-pause-exit,vmx-secondary-ctls,vmx-exit-nosave-debugctl,vmx-exit-load-perf-global-ctrl,vmx-exit-ack-i
ntr,vmx-exit-save-pat,vmx-exit-load-pat,vmx-exit-save-efer,vmx-exit-load-efer,vmx-exit-save-preemption-timer,vmx-entry-noload-debugctl,vmx-entry-ia32e-mode,vmx-entry-load-perf-global-ctrl,vmx-entry-load-pat,vmx-entry-load-efer,vmx-eptp-switching
Then, trying to resume from the "virtual Machine Manager" UI gets the message:
=====
Error unpausing domain: Requested
operation is not valid: domain is not
running
Error unpausing domain: Requested operation is not valid: domain is not running
Traceback (most recent call last):
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 72, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File "/usr/share/virt-manager/virtManager/asyncjob.py", line 108, in tmpcb
callback(*args, **kwargs)
File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 57, in newfn
ret = fn(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/virt-manager/virtManager/object/domain.py", line 1437, in resume
self._backend.resume()
File "/usr/lib64/python3.12/site-packages/libvirt.py", line 2425, in resume
raise libvirtError('virDomainResume() failed')
libvirt.libvirtError: Requested operation is not valid: domain is not running
I searched for a solution and most say to fiddle with some settings and reboot. I cannot reboot, I want to resume (unpause).
How can I restore the vm without crashing it (throwing away the saved memory), if it even boots this way?
TIA
2 weeks, 6 days
autostart fails on host boot. NAT-bridge doesn't come up fast enough
by Fred Clift
I have 18 VMs that are all supposed to attach to a NAT-bridge.
The bridge definition seems to be ok - it used to work when I didn't have
any guests defined. It worked when I had only a couple guests defined.
Now that I have 18 of them, it seems to take more than a minute for virbr0
to come up, and the guests all fail to autostart because the bridge doesn't
exist yet.
The host system (AlmaLinux 9.4) has dual Xeon(R) CPU E5-2680v4 with plenty
of cores and ram.
Is there some easy way to delay the start of the guests to wait for virbr0
to be up? Or can I specify a boot order? I assume that libvirtd
autostarts networks before guests but I really have no idea... Can I
specify an autostart dependency?
As a stopgap measure I turned 'autostart' off on these and made a
systemd-run bash script that just starts the network, and then starts the
guests all serially. It has worked 10 or so test boots in a row. I'd
prefer not to have to use an external script if possible.
Fred Clift
2 weeks, 6 days
splitting a bt wifi combo between two vms
by daggs
Greetings,
I have the following combo device:
dagg@NCC-5001D ~ $ lsusb -d8087:0029
Bus 001 Device 002: ID 8087:0029 Intel Corp. AX200 Bluetooth
dagg@NCC-5001D ~ $ lspci -vv -s 03:00.0
03:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
Subsystem: Intel Corporation Wi-Fi 6 AX200NGW
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 34
IOMMU group: 13
Region 0: Memory at fc600000 (64-bit, non-prefetchable) [size=16K]
Capabilities: <access denied>
Kernel modules: iwlwifi
I want to pass each of them to a different vm.
should I expect problems?
Thanks,
Dagg
2 weeks, 6 days
Updating authentication for a Ceph (RBD) disk in a live domain
by will.gorman＠joyent.com
Is it possible to update and change the <auth/> for an RBD network disk while the domain the disk is attached to is running and without detaching/reattaching the disk? For example if I have a disk attached like the following:
<disk type='network' device='disk'>
<driver name='qemu' type='raw'/>
<auth username='someuser'>
<secret type='ceph' usage='someuser key'/>
</auth>
<source protocol='rbd' name='somepool/someimage'>
<host name='127.0.0.1' port='3300'/>
</source>
<target dev='sdd' bus='scsi'/>
<alias name='scsi0-0-0-3'/>
<address type='drive' controller='0' bus='0' target='0' unit='3'/>
</disk>
If I want to change the auth to
<auth username='someotheruser'>
<secret type='ceph' usage='someotheruser key'/>
</auth>
can I do that without either attaching/detaching the disk or stopping/restarting the domain?
I've tried `virsh update-device domain disk.xml --live --persistent` using xml identical to the current disk except for the auth and it says "Device updated successfully" but when I check the domain with `dumpxml` I can still see the original auth settings for the disk.
3 weeks, 5 days
cap seach for session libvirt
by daggs
Greetings,
I'm working on allowing a session vm to create a tap iface.
the vm has this defintion:
<interface type='ethernet'>
<mac address='52:54:00:a7:79:6b'/>
<target dev='veth0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</interface>
when I try to start the vm, I get this error: Unable to create tap device veth0: Operation not permitted
searching the code led me to this line: https://github.com/libvirt/libvirt/blob/0caacf47d7b423db9126660fb0382ed56...
I've looked on line and found out I need the net_admin_cap set. so I took the relevant code in to a dedicated test file and using pam_cap I've defined such cap
for the test file, all went well.
so I took it back to virsh and defined that cap to virsh but I'm still getting the same issue, see: https://ibb.co/zHggRQZ
the os is debian 12
any ideas why I'm still getting this error?
Thanks,
Dagg
1 month, 1 week
