- Users - libvirt List Archives

Cannot create vm with machine type raspi2b
by Niccolò Belli 22 Oct '22

22 Oct '22

Hi, I'm trying to use virt-manager to create a Raspberry Pi 2 virtual machine on my Talos 2 ppc64le workstation. In virt-manager I click on "New", then "Manual install", architecture "arm" and machine type "raspi2b". Next I set either "Generic or unknown OS" or "Debian 11" as operating system, I keep the default 1024 memory and 1 cpu and then I disable storage for this virtual machine (because I want to use an existing image with direct kernel boot, see https://raspi.debian.net/tested-images/) I check "Customize configuration before install" and I click "Finish". Unfortunately I get the following error: Error starting installation: Did not find any UEFI binary path for arch 'armv7l' So I've decided to start from scratch and select "virt" as machine type instead of "raspi2b" and change it back later. With "virt" I manage to reach the configuration overview before the installation begins. At this point I have three options for Firmware: Default UEFI UEFI armv7l:/usr/share/qemu/edk2-arm-code.fd UEFI is the default and if I change the machine type to raspi2b I still have the same three options (/usr/share/qemu/edk2-arm-code.fd does indeed exist in my file system). Unfortunately clicking on "Begin Installation" leads me to yet another error: Unable to complete install: 'internal error: Required option 'modelName' is not set for PCI controller with index '1', model 'pcie-root-port' and modelName 'none'' If I change the machine type back to "virt" I don't get any error. I'm using app-emulation/qemu 7.1.0, app-emulation/libvirt-8.8.0 and virt-manager-4.1.0 on Gentoo Linux ppc64le with a 4K page size kernel. Thanks, Niccolo' Belli

1 0

How to merge incremental backups generated with virsh backup-begin?
by Jorge Luiz Correa 19 Oct '22

19 Oct '22

I'm trying to create incremental backups that I could restore when necessary. All backups are being generated fine but I couldn't find a way to recreate an image using all or some of the backup files. For example, my domain is called jammy and is running. jammy-backup.xml: <domainbackup> <incremental></incremental> </domainbackup> jammy-checkpoint.xml <domaincheckpoint> <disks> <disk name="vda" checkpoint="bitmap"/> </disks> </domaincheckpoint> ~# virsh backup-begin jammy jammy-backup.xml jammy-checkpoint.xml * This command with these files generates backup file in /var/lib/libvirt/images appending the checkpoint timestamp in file name (jammy.qcow2 -> jammy.qcow2.TIMESTAMP). ~# virsh checkpoint-list jammy Name Creation Time ----------------------------------------- 1666006874 2022-10-17 08:41:14 -0300 ~# ls -lh /var/lib/libvirt/images -rw-r--r-- 1 libvirt-qemu kvm 2,6G out 18 14:38 jammy.qcow2 -rw------- 1 root root 1,5G out 17 08:41 jammy.qcow2.1666006874 If I create a new domain using jammy.qcow2.1666006874 as disk, everything works good. Then, I've created some incremental backups. jammy-backup.xml: <domainbackup> <incremental>1666006874</incremental> </domainbackup> ~# virsh backup-begin jammy jammy-backup.xml jammy-checkpoint.xml ~# ls -lh /var/lib/libvirt/images -rw-r--r-- 1 libvirt-qemu kvm 2,6G out 18 14:38 jammy.qcow2 -rw------- 1 root root 1,5G out 17 08:41 jammy.qcow2.1666006874 -rw------- 1 root root 247M out 17 14:42 jammy.qcow2.1666010735 At this point, if I need to restore the backup with checkpoint 1666010735, how can I create a new image using jammy.qcow2.1666010735 and jammy.qcow2.1666006874? I would like to merge the incremental backup jammy.qcow2.1666010735 into the full backup jammy.qcow2.1666006874, to get a new image so I can create a new domain using it as disk. Am I doing it the right way? Ubuntu 22.04 LTS KVM libvirt 8.0.0-1ubuntu7.1 qemu 1:6.2+dfsg-2ubuntu6.4 Thank you! -- Jorge Luiz Corrêa Embrapa Agricultura Digital echo "CkpvcmdlIEx1aXogQ29ycmVhCkFu YWxpc3RhIGRlIFJlZGVzIGUgU2VndXJhbm NhCkVtYnJhcGEgQWdyaWN1bHR1cmEgRGln aXRhbCAtIE5USQpBdi4gQW5kcmUgVG9zZW xsbywgMjA5IChCYXJhbyBHZXJhbGRvKQpD RVAgMTMwODMtODg2IC0gQ2FtcGluYXMsIF NQClRlbGVmb25lOiAoMTkpIDMyMTEtNTg4 Mgpqb3JnZS5sLmNvcnJlYUBlbWJyYXBhLm JyCgo="|base64 -d -- __________________________ Aviso de confidencialidade Esta mensagem da Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), empresa publica federal regida pelo disposto na Lei Federal no. 5.851, de 7 de dezembro de 1972, e enviada exclusivamente a seu destinatario e pode conter informacoes confidenciais, protegidas por sigilo profissional. Sua utilizacao desautorizada e ilegal e sujeita o infrator as penas da lei. Se voce a recebeu indevidamente, queira, por gentileza, reenvia-la ao emitente, esclarecendo o equivoco. Confidentiality note This message from Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), a government company established under Brazilian law (5.851/72), is directed exclusively to its addressee and may contain confidential data, protected under professional secrecy rules. Its unauthorized use is illegal and may subject the transgressor to the law's penalties. If you are not the addressee, please send it back, elucidating the failure.

2 1

Re: MemoryBacking options
by Michal Prívozník 17 Oct '22

17 Oct '22

[please keep the list on CC for benefit of the community] On 10/16/22 09:58, Simon Fairweather wrote: > Thanks Michal, Are there any known issues with Memory Backing. We have > QEMU 7.1 and libvirt 8.7. > > If just Memory Backing is set the VM Freezes after a couple of hours and > the allocated CPUs show 100% on the host. > > Seems to happen more on AMD Platforms than Intel. > > Please advise on any info you need, likely this is a QEMU issue. > > <memoryBacking> > <source type='memfd'/> > <access mode='shared'/> > </memoryBacking> Agreed, this smells like QEMU issue. There's nothing obviously wrong with this snipped. I have a VM with just that and run it just fine. Although, I run it for couple of hours max. Michal

1 0

Qeustion about how to use domsetlaunchsecstate command correctly.
by 贺培轩 14 Oct '22

14 Oct '22

Hello, I'm new to libvirt. I have tried to launch a sev vm with secret injection recently, and I found the command domsetlaunchsecstate is what I need. But I had some problem to make it work. Here is what I did to use this command. 1. run command: virsh create sev-guest.xml 2. create secret header file and secret file. 3. run command: virsh domsetlaunchsecstate sev-guest-1 --secrethdr <hdr-filename> --secret <secret-filename> . But it will report this error: SEV: not in correct state. I think it is because the vm is not in a paused state. So how can I launch a sev vm which is in a paused state? How should I revise my xml file? The sev-guest.xml I use is as follows: <domain type="kvm"> <name>sev-guest-1</name> <uuid>d50a4205-40e0-4482-b0dc-f26bb4a1a9ff</uuid> <metadata> <libosinfo:libosinfo xmlns:libosinfo=" http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://ubuntu.com/ubuntu/16.04"/> </libosinfo:libosinfo> </metadata> <memory>4194304</memory> <currentMemory>4194304</currentMemory> <memtune> <hard_limit>4563402</hard_limit> </memtune> <vcpu>32</vcpu> <cpu mode='custom' match='exact' check='partial'> <model fallback='forbid'>EPYC</model> </cpu> <os> <type arch="x86_64" machine="q35">hvm</type> <loader readonly="yes" type="pflash">/data01/OVMF.fd</loader> <nvram template="/data01/OVMF.fd">/var/lib/libvirt/qemu/nvram/sev-guest-1_VARS.fd</nvram> <boot dev="hd"/> </os> <features> <acpi/> <apic/> </features> <clock offset="utc"> <timer name="rtc" tickpolicy="catchup"/> <timer name="pit" tickpolicy="delay"/> <timer name="hpet" present="no"/> </clock> <pm> <suspend-to-mem enabled="no"/> <suspend-to-disk enabled="no"/> </pm> <devices> <emulator>/usr/local/bin/qemu-system-x86_64</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/data01/AMDSEV/sev-guest-1.qcow2"/> <target dev="sda" bus="scsi"/> </disk> <controller type="scsi" index="0" model="virtio-scsi"> <driver iommu="on"/> </controller> <controller type="virtio-serial" index="0"> <driver iommu="on"/> </controller> <controller type="usb" index="0" model="ich9-ehci1"/> <controller type="usb" index="0" model="ich9-uhci1"> <master startport="0"/> </controller> <controller type="usb" index="0" model="ich9-uhci2"> <master startport="2"/> </controller> <controller type="usb" index="0" model="ich9-uhci3"> <master startport="4"/> </controller> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x8'/> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x9'/> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0xa'/> <alias name='pci.3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0xb'/> <alias name='pci.4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <console type="pty"/> <input type="tablet" bus="usb"/> <graphics type="vnc" port="-1" listen="127.0.0.1"/> <video> <model type="vga"/> <address type='pci' slot='0x07'/> </video> <memballoon model="virtio"> <driver iommu="on"/> </memballoon> </devices> <launchSecurity type="sev"> <cbitpos>51</cbitpos> <reducedPhysBits>1</reducedPhysBits> <policy>0x05</policy> <dhCert>XXXXXXXXX</dhCert> <session>XXXXXXXXXXXXXXXXx</session> </launchSecurity> </domain> Thank you in advance, Peixuan

2 1

Caching qemu capabilities and KubeVirt
by Roman Mohr 13 Oct '22

13 Oct '22

Hi, I have a question regarding capability caching in the context of KubeVirt. Since we start in KubeVirt one libvirt instance per VM, libvirt has to re-discover on every VM start the qemu capabilities which leads to a 1-2s+ delay in startup. We already discover the features in a dedicated KubeVirt pod on each node. Therefore I tried to copy the capabilities over to see if that would work. It looks like in general it could work, but libvirt seems to detect a mismatch in the exposed KVM CPU ID in every pod. Therefore it invalidates the cache. The recreated capability cache looks esctly like the original one though ... The check responsible for the invalidation is this: ``` Outdated capabilities for '%s': host cpuid changed ``` So the KVM_GET_SUPPORTED_CPUID call seems to return slightly different values in different containers. After trying out the attached golang scripts in different containers, I could indeed see differences. I can however not really judge what the differences in these KVM function registers mean and I am curious if someone else knows. The files are attached too (as json for easy diffing). Best regards, Roman

2 9

MemoryBacking options
by Simon Fairweather 10 Oct '22

10 Oct '22

Hi, Is there any documentation that states which options can be used together with each other. <domain> ... <memoryBacking> <hugepages> <page size="1" unit="G" nodeset="0-3,5"/> <page size="2" unit="M" nodeset="4"/> </hugepages> <nosharepages/> <locked/> <source type="file|anonymous|memfd"/> <access mode="shared|private"/> <allocation mode="immediate|ondemand" threads='8'/> <discard/> </memoryBacking> ... </domain>

2 1

Re: Why host device disappear in libvirt doman xml?
by Michal Prívozník 29 Sep '22

29 Sep '22

[Once again, please keep the list on CC] On 9/29/22 04:59, 陈新隆 wrote: > Thanks for the detail explanation, it's very helpful. > > I guess : > 1. virsh dumpxml <domain> outputs the live xml(or active xml) Yes, if domain is running then it outputs the live XML, otherwise it outputs the inactive XML. For running domains you can use --inactive flag to get the inactive XML. > 2. virsh edit <domain> edits the inactive xml, so I should cold boot > domain to take effect Correct. Michal

1 0

Re: Why host device disappear in libvirt doman xml?
by Michal Prívozník 28 Sep '22

28 Sep '22

[Please, keep the list on CC for benefit of others] On 9/28/22 10:13, 陈新隆 wrote: > I'm using the `virsh dumpxml <domain name>` to check the xml. When I > first execute this command, there's two hostdev(GPU) in the xml, but > next time I execute the same command these two hostdev elements > disappeared. During this time, the vm didn't restart or stop. Also, > within the vm `lspci | grep -i nvidia` command did not print GPU infos. > > So I was wondering if there's a mechanism in libvirtd detach the hostdev > element without the vm stop or restart. This problem doesn't happen > often, so I am looking a way to reproduce it. > > Can you help me with these questions : > > 1. If I edit the xml manually then detached the hostdev elements from > xml by invoke libvirtd apis, will the vm apply it immediately without > stop or restart? Or I must restart the vm to apply the latest xml ? I'm not sure what you mean. Editing XML manually is different to using libvirt APIs to detach hostdevs. Here's how it works: 1) a domain is defined (say using virsh define file.xml), libvirt parses this XML, keeps it in a memory and stores it "somewhere" (it's under /etc/libvirt/qemu/ but we do not want users to hand edit those files manually as libvirt reads them only on libvirtd/virtqemud restart). This is called inactive XML, because it reflects the inactive state of domain. Sometimes it's also called config XML. 2) when the domain is started (e.g. virsh start), libvirt creates a copy of inactive XML, populates it with runtime information and saves it elsewhere (/run/libvirt/qemu/, but again, we do not want users to hand edit those files). This copy is referred to as live XML. 3) users can alter the live XML using APIs (e.g. to hotplug a device or hotunplug it). The inactive XML can be altered by providing altered XML and defining it again (here, domain name and UUID must match already existing domain). 4) upon domain shutoff, the live XML is thrown away, and finally 5) the inactive XML is never thrown away, until virsh undefine is called. Now, you can see that there's no real connection between live and inactive XMLs and changing one has no affect on the other, except when the domain is cold booted again. Live and inactive XMLs can vary wildly. Therefore, you can have inactive XML with two <hostdev/>-s, and active XML with no <hostdev> at all. NB, hotplug APIs can also be used to alter inactive XML (virsh attach-device --config / virsh detach-device --config / ...) or both at the same time (virsh attach-device --config --live / virsh attach-device --config --live / ...). And what you describe sounds as if those two <hostdev/>-s you saw at domain startup were hotunplugged. The fact that even 'lspci' ran from inside the domain can't find them only supports this theory. And no, libvirt never tries to bring inactive and live XMLs together. It has no intelligence built in to do that and we, developers, do not want such thing either. We might change something that user specifically wanted and I believe nobody likes those "smart" tools that get in your way. > 2. After a host device is hot-unplugged, will libvirtd aware of it then > remove the related `hostdev` element from xml ? Yes. I believe the reasoning is seen in the previous block of my reply. > > On Mon, Sep 26, 2022 at 10:10 PM Michal Prívozník <mprivozn(a)redhat.com > <mailto:mprivozn@redhat.com>> wrote: > > On 9/26/22 15:06, 陈新隆 wrote: > > > > <https://stackoverflow.com/posts/73854544/timeline > <https://stackoverflow.com/posts/73854544/timeline>> > > > > I'm using Kubevirt to manage my virtual machine instances. When I > using > > Kubevirt to create a vm(with two GPUs), kubevirt will generate a > libvirt > > guest domain xml for this vm which includes two GPUs, the domain > xml as > > follow : > > > > |<hostdev mode='subsystem' type='pci' managed='no'> <driver > > name='vfio'/> <source> <address domain='0x0000' bus='0x83' slot='0x00' > > function='0x0'/> </source> <alias name='ua-gpu-gpu0'/> <address > > type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> > > </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <driver > > name='vfio'/> <source> <address domain='0x0000' bus='0x84' slot='0x00' > > function='0x0'/> </source> <alias name='ua-gpu-gpu1'/> <address > > type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> > > </hostdev> | > > > > No one ever edit this domain xml, but these two |hostdev| element > > disappeared in the domain xml. During this time, I've run > > the |gpu_burn| command to do a stress test for these two GPUs. > > > > My question is : > > > > * when will libvirtd change the guest domain xml ? > > * why libvirtd delete these two |hostdev| from domain xml ? > > > > Libvirt does not remove anything from domain XML (except for the > elements it does not understand, but this is not the case). My suspicion > is that you're looking at live XML instead of inactive XML or vice > versa. Libvirt allows guests to be defined (i.e. libvirt manages their > inactive definition). However, a guest can be started with wildly > different configuration (e.g. without those two <hostdev/>-s). OR, they > might have been hot-unplugged. > > I still recommend reading this link: > This article can explain more details: > https://wiki.libvirt.org/page/VM_lifecycle > <https://wiki.libvirt.org/page/VM_lifecycle> Michal

1 0

[Question] Should libvirt update live xml accordingly when guest mac is changed?
by Fangge Jin 26 Sep '22

26 Sep '22

Hi I met an issue when testing trustGuestRxFilters: Attach a macvtap interface with trustGuestRxFilters=’yes’ to vm, then change interface mac address in vm. Should libvirt update interface mac in live vm xml accordingly? If not, vm network will be broken after managedsaving and restoring vm. BR, Fangge Jin Steps: 1. Start a vm 2. Attach a macvtap interface with trustGuestRxFilters=’yes’ to vm <interface type='direct' trustGuestRxFilters='yes'> <source dev='enp175s0v0' mode='passthrough'/> <target dev='macvtap0'/> <model type='virtio'/> <alias name='net1'/> </interface> 3. Check vm xml: # virsh dumpxml uefi --xpath //interface <interface type="direct" trustGuestRxFilters="yes"> <mac address="52:54:00:46:88:8b"/> <source dev="enp175s0v0" mode="passthrough"/> <target dev="macvtap2"/> <model type="virtio"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface> 4. Change interface mac in guest: # ip link set dev enp1s0 address 52:54:00:9d:a1:1e # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 4: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:9d:a1:1e brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:46:88:8b inet 192.168.124.5/24 scope global enp1s0 valid_lft forever preferred_lft forever # ping 192.168.124.4 PING 192.168.124.4 (192.168.124.4) 56(84) bytes of data. 64 bytes from 192.168.124.4: icmp_seq=1 ttl=64 time=0.240 ms 64 bytes from 192.168.124.4: icmp_seq=2 ttl=64 time=0.138 ms --- 192.168.124.4 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms 5. Check vm xml: # virsh dumpxml uefi --xpath //interface <interface type="direct" trustGuestRxFilters="yes"> <mac address="52:54:00:46:88:8b"/> <source dev="enp175s0v0" mode="passthrough"/> <target dev="macvtap2"/> <model type="virtio"/> <alias name="net0"/> <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/> </interface> 6. Check on host: 16: macvtap1@enp175s0v0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 500 link/ether 52:54:00:9d:a1:1e brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe46:888b/64 scope link valid_lft forever preferred_lft forever 7. Do managedsave and restore # virsh managedsave uefi Domain 'uefi' state saved by libvirt # virsh start uefi Domain 'uefi' started 8. Check vm network function: # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 4: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:9d:a1:1e brd ff:ff:ff:ff:ff:ff permaddr 52:54:00:46:88:8b inet 192.168.124.5/24 scope global enp1s0 valid_lft forever preferred_lft forever # ping 192.168.124.4 PING 192.168.124.4 (192.168.124.4) 56(84) bytes of data. --- 192.168.124.4 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2036ms

2 1

Why host device disappear in libvirt doman xml?
by 陈新隆 26 Sep '22

26 Sep '22

<https://stackoverflow.com/posts/73854544/timeline> I'm using Kubevirt to manage my virtual machine instances. When I using Kubevirt to create a vm(with two GPUs), kubevirt will generate a libvirt guest domain xml for this vm which includes two GPUs, the domain xml as follow : <hostdev mode='subsystem' type='pci' managed='no'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x83' slot='0x00' function='0x0'/> </source> <alias name='ua-gpu-gpu0'/> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </hostdev> <hostdev mode='subsystem' type='pci' managed='no'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x84' slot='0x00' function='0x0'/> </source> <alias name='ua-gpu-gpu1'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </hostdev> No one ever edit this domain xml, but these two hostdev element disappeared in the domain xml. During this time, I've run the gpu_burn command to do a stress test for these two GPUs. My question is : - when will libvirtd change the guest domain xml ? - why libvirtd delete these two hostdev from domain xml ?

2 1