- Users - libvirt List Archives

Automate VM migration
by aheath1992＠gmail.com 20 Mar '24

20 Mar '24

Are their any tools that can automatically migrate VMs from one host to another, like in VMware, proxmox, and oVirt?

4 5

Info regarding AMX support and libvirt implications
by Gianluca Cecchi 09 Mar '24

09 Mar '24

Hello, I'm trying to use AMX in my virtual machines. More info on AMX: https://www.intel.com/content/www/us/en/products/docs/accelerator-engines/a… My system in test is currently SLES 15 SP5. I'm also verifying in parallel with Suse (especially regarding the backported features in their 5.14 based kernel), but in the meantime I would like to understand implication, if any, of libvirt in the certification loop I have to analyse. From what I see we have in upstream: . support in the KVM kernel module since 5.17 . support of cpu model SapphireRapids, the first offering AMX as an ISA extension, in QEMU since 7.0 Is there any dependance to check on libvirt too? When I run virsh cpu-models x86_64 Is libvirt sw stack querying qemu directly? Or the kvm kernel module? Or any internal "database" file? From man page it is not clear to me what "known" means: " cpu-models Syntax: cpu-models arch Print the list of CPU models known by libvirt for the specified architecture. Whether a specific hypervisor is able to create a domain which uses any of the printed CPU models is a separate question which can be answered by looking at the domain capabilities XML returned by domcapabilities command. Moreover, for some architectures libvirt does not know any CPU models and the usable CPU models are only limited by the hypervisor. This command will print that all CPU models are accepted for these architectures and the actual list of supported CPU models can be checked in the domain capabilities XML. " In SLES 15 SP5 with: qemu-7.1.0-150500.49.9.2.x86_64 kernel-default-5.14.21-150500.55.49.1.x86_64 libvirtd-*-9.0.0-150500.6.11.1.x86_64 I get # virsh cpu-models x86_64 ... Cascadelake-Server Cascadelake-Server-noTSX Icelake-Client Icelake-Client-noTSX Icelake-Server Icelake-Server-noTSX Cooperlake Snowridge athlon phenom Opteron_G1 Opteron_G2 ... # virsh domcapabilities | grep -i sapphirerapid # In fedora39 with qemu-8.1.3-4.fc39.x86_64 kernel-6.7.5-200.fc39.x86_64 libvirt-*-9.7.0-2.fc39.x86_64 I get # virsh cpu-models x86_64 ... Cascadelake-Server Cascadelake-Server-noTSX Icelake-Client Icelake-Client-noTSX Icelake-Server Icelake-Server-noTSX Cooperlake Snowridge SapphireRapids athlon phenom Opteron_G1 Opteron_G2 ... # virsh domcapabilities | grep -i sapphirerapids <model usable='no' vendor='Intel'>SapphireRapids</model> # because I'm running on a client system without AMX support Thanks in advance, Gianluca

2 3

Raspberry Pi 5 suggestion
by K otgc 07 Mar '24

07 Mar '24

Hello, the libvirt documentation <https://libvirt.org/apps.html> has nothing on installing libvirt to Raspberry Pi 5, so the documentation says to email <https://libvirt.org/contact.html> any suggestions. Cheers, Ken.

3 3

virsh undefine --nvram
by Chuck Lever 06 Mar '24

06 Mar '24

Hello - The virsh(1) man page says: > --nvram and --keep-nvram specify accordingly to delete or keep nvram (/domain/os/nvram/) file. However on my Fedora 39 system with libvirt 9.7.0, using "virsh undefine --nvram" option appears to leave the NVRAM backing file: cel@boudin:~$ ls ~/.config/libvirt/qemu/nvram guestfs-02a4oblo9vg7888d_VARS.qcow2 guestfs-7hmiolqcqwz5b0qs_VARS.qcow2 guestfs-el7lqymnwgzpnr38_VARS.qcow2 guestfs-jp0tfneh205xjrh8_VARS.qcow2 guestfs-mhqbp4gwmqsuu9w3_VARS.qcow2 guestfs-sat7x2kgpg9opjds_VARS.qcow2 guestfs-vhhi476rsq7vh83z_VARS.qcow2 guestfs-177mrq9xrb5589wl_VARS.qcow2 guestfs-88p8heyq1j0sy8bk_VARS.qcow2 guestfs-gxul3j9e1h3po19o_VARS.qcow2 guestfs-k0jf8msw4kh3yuff_VARS.qcow2 guestfs-oakcwpbozezvids8_VARS.qcow2 guestfs-sinmkv97r4k4dzcd_VARS.qcow2 guestfs-vj5cu9hgwcv0f18u_VARS.qcow2 guestfs-6jnim67ceme3h369_VARS.qcow2 guestfs-89g707uzdttxb9hv_VARS.qcow2 guestfs-h6vzt9zqxhoxx0hj_VARS.qcow2 guestfs-k4hw4gkcq8u0eebj_VARS.qcow2 guestfs-pljksgeblxgyf0pw_VARS.qcow2 guestfs-t8bnntykpzm3qvh2_VARS.qcow2 guestfs-ynbxqtj01sn4ovwy_VARS.qcow2 guestfs-6sqhenv6ditrq410_VARS.qcow2 guestfs-at8lndx32y69bjmg_VARS.qcow2 guestfs-ht0uycdn3vwdozad_VARS.qcow2 guestfs-lfy19ftph83jo4v0_VARS.qcow2 guestfs-qj0c2u9w2k4ors6h_VARS.qcow2 guestfs-tmq0h36spng4vtwk_VARS.qcow2 guestfs-7erqks7ks5c8rj38_VARS.qcow2 guestfs-ba2kzqxb1pna12l5_VARS.qcow2 guestfs-is7avsxjtnig5kgb_VARS.qcow2 guestfs-m3nrhs3x58rlget9_VARS.qcow2 guestfs-rilsled4ff9wvk6v_VARS.qcow2 guestfs-veebmlno2okc7c8h_VARS.qcow2 cel@boudin:~$ Or perhaps I do not understand what these files are. -- Chuck Lever

4 3

山东省女子监狱利用犯
by Bai Guoliang 06 Mar '24

06 Mar '24

*你好!* 山东省女子监狱十一监区利用犯人迫害学员事实补充。狱警纵容犯人打人，这就是犯法，犯人在狱中用暴力手段打人本身就是狱内在犯罪，作为监狱警察不但不制止，还纵容打人，隐瞒事实，找假证据诬赖被害人，次数不断增加，致使宋春梅越来越猖狂，这是罪上加罪。犯人是狱内在犯罪，按照监狱内部规定这必须要关禁闭高戒备，但监狱不仅不管还隐瞒事实，找人制造假证据反制被害者。这种行为太恶劣，构成犯罪。 *👉 详情请看附件。* 感谢你阅读这篇文章！ --- *法轮大法是佛法, 是正法。诚念九字真言得福报。* *法轮大法好，真善忍好* 几十年来，在中共的洗脑灌输下，大量的中国人被迫加入了共产党和其附属组织（共青团、少先队）。在「全球退党运动」中，中国人民正在从中共的谎言中觉醒。他们公开声明退出中共，远离邪恶。此时此刻，我们要让世界听到我们声音: 打倒中共恶魔！结束中共！迄今为止, 2024年2月12日 426,111,490 中国民众退出共产党，及其附属组织

1 0

How to monitor domains in regards steal time and other important metrics (VIR_DOMAIN_STATS_VCPU) ?
by Christian Rohmann 02 Mar '24

02 Mar '24

Hey libvirt-users, first allow me to give a little background. We monitor performance metrics of OpenStack Nova VMs using libvirt as hypervisor. We used to run the libvirt prometheus exporter written by zhangjianweibj [1]. This exporter, compared to the one from kumina / tinkoff ([2]) makes use of the DigitalOcean go-libvirt [3], but that should not make much of a difference for my questions. Since the development of that exporter seems to have stalled and we wanted to rework and contribute new features to it, we created a fork [4]. After working trough the various ideas we had and applying them to the code, we proposed the prometheus-community to adopt the exporter [5] to ensure it is maintained and to serve as a reference exporter even. Now to my actual question ... Libvirt exposes per VCPU stats for domains via [6]. I'd like to be able to export those via the exporter. One important metric to me would be things like the steal time (vcpu.<num>.delay), to determine is domains are starting to get cut short or even starve on cpu time. Apparently those metrics are / cannot be expose anymore since the switch to CGroupsV2? Reading [7] or [8] others seem to have run into this. Is this actually still the case, even for more recent kernels? If so, I am wondering if there is an issue being tracked to implement this functionality? How is the steal time reported to the guest if the hypervisor is unable to export this info? Then there are other approaches like vmtop by Digital Ocean [9], which does use info and metrics available via /proc to determine steal time and other vcpu based metrics. So it seems the required data is somewhat available from the kernel? Last but not least I'd like your opinion on what other key metrics are important to monitoring on hypervisors and their guests? Regards Christian [1] https://github.com/zhangjianweibj/prometheus-libvirt-exporter [2] https://github.com/Tinkoff/libvirt-exporter [3] https://github.com/digitalocean/go-libvirt [4] https://github.com/inovex/prometheus-libvirt-exporter [5] https://github.com/prometheus-community/community/issues/50 [6] https://libvirt.org/html/libvirt-libvirt-domain.html#VIR_DOMAIN_STATS_VCPU [7] https://bugzilla.redhat.com/show_bug.cgi?id=2015763 [8] https://bugzilla.redhat.com/show_bug.cgi?id=1796543 [9] https://github.com/digitalocean/vmtop/

3 7

Storage migration Error libvirt
by aheath1992＠gmail.com 02 Mar '24

02 Mar '24

I am trying to migrate on libvirt VM to a separate system with different storage, but keep getting the error `error: internal error: unable to execute QEMU command 'blockdev-add'` I have checked SELinux and it not flagging anything, and I have checked the logs but they are not providing anything useful. Has anyone seen this issue before?

1 1

non-root bridge set-up on Fedora 39 aarch64
by Chuck Lever 29 Feb '24

29 Feb '24

Hello- I'm somewhat new to the libvirt world, and I've encountered a problem that needs better troubleshooting skills than I have. I've searched Google/Ecosia and stackoverflow without finding a solution. I set up libvirt on an x86_64 system without a problem, but on my new aarch64 / Fedora 39 system, virsh doesn't seem to want to start virbr0 when run from my own user account: cel@boudin:~/kdevops$ virsh net-start default error: Failed to start network default error: error creating bridge interface virbr0: Operation not permitted cel@boudin:~/kdevops$ cat /etc/qemu/bridge.conf allow virbr0 cel@boudin:~/kdevops$ Where can I look next? -- Chuck Lever

3 12

Re: restarting libvirtd with sr-iov
by Paul B. Henson 28 Feb '24

28 Feb '24

To reply to myself, I see that the sr-iov pool is initialized by networkCreateInterfacePool in network/bridge_driver.c, and it looks like ports are allocated by networkAllocatePort. The latter looks for a device with 0 connections as defined by netdef->forward.ifs[i].connections, and later bumps that count if the device is successfully allocated. So it seems the answer to my question is that libvirt does maintain this state in memory only, and does not try to re-create it if restarted. As such I don't think there's any way to recover from my situation currently short of shutting down everything :(. networkCreateInterfacePool iterates over all the vf's while configuring the pool. Would there by any way for it to check to see if a vf is already is use while doing so, and initialize connections to 1 so it won't be used until the running vm releases it, or at least generate a warning that the vf is in use and *not* add it to the pool? Thanks...

1 0

restarting libvirtd with sr-iov
by Paul B. Henson 28 Feb '24

28 Feb '24

We're running libvirt under Debian 12, package version 9.0.0-4. Earlier today I made a configuration change and restarted libvirtd. I've done this for years and never had a problem, after restarting it shows all the active storage pools, networks, and virtual machines and worked fine. However, I guess this is the first time I've done it on a system with an sr-iov network pool. After restarting, I was unable to initialize any virtual machines, as it would try to reallocate vf's that were in use by existing machines already running, resulting in an error from qemu. I spent a fair amount of time trying to recover from this, ideally with some way to make libvirt scan existing vm's and update the sr-iov pool in use status, or even some manual way to tell it which ones were in use. Unfortunately, I couldn't find anything and ended up having to shut down all the vm's and then restart them to fix it. Where is the sr-iov pool state stored? Is it just in an in-memory data structure that goes away when libvirt restarts? libvirt doesn't inventory existing vm's and figure out what's in use at startup if that's the case? Is there any way to recover from this situation short of the nuclear "shut down and restart everything" option? Thanks much...

1 0