August 2019 - Users - libvirt List Archives

[libvirt-users] [virtual interface] detach interface during boot succeed with no changes
by Yalan Zhang 08 Sep '22

08 Sep '22

Hi guys, when I detach an interface from vm during boot (vm boot not finished), it always fail. I'm not sure if there is an existing bug. I have confirmed with someone that for disk, there is similar behavior, if this is also acceptable? # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 2; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </controller> <interface type='network'> <mac address='52:54:00:98:c4:a0'/> <source network='default' bridge='virbr0'/> <target dev='vnet0'/> <model type='rtl8139'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> When I detach after the vm boot, expand the sleep time to 10, it will succeed. # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 10; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully ------- Best Regards, Yalan Zhang IRC: yalzhang Internal phone: 8389413

3 4

[libvirt-users] Add Virtlyst to the list of libvirt users
by Daniel Nicoletti 23 Sep '20

23 Sep '20

Hi, I've just released v1.0.0 of Virtlyst[1] a web tool to manage VMs using Qt/C++/Cutelyst https://dantti.wordpress.com/2018/04/19/announcing-virtlyst-a-web-interface… Thanks. 1 - https://github.com/cutelyst/Virtlyst -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com

4 5

[libvirt-users] Question about disabling UFO on guest
by Bao Nguyen 05 Aug '20

05 Aug '20

Hello everyone, I would like to ask a question regarding to disable UFO of virtio vNIC in my guest. I have read the document at https://libvirt.org/formatdomain.html *host* The csum, gso, tso4, tso6, ecn and ufo attributes with possible values on and off can be used to turn off host offloading options. By default, the supported offloads are enabled by QEMU. *Since 1.2.9 (QEMU only)* The mrg_rxbuf attribute can be used to control mergeable rx buffers on the host side. Possible values are on (default) and off. *Since 1.2.13 (QEMU only)* *guest* The csum, tso4, tso6, ecn and ufo attributes with possible values on and off can be used to turn off guest offloading options. By default, the supported offloads are enabl ed by QEMU. *Since 1.2.9 (QEMU only)* Then I disabled UFO on my vNIC on guest as the following configuration <devices> <interface type='network'> <source network='default'/> <target dev='vnet1'/> <model type='virtio'/> <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'> *<host gso='off' ufo='off' />* *<guest ufo='off'/>* </driver> </interface> </devices> Then I reboot my node to get the change effect and it works. However, can I disable the UFO without touching the host OS? or it always has to disable on both host and guest like that? Thanks, Brs, Natsu

2 1

[libvirt-users] Add support for vhost-user-scsi-pci/vhost-user-blk-pci
by Li Feng 22 Dec '19

22 Dec '19

Hi Guys, And I want to add the vhost-user-scsi-pci/vhost-user-blk-pci support for libvirt. The usage in qemu like this: Vhost-SCSI -chardev socket,id=char0,path=/var/tmp/vhost.0 -device vhost-user-scsi-pci,id=scsi0,chardev=char0 Vhost-BLK -chardev socket,id=char1,path=/var/tmp/vhost.1 -device vhost-user-blk-pci,id=blk0,chardev=char1 What type should I add for libvirt. Type1: <hostdev mode='subsystem' type='vhost-user'> <source protocol='vhost-user-scsi' path='/tmp/vhost-scsi.sock'></source> <alias name="vhost-user-scsi-disk1"/> </hostdev> Type2: <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source protocol='vhost-user' path='/tmp/vhost-scsi.sock'> </source> <target dev='sdb' bus='vhost-user-scsi'/> <boot order='3'/> <alias name='scsi0-0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source protocol='vhost-user' path='/tmp/vhost-blk.sock'> </source> <target dev='vda' bus='vhost-user-blk'/> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> Could anyone give some suggestions? Thanks, Feng Li -- The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

2 5

[libvirt-users] About vhost-user-blk support
by Su Hua 12 Sep '19

12 Sep '19

Hi, everyone, ask a question, which version can fully support the device type of qemu hw/block/vhost-user-blk.c? If so, what should the format of the xml file look like? Regards Su

3 5

[libvirt-users] Recover snapshots from qcow images
by Petr Stodulka 06 Sep '19

06 Sep '19

Hi guys, I had to move to the new laptop week ago and I screw migration of my virtual machines. I recovered my virtual machines on the new laptop (virsh define) using the backed up xml files, but I am missing any file with metadata about snapshots. The original storage is cleaned so I cannot take these files anymore. Using qemu-info I can see my snapshots inside the qcow images, but libvirt doesn't know about them: ########################################### # virsh snapshot-list rhlvm Name Creation Time State ------------------------------- # qemu-img info rhlvm.qcow2 image: rhlvm.qcow2 file format: qcow2 virtual size: 25G (26843545600 bytes) disk size: 2.9G cluster_size: 65536 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 prepared 0 2018-09-05 11:06:06 00:00:00.000 Format specific information: compat: 1.1 lazy refcounts: true refcount bits: 16 corrupt: false ########################################### Is there any nice way to regenerate snapshot metadata for libvirt from the data inside qcow images? I have bunch of VMs so if there is nice way how to recover those data, you will make me really happy :) Thanks, -- Petr Stodulka OS & Application Modernization IRC nicks: pstodulk, skytak Software Engineer Red Hat Czech s.r.o.

2 2

[libvirt-users] virsh list -all zombies
by ofoerster＠posteo.de 29 Aug '19

29 Aug '19

Hello Community, I have a problem with two deleted KVMs. So far we have done everything that is necessary: virsh undefine [kvm-name] --managed-save --snapshots-metadata --remove-all-storage --nvram virsh destroy [kvm-name] and we delete the xml-File and the .img. If we now call "virsh list -all", then the machines are also gone. However, after a restart of libvirtd they reappear in the list, like zombies. What did we do wrong? Or is that even a bug? Kind regards, Oliver

1 1

[libvirt-users] [libvirtd] qemu_process: reset CPU affinity to all enabled CPUs, when runs in custom cpuset
by Valentina Krasnobaeva 29 Aug '19

29 Aug '19

Hello All, Since 4.5.0-23.el7 version (Red Hat 7.7), when I launch pinned VM, libvirtd reset CPU affinity to all enabled in host CPUs, if it runs in custom cpuset. I can't reproduce this behavior with 4.5.0-10.el7_6.12 with the same kernel version (Red Hat 7.7). Libvirt runs in a custom cpuset 'libvirt', where the number of available cpus is restricted to 0,2,4,6,8. And this 'libvirt' cpuset is created in a system with total cpus number: 40 (all cpus are enabled in BIOS) and I have '0-39' range in /sys/fs/cgroup/cpuset/cpuset.cpus. When a VM with pinned vcpus is launched, VM XML config is in attachement: <vcpu placement='static'>2</vcpu> <vcpupin vcpu='0' cpuset='4'/> <vcpupin vcpu='1' cpuset='6'/> I have the following error: # virsh create /tmp/vm1_2vms-2cores_host1.xml error: Failed to create domain from /tmp/vm1_2vms-2cores_host1.xml error: Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied And in the debug log I can see, that /sys/fs/cgroup/cpuset/machine.slice was created with a proper cpuset list: 0,2,4,6,8. This list at the beginning was successfully inherited and set in /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus. But finally libvirtd tries to reset inherited cpus and set there: "0-39" 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 0:cpu at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 1:cpuacct at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1049 : Make group /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupCpuSetInherit:989 : Setting up inheritance /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope -> /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.cpus 2019-08-28 15:11:03.357+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.cpus = 0,2,4,6,8 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus' to '0,2,4,6,8' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.mems 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.mems = 0-1 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.mems' to '0-1' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.memory_migrate 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.memory_migrate = 1 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.memory_migrate' to '1' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller memory 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller devices 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller freezer 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller blkio 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller net_cls 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller perf_event 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1055 : Not creating systemd controller group 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1126 : Done making controllers for group 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus' to '0-39' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: error : virCgroupSetValueStr:806 : Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied The same log snippet with 4.5.0-10.el7_6.12 version, everything works well: 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 0:cpu at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 1:cpuacct at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1049 : Make group /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:989 : Setting up inheritance /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope -> /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.cpus 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.cpus = 0,2,4,6,8 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.cpus' to '0,2,4,6,8' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.mems 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.mems = 0-1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.mems' to '0-1' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.memory_migrate 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.memory_migrate = 1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.memory_migrate' to '1' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller memory 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller devices 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller freezer 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller blkio 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller net_cls 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller perf_event 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1055 : Not creating systemd controller group 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1126 : Done making controllers for group 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.mems' to '0' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/tasks' to '27045' Would someone give me a hint, please, if we need to change settings lied to NUMA or vCPUs in VM XML config according to last changed in libvirtd ? Please, find a bug report about this issue here: https://bugzilla.redhat.com/show_bug.cgi?id=1746517 I suppose, that this may be an impact of one of the following patches applied to 4.5.0-10.el7_6.12 source code version: libvirt-qemu-Rework-setting-process-affinity.patch libvirt-qemu-Fix-qemuProcessInitCpuAffinity.patch libvirt-qemu-Fix-NULL-pointer-access-in-qemuProcessInitCpuAffinity.patch libvirt-qemu-Fix-leak-in-qemuProcessInitCpuAffinity.patch How reproducible: Always Steps to Reproduce: 1. create a cpuset with reduced cpus list and launch libvirtd in it 2. prepare VM XML with pinned vcpus and <numatune> setting 3. virsh create vm.xml Actual results: error: Failed to create domain from /tmp/vm1_2vms-2cores_host1.xml error: Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied Expected results: should work as with a previous version 4.5.0-10.el7_6.12. Additional info: # ps auxf | grep libvirt root 24571 2.3 0.0 1895564 28052 ? Ssl 16:52 0:00 /usr/sbin/libvirtd # grep Cpus_allowed /proc/24571/task/*/status /proc/24571/task/24571/status:Cpus_allowed: 00,00000554 /proc/24571/task/24571/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24572/status:Cpus_allowed: 00,00000554 /proc/24571/task/24572/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24573/status:Cpus_allowed: 00,00000554 /proc/24571/task/24573/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24574/status:Cpus_allowed: 00,00000554 /proc/24571/task/24574/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24575/status:Cpus_allowed: 00,00000554 .. for all threads Processor: "Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz" Sockets: 2, Cores: 20, HyperThreads: 40 socket 0 socket 1 +---------------------+ +---------------------+ | c0 c1 | | c0 c1 | | +-------+ +-------+ | | +-------+ +-------+ | | | 0| 20| | 2| 22| | | | 1| 21| | 3| 23| | | +-------+ +-------+ | | +-------+ +-------+ | | c2 c3 | | c2 c3 | | +-------+ +-------+ | | +-------+ +-------+ | | | 4| 24| | 6| 26| | | | 5| 25| | 7| 27| | | +-------+ +-------+ | | +-------+ +-------+ | | c4 c8 | | c4 c8 | | +-------+ +-------+ | | +-------+ +-------+ | | | 8| 28| | 10| 30| | | | 9| 29| | 11| 31| | | +-------+ +-------+ | | +-------+ +-------+ | | c9 c10 | | c9 c10 | | +-------+ +-------+ | | +-------+ +-------+ | | | 12| 32| | 14| 34| | | | 13| 33| | 15| 35| | | +-------+ +-------+ | | +-------+ +-------+ | | c11 c12 | | c11 c12 | | +-------+ +-------+ | | +-------+ +-------+ | | | 16| 36| | 18| 38| | | | 17| 37| | 19| 39| | | +-------+ +-------+ | | +-------+ +-------+ | +---------------------+ +---------------------+ # cat /sys/fs/cgroup/cpuset/cpuset.cpus 0-39 # cat /sys/fs/cgroup/cpuset/libvirt/cpuset.cpus 2,4,6,8,10 # cat /sys/fs/cgroup/cpuset/machine.slice/cpuset.cpus 0,2,4,6,8

1 0

[libvirt-users] CPU and Memory requirements for host OS ( CentOS 7.6) on Dell Poweredge R630 server
by Kaushal Shriyan 27 Aug '19

27 Aug '19

Hi, I am running Dell R630 Poweredge 1U with 32 cores vCPU's and 96 GB RAM. What should be the minimum numbers of CPU cores and memory that should be reserved for host OS (CentOS 7.6) and the remaining CPU cores and memory resources to be allocated for Guest OS? I look forward to hearing from you and thanks in advance. Best Regards, Kaushal

3 3

[libvirt-users] RLIMIT_MEMLOCK in container environment
by Ihar Hrachyshka 24 Aug '19

24 Aug '19

Hi all, KubeVirt uses libvirtd to manage qemu VMs represented as Kubernetes API resources. In this case, libvirtd is running inside an unprivileged pod, with some host mounts / capabilities added to the pod, needed by libvirtd and other services. One of the capabilities libvirtd requires for successful startup inside a pod is SYS_RESOURCE. This capability is used to adjust RLIMIT_MEMLOCK ulimit value depending on devices attached to the managed guest, both on startup and during hotplug. AFAIU the need to lock the memory is to avoid pages being pushed out from RAM into swap. In KubeVirt world, several libvirtd assumptions do not apply: 1. In Kubernetes environments, swap is usually disabled. (e.g. kubeadm official deployment tool won't even initialize a cluster until you disable it.) This is documented in lots of places, f.e.: https://docs.platform9.com/kubernetes/disabling-swap-kubernetes-node/ (note: while it's vendor docs, regardless it's well known community recommendation.) 2. hotplug is not supported. Domain definition is stable through its whole lifetime. We are working on a series of patches that would remove the need for SYS_RESOURCE capability from the pod running libvirtd: https://github.com/kubevirt/kubevirt/pull/2584 We achieve it by making another, *privileged* component to set RLIMIT_MEMLOCK for libvirtd process using prlimit() syscall, using the value that is higher than the final value libvirtd uses with setrlimit() [Linux kernel will allow to lower the value without the capability.] Since the formula to calculate the actual MEMLOCK value is embedded in libvirt and is not simple to reproduce outside, we pick the upper limit value set for libvirtd process quite conservatively even if ideally we would use the exact same value as libvirtd would do. The estimation code is here: https://github.com/kubevirt/kubevirt/pull/2584/files#diff-6edccf5f0d11c09e7… While the solution works, there are some drawbacks: 1. the value we use for prlimit() is not exactly equal to the final value used by libvirtd; 2. we are doing all this work in environment that is not prone to issues because of disabled swap space. I believe we would benefit from one of the following features on libvirt side (or both): a) expose the memory lock value calculated by libvirtd through libvirt ABI so that we can use it when calling prlimit() on libvirtd process; b) allow to disable setrlimit() calls via libvirtd config file knob or domain definition. Do you think it would be acceptable to have one of these enhancements in libvirtd, or perhaps both, for degenerate cases like KubeVirt? Thanks for attention, Ihar

4 7