August 2019 - Users - libvirt List Archives

[libvirt-users] [virtual interface] detach interface during boot succeed with no changes
by Yalan Zhang 08 Sep '22

08 Sep '22

Hi guys, when I detach an interface from vm during boot (vm boot not finished), it always fail. I'm not sure if there is an existing bug. I have confirmed with someone that for disk, there is similar behavior, if this is also acceptable? # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 2; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </controller> <interface type='network'> <mac address='52:54:00:98:c4:a0'/> <source network='default' bridge='virbr0'/> <target dev='vnet0'/> <model type='rtl8139'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> When I detach after the vm boot, expand the sleep time to 10, it will succeed. # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 10; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully ------- Best Regards, Yalan Zhang IRC: yalzhang Internal phone: 8389413

3 4

[libvirt-users] Add Virtlyst to the list of libvirt users
by Daniel Nicoletti 23 Sep '20

23 Sep '20

Hi, I've just released v1.0.0 of Virtlyst[1] a web tool to manage VMs using Qt/C++/Cutelyst https://dantti.wordpress.com/2018/04/19/announcing-virtlyst-a-web-interface… Thanks. 1 - https://github.com/cutelyst/Virtlyst -- Daniel Nicoletti KDE Developer - http://dantti.wordpress.com

4 5

[libvirt-users] Question about disabling UFO on guest
by Bao Nguyen 05 Aug '20

05 Aug '20

Hello everyone, I would like to ask a question regarding to disable UFO of virtio vNIC in my guest. I have read the document at https://libvirt.org/formatdomain.html *host* The csum, gso, tso4, tso6, ecn and ufo attributes with possible values on and off can be used to turn off host offloading options. By default, the supported offloads are enabled by QEMU. *Since 1.2.9 (QEMU only)* The mrg_rxbuf attribute can be used to control mergeable rx buffers on the host side. Possible values are on (default) and off. *Since 1.2.13 (QEMU only)* *guest* The csum, tso4, tso6, ecn and ufo attributes with possible values on and off can be used to turn off guest offloading options. By default, the supported offloads are enabl ed by QEMU. *Since 1.2.9 (QEMU only)* Then I disabled UFO on my vNIC on guest as the following configuration <devices> <interface type='network'> <source network='default'/> <target dev='vnet1'/> <model type='virtio'/> <driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'> *<host gso='off' ufo='off' />* *<guest ufo='off'/>* </driver> </interface> </devices> Then I reboot my node to get the change effect and it works. However, can I disable the UFO without touching the host OS? or it always has to disable on both host and guest like that? Thanks, Brs, Natsu

2 1

[libvirt-users] Add support for vhost-user-scsi-pci/vhost-user-blk-pci
by Li Feng 22 Dec '19

22 Dec '19

Hi Guys, And I want to add the vhost-user-scsi-pci/vhost-user-blk-pci support for libvirt. The usage in qemu like this: Vhost-SCSI -chardev socket,id=char0,path=/var/tmp/vhost.0 -device vhost-user-scsi-pci,id=scsi0,chardev=char0 Vhost-BLK -chardev socket,id=char1,path=/var/tmp/vhost.1 -device vhost-user-blk-pci,id=blk0,chardev=char1 What type should I add for libvirt. Type1: <hostdev mode='subsystem' type='vhost-user'> <source protocol='vhost-user-scsi' path='/tmp/vhost-scsi.sock'></source> <alias name="vhost-user-scsi-disk1"/> </hostdev> Type2: <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source protocol='vhost-user' path='/tmp/vhost-scsi.sock'> </source> <target dev='sdb' bus='vhost-user-scsi'/> <boot order='3'/> <alias name='scsi0-0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source protocol='vhost-user' path='/tmp/vhost-blk.sock'> </source> <target dev='vda' bus='vhost-user-blk'/> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> Could anyone give some suggestions? Thanks, Feng Li -- The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

2 5

[libvirt-users] About vhost-user-blk support
by Su Hua 12 Sep '19

12 Sep '19

Hi, everyone, ask a question, which version can fully support the device type of qemu hw/block/vhost-user-blk.c? If so, what should the format of the xml file look like? Regards Su

3 5

[libvirt-users] Recover snapshots from qcow images
by Petr Stodulka 06 Sep '19

06 Sep '19

Hi guys, I had to move to the new laptop week ago and I screw migration of my virtual machines. I recovered my virtual machines on the new laptop (virsh define) using the backed up xml files, but I am missing any file with metadata about snapshots. The original storage is cleaned so I cannot take these files anymore. Using qemu-info I can see my snapshots inside the qcow images, but libvirt doesn't know about them: ########################################### # virsh snapshot-list rhlvm Name Creation Time State ------------------------------- # qemu-img info rhlvm.qcow2 image: rhlvm.qcow2 file format: qcow2 virtual size: 25G (26843545600 bytes) disk size: 2.9G cluster_size: 65536 Snapshot list: ID TAG VM SIZE DATE VM CLOCK 1 prepared 0 2018-09-05 11:06:06 00:00:00.000 Format specific information: compat: 1.1 lazy refcounts: true refcount bits: 16 corrupt: false ########################################### Is there any nice way to regenerate snapshot metadata for libvirt from the data inside qcow images? I have bunch of VMs so if there is nice way how to recover those data, you will make me really happy :) Thanks, -- Petr Stodulka OS & Application Modernization IRC nicks: pstodulk, skytak Software Engineer Red Hat Czech s.r.o.

2 2

[libvirt-users] virsh list -all zombies
by ofoerster＠posteo.de 29 Aug '19

29 Aug '19

Hello Community, I have a problem with two deleted KVMs. So far we have done everything that is necessary: virsh undefine [kvm-name] --managed-save --snapshots-metadata --remove-all-storage --nvram virsh destroy [kvm-name] and we delete the xml-File and the .img. If we now call "virsh list -all", then the machines are also gone. However, after a restart of libvirtd they reappear in the list, like zombies. What did we do wrong? Or is that even a bug? Kind regards, Oliver

1 1

[libvirt-users] [libvirtd] qemu_process: reset CPU affinity to all enabled CPUs, when runs in custom cpuset
by Valentina Krasnobaeva 29 Aug '19

29 Aug '19

Hello All, Since 4.5.0-23.el7 version (Red Hat 7.7), when I launch pinned VM, libvirtd reset CPU affinity to all enabled in host CPUs, if it runs in custom cpuset. I can't reproduce this behavior with 4.5.0-10.el7_6.12 with the same kernel version (Red Hat 7.7). Libvirt runs in a custom cpuset 'libvirt', where the number of available cpus is restricted to 0,2,4,6,8. And this 'libvirt' cpuset is created in a system with total cpus number: 40 (all cpus are enabled in BIOS) and I have '0-39' range in /sys/fs/cgroup/cpuset/cpuset.cpus. When a VM with pinned vcpus is launched, VM XML config is in attachement: <vcpu placement='static'>2</vcpu> <vcpupin vcpu='0' cpuset='4'/> <vcpupin vcpu='1' cpuset='6'/> I have the following error: # virsh create /tmp/vm1_2vms-2cores_host1.xml error: Failed to create domain from /tmp/vm1_2vms-2cores_host1.xml error: Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied And in the debug log I can see, that /sys/fs/cgroup/cpuset/machine.slice was created with a proper cpuset list: 0,2,4,6,8. This list at the beginning was successfully inherited and set in /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus. But finally libvirtd tries to reset inherited cpus and set there: "0-39" 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 0:cpu at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 1:cpuacct at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupDetect:747 : Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator for pid -1 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1049 : Make group /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/ 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupCpuSetInherit:989 : Setting up inheritance /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope -> /machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.cpus 2019-08-28 15:11:03.357+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.cpus = 0,2,4,6,8 2019-08-28 15:11:03.357+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus' to '0,2,4,6,8' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.mems 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.mems = 0-1 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.mems' to '0-1' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/cpuset.memory_migrate 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.memory_migrate = 1 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.memory_migrate' to '1' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller memory 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller devices 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller freezer 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller blkio 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller net_cls 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller perf_event 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1055 : Not creating systemd controller group 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupMakeGroup:1126 : Done making controllers for group 2019-08-28 15:11:03.358+0000: 25536: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus' to '0-39' 2019-08-28 15:11:03.358+0000: 25536: debug : virFileClose:111 : Closed fd 27 2019-08-28 15:11:03.358+0000: 25536: error : virCgroupSetValueStr:806 : Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d4\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied The same log snippet with 4.5.0-10.el7_6.12 version, everything works well: 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 0:cpu at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 1:cpuacct at /sys/fs/cgroup/cpu,cpuacct in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupDetect:747 : Detected mount/mapping 2:cpuset at /sys/fs/cgroup/cpuset in /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 for pid -1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1049 : Make group /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.837+0000: 26937: debug : virCgroupMakeGroup:1073 : Make controller /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/ 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:989 : Setting up inheritance /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope -> /machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.cpus 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.cpus = 0,2,4,6,8 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.cpus' to '0,2,4,6,8' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.mems 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.mems = 0-1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.mems' to '0-1' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupGetValueStr:832 : Get value /sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/cpuset.memory_migrate 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupCpuSetInherit:999 : Inherit cpuset.memory_migrate = 1 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.memory_migrate' to '1' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller memory 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller devices 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller freezer 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller blkio 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller net_cls 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1062 : Skipping unmounted controller perf_event 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1055 : Not creating systemd controller group 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupMakeGroup:1126 : Done making controllers for group 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/cpuset.mems' to '0' 2019-08-28 16:13:22.838+0000: 26937: debug : virFileClose:111 : Closed fd 29 2019-08-28 16:13:22.838+0000: 26937: debug : virCgroupSetValueStr:796 : Set value '/sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/vcpu1/tasks' to '27045' Would someone give me a hint, please, if we need to change settings lied to NUMA or vCPUs in VM XML config according to last changed in libvirtd ? Please, find a bug report about this issue here: https://bugzilla.redhat.com/show_bug.cgi?id=1746517 I suppose, that this may be an impact of one of the following patches applied to 4.5.0-10.el7_6.12 source code version: libvirt-qemu-Rework-setting-process-affinity.patch libvirt-qemu-Fix-qemuProcessInitCpuAffinity.patch libvirt-qemu-Fix-NULL-pointer-access-in-qemuProcessInitCpuAffinity.patch libvirt-qemu-Fix-leak-in-qemuProcessInitCpuAffinity.patch How reproducible: Always Steps to Reproduce: 1. create a cpuset with reduced cpus list and launch libvirtd in it 2. prepare VM XML with pinned vcpus and <numatune> setting 3. virsh create vm.xml Actual results: error: Failed to create domain from /tmp/vm1_2vms-2cores_host1.xml error: Unable to write to '/sys/fs/cgroup/cpuset/machine.slice/machine-qemu\x2d1\x2dvm12vms\x2d2coreshost1.scope/emulator/cpuset.cpus': Permission denied Expected results: should work as with a previous version 4.5.0-10.el7_6.12. Additional info: # ps auxf | grep libvirt root 24571 2.3 0.0 1895564 28052 ? Ssl 16:52 0:00 /usr/sbin/libvirtd # grep Cpus_allowed /proc/24571/task/*/status /proc/24571/task/24571/status:Cpus_allowed: 00,00000554 /proc/24571/task/24571/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24572/status:Cpus_allowed: 00,00000554 /proc/24571/task/24572/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24573/status:Cpus_allowed: 00,00000554 /proc/24571/task/24573/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24574/status:Cpus_allowed: 00,00000554 /proc/24571/task/24574/status:Cpus_allowed_list: 2,4,6,8,10 /proc/24571/task/24575/status:Cpus_allowed: 00,00000554 .. for all threads Processor: "Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz" Sockets: 2, Cores: 20, HyperThreads: 40 socket 0 socket 1 +---------------------+ +---------------------+ | c0 c1 | | c0 c1 | | +-------+ +-------+ | | +-------+ +-------+ | | | 0| 20| | 2| 22| | | | 1| 21| | 3| 23| | | +-------+ +-------+ | | +-------+ +-------+ | | c2 c3 | | c2 c3 | | +-------+ +-------+ | | +-------+ +-------+ | | | 4| 24| | 6| 26| | | | 5| 25| | 7| 27| | | +-------+ +-------+ | | +-------+ +-------+ | | c4 c8 | | c4 c8 | | +-------+ +-------+ | | +-------+ +-------+ | | | 8| 28| | 10| 30| | | | 9| 29| | 11| 31| | | +-------+ +-------+ | | +-------+ +-------+ | | c9 c10 | | c9 c10 | | +-------+ +-------+ | | +-------+ +-------+ | | | 12| 32| | 14| 34| | | | 13| 33| | 15| 35| | | +-------+ +-------+ | | +-------+ +-------+ | | c11 c12 | | c11 c12 | | +-------+ +-------+ | | +-------+ +-------+ | | | 16| 36| | 18| 38| | | | 17| 37| | 19| 39| | | +-------+ +-------+ | | +-------+ +-------+ | +---------------------+ +---------------------+ # cat /sys/fs/cgroup/cpuset/cpuset.cpus 0-39 # cat /sys/fs/cgroup/cpuset/libvirt/cpuset.cpus 2,4,6,8,10 # cat /sys/fs/cgroup/cpuset/machine.slice/cpuset.cpus 0,2,4,6,8

1 0

[libvirt-users] CPU and Memory requirements for host OS ( CentOS 7.6) on Dell Poweredge R630 server
by Kaushal Shriyan 27 Aug '19

27 Aug '19

Hi, I am running Dell R630 Poweredge 1U with 32 cores vCPU's and 96 GB RAM. What should be the minimum numbers of CPU cores and memory that should be reserved for host OS (CentOS 7.6) and the remaining CPU cores and memory resources to be allocated for Guest OS? I look forward to hearing from you and thanks in advance. Best Regards, Kaushal

3 3

[libvirt-users] RLIMIT_MEMLOCK in container environment
by Ihar Hrachyshka 24 Aug '19

24 Aug '19

Hi all, KubeVirt uses libvirtd to manage qemu VMs represented as Kubernetes API resources. In this case, libvirtd is running inside an unprivileged pod, with some host mounts / capabilities added to the pod, needed by libvirtd and other services. One of the capabilities libvirtd requires for successful startup inside a pod is SYS_RESOURCE. This capability is used to adjust RLIMIT_MEMLOCK ulimit value depending on devices attached to the managed guest, both on startup and during hotplug. AFAIU the need to lock the memory is to avoid pages being pushed out from RAM into swap. In KubeVirt world, several libvirtd assumptions do not apply: 1. In Kubernetes environments, swap is usually disabled. (e.g. kubeadm official deployment tool won't even initialize a cluster until you disable it.) This is documented in lots of places, f.e.: https://docs.platform9.com/kubernetes/disabling-swap-kubernetes-node/ (note: while it's vendor docs, regardless it's well known community recommendation.) 2. hotplug is not supported. Domain definition is stable through its whole lifetime. We are working on a series of patches that would remove the need for SYS_RESOURCE capability from the pod running libvirtd: https://github.com/kubevirt/kubevirt/pull/2584 We achieve it by making another, *privileged* component to set RLIMIT_MEMLOCK for libvirtd process using prlimit() syscall, using the value that is higher than the final value libvirtd uses with setrlimit() [Linux kernel will allow to lower the value without the capability.] Since the formula to calculate the actual MEMLOCK value is embedded in libvirt and is not simple to reproduce outside, we pick the upper limit value set for libvirtd process quite conservatively even if ideally we would use the exact same value as libvirtd would do. The estimation code is here: https://github.com/kubevirt/kubevirt/pull/2584/files#diff-6edccf5f0d11c09e7… While the solution works, there are some drawbacks: 1. the value we use for prlimit() is not exactly equal to the final value used by libvirtd; 2. we are doing all this work in environment that is not prone to issues because of disabled swap space. I believe we would benefit from one of the following features on libvirt side (or both): a) expose the memory lock value calculated by libvirtd through libvirt ABI so that we can use it when calling prlimit() on libvirtd process; b) allow to disable setrlimit() calls via libvirtd config file knob or domain definition. Do you think it would be acceptable to have one of these enhancements in libvirtd, or perhaps both, for degenerate cases like KubeVirt? Thanks for attention, Ihar

4 7

Re: [libvirt-users] Compiling Libvirt on Windows for Hyper V support
by Daniel P. Berrangé 21 Aug '19

21 Aug '19

Re-adding the mailing list CC - please don't drop it. On Tue, Aug 20, 2019 at 11:54:34AM -0400, reza shahriari wrote: > Hi, > > I am using msys2. When I run `./configure` without any parameters and I still see missing pthreads impl. > > I have installed mingw64 cross compiler toolchain. Here is the output of pacman I see when searching for pthreads (they are all installed). > > mingw64/mingw-w64-x86_64-libwinpthread-git 7.0.0.5480.e14d23be-1 (mingw-w64-x86_64-toolchain) [installed] > MinGW-w64 winpthreads library > mingw64/mingw-w64-x86_64-winpthreads-git 7.0.0.5480.e14d23be-1 (mingw-w64-x86_64-toolchain) [installed] > MinGW-w64 winpthreads library > msys/mingw-w64-cross-winpthreads-git 7.0.0.5480.b627284b-1 (mingw-w64-cross-toolchain mingw-w64-cross) [installed] > MinGW-w64 winpthreads library for cross-compiler > > > Also, is there any another way to enable hyper-v domains on Libvirt ? The pthreads library is a mandatory part of libvirt, so we can't skip that. Can you provide your "config.log" file - if it is larger than 200 KB, please compress it, or upload it somewhere Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|

2 6

[libvirt-users] Starting VM fails with: "Setting different DAC user or group on /path... which is already in use" after upgrading to libvirt 5.6.0-1
by Nir Soffer 20 Aug '19

20 Aug '19

Hi, I upgraded to a Fedora 29 host using virt-preview repo to libvirt-daemon-5.6.0-1.fc29.x86_64 The host was using plain Fedora 29 without virt-preview before that. After the upgrade, starting some vms that were running fine fail now with this error: Error starting domain: internal error: child reported (status=125): Requested operation is not valid: Setting different DAC user or group on /home/libvirt/images/voodoo4-os.img which is already in use Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn ret = fn(self, *args, **kwargs) File "/usr/share/virt-manager/virtManager/object/domain.py", line 1279, in startup self._backend.create() File "/usr/lib64/python3.7/site-packages/libvirt.py", line 1089, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirt.libvirtError: internal error: child reported (status=125): Requested operation is not valid: Setting different DAC user or group on /home/libvirt/images/voodoo4-os.img which is already in use These vms we created by creating one vm, and the cloning the vms. I tried to delete the disks and add them back in one of the vms, but the vm still fail with the same error. I hope that someone have a clue what is the issue, and how it can be fixed. Here some details about the setup: vm1: <domain type='kvm'> <name>voodoo4</name> <uuid>0b3aa57a-00b6-4e99-81f9-8f216f85ccaf</uuid> <title>voodoo4 (fedora 29, gluster)</title> <metadata> <libosinfo:libosinfo xmlns:libosinfo=" http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://fedoraproject.org/fedora/29"/> </libosinfo:libosinfo> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <os> <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <vmport state='off'/> </features> <cpu mode='host-model' check='partial'> <model fallback='allow'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sda' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo4-os.img'/> <target dev='vda' bus='virtio'/> <serial>os</serial> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo4-gv0.img'/> <target dev='vdb' bus='virtio'/> <serial>gv0</serial> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo4-gv1.img'/> <target dev='vdc' bus='virtio'/> <serial>gv1</serial> <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0x16'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/> </controller> <controller type='pci' index='8' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='8' port='0x17'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:dd:a4:5c'/> <source bridge='ovirt'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='spice' autoport='yes'> <listen type='address'/> </graphics> <sound model='ich9'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/> </sound> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='2'/> </redirdev> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='3'/> </redirdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <backend model='random'>/dev/urandom</backend> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </rng> </devices> </domain> vm 2: <domain type='kvm'> <name>voodoo5</name> <uuid>8ded8ea2-6524-4fc0-94f6-31667338a5f2</uuid> <title>voodoo5 (fedora 29, gluster)</title> <metadata> <libosinfo:libosinfo xmlns:libosinfo=" http://libosinfo.org/xmlns/libvirt/domain/1.0"> <libosinfo:os id="http://fedoraproject.org/fedora/29"/> </libosinfo:libosinfo> </metadata> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <os> <type arch='x86_64' machine='pc-q35-3.0'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <vmport state='off'/> </features> <cpu mode='host-model' check='partial'> <model fallback='allow'/> </cpu> <clock offset='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo5-os.img'/> <target dev='vda' bus='virtio'/> <serial>os</serial> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo5-gv0.img'/> <target dev='vdb' bus='virtio'/> <serial>gv0</serial> <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/home/libvirt/images/voodoo5-gv1.img'/> <target dev='vdc' bus='virtio'/> <serial>gv1</serial> <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sda' bus='sata'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='usb' index='0' model='qemu-xhci' ports='15'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <controller type='pci' index='5' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='5' port='0x14'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/> </controller> <controller type='pci' index='6' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='6' port='0x15'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/> </controller> <controller type='pci' index='7' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='7' port='0x16'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/> </controller> <controller type='pci' index='8' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='8' port='0x17'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/> </controller> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <interface type='bridge'> <mac address='52:54:00:53:6a:b4'/> <source bridge='ovirt'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='spice' autoport='yes'> <listen type='address'/> </graphics> <sound model='ich9'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/> </sound> <video> <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='2'/> </redirdev> <redirdev bus='usb' type='spicevmc'> <address type='usb' bus='0' port='3'/> </redirdev> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </memballoon> <rng model='virtio'> <backend model='random'>/dev/urandom</backend> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </rng> </devices> </domain> ls -lhZ /home/libvirt/images/voodoo4* # ls -lhZ /home/libvirt/images/voodoo4* -rw-------. 1 root root system_u:object_r:virt_image_t:s0 20G Aug 17 03:55 /home/libvirt/images/voodoo4-gv0.img -rw-------. 1 root root system_u:object_r:virt_image_t:s0 20G Aug 17 03:55 /home/libvirt/images/voodoo4-gv1.img -rw-------. 1 root root system_u:object_r:virt_image_t:s0 50G Aug 17 03:52 /home/libvirt/images/voodoo4-os.img cat /etc/libvirt/storage/images.xml  The related pool: <pool type='dir'> <name>images</name> <uuid>f7190095-947d-442b-b94b-4a99790795bc</uuid> <capacity unit='bytes'>0</capacity> <allocation unit='bytes'>0</allocation> <available unit='bytes'>0</available> <source> </source> <target> <path>/home/libvirt/images</path> <permissions> <mode>0755</mode> <owner>-1</owner> <group>-1</group> </permissions> </target> </pool> Nir

2 2

[libvirt-users] Compiling Libvirt on Windows for Hyper V support
by reza shahriari 20 Aug '19

20 Aug '19

Hi, I am trying to compile Libvirt from the source code on windows using msys2 but keep hitting issues while running `./configure`. …. > checking whether C compiler handles -Wno-suggest-attribute=pure… yes > checking whether C compiler handles -Wno-suggest-attribute=const... yes > checking for how to force completely read-only GOT table… > checking for how to avoid indirect lib deps... -Wl,--no-copy-dt-needed-entries > checking for how to stop undefined symbols at link time… > checking sys/acl.h usability... no > checking sys/acl.h presence… no > checking for sys/acl.h… no > checking for aa_change_profile in -lapparmor… no > checking for pthread_mutexattr_init… no > checking for pthread.h... (cached) no > configure: error: A pthreads impl is required for building libvirt ... Does anyone have any experience compiling Libvirt on a windows machine ? Specifically I am trying to enable hyper-v domain on Libvirt. Thanks, Reza Shahriari

3 4

[libvirt-users] Libvirt API for attaching volume to domain
by Varsha Verma 15 Aug '19

15 Aug '19

Hi, I wanted to know if there is any Python API exposed for libvirt using which I can attach a libvirt volume to a libvirt domain/VM. I intend to do something similar to the `attach-disk` command of virsh using python. -- *Regards,* *Varsha Verma* *Final Year Undergraduate* *Department of Electrical Engineering* *IIT-BHU, Varanasi*

3 4

[libvirt-users] does virsh have a history with timestamps ?
by Lentes, Bernd 13 Aug '19

13 Aug '19

Hi, i knwo that virsh has its own history, but are somewhere the respective timestamps logged ? Bernd -- Bernd Lentes Systemadministration Institut für Entwicklungsgenetik Gebäude 35.34 - Raum 208 HelmholtzZentrum münchen bernd.lentes(a)helmholtz-muenchen.de phone: +49 89 3187 1241 phone: +49 89 3187 3827 fax: +49 89 3187 2294 http://www.helmholtz-muenchen.de/idg Perfekt ist wer keine Fehler macht Also sind Tote perfekt Helmholtz Zentrum Muenchen Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH) Ingolstaedter Landstr. 1 85764 Neuherberg www.helmholtz-muenchen.de Aufsichtsratsvorsitzende: MinDir'in Prof. Dr. Veronika von Messling Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich Bassler, Kerstin Guenther Registergericht: Amtsgericht Muenchen HRB 6466 USt-IdNr: DE 129521671

2 1

[libvirt-users] OVS / KVM / libvirt / MTU
by Sven Vogel 09 Aug '19

09 Aug '19

Hey there, I hope anyone can bring some light in the following problem. <interface type='bridge'> <source bridge='cloudbr0'/> <mtu size='2000'/> <mac address='02:00:74:76:00:01'/> <model type='virtio'/> <virtualport type='openvswitch'> </virtualport> <vlan trunk='no'> <tag id='2097'/> </vlan><link state='up'/> </interface> we have an base bridge for example and cloudbr0. After we add an mtu to the vm bridge here it seems the base bridge gets the same mtu like the vnet adapter. Is this normal behaviour of libvirt together with OVS? —ovs-vsctl show 5b154321-534d-413e-9761-60476ae06640 Bridge "cloudbr0" Port "cloudbr0" Interface "cloudbr0" type: internal —MTU from the bridge after set an MTU into the XML File (before we had here 9000) mtu : 2000 mtu_request : [] name : "cloudbr0" ofport : 65534 —MTU of the vnet interface mac_in_use : "fe:00:74:76:00:01" mtu : 1450 mtu_request : 1450 name : „vnet2" Thanks for help… __ Sven Vogel Teamlead Platform EWERK RZ GmbH Brühl 24, D-04109 Leipzig P +49 341 42649 - 11 F +49 341 42649 - 18 S.Vogel(a)ewerk.com www.ewerk.com Geschäftsführer: Dr. Erik Wende, Hendrik Schubert, Frank Richter Registergericht: Leipzig HRB 17023 Zertifiziert nach: ISO/IEC 27001:2013 DIN EN ISO 9001:2015 DIN ISO/IEC 20000-1:2011 EWERK-Blog | LinkedIn | Xing | Twitter | Facebook Auskünfte und Angebote per Mail sind freibleibend und unverbindlich. Disclaimer Privacy: Der Inhalt dieser E-Mail (einschließlich etwaiger beigefügter Dateien) ist vertraulich und nur für den Empfänger bestimmt. Sollten Sie nicht der bestimmungsgemäße Empfänger sein, ist Ihnen jegliche Offenlegung, Vervielfältigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte informieren Sie in diesem Fall unverzüglich den Absender und löschen Sie die E-Mail (einschließlich etwaiger beigefügter Dateien) von Ihrem System. Vielen Dank. The contents of this e-mail (including any attachments) are confidential and may be legally privileged. If you are not the intended recipient of this e-mail, any disclosure, copying, distribution or use of its contents is strictly prohibited, and you should please notify the sender immediately and then delete it (including any attachments) from your system. Thank you.

4 7

[libvirt-users] Vm in state "in shutdown"
by 马昊骢 Ianmalcolm Ma 09 Aug '19

09 Aug '19

Description of problem: libvirt 3.9 on CentOS Linux release 7.4.1708 (kernel 3.10.0-693.21.1.el7.x86_64) on Qemu version 2.10.0 I’m currently facing a strange situation. Sometimes my vm is shown by ‘virsh list’ as in state “in shutdown” but there is no qemu-kvm process linked to it. Libvirt log when “in shutdown” state occur is as follows: “d470c3b284425b9bacb34d3b5f3845fe” is vm’s name, remoteDispatchDomainMemoryStats API is called by ‘collectd’, which is used to collect some vm running states and host information once every 30’s. 2019-07-25 14:23:58.706+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: POWERDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.413 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.202 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: STOP, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.203 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.203 secs 2019-07-25 14:23:59.629+0000: 15818: error : qemuMonitorIORead:597 : Unable to read from monitor: Connection reset by peer 2019-07-25 14:23:59.629+0000: 121081: warning : qemuProcessEventHandler:4840 : vm: d470c3b284425b9bacb34d3b5f3845fe, event: 6 locked 2019-07-25 14:23:59.629+0000: 15822: error : qemuMonitorJSONCommandWithFd:364 : internal error: Missing monitor reply object 2019-07-25 14:24:29.483+0000: 15821: warning : qemuGetProcessInfo:1468 : cannot parse process status data 2019-07-25 14:24:29.829+0000: 15823: warning : qemuDomainObjBeginJobInternal:4391 : Cannot start job (modify, none) for domain d470c3b284425b9bacb34d3b5f3845fe; current job is (query, none) owned by (15822 remoteDispatchDomainMemoryStats, 0 <null>) for (30s, 0s) 2019-07-25 14:24:29.829+0000: 15823: error : qemuDomainObjBeginJobInternal:4403 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-07-25 14:24:29.829+0000: 121081: warning : qemuDomainObjBeginJobInternal:4391 : Cannot start job (destroy, none) for domain d470c3b284425b9bacb34d3b5f3845fe; current job is (query, none) owned by (15822 remoteDispatchDomainMemoryStats, 0 <null>) for (30s, 0s) 2019-07-25 14:24:29.829+0000: 121081: error : qemuDomainObjBeginJobInternal:4403 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-07-25 14:24:29.829+0000: 121081: warning : qemuProcessEventHandler:4875 : vm: d470c3b284425b9bacb34d3b5f3845fe, event: 6, cost 31.459 secs I’ve tried to find out how did this problem happened. I analyzed the execution process of the job and speculated that the problem occurred as follows: step one: libvirt send command 'system_powerdown' to qemu. step two: libvirt receive qemu monitor close event, and then deal EOF event. step three: remoteDispatchDomainMemoryStats job start on the same vm. step four: work thread dealing stop job is waited on job.cond, the timeout is 30s. It seems that the remoteDispatchDomainMemoryStats job is too slow to let the stop job wait timeout. Then I tried to reproduce this process. The step is as follows: First step: add a sleep on ‘qemuProcessEventHandler' by using pthread_cond_timedwait. So can execute 'virsh dommemstat active’ command at this time interval. Second step: start a vm to test Third step: execute 'virsh shutdown active’ to shutdown the vm Fourth step: execute 'virsh dommemstat active’ when stop job is sleep. Then it works. The test vm state became to 'in shutdown’, and the libvirt log is as follows: “active” is my test vm’s name. 2019-08-05 08:39:57.001+0000: 25889: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: modify (vm=0x7f7bbc145fe0 name=active, current job=none async=none) 2019-08-05 08:39:57.003+0000: 25889: warning : qemuDomainObjEndJob:4522 : Stopping job: modify (async=none vm=0x7f7bbc145fe0 name=active) 2019-08-05 08:39:57.003+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: POWERDOWN, vm: active, cost 0.008 secs 2019-08-05 08:39:57.854+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: active, cost 1.709 secs 2019-08-05 08:39:57.875+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: STOP, vm: active, cost 1.751 secs 2019-08-05 08:39:57.875+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: active, cost 1.751 secs 2019-08-05 08:39:57.915+0000: 25881: warning : qemuMonitorIO:756 : Error on monitor <null> 2019-08-05 08:39:57.915+0000: 25881: warning : qemuMonitorIO:777 : Triggering EOF callback 2019-08-05 08:39:57.915+0000: 26915: warning : qemuProcessEventHandler:4822 : usleep 20s 2019-08-05 08:40:01.004+0000: 25886: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: query (vm=0x7f7bbc145fe0 name=active, current job=none async=none) ^@2019-08-05 08:40:17.915+0000: 26915: warning : qemuProcessEventHandler:4845 : vm=0x7f7bbc145fe0, event=6 2019-08-05 08:40:17.915+0000: 26915: warning : qemuProcessEventHandler:4851 : vm: active, event: 6 locked 2019-08-05 08:40:17.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: destroy (vm=0x7f7bbc145fe0 name=active, current job=query async=none) 2019-08-05 08:40:17.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4344 : Waiting for job (vm=0x7f7bbc145fe0 name=active), job 26915 qemuProcessEventHandler, owned by 25886 remoteDispatchDomainMemoryStats ^@^@^@2019-08-05 08:40:47.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4405 : Cannot start job (destroy, none) for domain active; current job is (query, none) owned by (25886 remoteDispatchDomainMemoryStats, 0 <null>) for (46s, 0s) 2019-08-05 08:40:47.915+0000: 26915: error : qemuDomainObjBeginJobInternal:4417 : 操作超时：cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-08-05 08:40:47.915+0000: 26915: warning : qemuProcessEventHandler:4886 : vm: active, event: 6, cost 31.830 secs The threads backtraces is as follows: [Switching to thread 17 (Thread 0x7f12fddf5700 (LWP 32580))] #2 0x00007f12eb26df21 in qemuMonitorSend (mon=mon@entry=0x7f12d0013af0, msg=msg@entry=0x7f12fddf46e0) at qemu/qemu_monitor.c:1075 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { (gdb) l 1070 PROBE(QEMU_MONITOR_SEND_MSG, 1071 "mon=%p msg=%s fd=%d", 1072 mon, mon->msg->txBuffer, mon->msg->txFD); 1073 1074 while (mon && mon->msg && !mon->msg->finished) { 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { 1076 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1077 _("Unable to wait on monitor condition")); 1078 goto cleanup; 1079 } (gdb) p *msg $2 = {txFD = -1, txBuffer = 0x7f12b8000d50 "{\"execute\":\"qom-list\",\"arguments\":{\"path\":\"/machine/peripheral\"},\"id\":\"libvirt-17\"}\r\n", txOffset = 0, txLength = 85, rxBuffer = 0x0, rxLength = 0, rxObject = 0x0, finished = false, passwordHandler = 0x0, passwordOpaque = 0x0} (gdb) bt #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f130d738316 in virCondWait (c=c@entry=0x7f12d0013b28, m=m@entry=0x7f12d0013b00) at util/virthread.c:154 #2 0x00007f12eb26df21 in qemuMonitorSend (mon=mon@entry=0x7f12d0013af0, msg=msg@entry=0x7f12fddf46e0) at qemu/qemu_monitor.c:1075 #3 0x00007f12eb283b40 in qemuMonitorJSONCommandWithFd ( mon=mon@entry=0x7f12d0013af0, cmd=cmd@entry=0x7f12b80008c0, scm_fd=scm_fd@entry=-1, reply=reply@entry=0x7f12fddf4780) at qemu/qemu_monitor_json.c:355 #4 0x00007f12eb2904b9 in qemuMonitorJSONCommand (reply=0x7f12fddf4780, cmd=0x7f12b80008c0, mon=0x7f12d0013af0) at qemu/qemu_monitor_json.c:385 ... I found that on function ‘qemuMonitorUpdateWatch’, if mon->watch is zero, then event would’t be update. But 'qemuMonitorSend' would still wait on mon->notify. So the remoteDispatchDomainMemoryStats job is blocked, and then the stop job blocked too. 618 static void 619 qemuMonitorUpdateWatch(qemuMonitorPtr mon) 620 { 621 int events = 622 VIR_EVENT_HANDLE_HANGUP | 623 VIR_EVENT_HANDLE_ERROR; 624 625 if (!mon->watch) 626 return; 627 628 if (mon->lastError.code == VIR_ERR_OK) { 629 events |= VIR_EVENT_HANDLE_READABLE; 630 631 if ((mon->msg && mon->msg->txOffset < mon->msg->txLength) && 632 !mon->waitGreeting) 633 events |= VIR_EVENT_HANDLE_WRITABLE; 634 } 635 636 virEventUpdateHandle(mon->watch, events); 637 } I try to fix this bugs by add a judgment before qemuMonitorUpdateWatch, and it seem works. 1046 int 1047 qemuMonitorSend(qemuMonitorPtr mon, 1048 qemuMonitorMessagePtr msg) 1049 { 1050 int ret = -1; 1051 1052 /* Check whether qemu quit unexpectedly */ 1053 if (mon->lastError.code != VIR_ERR_OK) { 1054 VIR_DEBUG("Attempt to send command while error is set %s", 1055 NULLSTR(mon->lastError.message)); 1056 virSetError(&mon->lastError); 1057 return -1; 1058 } 1059 1060 if (!mon->watch) { 1061 VIR_WARN("Attempt to send command while mon->watch is zero"); 1062 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1063 _("attempt to send command when the monitor is closing")); 1064 return -1; 1065 } 1066 1067 mon->msg = msg; 1068 qemuMonitorUpdateWatch(mon); 1069 1070 PROBE(QEMU_MONITOR_SEND_MSG, 1071 "mon=%p msg=%s fd=%d", 1072 mon, mon->msg->txBuffer, mon->msg->txFD); 1073 1074 while (mon && mon->msg && !mon->msg->finished) { 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { 1076 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1077 _("Unable to wait on monitor condition")); 1078 goto cleanup; 1079 } 1080 } 1081 1082 if (mon->lastError.code != VIR_ERR_OK) { 1083 VIR_DEBUG("Send command resulted in error %s", 1084 NULLSTR(mon->lastError.message)); 1085 virSetError(&mon->lastError); 1086 goto cleanup; 1087 } 1088 1089 ret = 0; 1090 1091 cleanup: 1092 mon->msg = NULL; 1093 qemuMonitorUpdateWatch(mon); 1094 1095 return ret; 1096 } I’m not really sure that this change will affect other process. Thanks! Ma haocong

2 2

[libvirt-users] libvirt/dnsmasq is not adhering to static DHCP assignments
by Christian Kujau 09 Aug '19

09 Aug '19

This is basically a continuation of an older posting[0] I found, but apparently no solution has been posted. So, I'm trying to setup static DHCP leases with dnsmasq that is being started by libvirtd: ----------------------------------------------------------- $ sudo virsh net-dumpxml --network default <network> [...] <dns enable='no'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.130' end='192.168.122.250'/> <host mac='08:00:27:e2:81:39' name='f30' ip='192.168.56.139'/> ----------------------------------------------------------- And the domain is indeed being started with that hardware address: ----------------------------------------------------------- $ virsh dumpxml --domain f30 | grep -B1 -A3 mac\ a <interface type='bridge'> <mac address='08:00:27:e2:81:39'/> <source bridge='virbr0'/> <target dev='tap0'/> <model type='virtio'/> ----------------------------------------------------------- But for some reason the domain gets a different address assigned, albeit from the correct DHCP pool: ----------------------------------------------------------- $ ssh 192.168.122.233 "ip addr show" [...] 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 08:00:27:e2:81:39 brd ff:ff:ff:ff:ff:ff inet 192.168.122.233/24 brd 192.168.122.255 scope global dynamic enp1s0 ----------------------------------------------------------- See below for even more details. I've removed the virbr0.status file, restarted the network, still it gets this .233 address instead of the configured .139. Also, I'm not able to add a debug log to the libvirtd/dnsmasq instance, although it should be possible[1], the xmlns:dnsmasq stanza is overwritten as soon as the configuration is saved. Short of a debug (queries) log, strace on the libvirtd/dnsmasq process reveals that the client seems to request this strange IP address: 23002 19:01:59.053558 write(11, "<30>Jul 31 19:01:59 dnsmasq-dhcp[23002]: DHCPREQUEST(virbr0) 192.168.122.233 08:00:27:e2:81:39 ", 95) = 95 23002 19:01:59.054024 write(11, "<30>Jul 31 19:01:59 dnsmasq-dhcp[23002]: DHCPACK(virbr0) 192.168.122.233 08:00:27:e2:81:39 f30", 94) = 94 So I restarted the dhclient process in the domain (a freshly installed Fedora 30), removed all state files with "192.168.122.233" in them, but still the domains gets assigned the .233 instead of the .139 address. The next step would be to disable libvirtd/dnsmasq altogether and run my own dnsmasq instance, but I wanted to avoid that. Any idead on where to look next? Thanks, Christian. [0] https://www.redhat.com/archives/libvirt-users/2017-October/msg00070.html [1] https://libvirt.org/formatnetwork.html#elementsNamespaces # cd /var/lib/libvirt/dnsmasq/ # grep -r . . | fgrep -v \# ./default.conf:strict-order ./default.conf:port=0 ./default.conf:pid-file=/var/run/libvirt/network/default.pid ./default.conf:except-interface=lo ./default.conf:bind-dynamic ./default.conf:interface=virbr0 ./default.conf:dhcp-range=192.168.122.130,192.168.122.250,255.255.255.0 ./default.conf:dhcp-no-override ./default.conf:dhcp-authoritative ./default.conf:dhcp-lease-max=121 ./default.conf:dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile ./default.hostsfile:08:00:27:e2:81:39,192.168.56.139,f30 ./virbr0.status:[ ./virbr0.status: { ./virbr0.status: "ip-address": "192.168.122.233", ./virbr0.status: "mac-address": "08:00:27:e2:81:39", ./virbr0.status: "hostname": "f30", ./virbr0.status: "client-id": "ff:27:e2:81:39:00:04:8c:ad:c4:7d:04:e8:4b:de:93:4b:76:d8:75:82:86:c8", ./virbr0.status: "expiry-time": 1564628519 ./virbr0.status: } ./virbr0.status:] -- BOFH excuse #40: not enough memory, go get system upgrade

2 2

[libvirt-users] Reg: content of disk is not reflecting in host.
by bharath paulraj 07 Aug '19

07 Aug '19

Hi Team, I am doing a small testing and I don't know if my expectation is correct or not. Pardon me if I am ignorant. I created a VM and the VM is running. In the hypervisor I have created ".img" file and attached this .img file to the VM. My expectation is that, if VM is writing files to the attached disk, then it should reflect in the .img file created in the hypervisor. But It is not working as my expectation. Please correct me if my expectation is wrong. Steps: 1. Created disk.img in the hypervisor using the command: dd if=/dev/zero of=disk.img bs=1M count=50; mkfs ext3 -F disk.img 2. Attached the disk to the running VM using the command: virsh attach-disk <Domain-Name> --source disk.img --target vdb --live 3. In the VM, I mounted the disk and created few files. 4. In the hypervisor, I mounted the disk.img to check if the file created in the VM exists in the .img file. >> I am not able to see those files. Regards, Bharath

2 2

[libvirt-users] Dev wanted regarding Ceph and/or libxl related bugs and feature requests
by Julian Sternberg 03 Aug '19

03 Aug '19

Hi there, anybody with ceph api and/or libxl skills able to implement following feature and fix bug requests: https://bugzilla.redhat.com/show_bug.cgi?id=1729408 https://bugzilla.redhat.com/show_bug.cgi?id=1731159 https://bugzilla.redhat.com/show_bug.cgi?id=1732306

1 0

[libvirt-users] Detach disk from VM - virsh (working) vs. PHP (not working)
by Jan Horak 03 Aug '19

03 Aug '19

Hi all, i created a script in PHP for create a virtual server with two QCOW2 discs … one is our system for installation and second is target system. After successfully instalation (create a blank Debian system, prepare all files and grub partitions) i need a restart virtual without a installation disk. If i use Virsh: detach-disk --domain debian-test2 --persistent --target vda reset debian-test2 everything works well. If i use a PHP, there is „complicated“ way and „simple“ way: 1, complicated: libvirt_domain_destroy($domain); libvirt_domain_undefine($domain); $xml = domain_create_xml($name,$uuid,$memory,$cpu,$vnc,$mac); $domain = libvirt_domain_define_xml($server->conn, $xml); libvirt_domain_disk_add($domain, "/users/".$name.".img", "vdb", "virtio", "qcow2", NULL); libvirt_domain_create($domain); (or instead libvirt_domain_disk_add i can define disk directly in XML) But in this case, the server will not boot (GRUB error) 2, simple: libvirt_domain_disk_remove($domain,“vda“); libvirt_domain_reboot($domain); The problem of this solution is thats not working. The remove disk is failing with error „Unable attach disk“ - i looks to source code, and yes, there is a mystake with „attach“/„detach“, but main problem i see in log from libvirt: Aug 1 02:57:05 ry libvirtd[19051]: missing source information for device vda I try to put source detail to xml in source of PHP module libvirt-domain.c: 822 if (asprintf(&newXml, 823 " <disk type='file' device='disk'>\n" 824 " <target dev='%s'/>\n" 825 " </disk>", dev) < 0) { 826 set_error("Out of memory" TSRMLS_CC); 827 goto error; 828 } but my attempts was unsuccesfull (i’m not C programmer). Questions: A, why complicated way is not working and system dont want boot (GRUB error) if virsh works fine B, why libvirt_domain_disk_remove is not work ? I use libvirt and libvirt-php latest from git. Thank you, Jan

4 5

[libvirt-users] Researching why different cache modes result in 'some' guest filesystem corruption..
by vincent＠cojot.name 02 Aug '19

02 Aug '19

Hi All, I've been chasing down an issue in recent weeks (my own lab, so no prod here) and I'm reaching out in case someone might have some guidance to share. I'm running fairly large VMs (RHOSP underclouds - 8vcpu, 32gb ram, about 200gb single disk as a growable qcow2) on some RHEL7.6 hypervisors (kernel 3.10.0-927.2x.y, libvirt 4.5.0, qemu-kvm-1.5.3) on top of SSD/NVMe drives with various filesystems (vxfs, zfs, etc..) and using ECC RAM. The issue can be described as follows: - the guest VMs work fine for a while (days, weeks) but after a kernel update (z-stream) comes in, I am often greeted by the following message immediately after rebooting (or attempting to reboot into the new kernel): "error: not a correct xfs inode" - booting the previous kernel works fine and re-generating the initramfs for the new kernel (from the n-1 kernel) does not solve anything. - if booted from an ISO, xfs_repair does not find errors. - on ext4, there seems to be some kind of corruption there too. I'm building the initial guest image qcow2 for those guest VMs this way: 1) start with a rhel-guest image (currently rhel-server-7.6-update-5-x86_64-kvm.qcow2) 2) convert to LVM by doing this: qemu-img create -f qcow2 -o preallocation=metadata,cluster_size=1048576,lazy_refcounts=off final_guest.qcow2 512G virt-format -a final_guest.qcow2 --partition=mbr --lvm=/dev/rootdg/lv_root --filesystem=xfs guestfish --ro -a rhel_guest.qcow2 -m /dev/sda1 -- tar-out / - | \ guestfish --rw -a final_guest.qcow2 -m /dev/rootdg/lv_root -- tar-in - / 3) use "final_guest.qcow2" as the basis for my guests with LVM. After chasing down this issue some more and attempting various things (build the image on Fedora29, build a real XFS filesystem inside a VM and use the generated qcow2 as a basis instead of virt-format).. ..I've noticed that the SATA disk of each of those guests were using 'directsync' (instead of 'Hypervisor Default'). As soon as I switched to 'None', the XFS issues disappeared and I've now applied several consecutive kernel updates without issues. Also, 'directsync' or 'writethrough', while providing decent performance, both exhibited the XFS 'corruption' behaviour.. Only 'none' seem to have solved that. I've read the docs but I thought it was OK to use those modes (UPS, Battery-Backed RAID, etc..) Does anyone have any idea what's going on or what I may be doing wrong? Thanks for reading, Vincent

2 1

[libvirt-users] Why librbd disallow VM live migration if the disk cache mode is not none or directsync
by Ming-Hung Tsai 01 Aug '19

01 Aug '19

I'm curious that why librbd sets this limitation? The rule first appeared in librbd.git commit d57485f73ab. Theoretically, a write-through cache is also safe for VM migration, if the cache implementation guarantees that cache invalidation and disk write are synchronous operations. For example, I'm using Ceph RBD images as VM storage backend. The Ceph librbd supports synchronous write-through cache, by setting rbd_cache_max_dirty to zero, and setting rbd_cache_block_writes_upfront to true, thus it would be safe for VM migration. Is that true? Any suggestion would be appreciated. Thanks. Ming-Hung Tsai

2 3