- Users - libvirt List Archives

[libvirt-users] Vm in state "in shutdown"
by 马昊骢 Ianmalcolm Ma 09 Aug '19

09 Aug '19

Description of problem: libvirt 3.9 on CentOS Linux release 7.4.1708 (kernel 3.10.0-693.21.1.el7.x86_64) on Qemu version 2.10.0 I’m currently facing a strange situation. Sometimes my vm is shown by ‘virsh list’ as in state “in shutdown” but there is no qemu-kvm process linked to it. Libvirt log when “in shutdown” state occur is as follows: “d470c3b284425b9bacb34d3b5f3845fe” is vm’s name, remoteDispatchDomainMemoryStats API is called by ‘collectd’, which is used to collect some vm running states and host information once every 30’s. 2019-07-25 14:23:58.706+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: POWERDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.413 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.202 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: STOP, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.203 secs 2019-07-25 14:23:59.601+0000: 15818: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: d470c3b284425b9bacb34d3b5f3845fe, cost 1.203 secs 2019-07-25 14:23:59.629+0000: 15818: error : qemuMonitorIORead:597 : Unable to read from monitor: Connection reset by peer 2019-07-25 14:23:59.629+0000: 121081: warning : qemuProcessEventHandler:4840 : vm: d470c3b284425b9bacb34d3b5f3845fe, event: 6 locked 2019-07-25 14:23:59.629+0000: 15822: error : qemuMonitorJSONCommandWithFd:364 : internal error: Missing monitor reply object 2019-07-25 14:24:29.483+0000: 15821: warning : qemuGetProcessInfo:1468 : cannot parse process status data 2019-07-25 14:24:29.829+0000: 15823: warning : qemuDomainObjBeginJobInternal:4391 : Cannot start job (modify, none) for domain d470c3b284425b9bacb34d3b5f3845fe; current job is (query, none) owned by (15822 remoteDispatchDomainMemoryStats, 0 <null>) for (30s, 0s) 2019-07-25 14:24:29.829+0000: 15823: error : qemuDomainObjBeginJobInternal:4403 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-07-25 14:24:29.829+0000: 121081: warning : qemuDomainObjBeginJobInternal:4391 : Cannot start job (destroy, none) for domain d470c3b284425b9bacb34d3b5f3845fe; current job is (query, none) owned by (15822 remoteDispatchDomainMemoryStats, 0 <null>) for (30s, 0s) 2019-07-25 14:24:29.829+0000: 121081: error : qemuDomainObjBeginJobInternal:4403 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-07-25 14:24:29.829+0000: 121081: warning : qemuProcessEventHandler:4875 : vm: d470c3b284425b9bacb34d3b5f3845fe, event: 6, cost 31.459 secs I’ve tried to find out how did this problem happened. I analyzed the execution process of the job and speculated that the problem occurred as follows: step one: libvirt send command 'system_powerdown' to qemu. step two: libvirt receive qemu monitor close event, and then deal EOF event. step three: remoteDispatchDomainMemoryStats job start on the same vm. step four: work thread dealing stop job is waited on job.cond, the timeout is 30s. It seems that the remoteDispatchDomainMemoryStats job is too slow to let the stop job wait timeout. Then I tried to reproduce this process. The step is as follows: First step: add a sleep on ‘qemuProcessEventHandler' by using pthread_cond_timedwait. So can execute 'virsh dommemstat active’ command at this time interval. Second step: start a vm to test Third step: execute 'virsh shutdown active’ to shutdown the vm Fourth step: execute 'virsh dommemstat active’ when stop job is sleep. Then it works. The test vm state became to 'in shutdown’, and the libvirt log is as follows: “active” is my test vm’s name. 2019-08-05 08:39:57.001+0000: 25889: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: modify (vm=0x7f7bbc145fe0 name=active, current job=none async=none) 2019-08-05 08:39:57.003+0000: 25889: warning : qemuDomainObjEndJob:4522 : Stopping job: modify (async=none vm=0x7f7bbc145fe0 name=active) 2019-08-05 08:39:57.003+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: POWERDOWN, vm: active, cost 0.008 secs 2019-08-05 08:39:57.854+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: active, cost 1.709 secs 2019-08-05 08:39:57.875+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: STOP, vm: active, cost 1.751 secs 2019-08-05 08:39:57.875+0000: 25881: warning : qemuMonitorJSONIOProcessEvent:235 : type: SHUTDOWN, vm: active, cost 1.751 secs 2019-08-05 08:39:57.915+0000: 25881: warning : qemuMonitorIO:756 : Error on monitor <null> 2019-08-05 08:39:57.915+0000: 25881: warning : qemuMonitorIO:777 : Triggering EOF callback 2019-08-05 08:39:57.915+0000: 26915: warning : qemuProcessEventHandler:4822 : usleep 20s 2019-08-05 08:40:01.004+0000: 25886: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: query (vm=0x7f7bbc145fe0 name=active, current job=none async=none) ^@2019-08-05 08:40:17.915+0000: 26915: warning : qemuProcessEventHandler:4845 : vm=0x7f7bbc145fe0, event=6 2019-08-05 08:40:17.915+0000: 26915: warning : qemuProcessEventHandler:4851 : vm: active, event: 6 locked 2019-08-05 08:40:17.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4308 : Starting job: destroy (vm=0x7f7bbc145fe0 name=active, current job=query async=none) 2019-08-05 08:40:17.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4344 : Waiting for job (vm=0x7f7bbc145fe0 name=active), job 26915 qemuProcessEventHandler, owned by 25886 remoteDispatchDomainMemoryStats ^@^@^@2019-08-05 08:40:47.915+0000: 26915: warning : qemuDomainObjBeginJobInternal:4405 : Cannot start job (destroy, none) for domain active; current job is (query, none) owned by (25886 remoteDispatchDomainMemoryStats, 0 <null>) for (46s, 0s) 2019-08-05 08:40:47.915+0000: 26915: error : qemuDomainObjBeginJobInternal:4417 : 操作超时：cannot acquire state change lock (held by remoteDispatchDomainMemoryStats) 2019-08-05 08:40:47.915+0000: 26915: warning : qemuProcessEventHandler:4886 : vm: active, event: 6, cost 31.830 secs The threads backtraces is as follows: [Switching to thread 17 (Thread 0x7f12fddf5700 (LWP 32580))] #2 0x00007f12eb26df21 in qemuMonitorSend (mon=mon@entry=0x7f12d0013af0, msg=msg@entry=0x7f12fddf46e0) at qemu/qemu_monitor.c:1075 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { (gdb) l 1070 PROBE(QEMU_MONITOR_SEND_MSG, 1071 "mon=%p msg=%s fd=%d", 1072 mon, mon->msg->txBuffer, mon->msg->txFD); 1073 1074 while (mon && mon->msg && !mon->msg->finished) { 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { 1076 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1077 _("Unable to wait on monitor condition")); 1078 goto cleanup; 1079 } (gdb) p *msg $2 = {txFD = -1, txBuffer = 0x7f12b8000d50 "{\"execute\":\"qom-list\",\"arguments\":{\"path\":\"/machine/peripheral\"},\"id\":\"libvirt-17\"}\r\n", txOffset = 0, txLength = 85, rxBuffer = 0x0, rxLength = 0, rxObject = 0x0, finished = false, passwordHandler = 0x0, passwordOpaque = 0x0} (gdb) bt #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f130d738316 in virCondWait (c=c@entry=0x7f12d0013b28, m=m@entry=0x7f12d0013b00) at util/virthread.c:154 #2 0x00007f12eb26df21 in qemuMonitorSend (mon=mon@entry=0x7f12d0013af0, msg=msg@entry=0x7f12fddf46e0) at qemu/qemu_monitor.c:1075 #3 0x00007f12eb283b40 in qemuMonitorJSONCommandWithFd ( mon=mon@entry=0x7f12d0013af0, cmd=cmd@entry=0x7f12b80008c0, scm_fd=scm_fd@entry=-1, reply=reply@entry=0x7f12fddf4780) at qemu/qemu_monitor_json.c:355 #4 0x00007f12eb2904b9 in qemuMonitorJSONCommand (reply=0x7f12fddf4780, cmd=0x7f12b80008c0, mon=0x7f12d0013af0) at qemu/qemu_monitor_json.c:385 ... I found that on function ‘qemuMonitorUpdateWatch’, if mon->watch is zero, then event would’t be update. But 'qemuMonitorSend' would still wait on mon->notify. So the remoteDispatchDomainMemoryStats job is blocked, and then the stop job blocked too. 618 static void 619 qemuMonitorUpdateWatch(qemuMonitorPtr mon) 620 { 621 int events = 622 VIR_EVENT_HANDLE_HANGUP | 623 VIR_EVENT_HANDLE_ERROR; 624 625 if (!mon->watch) 626 return; 627 628 if (mon->lastError.code == VIR_ERR_OK) { 629 events |= VIR_EVENT_HANDLE_READABLE; 630 631 if ((mon->msg && mon->msg->txOffset < mon->msg->txLength) && 632 !mon->waitGreeting) 633 events |= VIR_EVENT_HANDLE_WRITABLE; 634 } 635 636 virEventUpdateHandle(mon->watch, events); 637 } I try to fix this bugs by add a judgment before qemuMonitorUpdateWatch, and it seem works. 1046 int 1047 qemuMonitorSend(qemuMonitorPtr mon, 1048 qemuMonitorMessagePtr msg) 1049 { 1050 int ret = -1; 1051 1052 /* Check whether qemu quit unexpectedly */ 1053 if (mon->lastError.code != VIR_ERR_OK) { 1054 VIR_DEBUG("Attempt to send command while error is set %s", 1055 NULLSTR(mon->lastError.message)); 1056 virSetError(&mon->lastError); 1057 return -1; 1058 } 1059 1060 if (!mon->watch) { 1061 VIR_WARN("Attempt to send command while mon->watch is zero"); 1062 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1063 _("attempt to send command when the monitor is closing")); 1064 return -1; 1065 } 1066 1067 mon->msg = msg; 1068 qemuMonitorUpdateWatch(mon); 1069 1070 PROBE(QEMU_MONITOR_SEND_MSG, 1071 "mon=%p msg=%s fd=%d", 1072 mon, mon->msg->txBuffer, mon->msg->txFD); 1073 1074 while (mon && mon->msg && !mon->msg->finished) { 1075 if (virCondWait(&mon->notify, &mon->parent.lock) < 0) { 1076 virReportError(VIR_ERR_INTERNAL_ERROR, "%s", 1077 _("Unable to wait on monitor condition")); 1078 goto cleanup; 1079 } 1080 } 1081 1082 if (mon->lastError.code != VIR_ERR_OK) { 1083 VIR_DEBUG("Send command resulted in error %s", 1084 NULLSTR(mon->lastError.message)); 1085 virSetError(&mon->lastError); 1086 goto cleanup; 1087 } 1088 1089 ret = 0; 1090 1091 cleanup: 1092 mon->msg = NULL; 1093 qemuMonitorUpdateWatch(mon); 1094 1095 return ret; 1096 } I’m not really sure that this change will affect other process. Thanks! Ma haocong

2 2

[libvirt-users] libvirt/dnsmasq is not adhering to static DHCP assignments
by Christian Kujau 09 Aug '19

09 Aug '19

This is basically a continuation of an older posting[0] I found, but apparently no solution has been posted. So, I'm trying to setup static DHCP leases with dnsmasq that is being started by libvirtd: ----------------------------------------------------------- $ sudo virsh net-dumpxml --network default <network> [...] <dns enable='no'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.130' end='192.168.122.250'/> <host mac='08:00:27:e2:81:39' name='f30' ip='192.168.56.139'/> ----------------------------------------------------------- And the domain is indeed being started with that hardware address: ----------------------------------------------------------- $ virsh dumpxml --domain f30 | grep -B1 -A3 mac\ a <interface type='bridge'> <mac address='08:00:27:e2:81:39'/> <source bridge='virbr0'/> <target dev='tap0'/> <model type='virtio'/> ----------------------------------------------------------- But for some reason the domain gets a different address assigned, albeit from the correct DHCP pool: ----------------------------------------------------------- $ ssh 192.168.122.233 "ip addr show" [...] 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 08:00:27:e2:81:39 brd ff:ff:ff:ff:ff:ff inet 192.168.122.233/24 brd 192.168.122.255 scope global dynamic enp1s0 ----------------------------------------------------------- See below for even more details. I've removed the virbr0.status file, restarted the network, still it gets this .233 address instead of the configured .139. Also, I'm not able to add a debug log to the libvirtd/dnsmasq instance, although it should be possible[1], the xmlns:dnsmasq stanza is overwritten as soon as the configuration is saved. Short of a debug (queries) log, strace on the libvirtd/dnsmasq process reveals that the client seems to request this strange IP address: 23002 19:01:59.053558 write(11, "<30>Jul 31 19:01:59 dnsmasq-dhcp[23002]: DHCPREQUEST(virbr0) 192.168.122.233 08:00:27:e2:81:39 ", 95) = 95 23002 19:01:59.054024 write(11, "<30>Jul 31 19:01:59 dnsmasq-dhcp[23002]: DHCPACK(virbr0) 192.168.122.233 08:00:27:e2:81:39 f30", 94) = 94 So I restarted the dhclient process in the domain (a freshly installed Fedora 30), removed all state files with "192.168.122.233" in them, but still the domains gets assigned the .233 instead of the .139 address. The next step would be to disable libvirtd/dnsmasq altogether and run my own dnsmasq instance, but I wanted to avoid that. Any idead on where to look next? Thanks, Christian. [0] https://www.redhat.com/archives/libvirt-users/2017-October/msg00070.html [1] https://libvirt.org/formatnetwork.html#elementsNamespaces # cd /var/lib/libvirt/dnsmasq/ # grep -r . . | fgrep -v \# ./default.conf:strict-order ./default.conf:port=0 ./default.conf:pid-file=/var/run/libvirt/network/default.pid ./default.conf:except-interface=lo ./default.conf:bind-dynamic ./default.conf:interface=virbr0 ./default.conf:dhcp-range=192.168.122.130,192.168.122.250,255.255.255.0 ./default.conf:dhcp-no-override ./default.conf:dhcp-authoritative ./default.conf:dhcp-lease-max=121 ./default.conf:dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile ./default.hostsfile:08:00:27:e2:81:39,192.168.56.139,f30 ./virbr0.status:[ ./virbr0.status: { ./virbr0.status: "ip-address": "192.168.122.233", ./virbr0.status: "mac-address": "08:00:27:e2:81:39", ./virbr0.status: "hostname": "f30", ./virbr0.status: "client-id": "ff:27:e2:81:39:00:04:8c:ad:c4:7d:04:e8:4b:de:93:4b:76:d8:75:82:86:c8", ./virbr0.status: "expiry-time": 1564628519 ./virbr0.status: } ./virbr0.status:] -- BOFH excuse #40: not enough memory, go get system upgrade

2 2

[libvirt-users] Reg: content of disk is not reflecting in host.
by bharath paulraj 08 Aug '19

08 Aug '19

Hi Team, I am doing a small testing and I don't know if my expectation is correct or not. Pardon me if I am ignorant. I created a VM and the VM is running. In the hypervisor I have created ".img" file and attached this .img file to the VM. My expectation is that, if VM is writing files to the attached disk, then it should reflect in the .img file created in the hypervisor. But It is not working as my expectation. Please correct me if my expectation is wrong. Steps: 1. Created disk.img in the hypervisor using the command: dd if=/dev/zero of=disk.img bs=1M count=50; mkfs ext3 -F disk.img 2. Attached the disk to the running VM using the command: virsh attach-disk <Domain-Name> --source disk.img --target vdb --live 3. In the VM, I mounted the disk and created few files. 4. In the hypervisor, I mounted the disk.img to check if the file created in the VM exists in the .img file. >> I am not able to see those files. Regards, Bharath

2 2

[libvirt-users] Dev wanted regarding Ceph and/or libxl related bugs and feature requests
by Julian Sternberg 04 Aug '19

04 Aug '19

Hi there, anybody with ceph api and/or libxl skills able to implement following feature and fix bug requests: https://bugzilla.redhat.com/show_bug.cgi?id=1729408 https://bugzilla.redhat.com/show_bug.cgi?id=1731159 https://bugzilla.redhat.com/show_bug.cgi?id=1732306

1 0

[libvirt-users] Detach disk from VM - virsh (working) vs. PHP (not working)
by Jan Horak 03 Aug '19

03 Aug '19

Hi all, i created a script in PHP for create a virtual server with two QCOW2 discs … one is our system for installation and second is target system. After successfully instalation (create a blank Debian system, prepare all files and grub partitions) i need a restart virtual without a installation disk. If i use Virsh: detach-disk --domain debian-test2 --persistent --target vda reset debian-test2 everything works well. If i use a PHP, there is „complicated“ way and „simple“ way: 1, complicated: libvirt_domain_destroy($domain); libvirt_domain_undefine($domain); $xml = domain_create_xml($name,$uuid,$memory,$cpu,$vnc,$mac); $domain = libvirt_domain_define_xml($server->conn, $xml); libvirt_domain_disk_add($domain, "/users/".$name.".img", "vdb", "virtio", "qcow2", NULL); libvirt_domain_create($domain); (or instead libvirt_domain_disk_add i can define disk directly in XML) But in this case, the server will not boot (GRUB error) 2, simple: libvirt_domain_disk_remove($domain,“vda“); libvirt_domain_reboot($domain); The problem of this solution is thats not working. The remove disk is failing with error „Unable attach disk“ - i looks to source code, and yes, there is a mystake with „attach“/„detach“, but main problem i see in log from libvirt: Aug 1 02:57:05 ry libvirtd[19051]: missing source information for device vda I try to put source detail to xml in source of PHP module libvirt-domain.c: 822 if (asprintf(&newXml, 823 " <disk type='file' device='disk'>\n" 824 " <target dev='%s'/>\n" 825 " </disk>", dev) < 0) { 826 set_error("Out of memory" TSRMLS_CC); 827 goto error; 828 } but my attempts was unsuccesfull (i’m not C programmer). Questions: A, why complicated way is not working and system dont want boot (GRUB error) if virsh works fine B, why libvirt_domain_disk_remove is not work ? I use libvirt and libvirt-php latest from git. Thank you, Jan

4 5

[libvirt-users] Researching why different cache modes result in 'some' guest filesystem corruption..
by vincent＠cojot.name 02 Aug '19

02 Aug '19

Hi All, I've been chasing down an issue in recent weeks (my own lab, so no prod here) and I'm reaching out in case someone might have some guidance to share. I'm running fairly large VMs (RHOSP underclouds - 8vcpu, 32gb ram, about 200gb single disk as a growable qcow2) on some RHEL7.6 hypervisors (kernel 3.10.0-927.2x.y, libvirt 4.5.0, qemu-kvm-1.5.3) on top of SSD/NVMe drives with various filesystems (vxfs, zfs, etc..) and using ECC RAM. The issue can be described as follows: - the guest VMs work fine for a while (days, weeks) but after a kernel update (z-stream) comes in, I am often greeted by the following message immediately after rebooting (or attempting to reboot into the new kernel): "error: not a correct xfs inode" - booting the previous kernel works fine and re-generating the initramfs for the new kernel (from the n-1 kernel) does not solve anything. - if booted from an ISO, xfs_repair does not find errors. - on ext4, there seems to be some kind of corruption there too. I'm building the initial guest image qcow2 for those guest VMs this way: 1) start with a rhel-guest image (currently rhel-server-7.6-update-5-x86_64-kvm.qcow2) 2) convert to LVM by doing this: qemu-img create -f qcow2 -o preallocation=metadata,cluster_size=1048576,lazy_refcounts=off final_guest.qcow2 512G virt-format -a final_guest.qcow2 --partition=mbr --lvm=/dev/rootdg/lv_root --filesystem=xfs guestfish --ro -a rhel_guest.qcow2 -m /dev/sda1 -- tar-out / - | \ guestfish --rw -a final_guest.qcow2 -m /dev/rootdg/lv_root -- tar-in - / 3) use "final_guest.qcow2" as the basis for my guests with LVM. After chasing down this issue some more and attempting various things (build the image on Fedora29, build a real XFS filesystem inside a VM and use the generated qcow2 as a basis instead of virt-format).. ..I've noticed that the SATA disk of each of those guests were using 'directsync' (instead of 'Hypervisor Default'). As soon as I switched to 'None', the XFS issues disappeared and I've now applied several consecutive kernel updates without issues. Also, 'directsync' or 'writethrough', while providing decent performance, both exhibited the XFS 'corruption' behaviour.. Only 'none' seem to have solved that. I've read the docs but I thought it was OK to use those modes (UPS, Battery-Backed RAID, etc..) Does anyone have any idea what's going on or what I may be doing wrong? Thanks for reading, Vincent

2 1

[libvirt-users] Why librbd disallow VM live migration if the disk cache mode is not none or directsync
by Ming-Hung Tsai 01 Aug '19

01 Aug '19

I'm curious that why librbd sets this limitation? The rule first appeared in librbd.git commit d57485f73ab. Theoretically, a write-through cache is also safe for VM migration, if the cache implementation guarantees that cache invalidation and disk write are synchronous operations. For example, I'm using Ceph RBD images as VM storage backend. The Ceph librbd supports synchronous write-through cache, by setting rbd_cache_max_dirty to zero, and setting rbd_cache_block_writes_upfront to true, thus it would be safe for VM migration. Is that true? Any suggestion would be appreciated. Thanks. Ming-Hung Tsai

2 3

[libvirt-users] Libvirt Virtualization Problem
by 王金磊 30 Jul '19

30 Jul '19

Dear Sir/Madam: I use virt-manager to create a new virtual machine(it's name is generic), then I edit generic to set it's cpu_mode as 'host-model', and I start it, but when dumpxml generic, it's cpu_mode is changed as 'custom'. I want to know that why it is? And what's the principle of dumpxml? Thanks! A user of libvirt

2 1

[libvirt-users] virsh command to list CPU,Memory and Storage
by Kaushal Shriyan 30 Jul '19

30 Jul '19

Hi, Is there a way to find out total CPU's, Memory and Storage using virsh command? For example virsh list --all list out all VM's 11 dockerregistry01 running 12 gitlab running Any help will be highly appreciated. Thanks in Advance. Best Regards, Kaushal

3 2

[libvirt-users] <VM LIVE Migration> <Sync conntrack entries>
by bharath paulraj 26 Jul '19

26 Jul '19

Hi Team, I am using QEMU/KVM for launching VMs and libvirt to govern those VMs. I would like to synchronise the connection tracking entries specific to the VM during the VM LIVE migrations. It is required when the firewall is implemented at the host level like libvirt's "network filters". If stateful firewall is enabled, then unless these connection tracking entries are synchronised, all the connections to the VM are lost and all TCP connections should be reestablished. Is there any option already available? I don't think current libvirt hooks are helpful, as VM pause in the source hypervisor and VM on in the destination hypervisor is done by QEMU and it does not wait for any application that needs to sync-up some metadata — In my case, it is conntrack entries. Also I tried with the existing hooks - stop, release, startcpus and nothing worked well. Has anybody came across similar scenario? If yes, how you overcome this? -- Regards, Bharath

2 1

[libvirt-users] Define libvirt network portgroup with native untagged.
by Tanmoy Sinha 26 Jul '19

26 Jul '19

Hi, I have created a libvirt network on top of a OVS bridge (named vlan-br) which receives all VLAN tagged packets, i.e. connected to a trunk port. The definition xml is below. What I want to achieve in the portgroup definition 'trunk-native-1221' is to allow 1221 as untagged/native but rest all VLAN as tagged. The following portgroup definition works, but I don't want to enumerate all the tagged VLANs in the portgroup definition. I understand what libvirt does on the underlying OVS bridge, once a guest interface (say vNetX) is attached to the portgroup is to set the vnetX with tag=1221 and vlan_mode= native-untagged and it sets trunk = [1222,1221,1223,1224]. Now if I go and clear the trunk setting on the OVS bridge for that interface I am able to see both tagged and untagged (1221) packets on the guest. This is exactly what I want to achieve in the libvirt network definition, i.e. have one untagged VLAN and* allow all other VLANs without having to enumerate them in the portgroup definition, as that is hard to maintain. * <network> <name>kvm-core-net</name> <bridge name = 'vlan-br'/> <forward mode = 'bridge'/> <virtualport type='openvswitch'/> ....... <portgroup name='*trunk-native-1221*'> <vlan trunk='yes'> <tag id='1222'/> <tag id='1221' nativeMode='untagged'/> <tag id='1223'/> <tag id='1224'/> </vlan> </portgtroup> ..... </network> Regards Tanmoy Sinha

1 0

[libvirt-users] make install libvirt v5.5.0 failed
by netsurfed 24 Jul '19

24 Jul '19

Hi all: Unable to install libvirt v5.5.0 correctly through the source code which clone from git. The error message is shown below. libtool: install: (cd /home/libvirt/src; /bin/sh /home/libvirt/libtool --silent --tag CC --mode=relink gcc -std=gnu99 -I/usr/include/libxml2 -fno-common -W -Waddress -Waggressive-loop-optimizations -Wall -Wattributes -Wbad-function-cast -Wbuiltin-macro-redefined -Wcast-align -Wchar-subscripts -Wclobbered -Wcomment -Wcomments -Wcoverage-mismatch -Wcpp -Wdeprecated-declarations -Wdiv-by-zero -Wdouble-promotion -Wempty-body -Wendif-labels -Wextra -Wformat-contains-nul -Wformat-extra-args -Wformat-security -Wformat-y2k -Wformat-zero-length -Wfree-nonheap-object -Wignored-qualifiers -Wimplicit -Wimplicit-function-declaration -Wimplicit-int -Winit-self -Winline -Wint-to-pointer-cast -Winvalid-memory-model -Winvalid-pch -Wlogical-op -Wmain -Wmaybe-uninitialized -Wmissing-braces -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include-dirs -Wmissing-parameter-type -Wmissing-prototypes -Wmultichar -Wnarrowing -Wnested-externs -Wnonnull -Wold-style-declaration -Wold-style-definition -Woverflow -Woverride-init -Wpacked-bitfield-compat -Wparentheses -Wpointer-arith -Wpointer-sign -Wpointer-to-int-cast -Wpragmas -Wpsabi -Wreturn-local-addr -Wreturn-type -Wsequence-point -Wshadow -Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstrict-prototypes -Wsuggest-attribute=const -Wsuggest-attribute=format -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wswitch -Wsync-nand -Wtrampolines -Wtrigraphs -Wtype-limits -Wuninitialized -Wunknown-pragmas -Wunused -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-function -Wunused-label -Wunused-local-typedefs -Wunused-parameter -Wunused-result -Wunused-value -Wunused-variable -Wvarargs -Wvariadic-macros -Wvector-operation-performance -Wvolatile-register-var -Wwrite-strings -Wnormalized=nfc -Wno-sign-compare -Wjump-misses-init -Wswitch-enum -Wno-format-nonliteral -fstack-protector-strong -fexceptions -fasynchronous-unwind-tables -fipa-pure-const -Wno-suggest-attribute=pure -Wno-suggest-attribute=const -Werror -Wframe-larger-than=4096 -g -O2 -module -avoid-version -Wl,-z -Wl,nodelete -export-dynamic -Wl,-z -Wl,relro -Wl,-z -Wl,now -Wl,--no-copy-dt-needed-entries -Wl,-z -Wl,defs -o libvirt_driver_vbox.la -rpath /usr/lib/libvirt/connection-driver libvirt_driver_vbox_impl.la libvirt.la ../gnulib/lib/libgnu.la -ldl ) ./.libs/libvirt_driver_vbox_impl.a(libvirt_driver_vbox_impl_la-vbox_common.o): In function `vboxAttachDrives': /home/libvirt/src/vbox/vbox_common.c:1124: undefined reference to `virDomainDeviceFindSCSIController' ./.libs/libvirt_driver_vbox_impl.a(libvirt_driver_vbox_impl_la-vbox_common.o): In function `vboxDomainSnapshotCreateXML': vbox_common.c:(.text+0x698f): undefined reference to `virObjectAutoUnref' vbox_common.c:(.text+0x80fa): undefined reference to `virObjectAutoUnref' ./.libs/libvirt_driver_vbox_impl.a(libvirt_driver_vbox_impl_la-vbox_common.o): In function `vboxDomainSnapshotGetXMLDesc': vbox_common.c:(.text+0xc430): undefined reference to `virDomainSnapshotDefNew' vbox_common.c:(.text+0xc7bc): undefined reference to `virObjectAutoUnref' vbox_common.c:(.text+0xe100): undefined reference to `virObjectAutoUnref' ./.libs/libvirt_driver_vbox_impl.a(libvirt_driver_vbox_impl_la-vbox_common.o): In function `vboxAttachNetwork.isra.17': vbox_common.c:(.text+0x10656): undefined reference to `virDomainNetModelTypeToString' ./.libs/libvirt_driver_vbox_impl.a(libvirt_driver_vbox_impl_la-vbox_common.o): In function `vboxDomainGetXMLDesc': vbox_common.c:(.text+0x1574d): undefined reference to `virDomainHostdevDefNew' collect2: error: ld returned 1 exit status libtool: install: error: relink `libvirt_driver_vbox.la' with the above command before installing it make[3]: *** [install-modLTLIBRARIES] Error 1 make[3]: Leaving directory `/home/libvirt/src' make[2]: *** [install-am] Error 2 make[2]: Leaving directory `/home/libvirt/src' make[1]: *** [install] Error 2 make[1]: Leaving directory `/home/libvirt/src' make: *** [install-recursive] Error 1 Here's how I do it. [root@localhost libvirt]# uname -a Linux localhost.localdomain 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux [root@localhost libvirt]# git clone https://libvirt.org/git/libvirt.git [root@localhost libvirt]# cd libvirt [root@localhost libvirt]# git checkout v5.5.0 [root@localhost libvirt]# ./autogen.sh [root@localhost libvirt]# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var [root@localhost libvirt]# make -j32 [root@localhost libvirt]# make install

2 2

[libvirt-users] Installing QEMU from source causing error
by Probir Roy 22 Jul '19

22 Jul '19

Hi, I am trying to run Qemu-4.0 installed from source code. When I run virt-manager to create a VM, I get the following error: ``` Unable to complete install: 'internal error: process exited while connecting to monitor: 2019-07-19T17:06:35.954242Z qemu-system-x86_64: -enable-kvm: unsupported machine type Use -machine help to list supported machines' Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/create.py", line 2553, in _do_async_install guest.start_install(meter=meter) File "/usr/share/virt-manager/virtinst/guest.py", line 498, in start_install doboot, transient) File "/usr/share/virt-manager/virtinst/guest.py", line 434, in _create_guest domain = self.conn.createXML(install_xml or final_xml, 0) File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3603, in createXML if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: process exited while connecting to monitor: 2019-07-19T17:06:35.954242Z qemu-system-x86_64: -enable-kvm: unsupported machine type Use -machine help to list supported machines ``` I configured the Qemu with following command: ``` ./../configure --prefix=/usr --enable-spice --target-list=x86_64-softmmu --enable-kvm --enable-debug --enable-libssh2 ``` Is it that I missed enabling any configuration while installing Qemu? What is the expected QEMU configuration to run with libvirt? - Probir

3 4

[libvirt-users] virsh edit has no impact on domain
by Probir Roy 22 Jul '19

22 Jul '19

Hi, I am using "virsh edit" to edit a domain. After edit it says >> Domain ubuntu18.04 XML configuration edited. However, reopening the domain with "virsh edit" shows old file. The domain is also turned off. $virsh list --all --persistent Id Name State ---------------------------------------------------- - ubuntu18.04 shut off I have also tried running "virsh define" to set new configuration. $virsh define ubuntu18.04.xml >>> Domain ubuntu18.04 defined from ubuntu18.04.xml However, it also did not change the file. How can I edit the domain config file? OS: Ubuntu 18.04.1 virsh: 4.0.0 -- Probir

2 1

[libvirt-users] domain xml questions
by Vasiliy Tolstov 19 Jul '19

19 Jul '19

Hi! If i want to use bridged network or openvswitch, does guestfwd only works with slirp network? Or how it work in case of using bridge/openvswitch? <devices> <channel type='unix'> <source mode='bind' path='/tmp/guestfwd'/> <target type='guestfwd' address='10.0.2.1' port='4600'/> </channel> </devices> If this cant't work with provided network types, how can i create inside guest serial console on 10 tty that binded to listening unix domain socket on host system? -- Vasiliy Tolstov, e-mail: v.tolstov(a)selfip.ru

2 2

[libvirt-users] Preloading shared library with libVirt
by Probir Roy 18 Jul '19

18 Jul '19

Hi, I am trying to preload two shared libraries while running Qemu using libvirt. To preload the shared libraries I have added an environment variable in the $domain.xml file with the following command: ``` virt-xml generic --edit --confirm --qemu-commandline 'env=LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so' ``` However, while running the domain, I get the following message of failure to preload the shared files in the log. ``` 2019-07-17 17:35:22.939+0000: starting up libvirt version: 1.3.1, package: 1ubuntu10.27 (Marc Deslauriers <marc.deslauriers(a)ubuntu.com> Tue, 02 Jul 2019 09:22:37 -0400), qemu version: 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.39), hostname: probir-XPS-13-9343 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=spice LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so /usr/bin/kvm-spice -name generic -S -machine pc-i440fx-xenial,accel=kvm,usb=off -cpu Broadwell-noTSX-IBRS -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 8f755c9c-adf8-4baf-8d6a-947ebdf61ff8 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/var/lib/libvirt/images/generic.img,format=qcow2,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-0-1,readonly=on -device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=26,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:82:62:ba,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on Domain id=9 is tainted: custom-argv ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. ERROR: ld.so: object '/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored. char device redirected to /dev/pts/21 (label charserial0) ``` I further went ahead to verify the shared library with ldd and ls -l. Following results came out: ``` probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so -rwxrwxr-x 1 probir probir 834240 Jun 13 17:53 /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd /home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so linux-vdso.so.1 => (0x00007ffdd97d5000) libxed.so => /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/xed/kits/xed-install-base/lib/libxed.so (0x00007f706442e000) libmonitor.so.0 => /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so.0 (0x00007f706420a000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f7063fed000) libbfd-2.26.1-system.so => /usr/lib/x86_64-linux-gnu/libbfd-2.26.1-system.so (0x00007f7063ca8000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f7063926000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7063710000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7063346000) /lib64/ld-linux-x86-64.so.2 (0x00007f7064f48000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7063142000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7062f28000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7062c1f000) ``` and ``` probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so lrwxrwxrwx 1 probir probir 19 Mar 28 13:45 /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so -> libmonitor.so.0.0.0 probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd /home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so linux-vdso.so.1 => (0x00007ffc5d12b000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff8ae23e000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff8ade74000) /lib64/ld-linux-x86-64.so.2 (0x00007ff8ae666000) ``` I cannot figure out what is causing the failure of preload. Previously, I manually preloaded the same shared libraries for other application. Probir

3 2

[libvirt-users] filesharing file owner problem
by Vojto Ľach 17 Jul '19

17 Jul '19

Hello, I'm having an issue with usage of libvirt driven qemu where I'm sharing directory with guest machine. virsh domain filesharing configuration: <filesystem type='mount' accessmode='passthrough'> <driver type='path' wrpolicy='immediate'/> <source dir='/media/share'/> <target dir='data'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </filesystem> /etc/libvirt/qemu.conf : user = "root" group = "root" dynamic_ownership = 0 clear_emulator_capabilities = 0 even with this unsecure configuration I wasn't able to achieve that newly created files/dirs in guest machine have ownership of guest machine user but they are still created under root user id. Also strange is that group of those files/dirs are correct. Can somebody help me with this? Thank you. Vojtech

2 1

[libvirt-users] libvirtd failing on MacOS in setgroups
by Marcus Furlong 17 Jul '19

17 Jul '19

Hi, I get the following error when running libvirtd on MacOS as root: $ sudo libvirtd 2019-07-11 00:12:33.379+0000: 123145573953536: info : libvirt version: 5.5.0 2019-07-11 00:12:33.379+0000: 123145573953536: warning : virProcessGetStartTime:1070 : Process start time of pid 49746 not available on this platform 2019-07-11 00:12:33.379+0000: 123145573953536: error : virSysinfoReadDMI:1172 : internal error: Failed to find path for dmidecode binary 2019-07-11 00:12:33.380+0000: 123145573953536: error : virFileFindHugeTLBFS:3734 : this function is not supported by the connection driver: virFileFindHugeTLBFS 2019-07-11 00:12:33.382+0000: 123145573953536: warning : virQEMUCapsInit:919 : Failed to get host CPU cache info 2019-07-11 00:12:33.401+0000: 123145573953536: error : virHostCPUGetTscInfo:1405 : Probing TSC is not supported on this platform: Function not implemented 2019-07-11 00:12:33.401+0000: 123145573953536: error : virExec:521 : Cannot find 'pm-is-supported' in path: No such file or directory 2019-07-11 00:12:33.401+0000: 123145573953536: warning : virQEMUCapsInit:926 : Failed to get host power management capabilities 2019-07-11 00:12:33.401+0000: 123145573953536: error : virNumaGetPages:988 : Operation not supported: page info is not supported on this platform 2019-07-11 00:12:33.401+0000: 123145573953536: warning : virQEMUCapsInit:933 : Failed to get pages info 2019-07-11 00:12:33.407+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-alpha for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.407+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-alpha: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-alpha for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.413+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-arm for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.413+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-arm: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-arm for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.419+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-arm for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.419+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-arm: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-arm for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.424+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-aarch64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.424+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-aarch64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-aarch64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.431+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-cris for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.431+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-cris: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-cris for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.437+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-i386 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.437+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-i386: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-i386 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.443+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-lm32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.443+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-lm32: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-lm32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.449+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-m68k for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.449+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-m68k: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-m68k for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.497+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-microblaze for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.497+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-microblaze: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-microblaze for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.503+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-microblazeel for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.503+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-microblazeel: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-microblazeel for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.509+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.509+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-mips: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.515+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mipsel for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.515+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-mipsel: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mipsel for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.521+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.521+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-mips64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.527+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips64el for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.527+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-mips64el: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-mips64el for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.533+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.533+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-ppc: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.571+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.571+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-ppc64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.576+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.576+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-ppc64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-ppc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.582+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-riscv32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.582+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-riscv32: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-riscv32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.588+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-riscv64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.588+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-riscv64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-riscv64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.594+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-s390x for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.594+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-s390x: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-s390x for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.600+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sh4 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.600+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-sh4: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sh4 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.605+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sh4eb for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.605+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-sh4eb: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sh4eb for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.612+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.612+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-sparc: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.619+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.619+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-sparc64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-sparc64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.667+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-unicore32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.667+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-unicore32: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-unicore32 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.673+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.673+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-x86_64: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-x86_64 for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.679+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensa for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.679+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-xtensa: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensa for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.685+0000: 123145573953536: error : qemuProcessQMPLaunch:8501 : internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensaeb for probing: libvirt: error : cannot set supplemental groups: Invalid argument 2019-07-11 00:12:33.685+0000: 123145573953536: warning : virQEMUCapsLogProbeFailure:4578 : Failed to probe capabilities for /usr/local/bin/qemu-system-xtensaeb: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-xtensaeb for probing: libvirt: error : cannot set supplemental groups: Invalid argument As a result `virsh capabilities` as root returns nothing: $ sudo virsh capabilities | grep qemu <baselabel type='qemu'>+0:+0</baselabel> whereas running as a regular user works fine: $ virsh capabilites | grep qemu <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> <domain type='qemu'/> and by extension, running VMs as regular user works fine via qemu:///session , but qemu:///system does not work. It seems like setgroups is failing: https://github.com/libvirt/libvirt/blob/v5.5.0/src/util/virutil.c#L1045-L10… Is this the expected behaviour? Marcus. -- Marcus Furlong

1 0

[libvirt-users] Question regarding using multiple physical functions inside a network pool.
by Mohamed, Faris 16 Jul '19

16 Jul '19

Hello, I was planning on adding multiple physical functions using the built in SR-IOV features offered by libvirt to a create a bigger pool of virtual functions using the <forward mode='hostdev' managed='yes'> heading (I want to be able to use more virtual machines then there are virtual functions available on an ethernet port). As soon as I try to define multiple physical functions in a pool, I get this error: error: Failed to define network from biggernetwork.xml error: XML error: Only one <pf> element is allowed in <forward> of network bigger network when using this command: sudo virsh net-define biggernetwork.xml on the following XML document: <network> <name>bigger network</name> <forward mode='hostdev' managed='yes'> <pf dev='eth5'/> <pf dev='eth7'/> </forward> </network> It seems that you are not able to add multiple physical functions to this network to create a pool of more virtual functions. Is there anyway that you can do this using that I am missing? I seem to be looking at the documentation and there seems to be nothing related to this. I think this would be extremely useful and would make automate adding virtual functions to virtual machines much easier to manage. Thanks in advance, Faris ----------------------------------------------------------------------------------------------------------------------- Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments. -----------------------------------------------------------------------------------------------------------------------

2 1

Re: [libvirt-users] Strange issues with libvirt v5.5.0
by Peter Krempa 12 Jul '19

12 Jul '19

On Thu, Jul 11, 2019 at 16:46:29 +0200, Julian Sternberg wrote: > <html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>Hey Peter,</div> Hi, at first I'd like to ask you to CC the libvirt-users or libvir-list with such kind of questions (questions implying regression of behaviour are okay on the development list). I've CC'd libvirt-users with this reply to archive this. Also I'd strongly encourage you _not_ to use HTML formatting on thechnical communications. As you can see I'm using a plain-text mail client and it messes up. > > <div> </div> > > <div>can you give me some insight regarding this commit:<br/> > https://github.com/libvirt/libvirt/commit/7d5b283065e484645b6116397ac55b068…</div> That one does not seem to be related to your issue. It simply assumes that when starting a qemu process (so nothing really related to any of the vol-* and pool-* virsh APIs) when the disk is referencing a volume from the storage driver and the referenced volume is a directory, then we assume that qemu should use 'fat' as the driver. > > <div> </div> > > <div>I'm having strange issues since switching from libvirt 5.4.0 to 5.5.0 which is as follows:</div> > > <div> </div> I'll have to cut off everything here and replace it with a non-HTML version so there might be weird quoting: > > on libvirt 5.5.0 the volumes list only show the first image from the remote ceph rbd cluster: > > # virsh vol-list vm_volumes > Name Path > ------------------------------------------------- > vmanager101-root vm_volumes/vmanager101-root > > But the rbd command itself all seing them: > # rbd -p vm_volumes ls > vmanager101-root > vmanager102-root > vmanager103-root > vmanager105-root > > on libvirt5.4.0 everything looks correct and it shows all the existing ones: > > # virsh vol-list vm_volumes > Name Path > ------------------------------------------------- > vmanager101-root vm_volumes/vmanager101-root > vmanager102-root vm_volumes/vmanager102-root > vmanager103-root vm_volumes/vmanager103-root > vmanager105-root vm_volumes/vmanager105-root I've looked at the git history and didn't find anything that could be related here unfortunately. I'm also not very familiar with RBD in libvirt. > > Seconds strange behavior is as follow: > virsh vol-create-as vm_volumes test.img --capacity 1G --format raw > error: Failed to create vol test.img error: unsupported configuration: only RAW volumes are supported by this storage pool So this is really weird. The error message is produced if the target format is not 'raw' but the above command clearly supplies it. I'd suggest you collect debug logs while trying both and attach them here. Unfortunately I don't see what could be wrong here.

3 2

[libvirt-users] UDP broadcasts vs. nat Masquerading issue
by Nikolai Zhubr 06 Jul '19

06 Jul '19

Hi all, I'm observing an issue that as soon as libvirt starts, UPD broadcasts going through physical network (and unrelated to any virtualization) get broken. Specifically, windows neighbourhood browsing through samba's nmbd starts suffering badly (Samba is running on this same box). At the moment I'm running a quite outdated version 1.2.9 of libvirt, but other than this issue, it does its job pretty well, so I'd first consider some patching/backporting rather than totally replacing it with a new one. Anyway, I first need to better understand what is going on and what is wrong with it. This could also be related somewhat to https://www.redhat.com/archives/libvir-list/2013-September/msg01311.html but I suppose it is not exactly that thing. I've already figured the source of trouble is anyway related to these rules added: -A POSTROUTING -o br0 -j MASQUERADE -A POSTROUTING -o enp0s25 -j MASQUERADE -A POSTROUTING -o virbr2_nic -j MASQUERADE -A POSTROUTING -o vnet0 -j MASQUERADE Here, virbr2_nic and vnet0 are used by libvirt for arranging network configurations for VMs, ok. However, br0 is a main interface of this host with primary ip address, with enp0s25 being a physical nic of this host, and it is used for all sorts of regular (unrelated to virtualization) communications. Also, br0 is used for attaching bridged (as opposed to NATed) VMs managed by libvirt. Clearly, libvirt somehow chooses to set up masquerading for literally all existing network interfaces here (except lo), but I can't see a real reason for the first two rules in the list above. Furthermore, they corrupt UDP broadcats coming from outside and reaching this host (through enp0s25/br0) such that source address gets replaced by this hosts primary address (as per masquerading). I've verified this by arranging a hand-crafted UDP listener and printing the respective source addresses as seen by normal userspace. Now I've discovered that I can "eliminate" the problem by either: 1. Removing "-A POSTROUTING -o br0 -j MASQUERADE" (manually) 2. Inserting "-A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.255/32 -j ACCEPT" (Of course correcting rules by hand is not a solution, just a test) So question is, how the correct rules should ideally look like? And, is this issue known/fixed in most current libvirt? Thank you, Regards, Nikolai

3 3

[libvirt-users] libvirt-python Create External Snapshot
by Roger Jin 05 Jul '19

05 Jul '19

Hi Libvirt, Does libvirt-python currently support the creation of external snapshots? I ran domain.snapshotCreateXML(xml) where xml is <domainsnapshot> <name>snap1</name> <description>Snapshot of OS install and updates</description> <disks> <disk name='vda'> <source file='/home/potato/libvirt/backups/test-snap1.qcow2'/> </disk> </disks> </domainsnapshot> And got the following error: if ret is None:raise libvirtError('virDomainSnapshotCreateXML() failed', dom=self) libvirt.libvirtError: unsupported configuration: disk 'vda' must use snapshot mode 'internal' Is this expected? Thanks! Sincerely, Roger Jin

2 1

[libvirt-users] Blockpull while domain is shutoff
by Roger Jin 05 Jul '19

05 Jul '19

Hi Libvirt, Is there a command that performs the same operation as blockpull while a domain is shutoff? I have a backing chain base <-- A <-- B that I would like to shorten to base <-- B It seems like blockpull B, setting --base <path-to-base> does this, but only when the domain is running. Thanks! Sincerely, Roger Jin

2 1

[libvirt-users] libvirt-python blockPull
by Roger Jin 04 Jul '19

04 Jul '19

Hi Libvirt, Does libvirt-python currently support adding a base path to virDomain.blockPull()? In particular, I would like to have the behavior of the following virsh command (from https://kashyapc.wordpress.com/2013/01/22/live-backup-with-external-disk-sn…) in libvirt-python: [image: image.png] Thanks! Sincerely, Roger Jin

1 0

[libvirt-users] macvtap vlan and tcp header overhead (and mtu size)
by Marc Roos 30 Jun '19

30 Jun '19

I have a host setup for libvirt kvm/qemu vms. And I wonder a bit about the overhead of the macvtap and how to configure the mtu's properly. To be able to communicate with the host, I have moved the ip address of the host from the adapter to a macvtap to allow host communication. I have the below setup on hosts. +---------+ | macvtap0| +--| ip | | | mtu1500 | | +---------+ +---------+ | +---------+ |eth0.v100| | | macvtap1| +--| no ip +-+--| ip | +---------+ | | mtu1500 | | mtu1500 | | eth0 | | +---------+ +---------+ | +--+ | mtu9000 | | +---------+ +---------+ | |eth0.v101| +--| ip | | mtu9000 | +---------+ https://pastebin.com/9jJrMCTD I can do a ping -M do -s 9000 between hosts via the vlan interface eth0.v101. That is as expected. The ping -M do -s 1500 macvtap0 and another host or macvtap1 fails. The maximum size that does not fragment is 1472. That is 28 bytes??? Where have they gone? I am only using macvtap, can this be the combination of the parent interface being a vlan and that macvtap is not properly handling this? Anyone experienced something similar? Or can explain where these 28 bytes go?

2 1