- Users - libvirt List Archives

[libvirt-users] Live update network configuration
by Shashwat shagun 01 Feb '18

01 Feb '18

Hi, i want live update the <metadata/> in libvirt network xml config. is this possible? -- Regards, Shashwat Shagun

2 2

[libvirt-users] libvirtd hangs
by Artem Likhachev 01 Feb '18

01 Feb '18

Hello everybody! We have a cluster of servers managed by VMmanager 5 KVM (by ispsystem). A typical node: # cat /etc/redhat-release CentOS Linux release 7.3.1611 (Core) # uname -r 3.10.0-693.11.6.el7.x86_64 # rpm -qa |grep libvirt libvirt-daemon-driver-qemu-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-disk-3.7.0-1.el7.centos.x86_64 libvirt-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-core-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-nodedev-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-lxc-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-iscsi-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-gluster-3.7.0-1.el7.centos.x86_64 libvirt-daemon-kvm-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-network-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-interface-3.7.0-1.el7.centos.x86_64 libvirt-daemon-config-network-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-rbd-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-3.7.0-1.el7.centos.x86_64 libvirt-libs-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-nwfilter-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-secret-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-mpath-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-scsi-3.7.0-1.el7.centos.x86_64 libvirt-client-3.7.0-1.el7.centos.x86_64 libvirt-daemon-3.7.0-1.el7.centos.x86_64 libvirt-daemon-config-nwfilter-3.7.0-1.el7.centos.x86_64 libvirt-daemon-driver-storage-logical-3.7.0-1.el7.centos.x86_64 # rpm -qa |grep qemu qemu-kvm-common-rhev-2.6.0-27.1.el7.centos.maros.x86_64 ipxe-roms-qemu-20160127-5.git6366fa7a.el7.noarch qemu-img-rhev-2.6.0-27.1.el7.centos.maros.x86_64 qemu-kvm-rhev-2.6.0-27.1.el7.centos.maros.x86_64 # rpm -qa |grep ebtables ebtables-2.0.10-15.el7.centos.marosnet.x86_64 ebtables build with patch https://marc.info/?l=netfilter-devel&m=150728694430435 (described at https://bugzilla.redhat.com/show_bug.cgi?id=1495893) Sometimes libvirtd just hangs and stops answering for virsh requests (like `virsh list --all`). At those moments: # strace -p 5786 read(53, "\0\0\0\34", 4) = 4 read(53, "keep\0\0\0\1\0\0\0\2\0\0\0\2\0\0\0\0\0\0\0\0", 24) = 24 poll([{fd=5, events=POLLIN}, {fd=7, events=POLLIN}, {fd=12, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}, {fd=15, events=POLLIN}, {fd=19, events=POLLIN}, {fd=23, events=POLLIN|POLLERR|POLLHUP}, {fd=21, events=POLLIN|POLLERR|POLLHUP}, {fd=27, events=POLLIN|POLLERR|POLLHUP}, {fd=25, events=POLLIN|POLLERR|POLLHUP}, {fd=22, events=POLLIN|POLLERR|POLLHUP}, {fd=24, events=POLLIN|POLLERR|POLLHUP}, {fd=26, events=POLLIN|POLLERR|POLLHUP}, {fd=29, events=POLLIN|POLLERR|POLLHUP}, {fd=30, events=POLLIN|POLLERR|POLLHUP}, {fd=31, events=POLLIN|POLLERR|POLLHUP}, {fd=33, events=POLLIN|POLLERR|POLLHUP}, {fd=32, events=POLLIN|POLLERR|POLLHUP}, {fd=36, events=POLLIN|POLLERR|POLLHUP}, {fd=35, events=POLLIN|POLLERR|POLLHUP}, {fd=39, events=POLLIN|POLLERR|POLLHUP}, {fd=40, events=POLLIN|POLLERR|POLLHUP}, {fd=41, events=POLLIN|POLLERR|POLLHUP}, {fd=44, events=POLLIN|POLLERR|POLLHUP}, {fd=42, events=POLLIN|POLLERR|POLLHUP}, {fd=43, events=POLLIN|POLLERR|POLLHUP}, {fd=48, events=POLLIN|POLLERR|POLLHUP}, {fd=49, events=POLLIN|POLLERR|POLLHUP}, {fd=59, events=POLLIN|POLLERR|POLLHUP}, {fd=46, events=POLLIN|POLLERR|POLLHUP}, {fd=50, events=POLLIN|POLLERR|POLLHUP}, ...], 43, 5000 # gdb -p 5786 (gdb) thread apply all bt Thread 17 (Thread 0x7f411a9d7700 (LWP 5788)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f4129c2a2e6 in virCondWait (c=c@entry=0x7f412b18ebb8, m=m@entry=0x7f412b18eb90) at util/virthread.c:154 #2 0x00007f4129c2ada3 in virThreadPoolWorker (opaque=opaque@entry=0x7f412b183ab0) at util/virthreadpool.c:124 #3 0x00007f4129c2a078 in virThreadHelper (data=<optimized out>) at util/virthread.c:206 #4 0x00007f4127033dc5 in start_thread (arg=0x7f411a9d7700) at pthread_create.c:308 #5 0x00007f4126d6273d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 16 (Thread 0x7f411a1d6700 (LWP 5789)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f4129c2a2e6 in virCondWait (c=c@entry=0x7f412b18ebb8, m=m@entry=0x7f412b18eb90) at util/virthread.c:154 #2 0x00007f4129c2ada3 in virThreadPoolWorker (opaque=opaque@entry=0x7f412b183a00) at util/virthreadpool.c:124 #3 0x00007f4129c2a078 in virThreadHelper (data=<optimized out>) at util/virthread.c:206 #4 0x00007f4127033dc5 in start_thread (arg=0x7f411a1d6700) at pthread_create.c:308 #5 0x00007f4126d6273d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 15 (Thread 0x7f41199d5700 (LWP 5790)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f4129c2a2e6 in virCondWait (c=c@entry=0x7f412b18ebb8, m=m@entry=0x7f412b18eb90) at util/virthread.c:154 #2 0x00007f4129c2ada3 in virThreadPoolWorker (opaque=opaque@entry=0x7f412b183950) at util/virthreadpool.c:124 #3 0x00007f4129c2a078 in virThreadHelper (data=<optimized out>) at util/virthread.c:206 #4 0x00007f4127033dc5 in start_thread (arg=0x7f41199d5700) at pthread_create.c:308 #5 0x00007f4126d6273d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 14 (Thread 0x7f41191d4700 (LWP 5791)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 #1 0x00007f4129c2a2e6 in virCondWait (c=c@entry=0x7f412b18ebb8, m=m@entry=0x7f412b18eb90) at util/virthread.c:154 #2 0x00007f4129c2ada3 in virThreadPoolWorker (opaque=opaque@entry=0x7f412b1838a0) at util/virthreadpool.c:124 #3 0x00007f4129c2a078 in virThreadHelper (data=<optimized out>) at util/virthread.c:206 #4 0x00007f4127033dc5 in start_thread (arg=0x7f41191d4700) at pthread_create.c:308 #5 0x00007f4126d6273d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 Thread 13 (Thread 0x7f41189d3700 (LWP 5792)): ---Type <return> to continue, or q <return> to quit--- log_level = 3 at /etc/libvirt/libvirtd.conf doesn't help to detect the problem. Actually, libvirtd continues acting, but is not responding. It's like waiting for something... may be an answer. No zombieing, no cpu loading. This fixes the issue: rm -f /run/ebtables.lock ; killall -9 virsh; systemctl restart systemd-{journald,udevd,logind,machined} ; systemctl restart libvirtd The same situation appears with libvirt-3.2.0-14.el7_4.7.x86_64. Could anybody help to resolve this situation?

2 4

[libvirt-users] Could not destroy domain, current job is remoteDispatchConnectGetAllDomainStats
by Serhii Kharchenko 31 Jan '18

31 Jan '18

Hello libvirt-users list, We're catching the same bug since 3.4.0 version (3.3.0 works OK). So, we have process that is permanently connected to libvirtd via socket and it is collecting stats, listening to events and control the VPSes. When we try to 'shutdown' a number of VPSes we often catch the bug. One of VPSes sticks in 'in shutdown' state, no related 'qemu' process is present, and there is the next error in the log: Jan 17 13:54:20 server1 libvirtd[20437]: 2018-01-17 13:54:20.005+0000: 20438: warning : qemuGetProcessInfo:1460 : cannot parse process status data Jan 17 13:54:20 server1 libvirtd[20437]: 2018-01-17 13:54:20.006+0000: 20441: error : virFileReadAll:1420 : Failed to open file '/sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d36\x2dDOMAIN1.scope/cpuacct.usage': No such file or directory Jan 17 13:54:20 server1 libvirtd[20437]: 2018-01-17 13:54:20.006+0000: 20441: error : virCgroupGetValueStr:844 : Unable to read from '/sys/fs/cgroup/cpu,cpuacct/machine.slice/machine-qemu\x2d36\x2dDOMAIN1.scope/cpuacct.usage': No such file or directory Jan 17 13:54:20 server1 libvirtd[20437]: 2018-01-17 13:54:20.006+0000: 20441: error : virCgroupGetDomainTotalCpuStats:3319 : unable to get cpu account: Operation not permitted Jan 17 13:54:23 server1 libvirtd[20437]: 2018-01-17 13:54:23.805+0000: 20522: warning : qemuDomainObjBeginJobInternal:4862 : Cannot start job (destroy, none) for domain DOMAIN1; current job is (query, none) owned by (20440 remoteDispatchConnectGetAllDomainStats, 0 <null>) for (30s, 0s) Jan 17 13:54:23 server1 libvirtd[20437]: 2018-01-17 13:54:23.805+0000: 20522: error : qemuDomainObjBeginJobInternal:4874 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchConnectGetAllDomainStats) I think only the last line matters. The bug is highly reproducible. We can easily catch it even when we call multiple 'virsh shutdown' in shell one by one. When we shutdown the process connected to the socket - everything become OK and the bug is gone. The system is used is Gentoo Linux, tried all modern versions of libvirt (3.4.0, 3.7.0, 3.8.0, 3.9.0, 3.10.0, 4.0.0-rc2 (today's version from git)) and they have this bug. 3.3.0 works OK. Thanks for any help in advance. We can send any additional info if needed. ~Serhii

5 8

[libvirt-users] How to use /dev/net/tun from libvirt-lxc with user namespacing enabled
by Thiago Padilha 31 Jan '18

31 Jan '18

I have a container rootfs that I use to keep all work-related stuff. This container was originally created by lxd (which creates all containers for use with user namespacing), but now I decided to start using libvirt for container management since I already use it for virtual machines, which will spare me from dealing with multiple hypervisor technologies. I managed to create a working domain xml for the container, and everything seems to be working very well except one thing: I cannot start openconnect (VPN software) inside the container. I noticed that by default libvirt won't create /dev/net/tun for the container, so I added this to the domain xml: <filesystem type='mount'> <source dir='/dev/net'/> <target dir='/dev/net'/> </filesystem> This successfully created /dev/net/tun in the container, but openconnect still can't open it even though it has 666 permissions. It seems this is exactly what lxd does to allow VPNs for their unprivileged containers, as shown by the output of ls -l /dev/net total 0 crw-rw-rw- 1 nobody nogroup 10, 200 Jan 29 13:23 tun The same container can also be successfully booted with systemd-nspawn, also allowing openconnect to create its VPN (though systemd-nspawn appears to create a new device node, owned by root relative to the user namespace). I already tried setting security driver to "none" in /etc/libvirt/lxc.conf, but it had no effect. I get "Operation not permitted" when trying to open /dev/net/tun, which is also the message openconnect displays in its logs. Can someone guide me on how I might debug what is causing this error? BTW, here's the full xml: <domain type='lxc'> <name>work-stuff</name> <uuid>ffee008c-ec6b-48ab-af6d-4aba830847a1</uuid> <memory unit='KiB'>8388608</memory> <currentMemory unit='KiB'>8388608</currentMemory> <vcpu placement='static'>16</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64'>exe</type> <init>/sbin/init</init> </os> <idmap> <uid start='0' target='165536' count='65536'/> <gid start='0' target='165536' count='65536'/> </idmap> <cpu mode='host-model'> <model fallback='allow'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/lib/libvirt/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/var/lib/libvirt/containers/work-stuff/rootfs'/> <target dir='/'/> </filesystem> <filesystem type='mount'> <source dir='/dev/net'/> <target dir='/dev/net'/> </filesystem> <interface type='network'> <mac address='52:54:00:3e:59:e9'/> <source network='default'/> </interface> <console type='pty'> <target type='lxc' port='0'/> </console> </devices> </domain>

2 2

[libvirt-users] Cannot add metadata to network XML
by Shashwat shagun 30 Jan '18

30 Jan '18

Hi, i tried adding metadata to Network XML ( NOT DOMAIN XML) but it removes it as soon as i save here's my network XML <network> <name>bridgeTest0</name> <uuid>cf2aae7a-b459-459b-ba2c-399190607629</uuid> <metadata> <app1:foo xmlns:app1="http://app1.org/app1/"> <app1:test>testing!</app1:test> </app1:foo> </metadata> <forward mode='bridge'/> <bridge name='lxcbr0'/> </network> -- Regards, Shashwat Shagun

2 2

[libvirt-users] Connection to libvirt
by Ori Liel 30 Jan '18

30 Jan '18

My attempt to install a host using oVirt results in the following message in the host: libvirt-buests.sh[3679]: Unable to connect to libvirt currently. Retrying .. 10Please enter your authentication name: Please enter your password: after several such retries: libvirt-buests.sh[3679]: Unable to connect to libvirt currently. Retrying .. 1Can't connect to deafult. Skipping. What may be the cause of this? Thanks, Ori.

2 1

Re: [libvirt-users] issue with openssh-server running in a libvirt based centos virtual machine
by Peter Crowther 29 Jan '18

29 Jan '18

You say you can ping but not ssh. If you install tcpdump on the VM, can you see the ping packets arriving and leaving? If not, I suspect an address collision - especially if ping continues to work with the VM shut down. If you can't ping, check the other end of your bridge. I'm more familiar with open vSwitch, but I'm somewhat concerned that your bridge definition doesn't include a physical NIC as one of its connections. Peter On 27 Jan 2018 1:13 p.m., "Adrian Pascalau" <adrian27oradea(a)gmail.com> wrote: Hi, I have a strange issue in a libvirt environment, and I do not know how to solve it. I have two centos hosts: first one is a physical server called server1, that acts as a host for the second one, called centos1. The centos1 is a virtual machine (VM) running in server1. A linux bridge in forwarding mode is used to connect the centos1 VM network interface to the server1 network interface and to the external network. The centos1 VM and the linux bridge are managed with libvirt (well, the bridge itself in this case is created manually). # virsh net-dumpxml br0 <network connections='1'> <name>br0</name> <uuid>5aaf72a5-023d-4b84-9d7c-d68b0918f620</uuid> <forward mode='bridge'/> <bridge name='br0'/> </network> # brctl show bridge name bridge id STP enabled interfaces br0 8000.fc15b4137688 no eno1 vnet0 Both server1 and centos1 have IP addresses in the same subnet, and both are reachable with ping from every other host in my network. In both server1 and centos1, the openssh-server configuration in /etc/ssh/sshd_config is the default one, and has not been changed. When I ssh with Putty to the physical server server1 IP address, everything works as expected: I get a login prompt, I enter my password and I log in. However, when I use Putty to connect to the centos1 VM, I do not get a login prompt whatsoever. So I think there might be some issue in between the server1 physical interface and my centos VM. I used openssh-server in debug mode, so see where the ssh connection hangs, and here is what I get: [...] debug1: Server will not fork when running in debugging mode. debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug1: sshd version OpenSSH_7.4, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: private host key #0: ssh-rsa SHA256:pEuFQsodwK+0PoRzbVRba1ahHLEpwp8DG2KGQmxOGJk debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:F6HrSNWZhYaU7LMweI+RBviqTCHcTYyMBGPDz5OjT4c debug1: private host key #2: ssh-ed25519 SHA256:aG3V6jjPHXUnNeavbxT/xozqrb5q3yWDkkAmXBCdnGk debug1: inetd sockets after dupping: 3, 3 Connection from x.x.x.181 port 49436 on x.x.x.115 port 22 debug1: Client protocol version 2.0; client software version PuTTY_Release_0.70 debug1: no match: PuTTY_Release_0.70 debug1: Local version string SSH-2.0-OpenSSH_7.4 debug1: Enabling compatibility mode for protocol 2.0 debug1: SELinux support enabled [preauth] debug1: permanently_set_uid: 74/74 [preauth] debug1: list_hostkey_types: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] I tried with other windows based ssh clients (MobaXterm) and the same issue happens. I discussed this with people in the openssh mailing list, and they said this issue could most probably be caused by a path MTU/fragmentation problem... Then I moved my centos1 qcow2 image in another physical server called server2, with exactly the same hw specs and network connections, where I have installed an all-in-one OpenStack Pike. The network would be managed with neutron in this case, however I have configured neutron exactly so that the centos1 VM interface connects through a linux bridge (managed by neutron) to the server1 physical network interface, like in the libvirt case. # brctl show bridge name bridge id STP enabled interfaces brqa13eec69-a4 8000.0e7faabad6d4 no eno1 tap8cb53db0-fb tapb24a1cc5-20 Above the tapb24a1cc5-20 is the tap interface towards my centos1 VM. In this case, the Putty issue is gone, and I do not have any issue anymore. If I go back to the libvirt environment in server1, I get the same issue again. So I tend to think that my ssh connection issue is caused by the libvirt and the way networking is configured, however I do not know how to troubleshoot this further anymore. Any help is greatly appreciated. Adrian _______________________________________________ libvirt-users mailing list libvirt-users(a)redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users

3 5

[libvirt-users] How do i associate a VM to An IP
by Shashwat shagun 29 Jan '18

29 Jan '18

I want to block my VM with mac addr 52:54:00:a1:05:b6 from using IPs other than 192.168.124.130 is there any libvirt way to do this? -- Regards, Shashwat Shagun

3 2

[libvirt-users] Static DHCP assignments to VMs connected to bridge
by Dagmawi Biru 29 Jan '18

29 Jan '18

Hello libvirt-users, Given the following setup: --- [Rest of network/dhcp server] ------[ physical interface] ------ [ bridge] <----> | <---> [ VM ] I have an interface configured in my Domain XML like so: --- <interface type="bridge"> <source bridge="br20"> </interface> Bridge "br20" is connected to an external network that has a DHCP server. However, I want to set it so the interface for this VM has a static IP served to it, either by a locally running DHCP server or via some sort of static IP assignment in the XML configuration. Is this possible?

2 1

[libvirt-users] issue with openssh-server running in a libvirt based centos virtual machine
by Adrian Pascalau 27 Jan '18

27 Jan '18

Hi, I have a strange issue in a libvirt environment, and I do not know how to solve it. I have two centos hosts: first one is a physical server called server1, that acts as a host for the second one, called centos1. The centos1 is a virtual machine (VM) running in server1. A linux bridge in forwarding mode is used to connect the centos1 VM network interface to the server1 network interface and to the external network. The centos1 VM and the linux bridge are managed with libvirt (well, the bridge itself in this case is created manually). # virsh net-dumpxml br0 <network connections='1'> <name>br0</name> <uuid>5aaf72a5-023d-4b84-9d7c-d68b0918f620</uuid> <forward mode='bridge'/> <bridge name='br0'/> </network> # brctl show bridge name bridge id STP enabled interfaces br0 8000.fc15b4137688 no eno1 vnet0 Both server1 and centos1 have IP addresses in the same subnet, and both are reachable with ping from every other host in my network. In both server1 and centos1, the openssh-server configuration in /etc/ssh/sshd_config is the default one, and has not been changed. When I ssh with Putty to the physical server server1 IP address, everything works as expected: I get a login prompt, I enter my password and I log in. However, when I use Putty to connect to the centos1 VM, I do not get a login prompt whatsoever. So I think there might be some issue in between the server1 physical interface and my centos VM. I used openssh-server in debug mode, so see where the ssh connection hangs, and here is what I get: [...] debug1: Server will not fork when running in debugging mode. debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug1: sshd version OpenSSH_7.4, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: private host key #0: ssh-rsa SHA256:pEuFQsodwK+0PoRzbVRba1ahHLEpwp8DG2KGQmxOGJk debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:F6HrSNWZhYaU7LMweI+RBviqTCHcTYyMBGPDz5OjT4c debug1: private host key #2: ssh-ed25519 SHA256:aG3V6jjPHXUnNeavbxT/xozqrb5q3yWDkkAmXBCdnGk debug1: inetd sockets after dupping: 3, 3 Connection from x.x.x.181 port 49436 on x.x.x.115 port 22 debug1: Client protocol version 2.0; client software version PuTTY_Release_0.70 debug1: no match: PuTTY_Release_0.70 debug1: Local version string SSH-2.0-OpenSSH_7.4 debug1: Enabling compatibility mode for protocol 2.0 debug1: SELinux support enabled [preauth] debug1: permanently_set_uid: 74/74 [preauth] debug1: list_hostkey_types: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] I tried with other windows based ssh clients (MobaXterm) and the same issue happens. I discussed this with people in the openssh mailing list, and they said this issue could most probably be caused by a path MTU/fragmentation problem... Then I moved my centos1 qcow2 image in another physical server called server2, with exactly the same hw specs and network connections, where I have installed an all-in-one OpenStack Pike. The network would be managed with neutron in this case, however I have configured neutron exactly so that the centos1 VM interface connects through a linux bridge (managed by neutron) to the server1 physical network interface, like in the libvirt case. # brctl show bridge name bridge id STP enabled interfaces brqa13eec69-a4 8000.0e7faabad6d4 no eno1 tap8cb53db0-fb tapb24a1cc5-20 Above the tapb24a1cc5-20 is the tap interface towards my centos1 VM. In this case, the Putty issue is gone, and I do not have any issue anymore. If I go back to the libvirt environment in server1, I get the same issue again. So I tend to think that my ssh connection issue is caused by the libvirt and the way networking is configured, however I do not know how to troubleshoot this further anymore. Any help is greatly appreciated. Adrian

1 0

[libvirt-users] Create virtual machine failed using virDomainCreateXML
by netsurfed 25 Jan '18

25 Jan '18

Hi all, When I created a virtual machine using virDomainCreateXML, with virtualport type was openvswitch, and virtual machine creation failed. The error message is: internal error: Child process (/sbin/tc filter add dev vnet110 parent ffff: protocol all u32 match u32 0 0 police rate 0kbps burst 0kb mtu 64kb drop flowid :1) unexpected exit status 2: RTNETLINK answers: Invalid argument The Domain XML file the <interface> section like this: <interface type='bridge'><mac address='52:54:00:71:b1:b6'/><source bridge='ovsbr'/><virtualport type='openvswitch'/><address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/></interface> I looked at the system log and it looked like an ovs port problem: 4560 Jan 22 17:43:33 ubuntu-24 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --if-exists del-port vnet110 -- add-port lcs_br_1 vnet110 -- set Interface vnet110 "external- ids:attached-mac=\"52:54:00:21:63:25\"" -- set Interface vnet110 "external-ids:iface-id=\"d0e5c770-6f03-4703-b107-3b074f989b06\"" -- set Interface vnet110 "external-ids:vm-id=\"1c79066e- 46a2-4fa0-984c-a88a7fffcd2b\"" -- set Interface vnet110 external-ids:iface-status=active 4561 Jan 22 17:43:33 ubuntu-24 kernel: [29028.418651] device vnet110 entered promiscuous mode 4562 Jan 22 17:43:33 ubuntu-24 libvirtd[4307]: 2018-01-22 09:43:33.507+0000: 4310: error : virCommandWait:2572 : internal error: Child process (/sbin/tc filter add dev vnet110 parent ffff: pr otocol all u32 match u32 0 0 police rate 0kbps burst 0kb mtu 64kb drop flowid :1) unexpected exit status 2: RTNETLINK answers: Invalid argument 4563 Jan 22 17:43:33 ubuntu-24 libvirtd[4307]: We have an error talking to the kernel 4564 Jan 22 17:43:33 ubuntu-24 kernel: [29028.454028] device vnet110 left promiscuous mode 4565 Jan 22 17:43:33 ubuntu-24 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --if-exists del-port vnet110 However, when I use "tc" to manage "vnet*" of an existing virtual machine, it still returns the same error. root@ubuntu-24:~# /sbin/tc filter add dev vnet83 parent ffff: protocol all u32 match u32 0 0 police rate 0kbps burst 0kb mtu 64kb drop flowid :1 RTNETLINK answers: Invalid argument We have an error talking to the kernel Is there any reason for this problem? Thank you very much. Below some information about my machine: libvirt version: 3.4.0 ovs_version: "2.8.90" root@ubuntu-24:~# uname -a Linux ubuntu-24 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

2 3

[libvirt-users] Create virtual machine failed using virDomainCreateXML
by netsurfed 23 Jan '18

23 Jan '18

Hi all, When I created a virtual machine using virDomainCreateXML, with virtualport type was openvswitch, and virtual machine creation failed. The error message is: internal error: Child process (/sbin/tc filter add dev vnet110 parent ffff: protocol all u32 match u32 0 0 police rate 0kbps burst 0kb mtu 64kb drop flowid :1) unexpected exit status 2: RTNETLINK answers: Invalid argument The Domain XML file the <interface> section like this: I looked at the system log and it looked like an ovs port problem. However, when I use "tc" to manage "vnet*" of an existing virtual machine, it still returns the same error. root@ubuntu-24:~# /sbin/tc filter add dev vnet83 parent ffff: protocol all u32 match u32 0 0 police rate 0kbps burst 0kb mtu 64kb drop flowid :1 RTNETLINK answers: Invalid argument We have an error talking to the kernel Is there any reason for this problem? Thank you very much. Below some information about my machine: libvirt version: 3.4.0 ovs_version: "2.8.90" root@ubuntu-24:~# uname -a Linux ubuntu-24 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

1 0

[libvirt-users] Issues Connecting to Remote Host Through SSH
by T A 23 Jan '18

23 Jan '18

Hello, I've been unable to connect to a remote host from within my network using the following argument. virsh/virt-manager -c qemu+ssh://user@host:port/system I've tried libssh & libssh2 as well. Using ssh just prompts me for the host password indefinitely. When using libssh2 the connection is rejected. The host computer is using a custom ssh port which I've added to the above argument. Neither box has an enabled root user. Remote box $ cat /etc/debian_version 9.3 $ ssh -V OpenSSH_7.4p1 Debian-10+deb9u2, OpenSSL 1.0.2l 25 May 2017 $ virsh --version 3.0.0 Host box $ cat /etc/centos-release CentOS Linux release 7.4.1708 (Core) $ ssh -V OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 $ virsh --version 3.2.0 Any help is greatly appreciated.

3 2

[libvirt-users] debug kernel
by llilulu 20 Jan '18

20 Jan '18

libvirt 3.4.0 centos 7.4 Hi: I want usr libvrit create vm to debug linux kernel. Previous , I usr qemu cmd(-gdb tcp::1234) directly. When I search on Internet, I found I can use libvrt xml configure kgdb. <qemu:commandline> <qemu:arg value='-gdb'/> <qemu:arg value='tcp::1234'/> </qemu:commandline> But configure xml use kgdb is differnet on kernel boot. use qemu cmd, before I connect kernel use gdb , the kernel still not boot, When I use gdb connect the vm and put continue cmd on gdb, the kernel will start boot. But I usr libvirt xml configure kgdb, the kernel not stop, I want known that have some way I can use libvirt xml configure kgdb let kernel stop boot before I put continue on gdb? Thanks

1 0

[libvirt-users] debug kernel
by llilulu 20 Jan '18

20 Jan '18

libvirt 3.4.0 centos 7.4 Hi: I want usr libvrit create vm to debug linux kernel. Previous , I usr qemu cmd(-gdb tcp::1234) directly. When I search on Internet, I found I can usr libvrt xml configure kgdb. <qemu:commandline> <qemu:arg value='-gdb'/> <qemu:arg value='tcp::1234'/> </qemu:commandline> But configure xml usr kgdb is differnet on kernel boot. usr qemu cmd, before I connect kernel usr gdb , the kernel still not boot, When I usr gdb connect the vm and put continue cmd on gdb, the kernel will start boot. But I usr libvirt xml configure kgdb, the kernel not stop, I want known that have some way I can use libvirt xml configure kgdb let kernel stop boot before I put continue on gdb? Thanks

1 0

[libvirt-users] How do i programmatically connect to libvirt using tcp
by Shashwat shagun 16 Jan '18

16 Jan '18

Hi, How do i connect to libvirt using tcp with sasl authentication using libvirt-go(Golang) ? -- Regards, Shashwat Shagun

1 0

[libvirt-users] domain xml does not have <target dev='vnetX'/>
by Shashwat shagun 15 Jan '18

15 Jan '18

Hi I'm trying to monitor bandwidth usage of a KVM VM but couldn't find <target dev='vnet0'/> on virsh edit domain_name all i get is this <interface type='network'> <mac address='52:54:00:a1:05:b6'/> <source network='vmango'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> how do i determine which vnet is connected to which domain? -- Regards, Shashwat Shagun

2 1

[libvirt-users] Limiting instructions for guest to help with migration to different host
by R 15 Jan '18

15 Jan '18

Hello, I am migrating a suspended x86-64 guest (disk & state) across different x86-64 hosts with small differences in the available CPU instructions and when I try to resume the guest on the different host libvirt reports an error like this "CPU feature XXX not found" and fails. My question is, is there a way to limit the instructions that are used on the "origin" host when creating the guest to avoid this error? I am pretty sure there is a set of basic instructions that are available across all hosts which could be used for this purpose. Thank you! /R

2 2

Re: [libvirt-users] VM migration upon shutdown in centos 7
by Michal Privoznik 12 Jan '18

12 Jan '18

[Please keed the list CCed] On 01/10/2018 04:43 PM, Raman Gupta wrote: >> Does this command alone succeed? > Yes. I have used this command to migrate VMs successfully, without even > knowing that spelling has changed. > > >> I don't know enough about systemd but maybe it's not waiting for virsh to > finish? > Yes I also think virsh or libvirtd or related service does not wait > before /root/vm_migrate.sh > is called and hence live migration fails. > If I replace Live Migration with a simple ping to peer node in the > vPreShutdownHook.service, then ping goes thru successfully thus indicating > network service was UP at that time. > >> Can you try to get any logs to see what is going on actually? > Jan 8 15:30:58 desktop4 systemd: Stopping Session c1 of user gdm. > Jan 8 15:30:58 desktop4 gdm: Freeing conversation 'gdm-launch-environment' > with active job > Jan 8 15:30:58 desktop4 systemd: Stopped target Sound Card. > Jan 8 15:30:58 desktop4 systemd: Stopping Sound Card. > Jan 8 15:30:58 desktop4 systemd: Stopping LVM2 PV scan on device 8:3... > Jan 8 15:30:58 desktop4 systemd: Removed slice system-getty.slice. > Jan 8 15:30:58 desktop4 systemd: Stopping system-getty.slice. > Jan 8 15:30:58 desktop4 systemd: Stopping Authorization Manager... > Jan 8 15:30:58 desktop4 systemd: Stopping Virtual Machine and Container > Registration Service... > Jan 8 15:30:58 desktop4 systemd: Stopping GlusterFS brick processes > (stopping only)... > Jan 8 15:30:58 desktop4 systemd: Removed slice > system-selinux\x2dpolicy\x2dmigrate\x2dlocal\x2dchanges.slice. > Jan 8 15:30:58 desktop4 systemd: Stopping > system-selinux\x2dpolicy\x2dmigrate\x2dlocal\x2dchanges.slice. > Jan 8 15:30:58 desktop4 systemd: Stopping Availability of block devices... > Jan 8 15:30:58 desktop4 systemd: Stopping Virtual Machine > qemu-1-GsmController. > Jan 8 15:30:58 desktop4 alsactl[837]: alsactl daemon stopped > Jan 8 15:30:58 desktop4 systemd: Stopping Session 1 of user root. > Jan 8 15:30:58 desktop4 systemd: Stopping Manage Sound Card State (restore > and store)... I don't see your service being called. Anyway, look at libvirt-guests.service file. Looks like the following lines cause systemd to wait for a command to finish: [Service] EnvironmentFile=-/etc/sysconfig/libvirt-guests # Hack just call traditional service until we factor # out the code ExecStart=@libexecdir@/libvirt-guests.sh start ExecStop=@libexecdir@/libvirt-guests.sh stop Type=oneshot RemainAfterExit=yes StandardOutput=journal+console TimeoutStopSec=0 > > > > On Wed, Jan 10, 2018 at 8:46 PM, Michal Privoznik <mprivozn(a)redhat.com> > wrote: > >> On 01/05/2018 12:00 PM, Raman Gupta wrote: >>> Hi, >>> >>> I have CentOS 7, two node system which allows live VM migration between >>> them. Live migration triggered from virsh is happily happening. I am >> using >>> GlusterFS for replicating VM disk files. >>> >>> Now I want to automatically do the live migration at the time of >>> reboot/shutdown/halt of the host node and for this I have written a >> systemd >>> service unit [vPreShutdownHook.service] and placed the live migration >>> command in a migrate script which is invoked from this service unit. The >>> migrate script is invoked but migration does not happen. >>> If someone has some idea please help me to migrate VM upon shutdown. >>> >>> >>> ######## vPreShutdownHook.service ############### >>> >>> [Unit] >>> Description=vPreShutdownHook >>> Requires=network.target >>> Requires=libvirtd.service >>> Requires=dbus.service >>> Requires=glusterd.service >>> Requires=glusterfsd.service >>> DefaultDependencies=no >>> Before=shutdown.target reboot.target >>> >>> [Service] >>> Type=oneshot >>> RemainAfterExit=true >>> ExecStart=/bin/true >>> ExecStop=/root/vm_migrate.sh >>> >>> [Install] >>> WantedBy=multi-user.target >>> >>> >>> ########## Command to migrate ############ >>> /usr/bin/virsh migrate --verbose --p2p --tunneled --live --compressed >>> --comp-methods "mt" --comp-mt-level 5 --comp-mt-threads 5 >>> --comp-mt-dthreads 5 MY_VM qemu+ssh://root@$node2/system >> >> Does this command alone succeed? >> BTW: unless really needed --live will only make the migration take longer. >> >> I don't know enough about systemd but maybe it's not waiting for virsh >> to finish? Can you try to get any logs to see what is going on actually? >> >> Michal >> > Michal

2 2

[libvirt-users] Whether libvirt can support all backing chain layer are iscsi network disk type
by Meina Li 11 Jan '18

11 Jan '18

Hi, For backing chain that all images are iscsi network disk type , such as iscsi://ip/iqn../0(base image) <-iscsi://ip/iqn../1(active image). Currently, 'qemu-img info --backing-chain' command can display correct backing file info, but after starting guest with the active image in backing chain, it don't include the related <backingStore> element in dumpxml. So, Whether libvirt can support all backing chain layer are iscsi network disk type? Best Regards Meina Li On Wed, Jan 3, 2018 at 6:39 AM, Meina Li <meili(a)redhat.com> wrote: > Hi, > > I am a libvirt qe, I am testing the function about the new location of > disk auth element (as sub-element of the source element) in Backing Chain > Management. And I have a questions: > > When all the backing chain are iscsi network disk type(no matter if the > authentication is exist), start the guest , it will only has the top > level disk in xml and there are no other backingStore in it. The test step > is below. > > So can this function support all backing chain with network disk type > actually, or only use network authentication as the base image for backing > chain of file disk type? > > Can you help reviewing it? Thanks very much in advance! > > Test step: > 1. iSCSI server: > o- iscsi ............................................................ > ................................................ [Targets: 2] > | o- iqn.2003-01.org.linux-iscsi.localhost.x8664:sn.9cba196611e6 > ....................................................... [TPGs: 1] > | | o- tpg1 ..................................................................................... > [gen-acls, tpg-auth, 1-way auth] > | | o- acls .............................. > ............................................................................ > [ACLs: 0] > | | o- luns .............................. > ............................................................................ > [LUNs: 4] > | | | o- lun0 .............................. > .................................. [fileio/file1 (/tmp/lun1.img) > (default_tg_pt_gp)] > | | o- portals .............................. > ...................................................................... > [Portals: 1] > | | o- 0.0.0.0:3260 .............................. > ....................................................................... > [OK] > | o- iqn.2003-01.org.linux-iscsi.localhost.x8664:sn.a8d92ebb4ece > ....................................................... [TPGs: 1] > | o- tpg1 ............................................................ > ...................................... [gen-acls, no-auth] > | o- acls .............................. > ............................................................................ > [ACLs: 0] > | o- luns .............................. > ............................................................................ > [LUNs: 3] > | | o- lun0 .............................. > .................................. [fileio/file3 (/tmp/lun3.img) > (default_tg_pt_gp)] > | o- portals .............................. > ...................................................................... > [Portals: 1] > | o- 0.0.0.0:3260 .............................. > ....................................................................... > [OK] > > 2. Set iscsi secret. > # cat iscsi-secret.xml > <secret ephemeral='no' private='yes'> > <description>iSCSI secret</description> > <usage type='iscsi'> > <target>libvirtiscsi</target> > </usage> > </secret> > # virsh secret-define iscsi-secret.xml > Secret 47bd2f3e-023f-44ba-85a3-e8fa7f16ff23 created > # MYSECRET=`printf %s "redhat" | base64` > # virsh secret-set-value 47bd2f3e-023f-44ba-85a3-e8fa7f16ff23 $MYSECRET > Secret value set > > 3. Create backing chain. > # qemu-img create -f qcow2 -b iscsi://redhat:redhat@10.66.7. > 27:3260/iqn.2003-01.org.linux-iscsi.localhost.x8664:sn.9cba196611e6/0 > iscsi://10.66.7.27:3260/iqn.2003-01.org.linux-iscsi.localhost.x8664:sn. > a8d92ebb4ece/0 -o backing_fmt=qcow2 > > 4. Start guest. > # qemu-img info iscsi://10.66.7.27:3260/iqn.2003-01.org.linux-iscsi. > localhost.x8664:sn.a8d92ebb4ece/0 --backing-chainimage: json:{"driver": > "qcow2", "file": {"lun": "0", "portal": "10.66.7.27:3260", "driver": > "iscsi", "transport": "tcp", "target": "iqn.2003-01.org.linux-iscsi. > localhost.x8664:sn.a8d92ebb4ece"}} > file format: qcow2 > virtual size: 5.0G (5368709120 <0536%20870%209120> bytes) > disk size: unavailable > cluster_size: 65536 > backing file: iscsi://redhat:redhat@10.66.7.27:3260/iqn.2003-01.org.linux- > iscsi.localhost.x8664:sn.9cba196611e6/0 > backing file format: qcow2 > Format specific information: > compat: 1.1 > lazy refcounts: false > refcount bits: 16 > corrupt: false > > image: json:{"driver": "qcow2", "file": {"lun": "0", "portal": " > 10.66.7.27:3260", "driver": "iscsi", "transport": "tcp", "user": > "redhat", "password": "redhat", "target": "iqn.2003-01.org.linux-iscsi. > localhost.x8664:sn.9cba196611e6"}} > file format: qcow2 > virtual size: 5.0G (5368709120 <0536%20870%209120> bytes) > disk size: unavailable > cluster_size: 65536 > Format specific information: > compat: 1.1 > lazy refcounts: false > refcount bits: 16 > corrupt: false > > # virsh dumpxml rhel7 | grep disk -A 9 > ... > <disk type='network' device='disk'> > <driver name='qemu' type='qcow2'/> > <source protocol='iscsi' name='iqn.2003-01.org.linux- > iscsi.localhost.x8664:sn.a8d92ebb4ece/0'> > <host name='10.66.7.27' port='3260'/> > </source> > <target dev='vdb' bus='virtio'/> > <alias name='virtio-disk1'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x09' > function='0x0'/> > </disk> > ... > > > Best Regards > Meina Li >

2 1

[libvirt-users] Proxmox to libvirt conversion?
by Andre Goree 10 Jan '18

10 Jan '18

Hello all. I'm wondering if anyone on this list has come across the need to convert a Proxmox VM to a libvirt VM. Given that they use the same underlying technology (qemu/KVM), I'm pretty sure it's possible and just a matter of hashing out the differences in configuration (i.e. creating the necessary libvirt xml for the Proxmox disk), but figured I'd ask here first in case anyone has dealt with this (probably will find and ask on the Proxmox mailing list as well). Thanks in advance! -- Andre Goree -=-=-=-=-=- Email - andre at drenet.net Website - http://blog.drenet.net PGP key - http://www.drenet.net/pubkey.html -=-=-=-=-=-

1 0

[libvirt-users] VM migration upon shutdown in centos 7
by Raman Gupta 10 Jan '18

10 Jan '18

Hi, I have CentOS 7, two node system which allows live VM migration between them. Live migration triggered from virsh is happily happening. I am using GlusterFS for replicating VM disk files. Now I want to automatically do the live migration at the time of reboot/shutdown/halt of the host node and for this I have written a systemd service unit [vPreShutdownHook.service] and placed the live migration command in a migrate script which is invoked from this service unit. The migrate script is invoked but migration does not happen. If someone has some idea please help me to migrate VM upon shutdown. ######## vPreShutdownHook.service ############### [Unit] Description=vPreShutdownHook Requires=network.target Requires=libvirtd.service Requires=dbus.service Requires=glusterd.service Requires=glusterfsd.service DefaultDependencies=no Before=shutdown.target reboot.target [Service] Type=oneshot RemainAfterExit=true ExecStart=/bin/true ExecStop=/root/vm_migrate.sh [Install] WantedBy=multi-user.target ########## Command to migrate ############ /usr/bin/virsh migrate --verbose --p2p --tunneled --live --compressed --comp-methods "mt" --comp-mt-level 5 --comp-mt-threads 5 --comp-mt-dthreads 5 MY_VM qemu+ssh://root@$node2/system Thanks,

3 2

[libvirt-users] All VMs permanently 'paused'
by Mark Coolen 08 Jan '18

08 Jan '18

I set up Alpine Linux as a libvirt-based KVM hypervisor last week. Everything was working beautifully on Friday. I left two Windows VMs up and running and two 'Saved' on Friday afternoon. When I go in this morning the two running ones were 'Paused' and I can't unpause them with virt-manager or with virsh. The other two saved ones I can start, but they are immediately paused too. When I click to unpause in virt- manager nothing happens. When I 'resume' in virsh I get 'Domain ... resumed' but it's immediately paused again. I looked in /var/log/libvirt/libvirtd.log and qemu/<DOMAIN>.log and there doesn't seem to be anything happening. No errors or anything. As I say. Everything was working fine on Friday afternoon. Does anyone know what I should try next? I'll attach my logs and the XML of one of my machines. _ _ _ ___ _ )\/,) ___ __ )L, )) __ __ )) __ _ _ ((`(( ((_( (| ((\ ((__((_)((_)(( (('((\(

3 4

[libvirt-users] virsh and bash scripts
by Hetz Ben Hamo 08 Jan '18

08 Jan '18

Hi, I'm looking for a way to use virsh to connect to vCenter/VCSA and do some simple management of the VM's in case of power loss (I'm planning to run the script upon UPS/apcd event of power failure). Looking through the virsh documentation, I didn't find 2 things: 1. Login through a script (I don't see any user/password, specially since VCSA for example requires to use username(a)vsphere.local [or your AD domain insteadd of vsphere.local if you connected it to your AD domain]) 2. Get all the VMs running on the vCenter, not just specific hosts. Any suggestion? Thanks

2 2

[libvirt-users] Libvirt not terminating SSH processes
by Priyanka Gupta 08 Jan '18

08 Jan '18

Hi LibVirt team, I am facing an issue with libvert. I am closing a "qemu+ssh" connection but the initiated ssh process are going into sleep mode and not getting killed upon closing the connection. Can you please help me figure out the issue ? -- With Kind Regards Priyanka Gupta

2 1