- Users - libvirt List Archives

[libvirt-users] SR-IOV Trust Setting
by ]σ§έ Λιαςάη 10 Oct '18

10 Oct '18

Is there a setting to allow/disable spoof checking and trust on VF from SR-IOV device? Lookin at https://www.libvirt.org/formatdomain.html#elementsDriverBackendOptions can this be done through the driver element? *<driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver>* At the moment i have to manually run: ip link set $_dev vf $vf spoofchk off ip link set $_dev vf $vf trust on Is it possible to set this values from the xml file on a per interface basis?

2 1

[libvirt-users] [OCaml] Reset of the libvirt-ocaml repository
by Pino Toscano 10 Oct '18

10 Oct '18

Hi libvirt developers and users, TL;DR: you need to clone again the libvirt-ocaml repository, if you had one already Due to the way the repository was converted from another SCM in the past, some of the metadata of older commits was broken, and this made the repository unusable with more strict git configurations (e.g. as used on GitHub). You can read more details about this in https://www.redhat.com/archives/libvir-list/2018-September/msg00214.html The ocaml-libvirt repository from Rich Jones' website was fixed, and replaced the existing libvirt-ocaml on libvirt.org. The libvirt-ocaml repository is the official home now (or again) for the OCaml bindings of libvirt. This means that you need to clone again the libvirt-ocaml repository, in case you had already a clone of this repository. In case you had pending work on it, make sure to carry it over from an old clone to a newer clone. As usual, if you have patches for libvirt-ocaml, feel free to send them to libvir-list, tagging them as [ocaml]. -- Pino Toscano

1 0

[libvirt-users] how libvirt check qemu domain status?
by Dmitry Melekhov 09 Oct '18

09 Oct '18

Hello! Could you tell me how libvirtd check qemu domain status? Looks like this is something more then just looking for pid. Because we have VMs running over gluster storage using gfapi and if something goes wrong, then libvirtd became unresponsive often enough.. Thank you!

1 1

[libvirt-users] live migration and config
by Dmitry Melekhov 04 Oct '18

04 Oct '18

Hello! After some mistakes yesterday we ( me and my colleague ) think that it will be wise for libvirt to check config file existence on remote side and through error if not, before migrating, otherwise migration will fail and VM fs can be damaged, because it is sort of remove of power plug... We missed twice yesterday :-( Could you tell me is there already such option or any plans to implement this? Thank you!

2 9

[libvirt-users] Libvirt 4.2.0 hang on destination on live migration cancel
by Scott Sullivan 03 Oct '18

03 Oct '18

I'm trying the following command on the source machine: virsh migrate --live --copy-storage-all --verbose TEST qemu+ssh:// 10.30.76.66/system If I ssh into the destination machine when this command is running, I can see NBD copying data as expected, and if I wait long enough it completes and succeeds. However if I ctrl+c this command above before it completes, it causes virsh commands (like virsh list) to just hang seemingly forever on the destination machine. Here's the libvirt debug entries when I run virsh list in this state: [root@host ~]# export LIBVIRT_DEBUG=1 [root@host ~]# virsh list 2018-10-02 05:14:05.053+0000: 16207: debug : virGlobalInit:359 : register drivers 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddbb60 name=Test 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering Test as driver 0 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddc660 name=ESX 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering ESX as driver 12018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddd560 name=remote 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering remote as driver 2 2018-10-02 05:14:05.054+0000: 16207: debug : virEventRegisterDefaultImpl:280 : registering default event implementation 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollAddHandle:115 : Used 0 handle slots, adding at least 10 more 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollInterruptLocked:722 : Skip interrupt, 0 0 2018-10-02 05:14:05.054+0000: 16207: info : virEventPollAddHandle:140 : EVENT_POLL_ADD_HANDLE: watch=1 fd=4 events=1 cb=0x7f70c18791c0 opaque=(nil) ff=(nil) 2018-10-02 05:14:05.054+0000: 16207: debug : virEventRegisterImpl:241 : addHandle=0x7f70c187a8a0 updateHandle=0x7f70c1879500 removeHandle=0x7f70c1879360 addTimeout=0x7f70c187a660 updateTimeout=0x7f70c1879690 removeTimeout=0x7f70c1879220 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollAddTimeout:230 : Used 0 timeout slots, adding at least 10 more2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollInterruptLocked:722 : Skip interrupt, 0 0 2018-10-02 05:14:05.054+0000: 16207: info : virEventPollAddTimeout:253 : EVENT_POLL_ADD_TIMEOUT: timer=1 frequency=-1 cb=0x5559f8dba380 opaque=0x7ffe5b15e700 ff=(nil) 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenAuth:1218 : name=<null>, auth=0x7f70c1ddea00, flags=0x0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83a3f0 classname=virConnect 2018-10-02 05:14:05.054+0000: 16208: debug : virThreadJobSet:99 : Thread 16208 is now running job vshEventLoop 2018-10-02 05:14:05.054+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.054+0000: 16207: debug : virConfLoadConfig:1576 : Loading config file '/etc/libvirt/libvirt.conf' 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 1 2018-10-02 05:14:05.054+0000: 16207: debug : virConfReadFile:752 : filename=/etc/libvirt/libvirt.conf 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.054+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=1 timeout=-1 2018-10-02 05:14:05.054+0000: 16207: debug : virFileClose:110 : Closed fd 6 2018-10-02 05:14:05.054+0000: 16207: debug : virConfGetValueString:897 : Get value string (nil) 0 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1013 : no name, allowing driver auto-select 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 0 (Test) ... 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1071 : driver 0 Test returned DECLINED 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 1 (ESX) ... 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1071 : driver 1 ESX returned DECLINED 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 2 (remote) ... 2018-10-02 05:14:05.054+0000: 16207: debug : remoteConnectOpen:1350 : Auto-probe remote URI 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:916 : proceeding with name = 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:925 : Connecting with transport 1 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:1060 : Proceeding with sockname /var/run/libvirt/libvirt-sock 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNewConnectUNIX:641 : path=/var/run/libvirt/libvirt-sock spawnDaemon=0 binary=<null> 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNewConnectUNIX:705 : connect() succeeded 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNew:237 : localAddr=0x7ffe5b15dd80 remoteAddr=0x7ffe5b15dcf0 fd=6 errfd=-1 pid=0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83aa20 classname=virNetSocket 2018-10-02 05:14:05.054+0000: 16207: info : virNetSocketNew:293 : RPC_SOCKET_NEW: sock=0x5559fa83aa20 fd=6 errfd=-1 pid=0 localAddr=127.0.0.1;0, remoteAddr=127.0.0.1;0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83ae20 classname=virNetClient 2018-10-02 05:14:05.055+0000: 16207: info : virNetClientNew:328 : RPC_CLIENT_NEW: client=0x5559fa83ae20 sock=0x5559fa83aa20 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83aa20 2018-10-02 05:14:05.055+0000: 16207: debug : virEventPollInterruptLocked:726 : Interrupting 2018-10-02 05:14:05.055+0000: 16207: info : virEventPollAddHandle:140 : EVENT_POLL_ADD_HANDLE: watch=2 fd=6 events=1 cb=0x7f70c19b4df0 opaque=0x5559fa83aa20 ff=0x7f70c19b5320 2018-10-02 05:14:05.055+0000: 16207: debug : virKeepAliveNew:196 : client=0x5559fa83ae20, interval=-1, count=0 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b120 classname=virKeepAlive 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollRunOnce:650 : Poll got 1 event(s) 2018-10-02 05:14:05.055+0000: 16207: info : virKeepAliveNew:215 : RPC_KEEPALIVE_NEW: ka=0x5559fa83b120 client=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchTimeouts:432 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:478 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b340 classname=virConnectCloseCallbackData 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:492 : i=0 w=1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83b340 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=1 events=1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b040 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83ac00 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83a9b0 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83b040 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ac00 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83a9b0 2018-10-02 05:14:05.055+0000: 16207: debug : doRemoteOpen:1179 : Trying authentication 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=1 w=2, f=6 e=1 d=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetMessageNew:46 : msg=0x5559fa83b720 tracked=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.055+0000: 16207: debug : virNetMessageEncodePayload:389 : Encode length as 28 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=2 timeout=-1 2018-10-02 05:14:05.055+0000: 16207: info : virNetClientSendInternal:2121 : RPC_CLIENT_MSG_TX_QUEUE: client=0x5559fa83ae20 len=28 prog=536903814 vers=1 proc=66 type=0 status=0 serial=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientCallNew:2074 : New call 0x5559fa83b6a0: msg=0x5559fa83b720, expectReply=1, nonBlock=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientIO:1880 : Outgoing message prog=536903814 version=1 serial=0 proc=66 type=0 length=28 dispatch=(nil) 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientIO:1939 : We have the buck head=0x5559fa83b6a0 call=0x5559fa83b6a0 2018-10-02 05:14:05.055+0000: 16207: info : virEventPollUpdateHandle:152 : EVENT_POLL_UPDATE_HANDLE: watch=2 events=0 2018-10-02 05:14:05.055+0000: 16207: debug : virEventPollInterruptLocked:726 : Interrupting 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollRunOnce:650 : Poll got 1 event(s) 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchTimeouts:432 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:478 : Dispatch 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:492 : i=0 w=1 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=1 events=1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=1 w=2, f=6 e=0 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=1 timeout=-1 And here's some information from gbd when its in this state: [root@host ~]# ps axf | grep libvirtd 17010 pts/2 S+ 0:00 \_ grep libvirtd 15555 ? Sl 0:00 libvirtd --daemon [root@host ~]# gdb -p $(pgrep libvirtd) (gdb) t a a bt Thread 17 (Thread 0x2ae648a10700 (LWP 15559)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x2ae648c11700 (LWP 15560)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x2ae648e12700 (LWP 15561)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x2ae649013700 (LWP 15562)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 ---Type <return> to continue, or q <return> to quit--- #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x2ae649214700 (LWP 15563)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x2ae649415700 (LWP 15564)): ---Type <return> to continue, or q <return> to quit--- #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x2ae649616700 (LWP 15565)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 10 (Thread 0x2ae649817700 (LWP 15566)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x2ae649a18700 (LWP 15567)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x2ae649c19700 (LWP 15568)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x2ae64d0aa700 (LWP 15588)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 ---Type <return> to continue, or q <return> to quit--- #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x2ae64d2ab700 (LWP 15589)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x2ae64d4ac700 (LWP 15590)): ---Type <return> to continue, or q <return> to quit--- #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x2ae64d6ad700 (LWP 15591)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 3 (Thread 0x2ae64d8ae700 (LWP 15592)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x2ae64dec8700 (LWP 15632)): #0 0x00002ae63d97ed0d in recvmsg () from /lib64/libpthread.so.0 #1 0x00002ae63e9caef0 in udev_monitor_receive_device () from /lib64/libudev.so.0 #2 0x00002ae64be26864 in udevEventHandleThread (opaque=<value optimized out>) at node_device/node_device_udev.c:1623 #3 0x00002ae63b2c0075 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2ae6429759e0 (LWP 15555)): #0 0x00002ae63d97e334 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00002ae63d9795d8 in _L_lock_854 () from /lib64/libpthread.so.0 #2 0x00002ae63d9794a7 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00002ae64be2364b in udevEventHandleCallback (watch=<value optimized out>, fd=19, events=<value optimized out>, data=<value optimized out>) at node_device/node_device_udev.c:1666 #4 0x00002ae63b257564 in virEventPollDispatchHandles () at util/vireventpoll.c:508 #5 virEventPollRunOnce () at util/vireventpoll.c:657 #6 0x00002ae63b255eb0 in virEventRunDefaultImpl () at util/virevent.c:327 #7 0x00002ae63b3a338d in virNetDaemonRun (dmn=0x55b1d39818d0) at rpc/virnetdaemon.c:858 #8 0x000055b1d2a4d60b in main (argc=<value optimized out>, argv=<value optimized out>) at remote/remote_daemon.c:1475 (gdb) This issue I know is fixed in 4.7.0 because I run this version on my CentOS 7 machines and don't have this issue. However on my CentOS 6 machines I have 4.2.0, and do experience this issue. Upgrading to 4.7.0 on CentOS 6 is troublesome though, because of the requirement of gnutls >= 3.2. I'm hoping with the above output someone might be able to point me in the right direction to a commit that resolved this between 4.2.0 and 4.7.0, or a possible way to prevent this hang in 4.2.0 as described above ? Thanks in advance

1 1

Re: [libvirt-users] [virt-tools-list] Virt-manager: Restricted networks
by Cole Robinson 02 Oct '18

02 Oct '18

On 10/02/2018 08:50 AM, Olivier Léobal wrote: > Hello, > > 'Isolated' networks created in VMM (I’m running 1.4.3) still connect to > the host (as gateway). Is it possible to avoid this? It is my > understanding that QEMU provides a 'restrict' option for that, but I > don't understand it well, and can’t find it in VMM. > ccing libvirt-users That's expected of isolated mode, according to this: https://wiki.libvirt.org/page/VirtualNetworking#Isolated_mode I'm not sure if/how to go about creating a network that only VMs can communicate on - Cole

2 1

[libvirt-users] This QEMU doesn't support the LSI 53C895A SCSI controller
by Gionatan Danti 02 Oct '18

02 Oct '18

Hi all, trying to edit a domain xml to enable an LSI SCSI controller I get the following error: error: unsupported configuration: This QEMU doesn't support the LSI 53C895A SCSI controller It is my understanding that the error is raised due to RedHat disabling this controller in its own qemu-kvm builds. This seems an unfortunate decision, as it makes harder to migrate from VMWare (which uses LSI SCSI and SAS adapters) to RH+KVM. Can anyone elaborate on this decision? It is possible to enable LSI controller support in RedHat 7.x? Should I open a BZ against it? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 3

[libvirt-users] libvirt reported capabilities doesn't match /proc/cpuinfo while the model does match
by Braiam Peguero 01 Oct '18

01 Oct '18

Hi, According to virsh capabilities I only have the following cpu features: <cpu> <arch>x86_64</arch> <model>IvyBridge-IBRS</model> <vendor>Intel</vendor> <microcode version='32'/> <topology sockets='1' cores='4' threads='1'/> <feature name='ds'/> <feature name='acpi'/> <feature name='ss'/> <feature name='ht'/> <feature name='tm'/> <feature name='pbe'/> <feature name='dtes64'/> <feature name='monitor'/> <feature name='ds_cpl'/> <feature name='vmx'/> <feature name='smx'/> <feature name='est'/> <feature name='tm2'/> <feature name='xtpr'/> <feature name='pdcm'/> <feature name='pcid'/> <feature name='osxsave'/> <feature name='arat'/> <feature name='ssbd'/> <feature name='xsaveopt'/> <feature name='invtsc'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> </cpu> Meanwhile, cpuinfo reports the following: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf This results on my cpu being detected, if I allow host copy, as AMD chip and the guest becomes unbearably slow. The model of the host cpu is correct. I'm using Debian testing/unstable. Compiled against library: libvirt 4.7.0 Using library: libvirt 4.7.0 Using API: QEMU 4.7.0 Running hypervisor: QEMU 2.12.0 -- Braiam

2 1

[libvirt-users] Libvirt TLS with Short Lived Certificates
by Charles Urquiola 01 Oct '18

01 Oct '18

I want to use short lived certificates with libvirtd to provided TLS access to the daemon. New certificates are generated on a daily basis and delivered to the host. Does libvirtd re-read TLS certificates with a reload of the service, systemctl reload libvirtd, or with a SIGHUP or is a full restart of the daemon required? --charlie

2 1

[libvirt-users] NUMA issues on virtualized hosts
by Lukas Hejtmanek 20 Sep '18

20 Sep '18

Hello, I have cluster with AMD EPYC 7351 cpu. Two CPUs per node. I have performance 8-NUMA configuration: This is from hypervizor: [root@hde10 ~]# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Thread(s) per core: 2 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 8 Vendor ID: AuthenticAMD CPU family: 23 Model: 1 Model name: AMD EPYC 7351 16-Core Processor Stepping: 2 CPU MHz: 1800.000 CPU max MHz: 2400.0000 CPU min MHz: 1200.0000 BogoMIPS: 4800.05 Virtualization: AMD-V L1d cache: 32K L1i cache: 64K L2 cache: 512K L3 cache: 8192K NUMA node0 CPU(s): 0-3,32-35 NUMA node1 CPU(s): 4-7,36-39 NUMA node2 CPU(s): 8-11,40-43 NUMA node3 CPU(s): 12-15,44-47 NUMA node4 CPU(s): 16-19,48-51 NUMA node5 CPU(s): 20-23,52-55 NUMA node6 CPU(s): 24-27,56-59 NUMA node7 CPU(s): 28-31,60-63 I'm running one big virtual on this hypervizor - almost whole memory + all physical CPUs. This is what I'm seeing inside: root@zenon10:~# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 8 NUMA node(s): 8 Vendor ID: AuthenticAMD CPU family: 23 Model: 1 Model name: AMD EPYC 7351 16-Core Processor Stepping: 2 CPU MHz: 2400.000 BogoMIPS: 4800.00 Virtualization: AMD-V Hypervisor vendor: KVM Virtualization type: full L1d cache: 64K L1i cache: 64K L2 cache: 512K NUMA node0 CPU(s): 0-3 NUMA node1 CPU(s): 4-7 NUMA node2 CPU(s): 8-11 NUMA node3 CPU(s): 12-15 NUMA node4 CPU(s): 16-19 NUMA node5 CPU(s): 20-23 NUMA node6 CPU(s): 24-27 NUMA node7 CPU(s): 28-31 This is virtual node configuration: (i tried different numatune settings but it was still the same) <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>one-55782</name> <vcpu><![CDATA[32]]></vcpu> <cputune> <shares>32768</shares> </cputune> <memory>507904000</memory> <os> <type arch='x86_64'>hvm</type> </os> <devices> <emulator><![CDATA[/usr/bin/kvm]]></emulator> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.0'/> <target dev='vda'/> <driver name='qemu' type='qcow2' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.1'/> <target dev='vdc'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.2'/> <target dev='vdd'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.3'/> <target dev='vde'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='cdrom'> <source file='/opt/opennebula/var/datastores/108/55782/disk.4'/> <target dev='vdb'/> <readonly/> <driver name='qemu' type='raw'/> </disk> <interface type='bridge'> <source bridge='br0'/> <mac address='02:00:93:fb:3b:78'/> <target dev='one-55782-0'/> <model type='virtio'/> <filterref filter='no-arp-mac-spoofing'> <parameter name='IP' value='147.251.59.120'/> </filterref> </interface> </devices> <features> <pae/> <acpi/> </features>  <cpu mode='host-passthrough'><topology sockets='8' cores='4' threads='1'/><numa><cell cpus='0-3' memory='62000000' /><cell cpus='4-7' memory='62000000' /><cell cpus='8-11' memory='62000000' /><cell cpus='12-15' memory='62000000' /><cell cpus='16-19' memory='62000000' /><cell cpus='20-23' memory='62000000' /><cell cpus='24-27' memory='62000000' /><cell cpus='28-31' memory='62000000' /></numa></cpu> <cputune><vcpupin vcpu='0' cpuset='0' /><vcpupin vcpu='1' cpuset='2' /><vcpupin vcpu='2' cpuset='4' /><vcpupin vcpu='3' cpuset='6' /><vcpupin vcpu='4' cpuset='8' /><vcpupin vcpu='5' cpuset='10' /><vcpupin vcpu='6' cpuset='12' /><vcpupin vcpu='7' cpuset='14' /><vcpupin vcpu='8' cpuset='16' /><vcpupin vcpu='9' cpuset='18' /><vcpupin vcpu='10' cpuset='20' /><vcpupin vcpu='11' cpuset='22' /><vcpupin vcpu='12' cpuset='24' /><vcpupin vcpu='13' cpuset='26' /><vcpupin vcpu='14' cpuset='28' /><vcpupin vcpu='15' cpuset='30' /><vcpupin vcpu='16' cpuset='1' /><vcpupin vcpu='17' cpuset='3' /><vcpupin vcpu='18' cpuset='5' /><vcpupin vcpu='19' cpuset='7' /><vcpupin vcpu='20' cpuset='9' /><vcpupin vcpu='21' cpuset='11' /><vcpupin vcpu='22' cpuset='13' /><vcpupin vcpu='23' cpuset='15' /><vcpupin vcpu='24' cpuset='17' /><vcpupin vcpu='25' cpuset='19' /><vcpupin vcpu='26' cpuset='21' /><vcpupin vcpu='27' cpuset='23' /><vcpupin vcpu='28' cpuset='25' /><vcpupin vcpu='29' cpuset='27' /><vcpupin vcpu='30' cpuset='29' /><vcpupin vcpu='31' cpuset='31' /></cputune> <numatune><memory mode='preferred' nodeset='0'/></numatune>) <devices><serial type='pty'><target port='0'/></serial><console type='pty'><target type='serial' port='0'/></console><channel type='pty'><target type='virtio' name='org.qemu.guest_agent.0'/></channel></devices> <devices><hostdev mode='subsystem' type='pci' managed='yes'><source><address domain='0x0' bus='0x11' slot='0x0' function='0x1'/></source></hostdev></devices> <devices><controller type='pci' index='1' model='pci-bridge'/><controller type='pci' index='2' model='pci-bridge'/><controller type='pci' index='3' model='pci-bridge'/><controller type='pci' index='4' model='pci-bridge'/><controller type='pci' index='5' model='pci-bridge'/></devices> <metadata> <system_datastore><![CDATA[/opt/opennebula/var/datastores/108/55782]]> </system_datastore> </metadata> </domain> If I run e.g., spec2017 on the virtual, I can see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1350 root 20 0 843136 830068 2524 R 78.1 0.2 513:16.16 bwaves_r_base.m 2456 root 20 0 804608 791264 2524 R 76.6 0.2 491:39.92 bwaves_r_base.m 4631 root 20 0 843136 829892 2344 R 75.8 0.2 450:16.04 bwaves_r_base.m 6441 root 20 0 802580 790212 2532 R 75.0 0.2 120:37.54 bwaves_r_base.m 7991 root 20 0 784676 772092 2576 R 75.0 0.2 387:15.39 bwaves_r_base.m 8142 root 20 0 843136 830044 2496 R 75.0 0.2 384:39.02 bwaves_r_base.m 8234 root 20 0 843136 830064 2524 R 75.0 0.2 99:04.48 bwaves_r_base.m 8578 root 20 0 749240 736604 2468 R 73.4 0.2 375:45.66 bwaves_r_base.m 9974 root 20 0 784676 771984 2468 R 73.4 0.2 348:01.36 bwaves_r_base.m 10396 root 20 0 802580 790264 2576 R 73.4 0.2 340:08.40 bwaves_r_base.m 12932 root 20 0 843136 830024 2480 R 73.4 0.2 288:39.76 bwaves_r_base.m 13113 root 20 0 784676 771864 2348 R 71.9 0.2 284:47.34 bwaves_r_base.m 13518 root 20 0 784676 762816 2540 R 71.9 0.2 276:31.58 bwaves_r_base.m 14443 root 20 0 784676 771984 2468 R 71.9 0.2 260:01.82 bwaves_r_base.m 12791 root 20 0 784676 772060 2544 R 70.3 0.2 291:43.96 bwaves_r_base.m 10544 root 20 0 843136 830068 2520 R 68.8 0.2 336:47.43 bwaves_r_base.m 15464 root 20 0 784676 762880 2608 R 60.9 0.2 239:19.14 bwaves_r_base.m 15487 root 20 0 784676 772048 2532 R 60.2 0.2 238:37.07 bwaves_r_base.m 16824 root 20 0 784676 772120 2604 R 55.5 0.2 212:10.92 bwaves_r_base.m 17255 root 20 0 843136 830012 2468 R 54.7 0.2 203:22.89 bwaves_r_base.m 17962 root 20 0 784676 772004 2488 R 54.7 0.2 188:26.07 bwaves_r_base.m 17505 root 20 0 843136 830068 2520 R 53.1 0.2 198:04.25 bwaves_r_base.m 27767 root 20 0 784676 771860 2344 R 52.3 0.2 592:25.95 bwaves_r_base.m 24458 root 20 0 843136 829888 2344 R 50.8 0.2 658:23.70 bwaves_r_base.m 30746 root 20 0 747376 735160 2604 R 43.0 0.2 556:47.67 bwaves_r_base.m The CPU TIME should be roughly the same but huge differences are obvious. This is what I see on the hypervizor: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18201 oneadmin 20 0 474.0g 473.3g 1732 S 2459 94.0 33332:54 kvm 369 root 20 0 0 0 0 R 100.0 0.0 768:12.85 kswapd1 368 root 20 0 0 0 0 R 94.1 0.0 869:05.61 kswapd0 i.e., kswapd is eating whole CPU. Swap is turned off. [root@hde10 ~]# free total used free shared buff/cache available Mem: 528151432 503432580 1214048 34740 23504804 21907800 Swap: 0 0 0 Hypervisor is [root@hde10 ~]# cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core) qemu-kvm-1.5.3-156.el7_5.5.x86_64 Virtual is Debian 9. Moreover, I'm using this type of disks for virtuals: <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.3'/> <target dev='vde'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> If I keep cache='unsafe' and if I run iozone test on really big files (e.g., 8x 100GB), I can see huge cache pressure on the hypervizor - all 8 kswapd are running on 100 % percent and slowing things down. The disk under datastore is NVME SSD Intel 4500. If I set cache='none', kswaps are on idle, disk writes are pretty fast, however, with 8-NUMA configuration, writes slow down to less than 10MB/s as soon as the size of written data is roughly the same as memory size in the virtual node. iozone has 100 % CPU usage thereafter and it seems that it is traversing page lists. If I do the same with 1-NUMA configuration, everything is ok except performance penalty about 25 %. -- Lukáš Hejtmánek Linux Administrator only because Full Time Multitasking Ninja is not an official job title

2 7