- Users - libvirt List Archives

[libvirt-users] Hello World
by Ameya Sinha 05 Nov '18

05 Nov '18

Hello Everyone! I'm Ameya Sinha, a student, and I wanted to contribute to the libvirt organization. I have read through the contributing guidelines ( https://libvirt.org/contribute.html) and contributor guidelines ( https://libvirt.org/hacking.html) Can someone suggest how should I proceed from here? Thank You, Ameya Sinha

3 2

[libvirt-users] Windows guest skipping chkdsk if using Spice display
by Gionatan Danti 05 Nov '18

05 Nov '18

Hi all, as per subject, when a Windows guest using a Spice display is rebooted with a planned chkdsk, the consistency check is always skipped. It seems that the Spice display issues some "phantom" keystrokes, which are then intended by Windows as the "skip chkdsk at boot" signal. I tried to use a Spice display with no additional features (ie: USB redirection), but to no avail. To workaround the problem I must use a VNC display, which is less featureful than the Spice one. My system and package versions are: CentOS 7.5 x86-64 spice-gtk3-0.34-3.el7_5.1.x86_64 spice-server-0.14.0-2.el7_5.4.x86_64 spice-glib-0.34-3.el7_5.1.x86_64 libvirt-3.9.0-14.el7_5.7.x86_64 qemu-kvm-ev-2.10.0-21.el7_5.4.1.x86_64 It seems I am not the only one experiencing this problem: http://seifesrants.blogspot.com/2016/07/ghost-keystrokes-with-libvirt-kvm-s… Any suggestion? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 3

[libvirt-users] How to change the default connection of 'virsh' tool from 'qemu' to 'xen' ?
by Minjun Hong 02 Nov '18

02 Nov '18

Hi. Thanks to help of this mailing list (especially Jim Fehlig), I have finished setup of libvirt. However, there is something weird. It is that I installed libvirt on 5 servers but the default connection of one of them is 'qemu:///' even if those of the others are all 'xen:///'. I searched how to change it but, I only found a bypass adding " LIBVIRT_DEFAULT_URI='xen:///' " in my .bashrc file. Currently, value of the variable (LIBVIRT_DEFAULT_URI) is set to 'qemu:///'. Since the bypass makes the virtualization type of the server visible 'qemu' to the open stack, I have to change libvirt configuration. I also checked some configure files under '/etc/libvirt', such as libvirt.conf, libvirtd.conf and libxl.conf. But I have no idea, how and what to change. Please give me some advise. It will be a big help for me. Thanks.

4 5

[libvirt-users] Read CPU Performance Statistics
by Hana Yehia 01 Nov '18

01 Nov '18

Hi, I am in need for a help for how to use libvirt in python to read the the guest VMs performance statistics like cache hit and miss rates , I have KVM- Qemu as a virtual machine manager , I am looking for a sample code or the instructions needed to help me read these statistics . Thanks, hana

1 0

[libvirt-users] Xen via libvirt
by Minjun Hong 31 Oct '18

31 Oct '18

Hello. I want to use libvirt for toolstack of Xen (not XenServer) to set up Openstack system but, I'm new to libvirt so, I had some troubles on installing libvirt. I downloaded the source code, build and installed it, following here ( https://libvirt.org/compiling.html) However, when I type a command (# virsh 'xen:///system' list) to check if libvirt has been installed successfully, I can see an error message: error: failed to connect to the hypervisor > error: internal error: libxenlight state driver is not active Of course, I confirmed that Xen was working well: root@node1:/home/nickeys# xl list > Name ID Mem VCPUs State > Time(s) > Domain-0 0 95656 32 r----- > 5888.0 And I'm wondering why 'Using API' below is QEMU. How can I change that to Xen ? root@node1: /home/nickeys# virsh version > Compiled against library: libvirt 4.9.0 > Using library: libvirt 4.9.0 > Using API: QEMU 4.9.0 > Running hypervisor: QEMU 2.11.1 I'm using libvirt 4.9, Xen 4.12 (the current latest) and Linux kernel 4.16.1. I want to know how to make the 'libxenlight state driver' active. Please let me know where I should check first to fix this wrong state, or any tip is also okay. Thanks.

2 4

[libvirt-users] how to use LV(container) created by lxc-create?
by lejeczek 20 Oct '18

20 Oct '18

hi everyone, how can I use a LV created by lxc in libvirt guest domain? You know, you lxc-create with LV as backstore for a container(ubuntu in my case) and now you want to give to libvit. That, I thought would be a very common case but I failed to find a good howto on how one: lxc-create(lvm) => let libvirt take control over it. Or it's not how you would do it at all? many thanks, L.

2 1

[libvirt-users] LXC with --filesystem - control/impose filesystem(/ in container) size limits - how?
by lejeczek 19 Oct '18

19 Oct '18

hi guys container was created off a template with --dir. Is it even possible? I mean, as it is with filesystem, without moving/migrating to LVM, ZFS, etc? The very brute solution which comes to my mind - xfs project quotas - if it even could work. But that outside of libvirt/lxc. many thanks, L.

1 0

[libvirt-users] KVM + libvirt + nftables without iptables?
by Roman Vesely 19 Oct '18

19 Oct '18

Hi everyone, I use Debian 9.5 Stretch and NFTABLES as a firewall. Using NFTABLES together with IPTABLES is not recommended, but libvirt depends on IPTABLES. Is it safe to run libvirt + kvm + virsh without IPTABLES? By the doc https://libvirt.org/firewall.html, IPTABLES are used for settingup filtering which I do not need. Thanks, Roman

3 4

[libvirt-users] pcie-expander-bus doesn't support pcie-pci-bridge and pcie-switch-upstream-port
by Han Han 18 Oct '18

18 Oct '18

In libvirt, I found pcie-expander-bus controller doesn't support pcie-to-pci-bridge and pcie-switch-upstream-port. Version: libvirt-4.9 # cat /tmp/c.xml ... <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> ... # virsh -k0 -K0 define /tmp/c.xml error: Failed to define domain from /tmp/c.xml error: XML error: The device at PCI address 0000:01:00.0 cannot be plugged into the PCI controller with index='1'. It requires a controller that accepts a pcie-to-pci-bridge. # cat /tmp/b.xml ... <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='3'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </controller> <controller type='pci' index='2' model='pcie-switch-upstream-port'> <model name='x3130-upstream'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> ... # virsh -k0 -K0 define /tmp/b.xml error: Failed to define domain from /tmp/b.xml error: XML error: The device at PCI address 0000:01:00.0 cannot be plugged into the PCI controller with index='1'. It requires a controller that accepts a pci-switch-upstream-port. In function virDomainPCIAddressBusSetModel, I find pcie-expander-bus only supports pcie-root-port and dmi-to-pci-bridge 353 case VIR_DOMAIN_CONTROLLER_MODEL_PCIE_EXPANDER_BUS: 354 ┆ /* 32 slots, no hotplug, only accepts pcie-root-port or 355 ┆ ┆* dmi-to-pci-bridge 356 ┆ ┆*/ 357 ┆ bus->flags = (VIR_PCI_CONNECT_TYPE_PCIE_ROOT_PORT | 358 ┆ ┆ ┆ ┆ ┆ VIR_PCI_CONNECT_TYPE_DMI_TO_PCI_BRIDGE); 359 ┆ bus->minSlot = 0; 360 ┆ bus->maxSlot = VIR_PCI_ADDRESS_SLOT_LAST; 361 ┆ break; But it works in qemu: # /usr/libexec/qemu-kvm -machine q35 -m 1024 -device pxb-pcie,bus_nr=250,id=pci.1,bus=pcie.0,addr=0x4 -device pcie-pci-bridge,id=pci.250,bus=pci.1,addr=0x0 -device pcie-pci-bridge,id=pci.251,bus=pci.1,addr=0x1 -spice port=5902,addr=0.0.0.0,disable-ticketing /var/lib/libvirt/images/q35.qcow2 So why pcie-expander-bus doesn't these two controllers? For any concerns in libvirt? Thank you~ -- Best regards, ----------------------------------- Han Han Quality Engineer Redhat. Email: hhan(a)redhat.com Phone: +861065339333

4 4

[libvirt-users] Serial ports: vm vs bare metal
by Lars Kellogg-Stedman 16 Oct '18

16 Oct '18

I'm running libvirt under Fedora 28. I would like to attach a USB device to a VM, but when I select "Redirect USB Device" from the "Virtual Machine" menu in virt-manager and then select the device, I get the error: USB redirection error spice-client-error-quark: Could not redirect [device name] at 1-11: Error setting USB device node ACL: 'Not authorized' (0) There are no additional log messages acccompanying this in either the journal or in audit.log. It's not a selinux issue. The "Details" dropdown in the error dialog helpfully adds "USB redirection error". What do I do with this error message? I'm not even sure what it's trying to tell me. Thanks, -- Lars Kellogg-Stedman <lars(a)redhat.com> | larsks @ {irc,twitter,github} http://blog.oddbit.com/ |

3 4

[libvirt-users] snapshots with virsh in a pacemaker cluster
by Lentes, Bernd 15 Oct '18

15 Oct '18

Hi, i have a two node cluster with virtual guests as resources. I'd like to snapshot the guests once in the night and thought i had a procedure. But i realize that things in a cluster are a bit more complicated than expected :-)) I will shutdown the guests to have a clean snapshot. I can shutdown the guests via pacemaker. But then arises the first problem: When i issue a "virsh snapshot-create-as" libvirt needs the domain name as a parameter. but libvirt does not know the domains any longer. When the guests are shutdown a "virsh list --all" on both nodes does not show any domain. A look in the respective resource agent VirtualDomain explains why: The domain is started with virsh create: "# The 'create' command guarantees that the domain will be # undefined on shutdown, ... OK: Now i could of course define all domains with a virsh define. But then i have immediately the next problem. Now i'd create the snapshots with "virsh snapshot-create-as" and starts the domains afterwards via cluster. But let's assume i issue that on node 1 and some guests are started afterwards via pacemaker on node 2. I can't predict on which node the guests are starting. Then i don't get a snapshot, right ? What to do ? Bernd -- Bernd Lentes Systemadministration Institut für Entwicklungsgenetik Gebäude 35.34 - Raum 208 HelmholtzZentrum münchen [ mailto:bernd.lentes@helmholtz-muenchen.de | bernd.lentes(a)helmholtz-muenchen.de ] phone: +49 89 3187 1241 fax: +49 89 3187 2294 [ http://www.helmholtz-muenchen.de/idg | http://www.helmholtz-muenchen.de/idg ] wer Fehler macht kann etwas lernen wer nichts macht kann auch nichts lernen Helmholtz Zentrum Muenchen Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH) Ingolstaedter Landstr. 1 85764 Neuherberg www.helmholtz-muenchen.de Aufsichtsratsvorsitzende: NN Stellv.Aufsichtsratsvorsitzender: MinDirig. Dr. Manfred Wolter Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen Registergericht: Amtsgericht Muenchen HRB 6466 USt-IdNr: DE 129521671

3 3

[libvirt-users] Device lease hot unplug and events
by Milan Zamazal 15 Oct '18

15 Oct '18

Hi, when working on hot unplugs of various devices, I've found out that hot unplugging <lease> device doesn't generate VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED event. <lease> also doesn't have an alias, so it wouldn't be identifiable in the corresponding callback. Is this difference from other hotpluggable devices intentional? If yes, is there any better way of checking that <lease> removal is completed than querying and examining the domain XML? From user's point of view, it would be best if I could simply handle the device removal event the same way as with other devices. Thanks, Milan

2 4

[libvirt-users] How to explain this libvirt oddity w.r.t machine types?
by Kashyap Chamarthy 15 Oct '18

15 Oct '18

Context: The baremetal host previously had QEMU 2.11. But I manually downgraded the QEMU version (via `dnf downgrade qemu-system-x86`); now it is at 2.10: $ rpm -q qemu-system-x86 qemu-system-x86-2.10.2-1.fc27.x86_64 The guest is offline. Let's see (in a couple of ways) what machine type it has while it is dormant: # virsh dumpxml cirros | grep -i machine= <type arch='x86_64' machine='pc-i440fx-2.10'>hvm</type> # grep machine= /etc/libvirt/qemu/cirros.xml <type arch='x86_64' machine='pc-i440fx-2.10'>hvm</type> Okay, now edit the guest XML and carefully remove the "machine='pc-i440fx-2.10'" bit---to see what machine type will libvirt (libvirt-daemon-kvm-4.0.0-2.fc27.x86_64) default to: # virsh edit cirros Domain cirros XML configuration edited. Now check the machine type again. Bizarrely enough, libvirt "helpfully" auto-adds QEMU *2.11* machine type, which is obviously no longer on the system! # grep machine= /etc/libvirt/qemu/cirros.xml <type arch='x86_64' machine='pc-i440fx-2.11'>hvm</type> # virsh dumpxml cirros | grep -i machine= <type arch='x86_64' machine='pc-i440fx-2.11'>hvm</type> How to explain this? Is this even a "valid test"? (To undo the nuisance, obviously, I had to `virsh edit cirros` again and change it to 2.10.) Note, I *don't* have 2.11 QEMU on the system: # rpm -qa | grep -E 'qemu.*2.11' # echo $? 1 * * * It's getting late, and I should stop staring at screens. -- /kashyap

2 5

[libvirt-users] live migration via unix socket
by David Vossel 12 Oct '18

12 Oct '18

Hey, Over in KubeVirt we're investigating a use case where we'd like to perform a live migration within a network namespace that does not provide libvirtd with network access. In this scenario we would like to perform a live migration by proxying the migration through a unix socket to a process in another network namespace that does have network access. That external process would live on every node in the cluster and know how to correctly route connections between libvirtds. virsh example of an attempted migration via unix socket. virsh migrate --copy-storage-all --p2p --live --xml domain.xml my-vm qemu+unix:///system?socket=destination-host-proxy-sock In this example, the src libvirtd is able to establish a connection to the destination libvirtd via the unix socket proxy. However, the migration-uri appears to require either tcp or rdma network connection. If I force the migration-uri to be a unix socket, I receive an error [1] indicating that qemu+unix is not a valid transport. Technically with qemu+kvm I believe what we're attempting should be possible (even though it is inefficient). Please correct me if I'm wrong. Is there a way to achieve this migration via unix socket functionality this using Libvirt? Also, is there a reason why the migration uri is limited to tcp/rdma Thanks! - David [1] https://github.com/libvirt/libvirt/blob/master/src/qemu/qemu_migration.c#L2…

4 7

[libvirt-users] how "safe" is blockcommit ?
by Lentes, Bernd 12 Oct '18

12 Oct '18

Hi, currently i'm following https://wiki.libvirt.org/page/Live-disk-backup-with-active-blockcommit. I 'm playing around with it and it seems to be quite nice. What i want is a daily consistent backup of my image file of the guest. I have the idea of the following procedure: - Shutdown the guest (i can live with a downtime of a few minutes, it will happen in the night). And i think it's the only way to have a real clean snapshot - create a snapshot with snapshot-create-as: snapshot-create-as guest testsn --disk-only - start the guest again. Changes will now go into the overlay, as e.g. inserts in a database - rsync the base file to a cifs server. With rsync not the complete, likely big file is transferred but just the delta - blockcommit the overlay: blockcommit guest /path/to/testsn --active --wait --verbose --pivot - delete the snapshot: snapshot-delete guest --snapshotname testsn --metadata - remove the overlay Is that ok ? How "safe" is blockcommit on a running guest ? It's possible that during the rsync, when the guest is running, some inserts are done in a database. Is it safe to copy the new sectors (i assume that's what blockcommit does) under a running database ? Or is it only safe doing blockcommit on a stopped guest ? Thanks for any answer. Bernd P.S. Is the same procedure possible when the guest disk(s) reside directly in a plain logical volume, without a file system in-between ? -- Bernd Lentes Systemadministration Institut für Entwicklungsgenetik Gebäude 35.34 - Raum 208 HelmholtzZentrum münchen [ mailto:bernd.lentes@helmholtz-muenchen.de | bernd.lentes(a)helmholtz-muenchen.de ] phone: +49 89 3187 1241 fax: +49 89 3187 2294 [ http://www.helmholtz-muenchen.de/idg | http://www.helmholtz-muenchen.de/idg ] wer Fehler macht kann etwas lernen wer nichts macht kann auch nichts lernen Helmholtz Zentrum Muenchen Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH) Ingolstaedter Landstr. 1 85764 Neuherberg www.helmholtz-muenchen.de Aufsichtsratsvorsitzende: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrer: Prof. Dr. med. Dr. h.c. Matthias H. Tschoep, Heinrich Bassler, Dr. rer. nat. Alfons Enhsen Registergericht: Amtsgericht Muenchen HRB 6466 USt-IdNr: DE 129521671

4 5

[libvirt-users] SR-IOV Trust Setting
by ]σ§έ Λιαςάη 10 Oct '18

10 Oct '18

Is there a setting to allow/disable spoof checking and trust on VF from SR-IOV device? Lookin at https://www.libvirt.org/formatdomain.html#elementsDriverBackendOptions can this be done through the driver element? *<driver name='vhost' txmode='iothread' ioeventfd='on' event_idx='off' queues='5' rx_queue_size='256' tx_queue_size='256'> <host csum='off' gso='off' tso4='off' tso6='off' ecn='off' ufo='off' mrg_rxbuf='off'/> <guest csum='off' tso4='off' tso6='off' ecn='off' ufo='off'/> </driver>* At the moment i have to manually run: ip link set $_dev vf $vf spoofchk off ip link set $_dev vf $vf trust on Is it possible to set this values from the xml file on a per interface basis?

2 1

[libvirt-users] [OCaml] Reset of the libvirt-ocaml repository
by Pino Toscano 10 Oct '18

10 Oct '18

Hi libvirt developers and users, TL;DR: you need to clone again the libvirt-ocaml repository, if you had one already Due to the way the repository was converted from another SCM in the past, some of the metadata of older commits was broken, and this made the repository unusable with more strict git configurations (e.g. as used on GitHub). You can read more details about this in https://www.redhat.com/archives/libvir-list/2018-September/msg00214.html The ocaml-libvirt repository from Rich Jones' website was fixed, and replaced the existing libvirt-ocaml on libvirt.org. The libvirt-ocaml repository is the official home now (or again) for the OCaml bindings of libvirt. This means that you need to clone again the libvirt-ocaml repository, in case you had already a clone of this repository. In case you had pending work on it, make sure to carry it over from an old clone to a newer clone. As usual, if you have patches for libvirt-ocaml, feel free to send them to libvir-list, tagging them as [ocaml]. -- Pino Toscano

1 0

[libvirt-users] how libvirt check qemu domain status?
by Dmitry Melekhov 09 Oct '18

09 Oct '18

Hello! Could you tell me how libvirtd check qemu domain status? Looks like this is something more then just looking for pid. Because we have VMs running over gluster storage using gfapi and if something goes wrong, then libvirtd became unresponsive often enough.. Thank you!

1 1

[libvirt-users] live migration and config
by Dmitry Melekhov 04 Oct '18

04 Oct '18

Hello! After some mistakes yesterday we ( me and my colleague ) think that it will be wise for libvirt to check config file existence on remote side and through error if not, before migrating, otherwise migration will fail and VM fs can be damaged, because it is sort of remove of power plug... We missed twice yesterday :-( Could you tell me is there already such option or any plans to implement this? Thank you!

2 9

[libvirt-users] Libvirt 4.2.0 hang on destination on live migration cancel
by Scott Sullivan 03 Oct '18

03 Oct '18

I'm trying the following command on the source machine: virsh migrate --live --copy-storage-all --verbose TEST qemu+ssh:// 10.30.76.66/system If I ssh into the destination machine when this command is running, I can see NBD copying data as expected, and if I wait long enough it completes and succeeds. However if I ctrl+c this command above before it completes, it causes virsh commands (like virsh list) to just hang seemingly forever on the destination machine. Here's the libvirt debug entries when I run virsh list in this state: [root@host ~]# export LIBVIRT_DEBUG=1 [root@host ~]# virsh list 2018-10-02 05:14:05.053+0000: 16207: debug : virGlobalInit:359 : register drivers 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddbb60 name=Test 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering Test as driver 0 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddc660 name=ESX 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering ESX as driver 12018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:657 : driver=0x7f70c1ddd560 name=remote 2018-10-02 05:14:05.054+0000: 16207: debug : virRegisterConnectDriver:668 : registering remote as driver 2 2018-10-02 05:14:05.054+0000: 16207: debug : virEventRegisterDefaultImpl:280 : registering default event implementation 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollAddHandle:115 : Used 0 handle slots, adding at least 10 more 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollInterruptLocked:722 : Skip interrupt, 0 0 2018-10-02 05:14:05.054+0000: 16207: info : virEventPollAddHandle:140 : EVENT_POLL_ADD_HANDLE: watch=1 fd=4 events=1 cb=0x7f70c18791c0 opaque=(nil) ff=(nil) 2018-10-02 05:14:05.054+0000: 16207: debug : virEventRegisterImpl:241 : addHandle=0x7f70c187a8a0 updateHandle=0x7f70c1879500 removeHandle=0x7f70c1879360 addTimeout=0x7f70c187a660 updateTimeout=0x7f70c1879690 removeTimeout=0x7f70c1879220 2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollAddTimeout:230 : Used 0 timeout slots, adding at least 10 more2018-10-02 05:14:05.054+0000: 16207: debug : virEventPollInterruptLocked:722 : Skip interrupt, 0 0 2018-10-02 05:14:05.054+0000: 16207: info : virEventPollAddTimeout:253 : EVENT_POLL_ADD_TIMEOUT: timer=1 frequency=-1 cb=0x5559f8dba380 opaque=0x7ffe5b15e700 ff=(nil) 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenAuth:1218 : name=<null>, auth=0x7f70c1ddea00, flags=0x0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83a3f0 classname=virConnect 2018-10-02 05:14:05.054+0000: 16208: debug : virThreadJobSet:99 : Thread 16208 is now running job vshEventLoop 2018-10-02 05:14:05.054+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.054+0000: 16207: debug : virConfLoadConfig:1576 : Loading config file '/etc/libvirt/libvirt.conf' 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 1 2018-10-02 05:14:05.054+0000: 16207: debug : virConfReadFile:752 : filename=/etc/libvirt/libvirt.conf 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.054+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.054+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=1 timeout=-1 2018-10-02 05:14:05.054+0000: 16207: debug : virFileClose:110 : Closed fd 6 2018-10-02 05:14:05.054+0000: 16207: debug : virConfGetValueString:897 : Get value string (nil) 0 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1013 : no name, allowing driver auto-select 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 0 (Test) ... 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1071 : driver 0 Test returned DECLINED 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 1 (ESX) ... 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1071 : driver 1 ESX returned DECLINED 2018-10-02 05:14:05.054+0000: 16207: debug : virConnectOpenInternal:1056 : trying driver 2 (remote) ... 2018-10-02 05:14:05.054+0000: 16207: debug : remoteConnectOpen:1350 : Auto-probe remote URI 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:916 : proceeding with name = 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:925 : Connecting with transport 1 2018-10-02 05:14:05.054+0000: 16207: debug : doRemoteOpen:1060 : Proceeding with sockname /var/run/libvirt/libvirt-sock 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNewConnectUNIX:641 : path=/var/run/libvirt/libvirt-sock spawnDaemon=0 binary=<null> 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNewConnectUNIX:705 : connect() succeeded 2018-10-02 05:14:05.054+0000: 16207: debug : virNetSocketNew:237 : localAddr=0x7ffe5b15dd80 remoteAddr=0x7ffe5b15dcf0 fd=6 errfd=-1 pid=0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83aa20 classname=virNetSocket 2018-10-02 05:14:05.054+0000: 16207: info : virNetSocketNew:293 : RPC_SOCKET_NEW: sock=0x5559fa83aa20 fd=6 errfd=-1 pid=0 localAddr=127.0.0.1;0, remoteAddr=127.0.0.1;0 2018-10-02 05:14:05.054+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83ae20 classname=virNetClient 2018-10-02 05:14:05.055+0000: 16207: info : virNetClientNew:328 : RPC_CLIENT_NEW: client=0x5559fa83ae20 sock=0x5559fa83aa20 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83aa20 2018-10-02 05:14:05.055+0000: 16207: debug : virEventPollInterruptLocked:726 : Interrupting 2018-10-02 05:14:05.055+0000: 16207: info : virEventPollAddHandle:140 : EVENT_POLL_ADD_HANDLE: watch=2 fd=6 events=1 cb=0x7f70c19b4df0 opaque=0x5559fa83aa20 ff=0x7f70c19b5320 2018-10-02 05:14:05.055+0000: 16207: debug : virKeepAliveNew:196 : client=0x5559fa83ae20, interval=-1, count=0 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b120 classname=virKeepAlive 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollRunOnce:650 : Poll got 1 event(s) 2018-10-02 05:14:05.055+0000: 16207: info : virKeepAliveNew:215 : RPC_KEEPALIVE_NEW: ka=0x5559fa83b120 client=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchTimeouts:432 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ae20 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:478 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b340 classname=virConnectCloseCallbackData 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:492 : i=0 w=1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83b340 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=1 events=1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83b040 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83ac00 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16207: info : virObjectNew:254 : OBJECT_NEW: obj=0x5559fa83a9b0 classname=virNetClientProgram 2018-10-02 05:14:05.055+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83b040 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83ac00 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16207: info : virObjectRef:388 : OBJECT_REF: obj=0x5559fa83a9b0 2018-10-02 05:14:05.055+0000: 16207: debug : doRemoteOpen:1179 : Trying authentication 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=1 w=2, f=6 e=1 d=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetMessageNew:46 : msg=0x5559fa83b720 tracked=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.055+0000: 16207: debug : virNetMessageEncodePayload:389 : Encode length as 28 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=2 timeout=-1 2018-10-02 05:14:05.055+0000: 16207: info : virNetClientSendInternal:2121 : RPC_CLIENT_MSG_TX_QUEUE: client=0x5559fa83ae20 len=28 prog=536903814 vers=1 proc=66 type=0 status=0 serial=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientCallNew:2074 : New call 0x5559fa83b6a0: msg=0x5559fa83b720, expectReply=1, nonBlock=0 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientIO:1880 : Outgoing message prog=536903814 version=1 serial=0 proc=66 type=0 length=28 dispatch=(nil) 2018-10-02 05:14:05.055+0000: 16207: debug : virNetClientIO:1939 : We have the buck head=0x5559fa83b6a0 call=0x5559fa83b6a0 2018-10-02 05:14:05.055+0000: 16207: info : virEventPollUpdateHandle:152 : EVENT_POLL_UPDATE_HANDLE: watch=2 events=0 2018-10-02 05:14:05.055+0000: 16207: debug : virEventPollInterruptLocked:726 : Interrupting 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollRunOnce:650 : Poll got 1 event(s) 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchTimeouts:432 : Dispatch 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:478 : Dispatch 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollDispatchHandles:492 : i=0 w=1 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=1 events=1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventRunDefaultImpl:324 : running default event implementation 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupTimeouts:525 : Cleanup 1 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCleanupHandles:574 : Cleanup 2 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=0 w=1, f=4 e=1 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollMakePollFDs:401 : Prepare n=1 w=2, f=6 e=0 d=0 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 1 timers 2018-10-02 05:14:05.055+0000: 16208: debug : virEventPollCalculateTimeout:371 : No timeout is pending 2018-10-02 05:14:05.055+0000: 16208: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=1 timeout=-1 And here's some information from gbd when its in this state: [root@host ~]# ps axf | grep libvirtd 17010 pts/2 S+ 0:00 \_ grep libvirtd 15555 ? Sl 0:00 libvirtd --daemon [root@host ~]# gdb -p $(pgrep libvirtd) (gdb) t a a bt Thread 17 (Thread 0x2ae648a10700 (LWP 15559)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x2ae648c11700 (LWP 15560)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x2ae648e12700 (LWP 15561)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x2ae649013700 (LWP 15562)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 ---Type <return> to continue, or q <return> to quit--- #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x2ae649214700 (LWP 15563)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x2ae649415700 (LWP 15564)): ---Type <return> to continue, or q <return> to quit--- #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x2ae649616700 (LWP 15565)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 10 (Thread 0x2ae649817700 (LWP 15566)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x2ae649a18700 (LWP 15567)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x2ae649c19700 (LWP 15568)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13a8 in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x2ae64d0aa700 (LWP 15588)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 ---Type <return> to continue, or q <return> to quit--- #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x2ae64d2ab700 (LWP 15589)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x2ae64d4ac700 (LWP 15590)): ---Type <return> to continue, or q <return> to quit--- #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x2ae64d6ad700 (LWP 15591)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 3 (Thread 0x2ae64d8ae700 (LWP 15592)): #0 0x00002ae63d97b68c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00002ae63b2c01ba in virCondWait (c=<value optimized out>, m=<value optimized out>) at util/virthread.c:154 #2 0x00002ae63b2c13fb in virThreadPoolWorker (opaque=<value optimized out>) at util/virthreadpool.c:124 #3 0x00002ae63b2c0036 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x2ae64dec8700 (LWP 15632)): #0 0x00002ae63d97ed0d in recvmsg () from /lib64/libpthread.so.0 #1 0x00002ae63e9caef0 in udev_monitor_receive_device () from /lib64/libudev.so.0 #2 0x00002ae64be26864 in udevEventHandleThread (opaque=<value optimized out>) at node_device/node_device_udev.c:1623 #3 0x00002ae63b2c0075 in virThreadHelper (data=<value optimized out>) at util/virthread.c:206 ---Type <return> to continue, or q <return> to quit--- #4 0x00002ae63d977aa1 in start_thread () from /lib64/libpthread.so.0 #5 0x00002ae63e4d893d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x2ae6429759e0 (LWP 15555)): #0 0x00002ae63d97e334 in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00002ae63d9795d8 in _L_lock_854 () from /lib64/libpthread.so.0 #2 0x00002ae63d9794a7 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00002ae64be2364b in udevEventHandleCallback (watch=<value optimized out>, fd=19, events=<value optimized out>, data=<value optimized out>) at node_device/node_device_udev.c:1666 #4 0x00002ae63b257564 in virEventPollDispatchHandles () at util/vireventpoll.c:508 #5 virEventPollRunOnce () at util/vireventpoll.c:657 #6 0x00002ae63b255eb0 in virEventRunDefaultImpl () at util/virevent.c:327 #7 0x00002ae63b3a338d in virNetDaemonRun (dmn=0x55b1d39818d0) at rpc/virnetdaemon.c:858 #8 0x000055b1d2a4d60b in main (argc=<value optimized out>, argv=<value optimized out>) at remote/remote_daemon.c:1475 (gdb) This issue I know is fixed in 4.7.0 because I run this version on my CentOS 7 machines and don't have this issue. However on my CentOS 6 machines I have 4.2.0, and do experience this issue. Upgrading to 4.7.0 on CentOS 6 is troublesome though, because of the requirement of gnutls >= 3.2. I'm hoping with the above output someone might be able to point me in the right direction to a commit that resolved this between 4.2.0 and 4.7.0, or a possible way to prevent this hang in 4.2.0 as described above ? Thanks in advance

1 1

Re: [libvirt-users] [virt-tools-list] Virt-manager: Restricted networks
by Cole Robinson 02 Oct '18

02 Oct '18

On 10/02/2018 08:50 AM, Olivier Léobal wrote: > Hello, > > 'Isolated' networks created in VMM (I’m running 1.4.3) still connect to > the host (as gateway). Is it possible to avoid this? It is my > understanding that QEMU provides a 'restrict' option for that, but I > don't understand it well, and can’t find it in VMM. > ccing libvirt-users That's expected of isolated mode, according to this: https://wiki.libvirt.org/page/VirtualNetworking#Isolated_mode I'm not sure if/how to go about creating a network that only VMs can communicate on - Cole

2 1

[libvirt-users] This QEMU doesn't support the LSI 53C895A SCSI controller
by Gionatan Danti 02 Oct '18

02 Oct '18

Hi all, trying to edit a domain xml to enable an LSI SCSI controller I get the following error: error: unsupported configuration: This QEMU doesn't support the LSI 53C895A SCSI controller It is my understanding that the error is raised due to RedHat disabling this controller in its own qemu-kvm builds. This seems an unfortunate decision, as it makes harder to migrate from VMWare (which uses LSI SCSI and SAS adapters) to RH+KVM. Can anyone elaborate on this decision? It is possible to enable LSI controller support in RedHat 7.x? Should I open a BZ against it? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 3

[libvirt-users] libvirt reported capabilities doesn't match /proc/cpuinfo while the model does match
by Braiam Peguero 01 Oct '18

01 Oct '18

Hi, According to virsh capabilities I only have the following cpu features: <cpu> <arch>x86_64</arch> <model>IvyBridge-IBRS</model> <vendor>Intel</vendor> <microcode version='32'/> <topology sockets='1' cores='4' threads='1'/> <feature name='ds'/> <feature name='acpi'/> <feature name='ss'/> <feature name='ht'/> <feature name='tm'/> <feature name='pbe'/> <feature name='dtes64'/> <feature name='monitor'/> <feature name='ds_cpl'/> <feature name='vmx'/> <feature name='smx'/> <feature name='est'/> <feature name='tm2'/> <feature name='xtpr'/> <feature name='pdcm'/> <feature name='pcid'/> <feature name='osxsave'/> <feature name='arat'/> <feature name='ssbd'/> <feature name='xsaveopt'/> <feature name='invtsc'/> <pages unit='KiB' size='4'/> <pages unit='KiB' size='2048'/> </cpu> Meanwhile, cpuinfo reports the following: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf This results on my cpu being detected, if I allow host copy, as AMD chip and the guest becomes unbearably slow. The model of the host cpu is correct. I'm using Debian testing/unstable. Compiled against library: libvirt 4.7.0 Using library: libvirt 4.7.0 Using API: QEMU 4.7.0 Running hypervisor: QEMU 2.12.0 -- Braiam

2 1

[libvirt-users] Libvirt TLS with Short Lived Certificates
by Charles Urquiola 01 Oct '18

01 Oct '18

I want to use short lived certificates with libvirtd to provided TLS access to the daemon. New certificates are generated on a daily basis and delivered to the host. Does libvirtd re-read TLS certificates with a reload of the service, systemctl reload libvirtd, or with a SIGHUP or is a full restart of the daemon required? --charlie

2 1

[libvirt-users] NUMA issues on virtualized hosts
by Lukas Hejtmanek 20 Sep '18

20 Sep '18

Hello, I have cluster with AMD EPYC 7351 cpu. Two CPUs per node. I have performance 8-NUMA configuration: This is from hypervizor: [root@hde10 ~]# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Thread(s) per core: 2 Core(s) per socket: 16 Socket(s): 2 NUMA node(s): 8 Vendor ID: AuthenticAMD CPU family: 23 Model: 1 Model name: AMD EPYC 7351 16-Core Processor Stepping: 2 CPU MHz: 1800.000 CPU max MHz: 2400.0000 CPU min MHz: 1200.0000 BogoMIPS: 4800.05 Virtualization: AMD-V L1d cache: 32K L1i cache: 64K L2 cache: 512K L3 cache: 8192K NUMA node0 CPU(s): 0-3,32-35 NUMA node1 CPU(s): 4-7,36-39 NUMA node2 CPU(s): 8-11,40-43 NUMA node3 CPU(s): 12-15,44-47 NUMA node4 CPU(s): 16-19,48-51 NUMA node5 CPU(s): 20-23,52-55 NUMA node6 CPU(s): 24-27,56-59 NUMA node7 CPU(s): 28-31,60-63 I'm running one big virtual on this hypervizor - almost whole memory + all physical CPUs. This is what I'm seeing inside: root@zenon10:~# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 8 NUMA node(s): 8 Vendor ID: AuthenticAMD CPU family: 23 Model: 1 Model name: AMD EPYC 7351 16-Core Processor Stepping: 2 CPU MHz: 2400.000 BogoMIPS: 4800.00 Virtualization: AMD-V Hypervisor vendor: KVM Virtualization type: full L1d cache: 64K L1i cache: 64K L2 cache: 512K NUMA node0 CPU(s): 0-3 NUMA node1 CPU(s): 4-7 NUMA node2 CPU(s): 8-11 NUMA node3 CPU(s): 12-15 NUMA node4 CPU(s): 16-19 NUMA node5 CPU(s): 20-23 NUMA node6 CPU(s): 24-27 NUMA node7 CPU(s): 28-31 This is virtual node configuration: (i tried different numatune settings but it was still the same) <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>one-55782</name> <vcpu><![CDATA[32]]></vcpu> <cputune> <shares>32768</shares> </cputune> <memory>507904000</memory> <os> <type arch='x86_64'>hvm</type> </os> <devices> <emulator><![CDATA[/usr/bin/kvm]]></emulator> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.0'/> <target dev='vda'/> <driver name='qemu' type='qcow2' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.1'/> <target dev='vdc'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.2'/> <target dev='vdd'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.3'/> <target dev='vde'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> <disk type='file' device='cdrom'> <source file='/opt/opennebula/var/datastores/108/55782/disk.4'/> <target dev='vdb'/> <readonly/> <driver name='qemu' type='raw'/> </disk> <interface type='bridge'> <source bridge='br0'/> <mac address='02:00:93:fb:3b:78'/> <target dev='one-55782-0'/> <model type='virtio'/> <filterref filter='no-arp-mac-spoofing'> <parameter name='IP' value='147.251.59.120'/> </filterref> </interface> </devices> <features> <pae/> <acpi/> </features>  <cpu mode='host-passthrough'><topology sockets='8' cores='4' threads='1'/><numa><cell cpus='0-3' memory='62000000' /><cell cpus='4-7' memory='62000000' /><cell cpus='8-11' memory='62000000' /><cell cpus='12-15' memory='62000000' /><cell cpus='16-19' memory='62000000' /><cell cpus='20-23' memory='62000000' /><cell cpus='24-27' memory='62000000' /><cell cpus='28-31' memory='62000000' /></numa></cpu> <cputune><vcpupin vcpu='0' cpuset='0' /><vcpupin vcpu='1' cpuset='2' /><vcpupin vcpu='2' cpuset='4' /><vcpupin vcpu='3' cpuset='6' /><vcpupin vcpu='4' cpuset='8' /><vcpupin vcpu='5' cpuset='10' /><vcpupin vcpu='6' cpuset='12' /><vcpupin vcpu='7' cpuset='14' /><vcpupin vcpu='8' cpuset='16' /><vcpupin vcpu='9' cpuset='18' /><vcpupin vcpu='10' cpuset='20' /><vcpupin vcpu='11' cpuset='22' /><vcpupin vcpu='12' cpuset='24' /><vcpupin vcpu='13' cpuset='26' /><vcpupin vcpu='14' cpuset='28' /><vcpupin vcpu='15' cpuset='30' /><vcpupin vcpu='16' cpuset='1' /><vcpupin vcpu='17' cpuset='3' /><vcpupin vcpu='18' cpuset='5' /><vcpupin vcpu='19' cpuset='7' /><vcpupin vcpu='20' cpuset='9' /><vcpupin vcpu='21' cpuset='11' /><vcpupin vcpu='22' cpuset='13' /><vcpupin vcpu='23' cpuset='15' /><vcpupin vcpu='24' cpuset='17' /><vcpupin vcpu='25' cpuset='19' /><vcpupin vcpu='26' cpuset='21' /><vcpupin vcpu='27' cpuset='23' /><vcpupin vcpu='28' cpuset='25' /><vcpupin vcpu='29' cpuset='27' /><vcpupin vcpu='30' cpuset='29' /><vcpupin vcpu='31' cpuset='31' /></cputune> <numatune><memory mode='preferred' nodeset='0'/></numatune>) <devices><serial type='pty'><target port='0'/></serial><console type='pty'><target type='serial' port='0'/></console><channel type='pty'><target type='virtio' name='org.qemu.guest_agent.0'/></channel></devices> <devices><hostdev mode='subsystem' type='pci' managed='yes'><source><address domain='0x0' bus='0x11' slot='0x0' function='0x1'/></source></hostdev></devices> <devices><controller type='pci' index='1' model='pci-bridge'/><controller type='pci' index='2' model='pci-bridge'/><controller type='pci' index='3' model='pci-bridge'/><controller type='pci' index='4' model='pci-bridge'/><controller type='pci' index='5' model='pci-bridge'/></devices> <metadata> <system_datastore><![CDATA[/opt/opennebula/var/datastores/108/55782]]> </system_datastore> </metadata> </domain> If I run e.g., spec2017 on the virtual, I can see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1350 root 20 0 843136 830068 2524 R 78.1 0.2 513:16.16 bwaves_r_base.m 2456 root 20 0 804608 791264 2524 R 76.6 0.2 491:39.92 bwaves_r_base.m 4631 root 20 0 843136 829892 2344 R 75.8 0.2 450:16.04 bwaves_r_base.m 6441 root 20 0 802580 790212 2532 R 75.0 0.2 120:37.54 bwaves_r_base.m 7991 root 20 0 784676 772092 2576 R 75.0 0.2 387:15.39 bwaves_r_base.m 8142 root 20 0 843136 830044 2496 R 75.0 0.2 384:39.02 bwaves_r_base.m 8234 root 20 0 843136 830064 2524 R 75.0 0.2 99:04.48 bwaves_r_base.m 8578 root 20 0 749240 736604 2468 R 73.4 0.2 375:45.66 bwaves_r_base.m 9974 root 20 0 784676 771984 2468 R 73.4 0.2 348:01.36 bwaves_r_base.m 10396 root 20 0 802580 790264 2576 R 73.4 0.2 340:08.40 bwaves_r_base.m 12932 root 20 0 843136 830024 2480 R 73.4 0.2 288:39.76 bwaves_r_base.m 13113 root 20 0 784676 771864 2348 R 71.9 0.2 284:47.34 bwaves_r_base.m 13518 root 20 0 784676 762816 2540 R 71.9 0.2 276:31.58 bwaves_r_base.m 14443 root 20 0 784676 771984 2468 R 71.9 0.2 260:01.82 bwaves_r_base.m 12791 root 20 0 784676 772060 2544 R 70.3 0.2 291:43.96 bwaves_r_base.m 10544 root 20 0 843136 830068 2520 R 68.8 0.2 336:47.43 bwaves_r_base.m 15464 root 20 0 784676 762880 2608 R 60.9 0.2 239:19.14 bwaves_r_base.m 15487 root 20 0 784676 772048 2532 R 60.2 0.2 238:37.07 bwaves_r_base.m 16824 root 20 0 784676 772120 2604 R 55.5 0.2 212:10.92 bwaves_r_base.m 17255 root 20 0 843136 830012 2468 R 54.7 0.2 203:22.89 bwaves_r_base.m 17962 root 20 0 784676 772004 2488 R 54.7 0.2 188:26.07 bwaves_r_base.m 17505 root 20 0 843136 830068 2520 R 53.1 0.2 198:04.25 bwaves_r_base.m 27767 root 20 0 784676 771860 2344 R 52.3 0.2 592:25.95 bwaves_r_base.m 24458 root 20 0 843136 829888 2344 R 50.8 0.2 658:23.70 bwaves_r_base.m 30746 root 20 0 747376 735160 2604 R 43.0 0.2 556:47.67 bwaves_r_base.m The CPU TIME should be roughly the same but huge differences are obvious. This is what I see on the hypervizor: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18201 oneadmin 20 0 474.0g 473.3g 1732 S 2459 94.0 33332:54 kvm 369 root 20 0 0 0 0 R 100.0 0.0 768:12.85 kswapd1 368 root 20 0 0 0 0 R 94.1 0.0 869:05.61 kswapd0 i.e., kswapd is eating whole CPU. Swap is turned off. [root@hde10 ~]# free total used free shared buff/cache available Mem: 528151432 503432580 1214048 34740 23504804 21907800 Swap: 0 0 0 Hypervisor is [root@hde10 ~]# cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core) qemu-kvm-1.5.3-156.el7_5.5.x86_64 Virtual is Debian 9. Moreover, I'm using this type of disks for virtuals: <disk type='file' device='disk'> <source file='/opt/opennebula/var/datastores/108/55782/disk.3'/> <target dev='vde'/> <driver name='qemu' type='raw' cache='unsafe'/> </disk> If I keep cache='unsafe' and if I run iozone test on really big files (e.g., 8x 100GB), I can see huge cache pressure on the hypervizor - all 8 kswapd are running on 100 % percent and slowing things down. The disk under datastore is NVME SSD Intel 4500. If I set cache='none', kswaps are on idle, disk writes are pretty fast, however, with 8-NUMA configuration, writes slow down to less than 10MB/s as soon as the size of written data is roughly the same as memory size in the virtual node. iozone has 100 % CPU usage thereafter and it seems that it is traversing page lists. If I do the same with 1-NUMA configuration, everything is ok except performance penalty about 25 %. -- Lukáš Hejtmánek Linux Administrator only because Full Time Multitasking Ninja is not an official job title

2 7