October 2021 - Users - libvirt List Archives

[libvirt-users] [virtual interface] detach interface during boot succeed with no changes
by Yalan Zhang 08 Sep '22

08 Sep '22

Hi guys, when I detach an interface from vm during boot (vm boot not finished), it always fail. I'm not sure if there is an existing bug. I have confirmed with someone that for disk, there is similar behavior, if this is also acceptable? # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 2; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </controller> <interface type='network'> <mac address='52:54:00:98:c4:a0'/> <source network='default' bridge='virbr0'/> <target dev='vnet0'/> <model type='rtl8139'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> When I detach after the vm boot, expand the sleep time to 10, it will succeed. # virsh destroy rhel7.2; virsh start rhel7.2 ;sleep 10; virsh detach-interface rhel7.2 network 52:54:00:98:c4:a0; sleep 2; virsh dumpxml rhel7.2 |grep /interface -B9 Domain rhel7.2 destroyed Domain rhel7.2 started Interface detached successfully ------- Best Regards, Yalan Zhang IRC: yalzhang Internal phone: 8389413

3 4

a test email (& dmarc) @admins
by lejeczek 10 Dec '21

10 Dec '21

hi guys. I sent email a while ago and I wonder, also having not received a single reply, because I did not get that own email - is there are problem I'm having here or DMARC(yahoo & others?) are not fixed in by this mailing list - would an admin(anybody) know? many thanks, L.

2 4

another upgrade another vm issue
by daggs 31 Oct '21

31 Oct '21

Greetings, so I've upgraded my server and yet again, one of my vm lost a functionally. there is no usable sound card. xml: https://dpaste.com/CVR5M75VH in vm: https://snipboard.io/aZ7Dcf.jpg outputs: utils_server /home/igor # qemu-system-x86_64 --version QEMU emulator version 6.0.0 Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers utils_server /home/igor # libvirtd --version libvirtd (libvirt) 7.8.0 utils_server /home/igor # any ideas? Thanks.

4 6

Iptables, et al best practices for protecting KVM host sharing "hostdev" (ixgbe-vf) interfaces with guests
by Philip Prindeville 29 Oct '21

29 Oct '21

Hi, I'm a developer and I use VM's for testing patches before submitting them (for various linux-* projects, as well as downstream things like OpenWrt, EPEL, etc). I have a trunked 10Gb/s Ethernet connection coming into my KVM servers (all of which are Supermicro x86_servers running CentOS 8) that have igb and ixgbe NIC's, so SR-IOV capable. Here's the situation: * the host is on a secured internal network, called "sandbox", which is VLAN 4. * the "external" network (i.e. public facing on my ISP) is "dirty", and is VLAN 66. * the main subnet that laptops, WAPs, desktops, etc. sit on is "main", and is VLAN 1. The KVM hosts sit wholly inside "sandbox". They host, however, several guests some of which sit on "main", and some of which sit on "main" and "external" (for instance, if I'm testing an OpenWrt firewall configuration with patches and it needs to be publicly accessible so that my testers can peer with it via IPsec). Oh, and one production guest which hosts "git" and sits only on "sandbox". Guests can be CentOS 8S, Windows 11, Ubuntu 20, Debian whatever, Fedora 33 or 34, OpenWrt, etc. My question is this: what's best practices for making sure that a switch VLAN misconfiguration issue, a cabling to the wrong port, etc. doesn't compromise the KVM server itself? How do I allow my KVM server to *not* be on "external", but some of its guests to be, without compromising security? Being the paranoid sort, I'd like to use defense-in-depth so I'm not counting on any single step to protect me. For instance, I might use ebtables or iptables rules to block VLAN traffic on hosts/guests that I don't want to see certain VLAN's under any circumstances. But I might also add interfaces to the list of things that NetworkManager ignores as /etc/NetworkManager/NetworkManager.conf lines like: [keyfile] unmanaged-devices=interface-name:eno*,except:interface-name:eno1 unmanaged-devices=interface-name:enp* Thinking about that, I'm wondering what the notation is for telling NetworkManager to ignore all interfaces except those on VLAN 3, for instance. Would that be: unmanaged-devices=interface-name:eno*.*,except:interface-name:eno*.3 Or something else? And what else can I do to further secure my KVM host? I thought about using one untagged (non VLAN'd) Ethernet for the host, and another tagged Ethernet for the guests, but that just seemed like a false sense of security... Apologies for the cross posting... these questions seems to touch on iptables, libvirt, and NetworkManager all at the same time... Thanks, -Philip

3 2

RE: apt libvirt package
by admin＠foundryserver.com 26 Oct '21

26 Oct '21

Hello again, So I have been struggling to compile the libvirt newest version 7.8.0. I am running CentOS 8 and during the meson build process I am getting non stop dependency fails. I keep installing the necessary packages and slowly progressing. Now the last one that stopped was gnutls. When I go to install that package via yum install gnutls -y it says that it is already complete and installed. Sigh... I am very surprised that there is no package for libvirt via rpm,yum etc. the lastest one installed via yum was 6.0.0. Can someone help by pointing to some where that has these latest packages? Or how best to compile this from source. It is amazing how much is missing from this distro. Thanks again. Brad From: admin(a)foundryserver.com <admin(a)foundryserver.com> Sent: October 23, 2021 12:54 PM To: 'libvirt-users(a)redhat.com' <libvirt-users(a)redhat.com> Subject: apt libvirt package Hello, I am new to kvm/qemu/libvirt. I am loving it. I am working on setting my first vm with shared file system with virtio-fs. I got an error, and google foo said, my version was too old. When I checked it said. libvirt version: 6.0.0, package: 0ubuntu8.14 (Matthew Ruffell matthew.ruffell(a)canonical.com <mailto:matthew.ruffell@canonical.com> Tue, 14 Sep 2021 14:00:49 +1200) I used apt package manager to install libvirt. I am running on ubuntu 20.04. I have tried apt update libvirt and I get..no updates available. What do I do at this point? Do I have to build it from source? Any help would be greatly appreciated. Brad

4 5

how can i measure reliably the speed of a virtual disk ?
by Lentes, Bernd 25 Oct '21

25 Oct '21

Hi, we have some domains running on a two-node pacemaker cluster. The disks for the domains are raw files which reside on a SAN. I measured the speed of one domain inside the guest and got completely weird results: between 18MB/s and 750MB/s !?! I measured with hdparm -t. I think this is related because, from the host point of view, some data is on the disk and some data is on chache. The -t i used for hdparm does not influence the host, just the guest. Measures on the host give reliable and reasonable results about 350 MByte/s. How can i measure reliably the speed of a virtual disk ? Bernd -- Bernd Lentes System Administrator Institute for Metabolism and Cell Death (MCD) Building 25 - office 122 HelmholtzZentrum München bernd.lentes(a)helmholtz-muenchen.de phone: +49 89 3187 1241 phone: +49 89 3187 3827 fax: +49 89 3187 2294 http://www.helmholtz-muenchen.de/mcd Public key: 30 82 01 0a 02 82 01 01 00 b3 72 3e ce 2c 0a 6f 58 49 2c 92 23 c7 b9 c1 ff 6c 3a 53 be f7 9e e9 24 b7 49 fa 3c e8 de 28 85 2c d3 ed f7 70 03 3f 4d 82 fc cc 96 4f 18 27 1f df 25 b3 13 00 db 4b 1d ec 7f 1b cf f9 cd e8 5b 1f 11 b3 a7 48 f8 c8 37 ed 41 ff 18 9f d7 83 51 a9 bd 86 c2 32 b3 d6 2d 77 ff 32 83 92 67 9e ae ae 9c 99 ce 42 27 6f bf d8 c2 a1 54 fd 2b 6b 12 65 0e 8a 79 56 be 53 89 70 51 02 6a eb 76 b8 92 25 2d 88 aa 57 08 42 ef 57 fb fe 00 71 8e 90 ef b2 e3 22 f3 34 4f 7b f1 c4 b1 7c 2f 1d 6f bd c8 a6 a1 1f 25 f3 e4 4b 6a 23 d3 d2 fa 27 ae 97 80 a3 f0 5a c4 50 4a 45 e3 45 4d 82 9f 8b 87 90 d0 f9 92 2d a7 d2 67 53 e6 ae 1e 72 3e e9 e0 c9 d3 1c 23 e0 75 78 4a 45 60 94 f8 e3 03 0b 09 85 08 d0 6c f3 ff ce fa 50 25 d9 da 81 7b 2a dc 9e 28 8b 83 04 b4 0a 9f 37 b8 ac 58 f1 38 43 0e 72 af 02 03 01 00 01

1 0

apt libvirt package
by admin＠foundryserver.com 23 Oct '21

23 Oct '21

Hello, I am new to kvm/qemu/libvirt. I am loving it. I am working on setting my first vm with shared file system with virtio-fs. I got an error, and google foo said, my version was too old. When I checked it said. libvirt version: 6.0.0, package: 0ubuntu8.14 (Matthew Ruffell matthew.ruffell(a)canonical.com <mailto:matthew.ruffell@canonical.com> Tue, 14 Sep 2021 14:00:49 +1200) I used apt package manager to install libvirt. I am running on ubuntu 20.04. I have tried apt update libvirt and I get..no updates available. What do I do at this point? Do I have to build it from source? Any help would be greatly appreciated. Brad

1 0

load monitoring differences
by Marc 22 Oct '21

22 Oct '21

I was wondering if it is normal that the load reported from within the guest differs a lot from the guest load monitored on the host? Eg. currently I have guest that reports a load of 12-17 (uptime) while on the host top shows 70% cpu usage, I would expect this to be also around 12-17%.

1 0

load monitoring differences
by Marc 22 Oct '21

22 Oct '21

I was wondering if it is normal that the load reported from within the guest differs a lot from the guest load monitored on the host? Eg. currently I have guest that reports a load of 12-17 (uptime) while on the host top shows 70% cpu usage, I would expect this to be also around 12-17%.

1 0

is there a way to stop a domain in 'D' process state (uninterruptible) ?
by Lentes, Bernd 21 Oct '21

21 Oct '21

Hi, how can i stop/shutdown a domain which is in process state 'D' ? 'D' means uninterruptible and a process in 'D' can't be terminated by kill, even not with kill -9. Bernd -- Bernd Lentes System Administrator Institute for Metabolism and Cell Death (MCD) Building 25 - office 122 HelmholtzZentrum München bernd.lentes(a)helmholtz-muenchen.de phone: +49 89 3187 1241 phone: +49 89 3187 3827 fax: +49 89 3187 2294 http://www.helmholtz-muenchen.de/mcd Public key: 30 82 01 0a 02 82 01 01 00 b3 72 3e ce 2c 0a 6f 58 49 2c 92 23 c7 b9 c1 ff 6c 3a 53 be f7 9e e9 24 b7 49 fa 3c e8 de 28 85 2c d3 ed f7 70 03 3f 4d 82 fc cc 96 4f 18 27 1f df 25 b3 13 00 db 4b 1d ec 7f 1b cf f9 cd e8 5b 1f 11 b3 a7 48 f8 c8 37 ed 41 ff 18 9f d7 83 51 a9 bd 86 c2 32 b3 d6 2d 77 ff 32 83 92 67 9e ae ae 9c 99 ce 42 27 6f bf d8 c2 a1 54 fd 2b 6b 12 65 0e 8a 79 56 be 53 89 70 51 02 6a eb 76 b8 92 25 2d 88 aa 57 08 42 ef 57 fb fe 00 71 8e 90 ef b2 e3 22 f3 34 4f 7b f1 c4 b1 7c 2f 1d 6f bd c8 a6 a1 1f 25 f3 e4 4b 6a 23 d3 d2 fa 27 ae 97 80 a3 f0 5a c4 50 4a 45 e3 45 4d 82 9f 8b 87 90 d0 f9 92 2d a7 d2 67 53 e6 ae 1e 72 3e e9 e0 c9 d3 1c 23 e0 75 78 4a 45 60 94 f8 e3 03 0b 09 85 08 d0 6c f3 ff ce fa 50 25 d9 da 81 7b 2a dc 9e 28 8b 83 04 b4 0a 9f 37 b8 ac 58 f1 38 43 0e 72 af 02 03 01 00 01

3 2

Re: about pcie-hot-plug on aarch64
by Andrea Bolognani 20 Oct '21

20 Oct '21

Forwarding to libvirt-users for visibility. On Wed, Oct 20, 2021 at 03:51:10PM +0800, Jaze Lee wrote: > For someone who meets the same problem. > > I upgrade stein to w, use one centos stream 8 vm as ctrl, and one > centos stream 8 server as cn, > then test it. There is no problem with attaching disks. All disks are > attached correctly. > But we do not know what's the problem with stein. If someone meets the > problem, you can update your openstack. > > Jaze Lee <jazeltq(a)gmail.com> 于2021年10月12日周二上午9:20写道： > > Andrea Bolognani <abologna(a)redhat.com> 于2021年10月12日周二上午12:07写道： > > > On Fri, Oct 08, 2021 at 04:54:37PM +0800, Jaze Lee wrote: > > > > Hello, > > > > We run Openstack Stein on arm. It runs nova-compute(use libvirt > > > > as virt driver) on arm host. We found when built with disks (use ceph > > > > rbd) on arm hosts, the vm can not attach all disk correctly. For > > > > example, built with six disks, the vm may attach three disks. No > > > > obvious error can be fond in nova-compute, libvirt. We compare aarch64 > > > > and x86, find when detach disk, the dmesg of the vm's os is different. > > > > May be the pciehg parameter is different? > > > > > > > > Did anyone met the problem? Or some suggestions? > > > > > > I think you might have just ran out of PCI ports available for > > > hotplug. Please try setting > > > > > > https://docs.openstack.org/nova/stein/configuration/config.html#libvirt.num… > > > > > > to a reasonable value and see whether that helps. > > > > Thanks. > > I already set the value to a reasonable value. It is 15 in our environment. > > If the value is not set correctly, the nova-compute will complain no > > available slot for pci device. > > But it is not the case I talked about here. -- Andrea Bolognani / Red Hat / Virtualization

1 0

VM paused during multipath iSCSI reservation
by Vojtech Juranek 14 Oct '21

14 Oct '21

Hi, I'm trying to find the root cause for BZ #1898049 [1]. When setting up Windows HA cluster on Windows Server VMs run on top of oVirt, Windows cluster validator runs couple of tests and fails during test "Validate SCSI-3 Persistent Reservation" and one of the VMs of the cluster is paused with IO error. Disk definition is as follows: <disk type='block' device='lun' sgio='unfiltered' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native'/> <source dev='/dev/mapper/3600a09803830447a4f244c4657616f6f' index='1'> <seclabel model='dac' relabel='no'/> <reservations managed='yes'> <source type='unix' path='/var/lib/libvirt/qemu/domain-1-Windows-2016-2/pr-helper0.sock' mode='client'/> </reservations> </source> <backingStore/> <target dev='sdb' bus='scsi'/> <shareable/> <alias name='ua-26b4975e-e1d4-4e27-b2c6-2ea0894a571b'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> and libvirt error I get is bellow [2]. When I try to create reservation from Windows VM manually, I get following error (but not sure I do it whole process correctly): .\sg_persist --out --register --param-sark=123abc e: QEMU QEMU HARDDISK 2.5+ Peripheral device type: disk PR out (Register): command not supported sg_persist failed: Illegal request, Invalid opcode Do you have any ideas what could be wrong or how to determine the root cause of this this issue? Thanks in advance. Vojta [1] https://bugzilla.redhat.com/1898049 [2] libvirt debug log: 2021-10-12 11:43:25.148+0000: 2006427: debug : qemuMonitorEmitIOError:1243 : mon=0x7fb02006a020 2021-10-12 11:43:25.148+0000: 2006427: info : virObjectRef:402 : OBJECT_REF: obj=0x7fb02006a020 2021-10-12 11:43:25.148+0000: 2006427: info : virObjectRef:402 : OBJECT_REF: obj=0x7fafd0130020 2021-10-12 11:43:25.148+0000: 2000208: info : virObjectRef:402 : OBJECT_REF: obj=0x7fafd010d340 2021-10-12 11:43:25.148+0000: 2006427: info : virObjectNew:258 : OBJECT_NEW: obj=0x7fb020082590 classname=virDomainEventIOError 2021-10-12 11:43:25.148+0000: 2000208: info : vir_object_finalize:321 : OBJECT_DISPOSE: obj=0x7fb020082500 2021-10-12 11:43:25.148+0000: 2006427: info : virObjectNew:258 : OBJECT_NEW: obj=0x7fb020082620 classname=virDomainEventIOError 2021-10-12 11:43:25.148+0000: 2000208: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fb020082500 2021-10-12 11:43:25.148+0000: 2006427: debug : qemuProcessHandleIOError:907 : Transitioned guest Windows-2016-2 to paused state due to IO error 2021-10-12 11:43:25.148+0000: 2000208: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fafd010d340 2021-10-12 11:43:25.148+0000: 2006427: info : virObjectNew:258 : OBJECT_NEW: obj=0x7fafbc1fb8c0 classname=virDomainEventLifecycle 2021-10-12 11:43:25.148+0000: 2006427: debug : virDomainLockProcessPause:204 : plugin=0x7fafd01272a0 dom=0x7fb01400f5e0 state=0x7fb02401d768 2021-10-12 11:43:25.148+0000: 2006427: debug : virDomainLockManagerNew:134 : plugin=0x7fafd01272a0 dom=0x7fb01400f5e0 withResources=1 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerPluginGetDriver:276 : plugin=0x7fafd01272a0 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerNew:300 : driver=0x7fafd444a000 type=0 nparams=5 params=0x7fafd77de640 flags=0x0 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerLogParams:97 : key=uuid type=uuid value=70eee88c-ba2c-4c6c-bd51-c2b663db27f8 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerLogParams:90 : key=name type=string value=Windows-2016-2 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerLogParams:78 : key=id type=uint value=1 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerLogParams:78 : key=pid type=uint value=2006418 2021-10-12 11:43:25.148+0000: 2006427: debug : virLockManagerLogParams:93 : key=uri type=cstring value=(null) 2021-10-12 11:43:25.148+0000: 2006427: debug : virDomainLockManagerNew:146 : Adding leases 2021-10-12 11:43:25.148+0000: 2006427: debug : virDomainLockManagerNew:151 : Adding disks 2021-10-12 11:43:25.149+0000: 2006427: debug : virDomainLockManagerAddImage:90 : Add disk /rhev/data-center/mnt/blockSD/7c4f09b6-9e87-436f-bda9-22d1f0b50955/images/f5d6e074-dfe9-462d-8cfd-3e14b0eb5aea/766e36b2-84a6-43e7-a48b-a5f47e669860 2021-10-12 11:43:25.149+0000: 2006427: debug : virLockManagerAddResource:326 : lock=0x7fafbc19e250 type=0 name=/rhev/data-center/mnt/blockSD/7c4f09b6-9e87-436f-bda9-22d1f0b50955/images/f5d6e074-dfe9-462d-8cfd-3e14b0eb5aea/766e36b2-84a6-43e7-a48b-a5f47e669860 nparams=0 params=(nil) flags=0x0 2021-10-12 11:43:25.149+0000: 2006427: debug : virDomainLockManagerAddImage:90 : Add disk /dev/mapper/3600a09803830447a4f244c4657616f6f 2021-10-12 11:43:25.149+0000: 2006427: debug : virLockManagerAddResource:326 : lock=0x7fafbc19e250 type=0 name=/dev/mapper/3600a09803830447a4f244c4657616f6f nparams=0 params=(nil) flags=0x2 2021-10-12 11:43:25.149+0000: 2006427: debug : virLockManagerRelease:359 : lock=0x7fafbc19e250 state=0x7fb02401d768 flags=0x0 2021-10-12 11:43:25.149+0000: 2006427: debug : virLockManagerFree:381 : lock=0x7fafbc19e250 2021-10-12 11:43:25.149+0000: 2006427: debug : qemuProcessHandleIOError:920 : Preserving lock state '<null>' 2021-10-12 11:43:25.150+0000: 2006427: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fafd0130020 2021-10-12 11:43:25.150+0000: 2006427: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fb02006a020 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectRef:402 : OBJECT_REF: obj=0x7fafd010d340 2021-10-12 11:43:25.150+0000: 2000208: info : vir_object_finalize:321 : OBJECT_DISPOSE: obj=0x7fb020082590 2021-10-12 11:43:25.150+0000: 2006427: info : virObjectRef:402 : OBJECT_REF: obj=0x7fb02006a020 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fb020082590 2021-10-12 11:43:25.150+0000: 2006427: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fb02006a020 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectNew:258 : OBJECT_NEW: obj=0x564a89a4cc60 classname=virDomain 2021-10-12 11:43:25.150+0000: 2006427: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x7fb02006a020 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectRef:402 : OBJECT_REF: obj=0x7fafd0018ca0 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectRef:402 : OBJECT_REF: obj=0x564a89978df0 2021-10-12 11:43:25.150+0000: 2000208: debug : virAccessManagerCheckDomain:238 : manager=0x564a89978df0(name=stack) driver=QEMU domain=0x7ffd2c677010 perm=0 2021-10-12 11:43:25.150+0000: 2000208: debug : virAccessManagerCheckDomain:238 : manager=0x564a89978e50(name=none) driver=QEMU domain=0x7ffd2c677010 perm=0 2021-10-12 11:43:25.150+0000: 2000208: info : virObjectUnref:380 : OBJECT_UNREF: obj=0x564a89978df0 2021-10-12 11:43:25.150+0000: 2000208: debug : remoteRelayDomainEventIOErrorReason:529 : Relaying domain io error Windows-2016-2 1 /dev/mapper/3600a09803830447a4f244c4657616f6f ua-26b4975e-e1d4-4e27-b2c6-2ea0894a571b 1 , callback 3

2 2

could not find capabilities for arch=x86_64 domaintype=kvm
by shafnamol N 13 Oct '21

13 Oct '21

Hi, I am new to Libvirt and Qemu.I have installed Libvirt 6.10 and qemu-kvm 4.2.0 on CentOS8. I configured and built libvirt based on instructions from https://libvirt.org/compiling.html. But when i tried to create a VM using virsh it shows the following error: *# virsh create /home/abc.xml* *error: Failed to create domain from /home/abc.xmlerror: invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm * When i check the hypervisor capabilities ,it doest show qemu in guest domain type. # virsh capabilities As per the reply from this mailing-list i passed an option while building the libvirt. ie., *meson build -Dsystem=true* It worked for me. But Now, i tried to migrate my work on a new machine with same os and same version as above. As part of,I configured and built libvirt.Also passed *meson build -Dsystem=true *while building libvirt. After installation I tried with *# virsh create /home/abc.xml* It results in the same error shown below. *error: Failed to create domain from /home/abc.xmlerror: invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm * I stuck here. Thanks for the help in advance. Shafnamol.N

1 0

about pcie-hot-plug on aarch64
by Jaze Lee 11 Oct '21

11 Oct '21

Hello, We run Openstack Stein on arm. It runs nova-compute(use libvirt as virt driver) on arm host. We found when built with disks (use ceph rbd) on arm hosts, the vm can not attach all disk correctly. For example, built with six disks, the vm may attach three disks. No obvious error can be fond in nova-compute, libvirt. We compare aarch64 and x86, find when detach disk, the dmesg of the vm's os is different. May be the pciehg parameter is different? Did anyone met the problem? Or some suggestions? x86: Nothing at all aarch64: Sep 29 15:28:55 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Attention button pressed Sep 29 15:28:55 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Powering off due to button press Sep 29 15:29:00 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Attention button pressed Sep 29 15:29:00 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Button cancel Sep 29 15:29:00 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Action canceled due to button press Sep 29 15:29:07 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Attention button pressed Sep 29 15:29:07 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Powering off due to button press Sep 29 15:29:13 * kernel: pciehp 0000:00:01.5:pcie004: Slot(0-5): Link Up Sep 29 15:29:16 * kernel: pciehp 0000:00:01.5:pcie004: Failed to check link status Sep 29 15:29:18 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Attention button pressed Sep 29 15:29:18 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Powering off due to button press Sep 29 15:29:23 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Attention button pressed Sep 29 15:29:23 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Button cancel Sep 29 15:29:23 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Action canceled due to button press Sep 29 15:29:30 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Attention button pressed Sep 29 15:29:30 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Powering off due to button press Sep 29 15:29:36 * kernel: pciehp 0000:00:01.6:pcie004: Slot(0-6): Link Up Sep 29 15:29:39 * kernel: pciehp 0000:00:01.6:pcie004: Failed to check link status Sep 29 15:29:39 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Attention button pressed Sep 29 15:29:39 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Powering off due to button press Sep 29 15:29:45 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Attention button pressed Sep 29 15:29:45 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Button cancel Sep 29 15:29:45 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Action canceled due to button press Sep 29 15:29:52 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Attention button pressed Sep 29 15:29:52 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Powering off due to button press Sep 29 15:29:58 * kernel: pciehp 0000:00:01.7:pcie004: Slot(0-7): Link Up Sep 29 15:30:01 * kernel: pciehp 0000:00:02.0:pcie004: Slot(0-8): Attention button pressed Sep 29 15:30:01 * kernel: pciehp 0000:00:02.0:pcie004: Slot(0-8): Powering off due to button press

3 3

re: nwfilter direction not being used when protocol all
by Jason Pyeron 11 Oct '21

11 Oct '21

> -----Original Message----- > From: Jason Pyeron > Sent: Monday, October 11, 2021 8:49 AM > To: Kyle Marek; Michael Watson Jr > Cc: libvirt-users > > Watson / Kyle: > > (note I coped the list) > > While I read https://libvirt.org/formatnwfilter.html#nwfelemsRulesProtoMisc , it is not > clear that it is intended to add the iptables action without regard to the rule’s > direction. > > Take the following rule scenarios: > > <rule action='accept' direction='in' priority='500' statematch='false'> > <tcp dstportstart='22'/> > </rule> > <rule action='drop' direction='in' priority='1000'> > <all/> > </rule> > > # iptables-save | grep vnet5 | tee in > :FI-vnet5 - [0:0] > :FO-vnet5 - [0:0] > :HI-vnet5 - [0:0] > -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A FI-vnet5 -j DROP > -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT > -A FO-vnet5 -j DROP > -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A HI-vnet5 -j DROP > -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 > -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 > -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT > -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 > > <rule action='accept' direction='in' priority='500' statematch='false'> > <tcp dstportstart='22'/> > </rule> > <rule action='drop' direction='out' priority='1000'> > <all/> > </rule> > > # iptables-save | grep vnet5 | tee out > :FI-vnet5 - [0:0] > :FO-vnet5 - [0:0] > :HI-vnet5 - [0:0] > -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A FI-vnet5 -j DROP > -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT > -A FO-vnet5 -j DROP > -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A HI-vnet5 -j DROP > -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 > -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 > -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT > -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 > > <rule action='accept' direction='in' priority='500' statematch='false'> > <tcp dstportstart='22'/> > </rule> > <rule action='drop' direction='inout' priority='1000'> > <all/> > </rule> > > # iptables-save | grep vnet5 | tee inout > :FI-vnet5 - [0:0] > :FO-vnet5 - [0:0] > :HI-vnet5 - [0:0] > -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A FI-vnet5 -j DROP > -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT > -A FO-vnet5 -j DROP > -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN > -A HI-vnet5 -j DROP > -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 > -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 > -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT > -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 > > We note that the > > -A HI-vnet5 -j DROP > -A FI-vnet5 -j DROP > -A FO-vnet5 -j DROP > > Is present without regards to the state of the direction attribute on the “default” drop > rule. > > If the direction is “in” then the “-A FI-vnet5 -j DROP” should not exists. > > What does the source code say? I worry that either the docs are imprecise and this is > desired, or there is a bug and I can end up like After looking at libvirt-4.5.0/src/nwfilter/nwfilter_ebiptables_driver.c's _iptablesCreateRuleInstance and iptablesCreateRuleInstanceStateCtrl, I saw the if statements like the below. 1598 if (directionIn && !inout) { 1599 if ((rule->flags & IPTABLES_STATE_FLAGS)) 1600 create = false; 1601 } 1629 if (!directionIn) { 1630 if ((rule->flags & IPTABLES_STATE_FLAGS)) 1631 create = false; 1632 } Is the only way to respect the direction is to have <all state='something...'/> ? If that is the case the docs, really need an update to note this. For others, my deny inbound, allow outbound was accomplished by: <rule action='accept' direction='in' priority='999'> <all state='ESTABLISHED,RELATED'/> </rule> <rule action='drop' direction='in' priority='1000'> <all state='NONE'/> </rule> -Jason

1 0

nwfilter direction not being used when protocol all
by Jason Pyeron 11 Oct '21

11 Oct '21

Watson / Kyle: (note I coped the list) While I read https://libvirt.org/formatnwfilter.html#nwfelemsRulesProtoMisc , it is not clear that it is intended to add the iptables action without regard to the rule’s direction. Take the following rule scenarios: <rule action='accept' direction='in' priority='500' statematch='false'> <tcp dstportstart='22'/> </rule> <rule action='drop' direction='in' priority='1000'> <all/> </rule> # iptables-save | grep vnet5 | tee in :FI-vnet5 - [0:0] :FO-vnet5 - [0:0] :HI-vnet5 - [0:0] -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A FI-vnet5 -j DROP -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT -A FO-vnet5 -j DROP -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A HI-vnet5 -j DROP -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 <rule action='accept' direction='in' priority='500' statematch='false'> <tcp dstportstart='22'/> </rule> <rule action='drop' direction='out' priority='1000'> <all/> </rule> # iptables-save | grep vnet5 | tee out :FI-vnet5 - [0:0] :FO-vnet5 - [0:0] :HI-vnet5 - [0:0] -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A FI-vnet5 -j DROP -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT -A FO-vnet5 -j DROP -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A HI-vnet5 -j DROP -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 <rule action='accept' direction='in' priority='500' statematch='false'> <tcp dstportstart='22'/> </rule> <rule action='drop' direction='inout' priority='1000'> <all/> </rule> # iptables-save | grep vnet5 | tee inout :FI-vnet5 - [0:0] :FO-vnet5 - [0:0] :HI-vnet5 - [0:0] -A FI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A FI-vnet5 -j DROP -A FO-vnet5 -p tcp -m tcp --dport 22 -j ACCEPT -A FO-vnet5 -j DROP -A HI-vnet5 -p tcp -m tcp --sport 22 -j RETURN -A HI-vnet5 -j DROP -A libvirt-host-in -m physdev --physdev-in vnet5 -g HI-vnet5 -A libvirt-in -m physdev --physdev-in vnet5 -g FI-vnet5 -A libvirt-in-post -m physdev --physdev-in vnet5 -j ACCEPT -A libvirt-out -m physdev --physdev-out vnet5 --physdev-is-bridged -g FO-vnet5 We note that the -A HI-vnet5 -j DROP -A FI-vnet5 -j DROP -A FO-vnet5 -j DROP Is present without regards to the state of the direction attribute on the “default” drop rule. If the direction is “in” then the “-A FI-vnet5 -j DROP” should not exists. What does the source code say? I worry that either the docs are imprecise and this is desired, or there is a bug and I can end up like https://superuser.com/questions/1660080/in-libvirt-network-filters-nwfilter… As this is going to be a generic rule, applied many times – I would prefer not to have mac based source allow rules. -Jason

1 0

how to get enabled features
by Jiatong Shen 11 Oct '21

11 Oct '21

Hello community, I am trying to learn how to use dtrace and systemtap for libvirt, as the documentation says it seems to enable `--with-dtrace` when compiling. I am installing libvirt using apt install and is it possible to determine if dtrace is built by running some libvirt commands? Thank you. -- Best Regards, Jiatong Shen

2 1

Need more doc for libvirt-console-proxy
by Guy Godfroy 05 Oct '21

05 Oct '21

Hello, I'm making a web app for my company that will enable different teams to manage their own VMs. I wish to make possible to interact with each VM console, so I plan to use some xterm.js with websockets. So I discovered libvirt-console-proxy [1] when I looked for something to put a libvirt console into a websocket. That seems like the right tool for the job. The only doc I found is this article from 2017 [2]. After trying to understand from this article and from --help, I still have many questions. I am really bad at reading code so I can't even get answers from the sources. My main concern is: How a client is supposed to talk to the proxy? It is said that a security token must be provided. How? HTTP header? Which header? Am I missing something in websocket protocol? I think an example client implementation would help a lot. Also, I tried to use virtconsoleresolveradm to set up metadata on my domains like explained in the article [1] : ./virtconsoleresolveradm enable milou Enabled access to domain 'milou' But that doesn't seem to do anything (except defining the metadata namespace in the XML): virsh metadata milou http://libvirt.org/schemas/console-proxy/1.0 <consoles/> I precise that I have already this in my XML: <serial type='pty'> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='pty'> <target type='serial' port='0'/> </console> Should I remove that? Should I edit that? Thanks for your help. Guy Godfroy [1] https://gitlab.com/libvirt/libvirt-console-proxy [2] https://www.berrange.com/posts/2017/01/26/announce-new-libvirt-console-prox…

1 1

how/where to configure access to libvirt-sock-ro
by Marc 04 Oct '21

04 Oct '21

I wanted to create a monitoring user that can do some reporting like this: runuser -u xxxxx -- prometheus-libvirt-exporter -libvirt.uri /var/run/libvirt/libvirt-sock-ro But I am getting the failed to connect: authentication required

2 1

how/where to configure access to libvirt-sock-ro
by Marc 03 Oct '21

03 Oct '21

I wanted to create a monitoring user that can do some reporting like this: runuser -u xxxxx -- prometheus-libvirt-exporter -libvirt.uri /var/run/libvirt/libvirt-sock-ro But I am getting the failed to connect: authentication required

1 0