- Users - libvirt List Archives

unable to migrate non shared storage in tunneled mode
by Vjaceslavs Klimovs 24 Aug '20

24 Aug '20

Hey all, With libvirt 6.5.0 and qemu 5.1.0 migration of non shared disks in tunneled mode does not work for me: virsh # migrate alpinelinux3.8 qemu+tls://ratchet.lan/system --live --persistent --undefinesource --copy-storage-all --tunneled --p2p error: internal error: qemu unexpectedly closed the monitor: Receiving block device images Error unknown block device 2020-08-15T21:21:48.995016Z qemu-system-x86_64: error while loading state section id 1(block) 2020-08-15T21:21:48.995193Z qemu-system-x86_64: load of migration failed: Invalid argument This is both with UEFI and BIOS guests. I understand that newer ways of migrating non shared disks is via NBD directly with QEMU, however I am certain that this used to work before libvirt 6.0. There is a series of commits to /src/qemu/qemu_migration.c on Dec 8, 2019, could they have something to do with this? Is migration of non shared disks supported and supposed to work in tunneled mode or is it not a supported configuration and native NBD directly with QEMU should be used in all cases? Thanks in advance! Full qemu log on receiving host: 2020-08-15 21:23:38.917+0000: starting up libvirt version: 6.5.0, qemu version: 5.1.0, kernel: 5.4.57-gentoo, hostname: ratchet.lan LC_ALL=C \ PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin \ HOME=/var/lib/libvirt/qemu/domain-4-alpinelinux3.8 \ USER=root \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-4-alpinelinux3.8/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-4-alpinelinux3.8/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-4-alpinelinux3.8/.config \ QEMU_AUDIO_DRV=none \ /usr/bin/qemu-system-x86_64 \ -name guest=alpinelinux3.8,debug-threads=on \ -S \ -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-4-alpinelinux3.8/master-key.aes \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/uefi/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/alpinelinux3.8_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-q35-5.1,accel=kvm,usb=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \ -cpu kvm64,ibpb=on,md-clear=on,spec-ctrl=on,ssbd=on,vme=on,x2apic=on,hypervisor=on \ -m 2048 \ -overcommit mem-lock=off \ -smp 2,sockets=2,cores=1,threads=1 \ -uuid 95286971-32fa-4138-be0e-519ec21af800 \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=35,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.1,addr=0x0 \ -blockdev '{"driver":"host_device","filename":"/dev/vg0/test","aio":"native","node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-2-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"raw","file":"libvirt-2-storage"}' \ -device virtio-blk-pci,bus=pci.2,addr=0x0,drive=libvirt-2-format,id=virtio-disk0,bootindex=1,write-cache=on \ -device ide-cd,bus=ide.0,id=sata0-0-0 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -vnc 127.0.0.1:2 \ -device VGA,id=video0,vgamem_mb=16,bus=pcie.0,addr=0x1 \ -incoming defer \ -device virtio-balloon-pci,id=balloon0,bus=pci.3,addr=0x0 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on char device redirected to /dev/pts/5 (label charserial0) Receiving block device images Error unknown block device 2020-08-15T21:23:39.278287Z qemu-system-x86_64: error while loading state section id 1(block) 2020-08-15T21:23:39.278422Z qemu-system-x86_64: load of migration failed: Invalid argument 2020-08-15 21:23:39.344+0000: shutting down, reason=failed

2 3

Conflicting parameters on qemu call
by Jan Walzer 21 Aug '20

21 Aug '20

Hi Lists, I currently have the issue of wanting to use emu-system-x86_64 on a ppc64le platform. It is imperative to pass the "-accel tcg,thread=multi” parameter to qemu when starting an instance, as without that, it will only use one thread and hence of limited/no use. The problem is, that libvirt itself, passes “-machine q35,accel=tcg” to qemu, which is a different parameter, that conflicts with the other one. Can we discuss, if I either have overlooked something, or is there a workaround, or is this a bug? I’m running: # uname -a Linux tiger-v4 5.7.0-1-powerpc64le #1 SMP Debian 5.7.6-1 (2020-06-24) ppc64le GNU/Linux # apt-cache policy libvirt-daemon libvirt-daemon: Installed: 6.5.0-1 Candidate: 6.5.0-1 Version table: *** 6.5.0-1 500 500 http://deb.debian.org/debian testing/main ppc64el Packages # apt-cache policy qemu-system-x86 qemu-system-x86: Installed: 1:5.0-14 Candidate: 1:5.0-14 Version table: *** 1:5.0-14 500 500 http://deb.debian.org/debian testing/main ppc64el Packages Do you need any more information? Feel free to only continue on the list that matters. Greetings, Jan

2 8

multiple vms with same PCI passthrough
by Daniel Black 19 Aug '20

19 Aug '20

In attempting to isolate vfio-pci problems between two different guest instances, the creation of a second guest (with existing guest shutdown) resulted in:. Aug 09 12:43:23 grit libvirtd[6716]: internal error: Device 0000:01:00.3 is already in use Aug 09 12:43:23 grit libvirtd[6716]: internal error: Device 0000:01:00.3 is already in use Aug 09 12:43:23 grit libvirtd[6716]: Failed to allocate PCI device list: internal error: Device 0000:01:00.3 is already in use Compiled against library: libvirt 6.1.0 Using library: libvirt 6.1.0 Using API: QEMU 6.1.0 Running hypervisor: QEMU 4.2.1 (fc32 default install) The upstream code seems also to test definitions rather than active uses of the PCI device. My potentially naive patch to correct this (but not the failing test cases) would be: diff --git a/src/util/virpci.c b/src/util/virpci.c index 47c671daa0..a00c5e6f44 100644 --- a/src/util/virpci.c +++ b/src/util/virpci.c @@ -1597,7 +1597,7 @@ int virPCIDeviceListAdd(virPCIDeviceListPtr list, virPCIDevicePtr dev) { - if (virPCIDeviceListFind(list, dev)) { + if (virPCIDeviceBusContainsActiveDevices(dev, list)) { virReportError(VIR_ERR_INTERNAL_ERROR, _("Device %s is already in use"), dev->name); return -1; Is this too simplistic or undesirable a feature request/implementation? I'd be more than grateful if someone carries this through as I'm unsure when I may get time for this.

2 5

KVM guest VM IP address
by Kaushal Shriyan 18 Aug '20

18 Aug '20

Hi, I am trying to find out the IP address of the KVM guest virtual machine. #virsh dumpxml newsoftlinedrupalpoc | grep "mac address" | awk -F\' '{ print $2}' 52:54:00:2c:7e:ff [root@baseserver1 ~]# arp -an | grep 52:54:00:2c:7e:ff [root@baseserver1 ~]# virsh domifaddr newsoftlinedrupalpoc Name MAC address Protocol Address ------------------------------------------------------------------------------- [root@baseserver1 ~]# It is not showing anything. I manually configure the IP using /etc/sysconfig/network-scripts/ifcfg-eth0 ONBOOT=yes IPADDR=192.168.0.189 PREFIX=24 GATEWAY=192.168.0.10 DNS1=8.8.8.8 DNS2=8.8.4.4 CentOS Linux release 7.6.1810 (Core) virt-install --version 1.5.0 virsh version Compiled against library: libvirt 4.5.0 Using library: libvirt 4.5.0 Using API: QEMU 4.5.0 Running hypervisor: QEMU 1.5.3 Am I missing something? Thanks in advance and I look forward to hearing from you. Best Regards,

2 1

ipv6 NAT; accept_ra errors and about network choice
by Ian Wienand 18 Aug '20

18 Aug '20

Hello, Firstly THANK YOU for the IPv6 NAT support merged in 6.5. It has been almost impossible to get IPv6 into a VM on a laptop that switches between wifi and wired (dock) connections, because you can not add a wifi interface to a bridge. I know NAT is against the IPv6 end-to-end xen but it makes this "just work" for the vast majority of people like me who need to ssh/curl/talk to ipv6 only hosts! So I installed 6.6.0 from the virt-preview repos on Fedora 32 to eagerly test it out. My network config looks like <network> <name>network</name> <uuid> ... </uuid> <forward mode='nat'> <nat ipv6='yes'/> </forward> <bridge name='virbr0' stp='on' delay='0'/> <mac address=' ... '/> <domain name='network'/> <ip address='192.168.100.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.100.128' end='192.168.100.254'/> </dhcp> </ip> <ip family='ipv6' address='fc00:dead:beef:55::' prefix='64'> </ip> </network> The first problem I hit was trying to start that network: error: internal error: Check the host setup: enabling IPv6 forwarding with RA routes without accept_ra set to 2 is likely to cause routes loss. Interfaces to look at: wlp4s0 wlp4s0 is my wifi card that is configured by NetworkManager in a completely unremarkable fashion. By default it gets an ipv6 via SLAAC from my router. This feels a bit like the unresolved bug [1] which says that systemd-networkd is handling the RA's in userspace for ... reasons [2]. It's unclear to me if NetworkManager is doing similar. I feel like this must be a red-herring. My wired interface has the same setting of 0 $ cat /proc/sys/net/ipv6/conf/enp0s31f6/accept_ra 0 and is similarly just a very standard auto-configured NetworkManager interface. When I "net-start" the network whilst on wifi libvirt doesn't seem to care about that interface (I presume it only looks at the active one?). When I dock and turn off wifi, ipv6 connectivity continues to work through enp0s31f6, so I don't think the accept_ra really matters in this case. I feel like this message is incorrect, and being as I've done nothing special to my underlying interfaces probably going to be wrong for a lot of people trying this? Does anyone know the details of this message and see why it would be required in this situation? The other thing that I'd like to expand the documentation on, if I can get some clarity, is the choice of network. It seems like it has to be a /64, and it seems like the best choice is within fc00::/7, or at least that is what has been assigned for private networks like this [3]? The only problem with this is that I think glibc filters this range so nothing prefers IPv6. Is this the range expected to be used for ipv6 NAT? If so, would a patch to drop some documentation breadcrumbs about setting gai.conf or something be useful? Or are there better choices for the network? Thanks! -i [1] https://bugzilla.redhat.com/show_bug.cgi?id=1639087 [2] https://github.com/systemd/systemd/commit/3b015d40c19d9338b66bf916d84dec601… [3] https://tools.ietf.org/html/rfc4193

3 4

Post-firewall hook to insert custom rules?
by Gunnar Niels 18 Aug '20

18 Aug '20

Hello, I have a set of iptables rules that I need to insert *after* libvirt has set up all of its firewall rules. Is there a hook that I can tap into in order to run something like a custom script to make sure this happens? Any ideas? -GN

2 2

Way to skip check for 'libvirt-qemu' group and user while installing guest
by john doe 17 Aug '20

17 Aug '20

Hi, Debian Buster here, I'm trying to install a guest on a sshfs mount, Libvirt is checking for the user and group 'libvirt-qemu'. Is there a way to avoid that check or has anyone a way to use a mount used by sshfs with Libvirt? The SSH server has no group with the UID/GID of 'libvirt-qemu'. -- John Doe

1 1

Constant error logging
by Digimer 17 Aug '20

17 Aug '20

Hi all, I've got a (pacemaker + drbd) cluster running VMs. When the VM is running on another node, the local storage is Secondary, meaning that it can't be read or otherwise touched. As such, virsh thinks there is no storage and constantly prints errors like: ==== Aug 14 19:25:24 mk-a02n01.digimer.ca libvirtd[1655]: 2020-08-14 23:25:24.812+0000: 1679: error : qemuOpenFileAs:3119 : Failed to open file '/dev/drbd/by-res/srv01-sql/0': No such file or directory Aug 14 19:25:24 mk-a02n01.digimer.ca libvirtd[1655]: 2020-08-14 23:25:24.812+0000: 1679: error : qemuOpenFileAs:3119 : Failed to open file '/dev/drbd/by-res/srv01-sql/1': No such file or directory Aug 14 19:25:24 mk-a02n01.digimer.ca libvirtd[1655]: 2020-08-14 23:25:24.812+0000: 1679: error : qemuOpenFileAs:3119 : Failed to open file '/dev/drbd/by-res/srv07-el6/0': Wrong medium type === To the system logs (journald), about once every second or two. Is there any way to tell libvirtd not to complain about this? Thanks -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould

2 1

Problem with xen config
by Christoph 15 Aug '20

15 Aug '20

Hi all, I 've such a config on xen (4.14): name = "marax.chao5.int" uuid = "e0de3cb7-3937-417d-8d63-b0993b377b6a" maxmem = 16384 memory = 16384 kernel = '/usr/lib64/xen/boot/hvmloader' vcpus = 16 rtc_timeoffset = 0 localtime = 1 on_poweroff = "destroy" on_reboot = "restart" on_crash = "restart" vif = [ "mac=00:16:3e:05:01:10,bridge=xenbr5,script=vif-bridge,model=e1000" ] parallel = "none" serial = "none" type = "hvm" loader = "/usr/lib64/xen/boot/hvmloader" disk = [ "phy:/dev/mapper/marax_c,hda,rw", "phy:/dev/vg_lilith/lv_marax_d,hdb,rw" ] max_grant_frames = "128" pci = [ "01:00.0", "01:00.1", "01:00.2", "01:00.3", "00:1f.3", "05:00.0" ] pci_permissive = 1 keymap = "de" vnclisten="0.0.0.0" pci_power_mgmt=1 xen_platform_pci=1 pci_msitranslate=1 viridian=1 hpet=1 acpi=1 apic=1 pae=1 I want to convert it to a xml format for use it with libvirt/virsh but I get an error: "error: An error occurred, but the cause is unknown" If I comment out the line with pci = [ "01:00.0", "01:00.1", "01:00.2", "01:00.3", "00:1f.3", "05:00.0" ] then it works but I dont see the settings like xen_platform_pci=1 pci_msitranslate=1 pci_permissive = 1 max_grant_frames = "128 and the pci passthrough devices are not there... Does libvirt doesnt support such a config? -- ------ Greetz

3 9

pass-though Intel gpu int o a vm
by daggs 13 Aug '20

13 Aug '20

Greetings, I have a server that run a vm with several nic pcie cards as pass-though. the board has an intel gpu and the hdmi output is not used. I thought of booting up another vm and pass the gpu into it. I found several discussions on the subject stating that it isn't possible but all of them are several years old. I wonder if that has changed? is there a way to pt the vga to a vm? Thanks, Dagg.

3 5