- Users - libvirt List Archives

Re: Issue
by Samuel Thampy 12 Mar '21

12 Mar '21

Is there any additional info anyone who needs? ________________________________ From: Samuel Thampy <samjiks(a)hotmail.com> Sent: 03 March 2021 22:11 To: libvirt-users(a)redhat.com <libvirt-users(a)redhat.com> Subject: Issue I have a storage backend san disks, I have five servers which is KVM hosted guests. I am looking to clone an image of a guest machine, so I can use a new VM form the image on a different server.. I am not sure what the best way. But when I do that, I get a memory error, and need to xfs_repair and damages the original machine. My steps are One vm machine on the same server * dd create original vm to template.img * lvcreate -L 9G -n <vol-name> <pool> * dd template.img to /dev/<pool>/<vol-name> * virsh define <vm_name>.xml another vm machine on the secondary server * copy template.img to another server * lvcreate -L 9G -n <vol-name> <pool> * dd if=template.img of=/dev/<pool>/<vol-name> * virsh define <vm_name>.xml May I know the steps are missing something because both are getting corrupted. Thanks for your help. Regards Samuel Thampy

2 1

Bridge and VLAN trunk
by Gionatan Danti 11 Mar '21

11 Mar '21

Dear list, I am a question about the best use of bridge, vlan trunk and libvirt. When dealing with virtual machies bound to specific vlan, I generally use a straightforward approach: eth -> bridge -> vm (for untagged traffic) eth -> eth.10 -> bridge -> vm eth -> eth.nn -> bridge -> vm Now I am faced with enabling vlan trunking for a specific vm (a virtualized firewall). The simpler approach would be: eth -> bridge -> vm (for the vm needing trunk) eth -> bridge.10 -> macvtap -> vm The issue with the above method is that any VM on the main untagged vlan needs to be bound to the "plain" bridge, having access to *any* traffic of *any* other vlan. If this is ok (and the desired behavior) for the firewall, it is clearly wrong for the other VMs. A simple fix would be to use ebtables to block/drop vlan tagged traffic on the main bridge for any virtual adapter except the required one (ie: the firewall virtual interface). It works, but I wounder if other preferred approaches exists. For example, I tested another more convoluted setup: eth -> bridge -> firewall vm eth -> bridge.10 -> macvtap -> vm eth -> bridge -> veth0 -> veth1 -> other bridge with vlan filtering on -> vm The last row show the use of veth virtual interface, configurable via ip link. Enabling vlan filtering on the second bridge (rather than on the first) is to keep vlan filtering simple: rathen than enabling all required vlan on the first bridge, I simply enable only untagged traffic on the second one. Does libvirt support bridge vlan filtering natively? Reading the docs, it seems to the supported only on OpenVSwitch or SRV-IO based adapter. Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

2 2

Snapshot operation aborted and volume usage
by Liran Rotenberg 11 Mar '21

11 Mar '21

We recently had this bug[1]. The thought that came from it is the handling of error code after running virDomainSnapshotCreateXML, we encountered VIR_ERR_OPERATION_ABORTED(78). Apparently, the new volume is in use. Are there cases where this will happen and the new volume won't appear in the volumes chain? Can we detect / know when? Thinking aloud, if we can detect such cases we can prevent rolling back by reporting it back from VDSM to ovirt. Or, if it can't be detected to go on the safe side in order to save data corruption and prevent the rollback as well. Currently, in ovirt, if the job is aborted, we will look into the chain to decide whether to rollback or not. Regards, Liran. [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1916122

2 2

virsh snapshot-create-as with quiesce option times out
by David Wells 05 Mar '21

05 Mar '21

Hi all! I'm still working on the live backup of a couple vm's and what happens most of the times es that when I execute virsh with the snapshot-create-as with the --quiesce option the process finishes with an error that reads > error: Timed out during operation: cannot acquire state change lock I tried turning up the debug level but found nothing that appears to be of interest and I can connect successfully to the qemu guest agent since the following command returns the correct value > sudo /usr/sbin/virsh domfsinfo slackware-current > Mountpoint Name Type Target > ------------------------------------------------------------------- > / vda1 ext4 vda I also tried running the qemu-ga agent on the guest with debugging enabled and when I issue this last command I can see the agent talking to the host but when I issue the quiesce option the guest agent shows nothing at all. Is this by any chance a known bug? Is there something obvious I'm missing? What else can I provide to help debug this issue? Thanks in advance! Best regards, David Wells.

2 1

virtio model type and netfilter masquerade not being applied
by Brad Jennings 04 Mar '21

04 Mar '21

Quick question for anyone in the know, I have a fairly basic setup (at least I think it is?) with an openvswitch, and the br0 port has an IP assigned in the same subnet as the VM to act as a gateway. |------ovs-------| eno2 <-- |--br0 | |--vnet0 - VM | |------------------| I would like the VM (vnet0) to use br0 as a gateway which local connectivity wise seems fine but the internet is a bit odd. I can ping for example 1.1.1.1 dns without any issues but anying udp/tcp is a no go. I checked the physical hosts interface(eno2) and br0 to find that the VM's packets were successfully heading to br0 but when leaving the physical host(eno2) the tcp/udp packets weren't being masqueraded. The rule is pretty straightforward and to test I plugged another device into the eno1 afxdp port and had no connectivity issues and packets were being masqueraded fine. I tried to set trustGuestRxFilters='yes' but that didn't work and the same state remained, the only thing that worked was using the "rtl8139" model type. I always remember using 'virtio' in the past and I must be missing something crucial in the somewhat lengthy libvirt documentation. Would be super helpful if someone can shed some light on this ? and possibly if I should be using virtio or the realtek driver ? Thanks ! (config below) Iptables: sudo iptables -t nat -A POSTROUTING -o eno2 -j MASQUERADE ovs-vsctl show ec13c3e2-6159-4019-984e-36cc90c59075 Bridge br0 fail_mode: standalone datapath_type: netdev Port vnet0 Interface vnet0 Port eno1 Interface eno1 type: afxdp Port br0 Interface br0 type: internal instance domain xml <interface type='bridge'> <mac address='52:54:00:77:fc:70'/> <source bridge='br0'/> <virtualport type='openvswitch'> <parameters interfaceid='2124ef39-e244-434c-8339-d2aa04d0d888'/> </virtualport> <model type='virtio'/> #rtl8139 works. <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> </interface>

1 0

Some confusion about lsilogic controller
by xingchaochao 03 Mar '21

03 Mar '21

Hello, I have been confused by such a phenomenon recently. Libvirt is the master branch , and the VM is centos8.2(kernel is 4.18.0-193.el8.aarch64). When I hot-plug the scsi disk for a virtual machine without a virtio-scsi controller, libvirt will automatically generate an lsilogic controller for the scsi disk. <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none' io='native'/> <source file='/Images/xcc/tmp.img'/> <backingStore/> <target dev='sdt' bus='scsi'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> linux-upcHIq:/Images/xcc # virsh list Id Name State ---------------------- 12 g1 running linux-upcHIq:/Images/xcc # virsh attach-device g1 disk.xml Device attached successfully linux-upcHIq:/Images/xcc # virsh dumpxml g1 | grep scsi <target dev='sdt' bus='scsi'/> <alias name='scsi0-0-0'/> <controller type='scsi' index='0' model='lsilogic'> <alias name='scsi0'/> But this scsi disk cannot be found through the lsblk command inside the virtual machine. [root@localhost ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 252:0 0 20G 0 disk ├─vda1 252:1 0 600M 0 part /boot/efi ├─vda2 252:2 0 1G 0 part /boot └─vda3 252:3 0 18.4G 0 part ├─cl-root 253:0 0 16.4G 0 lvm / └─cl-swap 253:1 0 2G 0 lvm [SWAP] After hot unplugging the scsi disk, I performed the hot unplug operation of the lsilogic controller. libvirt shows "Device detached successfully", but in fact, the lsilogic controller is not removed from the live XML and persistent XML. Through "virsh dumpxml vmname" and "virsh edit vmname", I can see <controller type='scsi' index='0' model='lsilogic'> is always there. linux-upcHIq:/Images/xcc # virsh detach-device g1 disk.xml Device detached successfully linux-upcHIq:/Images/xcc # virsh dumpxml g1 | grep scsi <controller type='scsi' index='0' model='lsilogic'> <alias name='scsi0'/> linux-upcHIq:/Images/xcc # linux-upcHIq:/Images/xcc # cat lsi.xml <controller type='scsi' index='0' model='lsilogic'> <alias name='scsi0'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x05' function='0x0'/> </controller> linux-upcHIq:/Images/xcc # virsh detach-device g1 lsi.xml Device detached successfully linux-upcHIq:/Images/xcc # virsh dumpxml g1 | grep scsi <controller type='scsi' index='0' model='lsilogic'> <alias name='scsi0'/> I am confused, why libvirt chooses to generate an lsilogic controller for the scsi disk when there is no scsi controller, instead of directly reporting an error and exiting the hot plug operation. After all, the scsi disk based on the lsilogic controller is not perceived inside the virtual machine, and lsilogic will remain in the XML file of the virtual machine.

2 2

Live backups create-snapshot-as and memspec
by David Wells 01 Mar '21

01 Mar '21

Hi all! I've been using libvirt for some time and until now I have treated backups of virtual computers as if they where physical computers installing the backup client on the guest. I am now however facing the need to backup a couple a couple of guest at the host level so I've been trying to catch up by reading, googling and by trial and error too. Up to now I've been able to backup a live machine whith a command like the following > virsh snapshot-create-as --domain test --name backup --atomic > --diskspec vda,snapshot=external --disk-only This command creates a file test.backup and in the meantime I can backup the original test.qcow2 but for what I saw this disk image is in a "dirty" state, as if the machine I could restore from this file had been turned off whithout a proper shutdown. I know that I can later restore the machine to its original state by issuing a command like this > virsh blockcommit --domain test vda --active --pivot > virsh snapshot-delete test --metadata backup I have seen that it is possible to create the snapshot using a memspec parameter which would make the backup of the guest as if it where in a clean state, however I haven't found the equivalent of the blockcommit for the memory file, in a sort of speak, to be able to restore the guest to it's original state. Thank you very much! Best regards. David Wells.

3 4

Upgrading the host
by Marcin Struzak 19 Feb '21

19 Feb '21

Hello-- I'm planning on upgrading the OS on my host machine by doing manual re-installation, which will trigger an upgrade of libvirt from 1.1.3 to 6.1.0. Hardware, storage volumes & paths, etc., will be the same after upgrade. I was going to redefine & start everything from .xml files, starting with the network virsh net-define <network XML> virsh net-autostart <network> virsh net-start <network> and following with all the guests virsh define <domain XML> virsh autostart <domain> virsh start <domain> I have the following questions: 1. Do I need to generate XML dumps manually, or is it ok to use the ones from host's /etc/libvirt? 2. Do I need to define & start storage pools & volumes explicitly, or will they be picked up from domain definitions? 3. Anything else I should worry about or prepare ahead? --Marcin

5 9

Static DHCP lease never distributed
by Brooks Swinnerton 10 Feb '21

10 Feb '21

Hi there, I have a libvirt network defined as so: <network> <name>customers</name> <bridge name='customers' macTableManager='libvirt' /> <port isolated='yes' /> <dns enable='no' /> <ip address='10.0.0.1' prefix='24'> <dhcp> <host mac='02:99:92:43:eb:b8' name='dhcp-test' ip='10.0.0.10' /> </dhcp> </ip> </network> And that network is attached to a virtual machine: <interface type='network'> <mac address='02:99:92:43:eb:b8'/> <source network='customers'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> But for some reason when the domain starts, it never gets an address. If I tcpdump the bridge that was created by the network I can see it sending out discover packets, but dnsmasq never seems to respond: 01:26:25.039987 02:99:92:43:eb:b8 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 342: (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:99:92:43:eb:b8, length 300, xid 0xc7283f76, secs 228, Flags [none] Client-Ethernet-Address 02:99:92:43:eb:b8 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Client-ID Option 61, length 7: ether 02:99:92:43:eb:b8 MSZ Option 57, length 2: 576 Parameter-Request Option 55, length 7: Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname Domain-Name, BR, NTP Vendor-Class Option 60, length 3: "d-i" Despite that appearing to be the correct mac address. Looking in /var/lib/libvirt/dnsmasq/customers.hostsfile, it's returning what I would expect to be there: 02:99:92:43:eb:b8,10.0.0.10,dhcp-test If I add a <range> stanza to the configuration, that does appear to work, so it seems this is only related to static addresses. This is libvirtd 5.8.0.

2 1

Re: [ovirt-devel] Issue: Device path changed after adding disks to guest VM
by Nir Soffer 02 Feb '21

02 Feb '21

On Wed, Dec 2, 2020 at 4:57 PM Joy Li <joooy.li(a)gmail.com> wrote: > > Thanks a lot Nir! Good to know that oVirt cannot guarantee the disk names so that I don't need to spend more time trying to enable such a feature. > > I can always reproduce the problem via my application, basically, the procedure is as following: > > 1. create a VM Which guest OS? Can you share the guest disk image or ISO image used to instal it? > 2. add disks to the VM (e.g.: disk names: disk1, disk3) > 3. check the disk mappings via `virsh domblklist ` Please shared the libvirt domain xml (virsh dumpxml vm-name) > 4. add another disk (let's say, disk2, give a name alphabetically before some existing disks) Add disk while the vm is running (hotplug)? > 5. shutdown the VM via hypervisor and start it again (reboot won't work) What do you mean by "reboot does not work?" > 6. `virsh domblklist` again, then you might see the problem I mentioned before Mapping is different compared with state before the reboot? > There are no virtio devices inside /dev/disk/by-id/xxx of my guest VM. Maybe you don't have systemd-udev installed? The links in /dev/disk/... are created by udev during startup, and when detecting a new disk. > And I just noticed that the disks mapping information given by hypervisor (from VM configuration or virsh command) is different from the reality inside the VM. The disk name inside the VM was actually not changed. > > So now my issue is that given a disk name (/dev/vdb) of a VM, how can I get its wwid? Before I got it from the hypervisor, but now the hypervisor's information is not reliable, and since the disk is unformatted, I cannot use UUID. You can use: # udevadm info /dev/vda P: /devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio4/block/vda N: vda L: 0 S: disk/by-path/pci-0000:06:00.0 S: disk/by-id/virtio-b97e68b2-87ea-45ca-9 S: disk/by-path/virtio-pci-0000:06:00.0 E: DEVPATH=/devices/pci0000:00/0000:00:02.5/0000:06:00.0/virtio4/block/vda E: DEVNAME=/dev/vda E: DEVTYPE=disk E: MAJOR=252 E: MINOR=0 E: SUBSYSTEM=block E: USEC_INITIALIZED=10518442 E: ID_SERIAL=b97e68b2-87ea-45ca-9 E: ID_PATH=pci-0000:06:00.0 E: ID_PATH_TAG=pci-0000_06_00_0 E: DEVLINKS=/dev/disk/by-path/pci-0000:06:00.0 /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 /dev/disk/by-path/virtio-pci-0000:06:00.0 E: TAGS=:systemd: I tried to reproduce you issue 4.4.5 development build: Starting vm with 2 direct LUN disks: # virsh -r dumpxml disk-mapping ... <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='qcow2' cache='none' error_policy='stop' io='native' discard='unmap'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5' index='3'> <seclabel model='dac' relabel='no'/> </source> <backingStore type='block' index='5'> <format type='qcow2'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/ad891316-b80a-4ab5-895b-400108bd0ca1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> </backingStore> <target dev='sda' bus='scsi'/> <serial>40018b33-2b11-4d10-82e4-604a5b135fb2</serial> <boot order='1'/> <alias name='ua-40018b33-2b11-4d10-82e4-604a5b135fb2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/3600140594af345ed76d42058f2b1a454' index='2'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <serial>b97e68b2-87ea-45ca-94fb-277d5b30baa2</serial> <alias name='ua-b97e68b2-87ea-45ca-94fb-277d5b30baa2'/> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5' index='1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vdb' bus='virtio'/> <serial>d9a29187-f492-4a0d-aea2-7d5216c957d7</serial> <alias name='ua-d9a29187-f492-4a0d-aea2-7d5216c957d7'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> ... # virsh -r domblklist disk-mapping Target Source --------------------------------------------------------------------------------------------------------------------------------------------------------------- sdc - sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 vda /dev/mapper/3600140594af345ed76d42058f2b1a454 vdb /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 In guest: # ls -lh /dev/disk/by-id/virtio-* lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb NOTE: "d9a29187-f492-4a0d-a" are the first characters of the disk id: "d9a29187-f492-4a0d-aea2-7d5216c957d7" seen in oVirt: https://my-engine/ovirt-engine/webadmin/?locale=en_US#disks-general;id=d9a2… Adding another disk in sorted in the middle (while the vm is running): # virsh -r dumpxml disk-mapping ... <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='qcow2' cache='none' error_policy='stop' io='native' discard='unmap'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5' index='3'> <seclabel model='dac' relabel='no'/> </source> <backingStore type='block' index='5'> <format type='qcow2'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/ad891316-b80a-4ab5-895b-400108bd0ca1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> </backingStore> <target dev='sda' bus='scsi'/> <serial>40018b33-2b11-4d10-82e4-604a5b135fb2</serial> <boot order='1'/> <alias name='ua-40018b33-2b11-4d10-82e4-604a5b135fb2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/3600140594af345ed76d42058f2b1a454' index='2'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <serial>b97e68b2-87ea-45ca-94fb-277d5b30baa2</serial> <alias name='ua-b97e68b2-87ea-45ca-94fb-277d5b30baa2'/> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5' index='1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vdb' bus='virtio'/> <serial>d9a29187-f492-4a0d-aea2-7d5216c957d7</serial> <alias name='ua-d9a29187-f492-4a0d-aea2-7d5216c957d7'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/36001405b4d0c0b7544d47438b21296ef' index='7'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vdc' bus='virtio'/> <serial>e801c2e4-dc2e-4c53-b17b-bf6de99f16ed</serial> <alias name='ua-e801c2e4-dc2e-4c53-b17b-bf6de99f16ed'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </disk> ... # virsh -r domblklist disk-mapping Target Source --------------------------------------------------------------------------------------------------------------------------------------------------------------- sdc - sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 vda /dev/mapper/3600140594af345ed76d42058f2b1a454 vdb /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 vdc /dev/mapper/36001405b4d0c0b7544d47438b21296ef In the guest: # ls -lh /dev/disk/by-id/virtio-* lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb lrwxrwxrwx. 1 root root 9 Jan 6 09:51 /dev/disk/by-id/virtio-e801c2e4-dc2e-4c53-b -> ../../vdc Shutdown VM and start it again # virsh -r dumpxml disk-mapping ... <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='qcow2' cache='none' error_policy='stop' io='native' discard='unmap'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5' index='4'> <seclabel model='dac' relabel='no'/> </source> <backingStore type='block' index='6'> <format type='qcow2'/> <source dev='/rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/ad891316-b80a-4ab5-895b-400108bd0ca1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> </backingStore> <target dev='sda' bus='scsi'/> <serial>40018b33-2b11-4d10-82e4-604a5b135fb2</serial> <boot order='1'/> <alias name='ua-40018b33-2b11-4d10-82e4-604a5b135fb2'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/3600140594af345ed76d42058f2b1a454' index='3'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vda' bus='virtio'/> <serial>b97e68b2-87ea-45ca-94fb-277d5b30baa2</serial> <alias name='ua-b97e68b2-87ea-45ca-94fb-277d5b30baa2'/> <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/36001405b4d0c0b7544d47438b21296ef' index='2'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vdb' bus='virtio'/> <serial>e801c2e4-dc2e-4c53-b17b-bf6de99f16ed</serial> <alias name='ua-e801c2e4-dc2e-4c53-b17b-bf6de99f16ed'/> <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/> </disk> <disk type='block' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='native' iothread='1'/> <source dev='/dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5' index='1'> <seclabel model='dac' relabel='no'/> </source> <backingStore/> <target dev='vdc' bus='virtio'/> <serial>d9a29187-f492-4a0d-aea2-7d5216c957d7</serial> <alias name='ua-d9a29187-f492-4a0d-aea2-7d5216c957d7'/> <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/> </disk> ... # virsh -r domblklist disk-mapping Target Source --------------------------------------------------------------------------------------------------------------------------------------------------------------- sdc - sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 vda /dev/mapper/3600140594af345ed76d42058f2b1a454 vdb /dev/mapper/36001405b4d0c0b7544d47438b21296ef vdc /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 In the guest: # ls -lh /dev/disk/by-id/virtio-* lrwxrwxrwx. 1 root root 9 Jan 6 09:55 /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda lrwxrwxrwx. 1 root root 9 Jan 6 09:55 /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb lrwxrwxrwx. 1 root root 9 Jan 6 09:55 /dev/disk/by-id/virtio-e801c2e4-dc2e-4c53-b -> ../../vdc Comparing to state before reboot: # virsh -r domblklist disk-mapping Target Source --------------------------------------------------------------------------------------------------------------------------------------------------------------- sdc - sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 vda /dev/mapper/3600140594af345ed76d42058f2b1a454 vdb /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 vdc /dev/mapper/36001405b4d0c0b7544d47438b21296ef # ls -lh /dev/disk/by-id/virtio-* lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda lrwxrwxrwx. 1 root root 9 Jan 6 09:42 /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb lrwxrwxrwx. 1 root root 9 Jan 6 09:51 /dev/disk/by-id/virtio-e801c2e4-dc2e-4c53-b -> ../../vdc In the guest disks are mapped to the same device name. It looks like libivrt domblklist is not correct - vdb and vdc are switched. Peter, this expected? Nir > > Joy > > On Wed, Dec 2, 2020 at 1:28 PM Nir Soffer <nsoffer(a)redhat.com> wrote: >> >> On Wed, Dec 2, 2020 at 10:27 AM Joy Li <joooy.li(a)gmail.com> wrote: >> > >> > Hi All, >> > >> > I'm facing the problem that after adding disks to guest VM, the device target path changed (My ovirt version is 4.3). For example: >> > >> > Before adding a disk: >> > >> > virsh # domblklist <vmname> >> > Target Source >> > --------------------------------------------------------- >> > hdc - >> > vda /dev/mapper/3600a09803830386546244a546d494f53 >> > vdb /dev/mapper/3600a09803830386546244a546d494f54 >> > vdc /dev/mapper/3600a09803830386546244a546d494f55 >> > vdd /dev/mapper/3600a09803830386546244a546d494f56 >> > vde /dev/mapper/3600a09803830386546244a546d494f57 >> > vdf /dev/mapper/3600a09803830386546244a546d494f58 >> > >> > After adding a disk, and then shutdown and start the VM: >> > >> > virsh # domblklist <vmname> >> > Target Source >> > --------------------------------------------------------- >> > hdc - >> > vda /dev/mapper/3600a09803830386546244a546d494f53 >> > vdb /dev/mapper/3600a09803830386546244a546d494f54 >> > vdc /dev/mapper/3600a09803830386546244a546d494f6c >> > vdd /dev/mapper/3600a09803830386546244a546d494f55 >> > vde /dev/mapper/3600a09803830386546244a546d494f56 >> > vdf /dev/mapper/3600a09803830386546244a546d494f57 >> > vdg /dev/mapper/3600a09803830386546244a546d494f58 >> > >> > The devices' multipath doesn't map to the same target path as before, so in my VM the /dev/vdc doesn't point to the old /dev/mapper/3600a09803830386546244a546d494f55 anymore. >> > >> > Anybody knows how can I make the device path mapping fixed without being changed after adding or removing disks. >> >> Device nodes are not stable, and oVirt cannot guarantee that you will >> get the same >> node in the guest in all runs. >> >> You should use /dev/disk/by-id/xxx links to located devices, and blkid to create >> fstab mounts that do not depend on node names. >> >> Regardless, oVirt try to keep devices stable as possible. Do you know >> how to reproduce >> this issue reliably? >> >> Nir >>

6 6