- Users - libvirt List Archives

Re: [libvirt-users] [Fwd: UEFI NVRAM variables]
by Laszlo Ersek 13 Jul '17

13 Jul '17

Hi, thank you, Andrea, for the forward. On 07/13/17 10:19, Andrea Bolognani wrote: > -------- Forwarded Message -------- > From: Thomas Meyer <thomas(a)m3y3r.de> > To: libvirt-users(a)redhat.com > Subject: [libvirt-users] UEFI NVRAM variables > Date: Wed, 12 Jul 2017 07:49:43 +0200 > >> Hi, >> >> how do I set the BootOrder variable in an NVRAM file for UEFI boot? This is one of the most frequently asked questions about OVMF. Short version: - alternative 1: use neither <boot dev="..."/> nor <boot index="..."/> in your domain XML, and manage the BootOrder and Boot#### variables entirely within the guest (using the OVMF Setup TUI and/or the efibootmgr utility). - alternative 2: use <boot index="..."/> in your domain XML, and it will mostly do what you want. Mid-size version: * Never use "-boot order=x,y,z" with OVMF, on the QEMU command line, because the guest-visible effects of this QEMU option are fully useless to UEFI guest firmware. For more details, please refer to <https://bugzilla.redhat.com/show_bug.cgi?id=1323085#c10>. Translating the above to libvirt, never use <boot dev='...'/> elements in your domain XML with UEFI guest firmware. (They are also discouraged by the libvirt documentation, for their non-unique meaning.) * The "-device xxx,bootindex=N" properties on the QEMU command line are usable and useful to UEFI guest firmware. (This is what libvirt's <boot order='N'/> per-device elements map to, which are also what the libvirt docs recommend.) However, the expressive power of these properties is still lesser than the expressive power of the UEFI boot options, therefore the best that OVMF can do here is a kind of auto-generation of all possible boot options, and filtering and reordering them based on translation and prefix-matching. For that, the long version: please refer to the OVMF whitepaper from a few years back, where I described this algorithm in excruciating detail: http://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt Search it for "Platform-specific boot policy". >> Is there a way to configure a domain with the bios qemu option instead of pflash? If you want to boot a guest with OVMF as firmware, but map the firmware as ROM, not as pflash, you can do it. For that, you have to use the unified OVMF image (called "OVMF.fd" most commonly), not the split image (called "OVMF_CODE.fd" and other variants). The <loader> element should look like: <loader>/usr/share/OVMF/OVMF.fd</loader> and the <nvram> element should not be present. This will map to "-bios OVMF.fd" on the QEMU command line. Be *strongly* warned though that by this, you are throwing away persistent non-volatile variables, and that the fake variable support that OVMF falls back to does not -- cannot -- conform to the UEFI spec. You will encounter obscure mis-behavior with variables. (One of the obvious glitches will be that Secure Boot settings will not survive a reboot.) >> >> Are there a tool available to manipulate the UEFI variables from the outside? No, there isn't, and as far as it's up to me, there won't be. (This is another of the most frequently asked questions.) The reason for this is that the representation of the variable store on the host filesystem (in /var/lib/libvirt/qemu/nvram/<guest>_VARS.fd) is the functional composition of three layers in the edk2 driver stack: - platform-specific pflash driver / layout - generic fault tolerant write driver (implementing a kind of journaled system, where you can "lose power" in the middle of a write operation, and your varstore won't be corrupted) - generic variable driver None of these are standardized in UEFI, and even restricting ourselves to edk2 (the reference implementation of PI/UEFI), only the last two layers are documented in Intel whitepapers. Regarding examples for the first layer, OVMF itself can be built with two distinct (and incompatible) variable store layouts, and the ArmVirtQemu varstore layout is again different. In other words, a host-side utility to parse and modify these files would have to reimplement the functionality of the above three layers, deal with, and clean up, interrupted writes (see the fault tolerant writer level), and generally keep chasing any pertinent changes in the edk2 codebase. (Upstream edk2 does not guarantee backwards compatibility with existent variable stores, and it does not provide conversion tools either.) If someone feels up to writing and continuously maintaining such a host-side utility, I won't try to prevent them. But, I won't be that person. Another approach would be to replace the above-mentioned variable driver stack in OVMF, with a custom "paravirt" variable driver implementation. This would be a large project, and even if I were sympathetic to it (which I'm not), I'm pretty sure it wouldn't be welcome to upstream edk2 (because competing, parallel implementations for the same thing are a big maintenance burden). I vaguely suspect that a giga-company whose name starts with G and refers to a very large number has done this, targeting their own (unreleased / proprietary) VMM, but to my knowledge their variable driver implementation is similarly unreleased / proprietary. Either way, massaging guest-produced data from the host side, be the data "UEFI variable content" or "disk image content", is also a security question. So the only one really feasible approach here would be a libguestfs-like tool that - booted a guest on top of the variable store, - implemented a kind of "firmware guest agent" that manipulated the variables "from the inside", - and used custom commands over virtio-serial, between host and guest. As I said, a large project. Thanks, Laszlo

1 1

[libvirt-users] VM freezes while insmod virtio_net.ko
by jason 13 Jul '17

13 Jul '17

Hi all, I encountered a tricky problem. I am using ovs+DPDK on the host side to build the virtual network, so on the VM side , the virtio_net.ko module is used to drive the nic. But VM always freezes while insmod virtio_net.ko (I found virtio_net.ko is located in initrams.img by defualt, so I had to delete all virtio modules from initramfs.img and insmod then one by one after system is loaded to confirm that). After such freeze, virsh reset can not recover it due to cannot acquire state change lock (held by remoteDispatchDomainReset) so I have to use virsh destroy instead. On the ovs-vswitchd side, the last log I can see is: VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK VHOST_CONFIG: vring kick idx:0 file:59 VHOST_CONFIG: virtio is not ready for processing. VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:0 file:60 VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK VHOST_CONFIG: vring kick idx:1 file:46 VHOST_CONFIG: virtio is not ready for processing. VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL VHOST_CONFIG: vring call idx:1 file:61 VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE VHOST_CONFIG: set queue enable: 1 to qp idx: 0 VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE VHOST_CONFIG: set queue enable: 1 to qp idx: 1 VHOST_CONFIG: read message VHOST_USER_SET_FEATURES Software version I used are: guest kernel: 3.10.0-514.26.2.el7.x86_64 host kernel: 3.10.0-514.26.2.el7.x86_64 host libvirt: 2.0.0-10.el7_3.9.x86_64 host qemu: 2.6.0-28.el7.10.1 host openvswitch: git 2.7.0 host DPDK: git 17.02.0 Please help to have a look at this issue. -- B.R., Jason

1 1

Re: [libvirt-users] Is there still no easier way to shrink a VM image?
by Leroy Tennison 12 Jul '17

12 Jul '17

Thanks for letting me know I'm not making myself clear, let me try again with a few more specifics, These are Windows VMs with three disk images for C:, D: and L:. To simplify I'll focus on the image for C: which had grown to 77G physical size on the hypervisor's ssd (I just double-checked that with 'ls -al' because qemu-img below says 70G, this image had two snapshots at one time which may be the reason for the discrepancy). qemu-img info reports: file format: qcow2 virtual size: 100G (107374182400 bytes) disk size: 70G cluster_size: 65536 Format specific information: compat: 0.10 I used Windows Server 2012r2 "Optimize" (defrag) and then reduced the C: partition to about 67G in Disk Administrator leaving the remaining 33G as unallocated. Afterward I tried a web reference technique and used Sysinternals SDelete to zero the free space then used 'qemu-img convert -O qcow2 <original.qcow2> <new.qcow2>' to produce a physical image size of 29G. qemu-img info reports on "new.qcow2": file format: qcow2 virtual size: 100G (107374182400 bytes) disk size: 29G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false The issue is that the virtual size is still 100G. I don't have the physical disk space to allow approximately 12 images (all configured for a virtual size of 100G) to grow to that size (6 VMs with C: and D: on a 1TB ssd, L: is on an hdd which isn't an issue). I need to change that virtual size number for this image to 67 or 68G. On the D: images I can drop to approximately 40G for an aggregate total of about 650G for all six VMs - well within the 875G physical size limitation that the ssd provides after overhead. That's a long background to why I need to change the virtual disk size. Again, any alternatives would be much appreciated. -----Original Message----- From: Martin Kletzander <mkletzan(a)redhat.com> To: Leroy Tennison <leroy.tennison(a)verizon.net> Cc: libvirt-users <libvirt-users(a)redhat.com> Sent: Tue, Jul 11, 2017 2:51 am Subject: Re: [libvirt-users] Is there still no easier way to shrink a VM image? On Tue, Jul 11, 2017 at 12:34:31AM -0500, Leroy Tennison wrote: >I have numerous qcow2 images which need to be reduced in size and have >their maximum size (virtual size) reduced. Physical disk space became >so low that VMs "auto-paused" themselves, I moved enough images to solve >the immediate problem but need to rectify the underlying issue. It >seems that qcow[2] files are grown in size such that the data inside of >them takes about 50-60% of the space (does anyone know the actual >algorithm or how to control it?). Given the total physical disk space >on the hypervisors, I need something more restrictive. > I don't get it. You have virtual size greater than the free space on the physical storage and instead of the VM finding out you want the guest OS to see it has no space at all? >Our hypervisors are a mix of Ubuntu 14 or 16 LTS (qemu-img 2.2 or 2.5). >After doing all the preparation (defragment, reduce OS partition size) >"qemu-img resize" reports that shrinking isn't supported yet. My web >research indicates that, to accomplish this, I have to: > > convert to raw > > shrink the image > > convert back to qcow[2] > > increase the image size to provide for some growth > >I'm hoping I've missed something in my research and that someone knows >an easier way. I don't feel constrained to qemu-img but this is a >production environment precluding consideration of experimental >software. Virt-resize, guestfish or any other reasonable option is fine >with me. Solutions or ideas? Thanks. > >_______________________________________________ >libvirt-users mailing list >libvirt-users(a)redhat.com >https://www.redhat.com/mailman/listinfo/libvirt-users

3 2

[libvirt-users] UEFI NVRAM variables
by Thomas Meyer 12 Jul '17

12 Jul '17

1 0

[libvirt-users] Is there still no easier way to shrink a VM image?
by Leroy Tennison 11 Jul '17

11 Jul '17

I have numerous qcow2 images which need to be reduced in size and have their maximum size (virtual size) reduced. Physical disk space became so low that VMs "auto-paused" themselves, I moved enough images to solve the immediate problem but need to rectify the underlying issue. It seems that qcow[2] files are grown in size such that the data inside of them takes about 50-60% of the space (does anyone know the actual algorithm or how to control it?). Given the total physical disk space on the hypervisors, I need something more restrictive. Our hypervisors are a mix of Ubuntu 14 or 16 LTS (qemu-img 2.2 or 2.5). After doing all the preparation (defragment, reduce OS partition size) "qemu-img resize" reports that shrinking isn't supported yet. My web research indicates that, to accomplish this, I have to: convert to raw shrink the image convert back to qcow[2] increase the image size to provide for some growth I'm hoping I've missed something in my research and that someone knows an easier way. I don't feel constrained to qemu-img but this is a production environment precluding consideration of experimental software. Virt-resize, guestfish or any other reasonable option is fine with me. Solutions or ideas? Thanks.

3 2

[libvirt-users] How to make gic_version=3 as defailt to qemu on arm64
by Vishnu Pajjuri 11 Jul '17

11 Jul '17

Hi I'm running Openstack which is installed by using devstack. But it is not launching VMs. >From command line with gic_version=3 option it is running. But openstack glance doesn't have any privilege to specify gic version. On my ARM64 board gicv2 is not supported, so i want to make gicv3 as default one to pass to qemu. Kindly suggest any specific version of vibvirt or patch such that libvirt should pass gicv3 as default one. Thanks in Advance -Vishnu.

1 0

[libvirt-users] Ubuntu 16.04 libvirt-guests.sh [6917] - running guests under URI address default: no running guests
by Jędrek Domański 11 Jul '17

11 Jul '17

Hi I have recently upgraded from Ubuntu 15.04 to 16.04 and now everytime I shutdown my PC I get a black screen where it says "libvirt-guests.sh [6917] - running guests under URI address default: no running guests" and the PC just hangs forever and will not shutdown. I need to manually kill it in order to shut it down which is a pain in the ass. I have done some investigation and here is what I've found out so far. I have uninstalled libvirt, kvm and qemu: sudo apt-get purge libvirt* kvm qemu* The problem has gone, however I noticed some error messages at boot saying something that "FAILED Qemu .... It flashes so quickly and I am not able to read what exactly it says. I installed libvirt again: sudo apt install qemu-kvm libvirt-bin and the problem is back! So is this script trying to shutdown my virtual machines? It looks like it does but since it does not find any it hangs? I don't have any machines, though. I used to have one, but I removed it. What is interesting, my PC hangs at shutdown randomly, that is sometimes it hangs for like 15 seconds and it shutsdown but in most cases it hangs forever. So this happens randomly. So, this investigation has brought me to you guys so please help me. I have attached a screen (sorry for polish error message but the translation is in the email title). I have also posted a thread on AskUbuntu on this: https://askubuntu.com/questions/925696/ubuntu-16-04- libvirt-guests-sh-6917-running-guests-under-uri-address-default Thank you, Jędrzej

3 22

[libvirt-users] Broken br0
by virt.persik＠9ox.net 10 Jul '17

10 Jul '17

Hello, I wanted to get an extra IP on my local NIC, so I ran `sudo ip addr add 192.168.1.130/24 dev enp4s0`. This didn't work as intended, so I thought I'd restart the Ubuntu system to have things back to how they were. Alas, this didn't happen. While the host still has network as usual, none of the VMs are able to get a DHCP lease from the router, or any connectivity at all. I can still see br0 with `ip addr`, but my guess is that it's misconfigured now. I tried adding a br1 with virt-manager (Edit > Connection details > Network interfaces > Add interface) and using that instead, but the app crashed halfway through; and I can't disable or remove the connection with the app now. I wonder if this isn't a routing issue, but I am not very proficient in that area to troubleshoot and fix. Where should I start? I'd like to use this as an opportunity to learn more about networking and VMs, so bear with me if I ask stupid/basic questions. Thanks!

3 2

[libvirt-users] Activation of org.freedesktop.machine1 timed out
by Masoud 10 Jul '17

10 Jul '17

Hi recently i have this error on our server.... how i can fix it? # systemctl status libvirtd ● libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2017-07-01 11:08:50 EDT; 2min 54s ago Docs: man:libvirtd(8) http://libvirt.org Main PID: 5233 (libvirtd) CGroup: /system.slice/libvirtd.service └─5233 /usr/sbin/libvirtd Jul 01 11:10:51 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:16 localhost.localdomain libvirtd[5233]: error from service: GetMachineByPID: Activation of org. Jul 01 11:11:42 localhost.localdomain libvirtd[5233]: Activation of org.freedesktop.machine1 timed out Jul 01 11:11:42 localhost.localdomain libvirtd[5233]: internal error: Failed to autostart VM 'v4375': Activat -- Chief Technology officer Masoud

2 1

[libvirt-users] virtual drive performance
by Dominik Psenner 07 Jul '17

07 Jul '17

Hi, I'm investigating a performance issue on a virtualized windows server host that is run on a ubuntu machine via libvirt/qemu. While the host can easily read/write on the raid drive with 100Mmb/s as observable with dd, the virtualized windows server running on that host is barely able to read/write with at most 8Mb/s and averages around 1.4Mb/s. This has grown to the extent that the virtualized host is often unresponsive and even unable to start up its services with system default timeouts. Any help to improve the situation is greatly appreciated. This is the configuration of the virtualized host: ~$ virsh dumpxml windows-server-2016-x64 <domain type='kvm' id='1'> <name>windows-server-2016-x64</name> <uuid>XXX</uuid> <memory unit='KiB'>2097152</memory> <currentMemory unit='KiB'>2097152</currentMemory> <vcpu placement='static'>2</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='x86_64' machine='pc-i440fx-xenial'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='custom' match='exact'> <model fallback='allow'>IvyBridge</model> <topology sockets='1' cores='2' threads='1'/> </cpu> <clock offset='localtime'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> <timer name='hypervclock' present='yes'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <pm> <suspend-to-mem enabled='no'/> <suspend-to-disk enabled='no'/> </pm> <devices> <emulator>/usr/bin/kvm-spice</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/data/virtuals/machines/windows-server-2016-x64/image.qcow2'/> <backingStore/> <target dev='hda' bus='ide'/> <alias name='ide0-0-0'/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/var/data/virtuals/machines/windows-server-2016-x64/dvd.iso'/> <backingStore/> <target dev='hdb' bus='ide'/> <readonly/> <alias name='ide0-0-1'/> <address type='drive' controller='0' bus='0' target='0' unit='1'/> </disk> <controller type='usb' index='0' model='ich9-ehci1'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/> </controller> <controller type='usb' index='0' model='ich9-uhci1'> <alias name='usb'/> <master startport='0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/> </controller> <controller type='usb' index='0' model='ich9-uhci2'> <alias name='usb'/> <master startport='2'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/> </controller> <controller type='usb' index='0' model='ich9-uhci3'> <alias name='usb'/> <master startport='4'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <interface type='bridge'> <mac address='52:54:00:0e:f2:23'/> <source bridge='br0'/> <target dev='vnet0'/> <model type='rtl8139'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/1'/> <target port='0'/> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/1'> <source path='/dev/pts/1'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <input type='tablet' bus='usb'> <alias name='input0'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'> <listen type='address' address='127.0.0.1'/> </graphics> <video> <model type='vga' vram='16384' heads='1'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='apparmor' relabel='yes'> <label>libvirt-XXX</label> <imagelabel>libvirt-XXX</imagelabel> </seclabel> </domain> Cheers, Dominik

6 17