July 2016 - Users - libvirt List Archives

[libvirt-users] How can I run command in containers on the host?
by John Y. 26 Jul '16

26 Jul '16

How can I run command in containers on the host? Just like the lxc command lxc-attach. I run : virsh -c lxc:/// lxc-enter-namespace fedora2 --noseclabel /bin/ls but get error: libvirt: error : Expected at least one file descriptor error: internal error: Child process (14930) unexpected exit status 125 Here is my libvirt.xml <domain type='lxc'> <name>fedora2</name> <memory>190000</memory> <vcpu>2</vcpu> <cputune> <shares>1000</shares> </cputune> <os> <features> <privnet/> <capabilities policy='allow'> </capabilities> </features> <type>exe</type> <init>/sbin/init</init> </os> <devices> <filesystem type="mount"> <source dir="/home/lxc-fedora"></source> <target dir="/"></target> </filesystem> <console type='pty'/> <interface type='bridge'> <mac address='52:54:00:e8:96:88'/> <source bridge='virbr0'/> <target dev='vnet0'/> <guest dev='eth0'/> </interface> </devices> </domain> version of libvirt: Compiled against library: libvirt 1.2.9 Using library: libvirt 1.2.9 Using API: QEMU 1.2.9 Running hypervisor: QEMU 2.1.2 Is there any mistake? Thanks, John

2 1

[libvirt-users] Blockpull behavior when interrupted
by Andrew Martin 25 Jul '16

25 Jul '16

Hello, I use snapshot-create-as followed by blockpull when creating external snapshots of VMs. This works well, however I am curious about the behavior of blockpull after an unexpected shutdown (or SIGKILL). If a blockpull is in progress and an unexpected power loss occurs, will the VM continue to reference the backing file for the parts of it that have not yet been copied? Or, will will the disk image no longer be usable? Thanks, Andrew

2 5

[libvirt-users] libvirtd terminates running instances after monitor's socket inode id is changed
by Gavin Nicholson 25 Jul '16

25 Jul '16

We accidentally moved /var/lib/libvirt/qemu to another directory which is on a different disk partition, then we moved it back; however, that caused changing the inode ids of the files inside /var/lib/libvirt/qemu including the socket files ( <instance>.monitor ) of the running VMs. The problem we are facing is that when libvirtd is restarted it terminates the running VMs with "error : qemuMonitorOpenUnix:315 : monitor socket did not show up.: Connection refused” We are trying to find a workaround so that when libvirtd is restarted it doesn’t stop the running VMs. We are running libvirt 0.9.13 Any help is appreciated. Thanks, Gavin

2 1

Re: [libvirt-users] [Qemu-devel] VFIO PCIe Extended Capabilities
by Spenser Gilliland 19 Jul '16

19 Jul '16

Hi Marcel, >> Indeed, if a device is attached to a PCI bus it makes no sense to advertise the extended configuration space. >> Can you please share the QEMU command line? Maybe is possible to make the device's bus PCIe in QEMU? Changing the following should make the tweak I proposed irrelevant. - device vfio-pci,host=03:00.0,id=hostdev0,bus=pci.2,addr=0x4 + device vfio-pci,host=03:00.0,id=hostdev0,bus=pcie.0,addr=0x4 Here's the full command for reference. 2016-07-19 18:42:22.110+0000: starting up libvirt version: 1.2.17, package: 13.el7 (CentOS BuildSystem <http://bugs.centos.org>, 2015-11-20-16:24:10, worker1.bsys.centos.org) qemu version: 2.3.0 (qemu-kvm-ev-2.3.0-31.el7_2.10.9) LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name instance-00000019 -S -machine pc-q35-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX,+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -m 8192 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid 8b6865eb-2118-430e-a7e0-2989696576b1 -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=13.1.0-1.el7,serial=a1068a93-24d1-4da2-8903-f9b8307fb0d8,uuid=8b6865eb-2118-430e-a7e0-2989696576b1,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-00000019/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 -drive file=/var/lib/nova/instances/8b6865eb-2118-430e-a7e0-2989696576b1/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x2,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/nova/instances/8b6865eb-2118-430e-a7e0-2989696576b1/disk.swap,if=none,id=drive-virtio-disk1,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x3,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=30 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:9c:cf:60,bus=pci.2,addr=0x1 -chardev file,id=charserial0,path=/var/lib/nova/instances/8b6865eb-2118-430e-a7e0-2989696576b1/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -vnc 0.0.0.0:0 -k en-us -device cirrus-vga,id=video0,bus=pcie.0,addr=0x1 -device vfio-pci,host=03:00.0,id=hostdev0,bus=pci.2,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x5 -msg timestamp=on char device redirected to /dev/pts/3 (label charserial1) It seems very reasonable to change the qemu command line. However, I'm using OpenStack Nova to launch the instance, which then uses libvirt, which finally uses qemu. So, the issues is very buried. > I think that any instance of a q35 machine where the assigned device is > placed on the conventional PCI bridge will create this scenario. It's > the default for attaching devices to a libvirt managed q35 VM AFAIK. > Yes, I discussed with Laine from libvirt the possibility to assign > devices to a PCIe port instead. Yes, I see this as the issue as well. The VM is being created by libvirt with the following xml. The problem is that the device is auto-assigned to the pci-bridge by default. <domain type="kvm"> <uuid>8b6865eb-2118-430e-a7e0-2989696576b1</uuid> <name>instance-00000019</name> <memory>8388608</memory> <vcpu>8</vcpu> ... <snip> ... <devices> ... <snip> ... <hostdev mode="subsystem" type="pci" managed="yes"> <source> <address bus="0x03" domain="0x0000" function="0x0" slot="0x00"/> </source> </hostdev> ... <snip> ... </devices> </domain> If I do a dumpxml I get the following <domain type="kvm"> <uuid>8b6865eb-2118-430e-a7e0-2989696576b1</uuid> <name>instance-00000019</name> <memory>8388608</memory> <vcpu>8</vcpu> ... <snip> ... <devices> ... <snip> ... <controller type='pci' index='0' model='pcie-root'> <alias name='pcie.0'/> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <alias name='pci.1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <alias name='pci.2'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> <interface type='bridge'> ... <snip> ... <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> </hostdev> </devices> </domain> If I manually change the domain as follows <hostdev mode='subsystem' type='pci' managed='yes'> <driver name='vfio'/> <source> <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </source> <alias name='hostdev0'/> - <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </hostdev> I can reimport and the device and it attaches successfully to the pcie-root hub. The problem is I need to either manually specify this in nova or update the behavior of libvirt to auto assign pcie devices to pcie busses. This would involve reading pci configuration space to check if the device is a pcie devices. Then attaching to the pcie-root, if possible. >>> >>> In fact, I've tried to fix this multiple times: >>> >>> https://lists.nongnu.org/archive/html/qemu-devel/2015-10/msg05384.html >>> https://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg02422.html >>> https://lists.nongnu.org/archive/html/qemu-devel/2016-01/msg03259.html >>> >>> Yet the patch remains unapplied :( >> >> I thought is it in already. Maybe Michael can add it as part of the hard freeze. >> And if the patch will be applied, the tweak above wouldn't help, right Alex? > > The tweak Spenser suggested above should be unnecessary with my proposed > patch applied. > I think I finally understand this. The bus is not pcie -> we return > from vfio_add_ext_cap without "breaking" the extended capabilities > chain and the bare metal SR-IOV capability will be visible to guest. > With your patch the PCI bridge will "interfere" and mask the extended > configuration space completely. > Only now searching for that patch did I notice Michael's > comment hidden at the bottom of his reply, which I assume is why it > never got applied: > > https://patchwork.kernel.org/patch/8057411/ > > I just saw it too! It seems Michael wants to cache this info > in device instead of re-calculating it every time. >> Anyway, the current behavior is clearly a bug, so QEMU hard freeze >> should be irrelevant. If anyone wants to take over the patch, feel >> free. Thanks, > I suppose I can handle it, but sadly not for 2.7. > If Spencer has some time now he can help by testing it and reviewing it quickly :) I'd be happy to help; but that patch really just more permanently breaks my current workaround ;-) . Thanks, Spenser This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.

2 2

[libvirt-users] Fwd: Please help see issue: conn.storageVolLookupByPath('/var/lib/xen/images/xen-pv-rhel5.9-x64-nolvm.img')
by Junqin Zhou 15 Jul '16

15 Jul '16

Hi all, Please see the attachment for vdsm.log details, thanks. Best Regards, juzhou. ----- Forwarded Message ----- > From: "Junqin Zhou" <juzhou(a)redhat.com> > To: libvirt-users(a)redhat.com > Cc: "Shahar Havivi" <shavivi(a)redhat.com>, "Tingting Zheng" <tzheng(a)redhat.com>, "Min Zhan" <mzhan(a)redhat.com>, "Ming > Xie" <mxie(a)redhat.com> > Sent: Wednesday, July 13, 2016 6:24:15 PM > Subject: Please help see issue: conn.storageVolLookupByPath('/var/lib/xen/images/xen-pv-rhel5.9-x64-nolvm.img') > > Hi guys, > While testing v2v integration to rhevm4.0, I met following error when import > virtual machine from xen server. > > I.rhevm4.0 prompt error message > Failed to communicate with the external provider, see log for additional > details > > Steps： > 1. In order to import VMs password-less SSH access has to be enabled between > VDSM host and the Xen host. The following steps needed to be done at the > VDSM host: > Refer to > https://github.com/oVirt/ovirt-site/blob/master/source/develop/release-mana… > > 2. Then login rhevm 4.0 and try to import virtual machine from Xen. > Virtual Machines-->Import-->Fill items on 'Import Virtual Machine(s)' window > with: > Data Center:xx > Source: Xen(via RHEL) > URI: xen+ssh://root@xxxxxx > Proxy Host: xxxxxx > > Result: > Step2 failed to load xen server's guests with error: > Failed to communicate with the external provider, see log for additional > details > > Part of vdsm.log > ... > jsonrpc.Executor/3::WARNING::2016-07-13 > 08:18:40,588::v2v::959::root::(_add_disks) Disk {'alias': > '/var/lib/xen/images/xen-pv-rhel5.9-x64-nolvm.img', 'type': 'disk', 'dev': > 'xvda'} has unsupported format: <built-in function format> > jsonrpc.Executor/3::ERROR::2016-07-13 > 08:18:41,504::v2v::923::root::(_add_disk_info) Error getting disk size > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 920, in > _add_disk_info > vol = conn.storageVolLookupByPath(disk['alias']) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in > storageVolLookupByPath > if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', > conn=self) > libvirtError: Storage volume not found: no storage vol with matching path > > Please help me check how to resolve this issue, thanks. > > > Best Regards, > juzhou. > > >

1 0

[libvirt-users] virt-login-shell: Security model none cannot be entered
by james＠lottspot.com 13 Jul '16

13 Jul '16

Hello! I am currently experimenting a bit with some of the LXC support under libvirt, and in trying to utilize the tool virt-login-shell, I encounter the following error: [james@lxchost ~]$ virt-login-shell libvirt: error : argument unsupported: Security model none cannot be entered Though it should be apparent from the lack of error, the domain is most definitely running. [root@lxchost ~]# virsh -c lxc:/// list Id Name State ---------------------------------------------------- 4909 james running Config information: [root@lxchost ~]# grep -vE '^(#|$)' /etc/libvirt/virt-login-shell.conf allowed_users = [ "*" ] OS information: [root@lxchost ~]# cat /etc/os-release NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" Am I simply missing something here? Any tips or pointers are greatly appreciated!

3 2

[libvirt-users] nested vms and nested macvtaps
by jsl6uy js16uy 11 Jul '16

11 Jul '16

Hello all, hope all is well this maybe outside of libvirt-users.... Can you nest macvtap devices to ultimately receive a real routable ip on the nested vm? I have a nested vm up and running. Both vm and nested vm are centos 7 on arch linux host. The first vm uses a macvtap in bridge mode receives dhcp from an external dhcp server. I start the second vm and dhclient hangs and never receives an offer. I stood up a static interface on the nested vm. If I initiate a ping from within the nested vm, I can see via tcpdump that the echo request is seen on the non-nested vm and the hosting server. The reply comes back and stops/is answered on the hosting OS interface. rp_filter is off. No firewall rules in play blocking dhcp or icmp Any/all help appreciated thanks very much

2 5

[libvirt-users] lxc containers won't start in a f24 custom install - odd cgroup fs layout observed
by Thierry Parmentelat 10 Jul '16

10 Jul '16

Hi folks I use libvirt to programmatically spawn lxc containers I am facing an issue when migrating from fedora23 to fedora24 I use the stock kernel and libvirt version on both deployments, i.e.: f23: libvirt-1.2.18.3-2.fc23.x86_64 - kernel 4.5.7-202.fc23.x86_64 f24: libvirt-1.3.3.1-4.fc24.x86_64 - kernel 4.6.3-300.fc24.x86_64 First off, I need to outline that the host installation is done through some ad hoc procedure, as all this happens in the context of the centrally-managed PlanetLab global infrastructure I could not reproduce this problem on a host that is installed with the standard fedora install However I could use any clue that would point at a possible reason for this IMHO odd behaviour from libvirt, so that I can narrow down the potential flaws in my host installation procedure At first sight, it looks like the f24 deployment uses a different cgroup fs layout - see below and I suspect this could be the problem, but I am not aware of anything in my install. procedure that could cause that change, and as per this document https://libvirt.org/cgroups.html#systemdScope this change in the naming scheme looks odd - unless the doc is out of date of course So again, any clue is most welcome; many thanks in advance -- Thierry ------ All is fine with f23/libvirt-1.2.18.3-2.fc23.x86_64 However with f24/libvirt-1.3.3.1-4.fc24.x86_64 my lxc containers won't start and I am seeing this message in the libvirtd logfile 2016-07-09 11:52:33.416+0000: 3479: debug : virCgroupValidateMachineGroup:317 : Name 'lxc-3555-inrisl1.libvirt-lxc' for controller 'cpu' does not match 'inri_sl1', 'lxc-3555-inrisl1', 'inri_sl1.libvirt-lxc', '\ machine-lxc\x2dinri_sl1.scope' or 'machine-lxc\x2d3555\x2dinrisl1.scope' 2016-07-09 11:52:33.416+0000: 3479: debug : virCgroupNewDetectMachine:1501 : Failed to validate machine name for 'inri_sl1' driver 'lxc' 2016-07-09 11:52:33.416+0000: 3479: error : virLXCProcessStart:1501 : internal error: No valid cgroup for machine inri_sl1 At first it looked like the issue could be linked to the domain name having an underscore (example below uses inri_sl1 as a domain name) But trying with a regular name 'plain' without an underscore exhibits a similar issue ------------------------- Here's the set of files under /sys/fs/cgroup/memory that contains 'inri' in their name (other susbsytems have similar naming schemes in both cases) ========= f23 / libvirt-1.2.18.3-2.fc23.x86_64 (works fine) [root@vnode06 log]# cat /etc/fedora-release Fedora release 23 (Twenty Three) [root@vnode06 log]# uname -r 4.5.7-202.fc23.x86_64 [root@vnode06 log]# rpm -q libvirt libvirt-1.2.18.3-2.fc23.x86_64 [root@vnode06 log]# find /sys/fs/cgroup/memory/ -name '*inri*' /sys/fs/cgroup/memory/machine.slice/machine-lxc\x2dinri_sl1.scope ========= f24 / libvirt-1.3.3.1-4.fc24.x86_64 (container won't start) [root@vnode05 libvirt]# cat /etc/fedora-release Fedora release 24 (Twenty Four) [root@vnode05 libvirt]# uname -r 4.6.3-300.fc24.x86_64 [root@vnode05 libvirt]# rpm -q libvirt libvirt-1.3.3.1-4.fc24.x86_64 [root@vnode05 libvirt]# find /sys/fs/cgroup/memory/ -name '*inri*' /sys/fs/cgroup/memory/machine/lxc-3237-inrisl1.libvirt-lxc /sys/fs/cgroup/memory/machine/lxc-3555-inrisl1.libvirt-lxc /sys/fs/cgroup/memory/machine/lxc-2989-inrisl1.libvirt-lxc

1 1

[libvirt-users] vfio driver using libvirt
by abhishek jain 07 Jul '16

07 Jul '16

Hi Team I need to run vfio driver using qemu and I'm using below xml file for this.. <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <name>instance</name> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <vcpu placement='static'>8</vcpu> <os> <type arch='aarch64' machine='virt'>hvm</type> <kernel>/home/root/Image</kernel> <initrd>/home/root/fsl-image-minimal-ls2085ardb.ext2.gz</initrd> <cmdline>'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000'</cmdline> </os> <cpu mode='custom' match='exact'> <model fallback='allow'>host</model> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-system-aarch64</emulator> <memballoon model='none'/> </devices> <qemu:commandline> <qemu:arg value='-mem-path'/> <qemu:arg value='/hugepages'/> <qemu:arg value='-device'/> <qemu:arg value='vfio-fsl-mc,host=dprc.2'/> </qemu:commandline> </domain> I'm able to launch VM succesfully using the above xml file and it produces the below process running in the background.. *ps -ef | grep qemu* root 6767 1 34 22:11 ? 00:00:02 /usr/bin/qemu-system-aarch64 -name instance -S -machine virt,accel=kvm,usb=off -cpu host -m 4096 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid 3b527012-c15d-46b3-862d-9a7fd3b04e01 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -kernel /home/root/Image -initrd /home/root/fsl-image-minimal-ls2085ardb.ext2.gz -append 'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000' -usb -mem-path /hugepages *-device vfio-fsl-mc,host=dprc.2* -msg timestamp=on Although the driver appears in the above process,actually that driver is not loaded as there is no output of cat /proc/interrupts | grep dpio on the host. However when I'm using the same command as that of the above process i.e .. /usr/bin/qemu-system-aarch64 -name instance -S -machine virt,accel=kvm,usb=off -cpu host -m 4096 -realtime mlock=off -smp 8,sockets=8,cores=1,threads=1 -uuid 3b527012-c15d-46b3-862d-9a7fd3b04e01 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -kernel /home/root/Image -initrd /home/root/fsl-image-minimal-ls2085ardb.ext2.gz -append 'root=/dev/ram0 rw console=ttyAMA0,115200 rootwait earlyprintk ramdisk_size=1000000' -usb -mem-path /hugepages *-device vfio-fsl-mc,host=dprc.2* -msg timestamp=on the driver is getting loaded and I'm able to see the output of cat /proc/interrupts | grep dpio on the host. Please help me regarding this. Thanks Abhishek Jain

1 0

[libvirt-users] Permanently change default virsh editor
by J Travis Lindsey 07 Jul '16

07 Jul '16

Hello, I found in the docs that I can change the editor I use on a per vm bases with the following: *EDITOR=$your-favorite-editor virsh edit $your-vm-name* But how do I change it permanently so when I run: *virsh edit $your-vm-name* It defaults to my favorite editor (vim)? Ref: http://wiki.libvirt.org/page/FAQ

4 4