- Users - libvirt List Archives

[libvirt-users] create a domain failed using virt-install
by lizhuoyao 08 Jun '18

08 Jun '18

hi everyone: Sorry to bother you again! my order: virt-install --virt-type kvm -n centos -r 1024 --disk centos.img,format=qcow2,size=10 --cdrom /home/CentOS-7-aarch64-Everything.iso the failed log: qemu-kvm: -device pcie-root-port,port=0x8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1:MSI-X is not supported by interrupt controller. my environment: arm64 platform;centos7.5 for arm; libvirt-3.9.0;qemu-2.10.0;virt-install-1.4.3 Do you have met the issue ever?Any suggestion? -- Have a good day

2 1

[libvirt-users] Are more then one overlay possible without recieving a "Permission denied"?
by Xen Mann 07 Jun '18

07 Jun '18

I've created a backing chain like this. FirstFollr.ovl is an overlay to Base.img and so on. Base.img (ok - Win 10 Installation starting) FirstFloor.ovl (ok - Win 10 Installation starting) SecondFloor.ovl (permission denied -> Base.img) => assumed Bug Roof.ovl (permission denied -> Base.img) => assumed Bug Using virsh / virt-manager, I can only start Base and FirstFloor. Starting Second and Roof are failing. What's wrong? Details are described in: https://bugzilla.redhat.com/show_bug.cgi?id=1588373 System infos: Compiled against library: libvirt 4.0.0 Using library: libvirt 4.0.0 Using API: QEMU 4.0.0 Running hypervisor: QEMU 2.11.1 Ubuntu 18.04 LTS (Dell Precission Tower 3620)

1 0

[libvirt-users] List-Archives ...
by thg 07 Jun '18

07 Jun '18

Hi everybody, actually I wanted to search the list archive, before asking, but unfortunately I don't "get" it: $ gunzip 2018-May.txt.gz gunzip: 2018-May.txt.gz: not in gzip format It seems, that in every archive-file there is always one message in plaintext and then a big binary block. Any hint? Thanks a lot, -- kind regards, thg

3 2

[libvirt-users] Vlan on vhostuser interfaces
by Pradeep K.S 05 Jun '18

05 Jun '18

I am planning to configure vlan on vhostuser interfaces in the libvirt xml, it throws out error. Does this configuration is supported ? I can install higher version if required. *Error:* unsupported configuration: an interface of type 'vhostuser' is requesting a vlan tag, but that is not supported for this type of connection *Libvirt version:* [redhathost@qemu]# libvirtd --version libvirtd (libvirt) 3.2.0 *Qemu version:* QEMU emulator version 2.9.0(qemu-kvm-ev-2.9.0-16.el7_4.11.1) *Domain XML* <interface type='vhostuser'> <mac address='02:3a:c1:4b:a1:0a'/> <source type='unix' path='/tmp/vhost-test' mode='server'/> <vlan trunk='yes'> <tag id='3' nativeMode='tagged'/> <tag id='4'/> </vlan> <model type='virtio'/> <driver rx_queue_size='1024'> <host mrg_rxbuf='on'/> </driver> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> -- Thanks and Regards, Pradeep.K.S.

2 1

[libvirt-users] Make discard='unmap' the default?
by Ian Pilcher 01 Jun '18

01 Jun '18

Is it possible to make discard='unmap' the default for virtio-scsi disks? (Related, is it possible to make virtio-scsi the default disk type, rather than virtio-blk?) Thanks! -- ======================================================================== Ian Pilcher arequipeno(a)gmail.com -------- "I grew up before Mark Zuckerberg invented friendship" -------- ========================================================================

3 6

[libvirt-users] virRandomBits - not very random
by Brian Rak 01 Jun '18

01 Jun '18

I just ran into an issue where I had about 30 guests get duplicate mac addresses assigned. These were scattered across 30 different machines. Some debugging revealed that: 1) All the host machines were restarted within a couple seconds of each other 2) All the host machines had fairly similar libvirtd pids (within ~100 PIDs of each other) 3) Libvirt seeds the RNG using 'time(NULL) ^ getpid()' This perfectly explains why I saw so many duplicate mac addresses. Why is the RNG seed such a predictable value? Surely there has to be a better source of a random seed then the timestamp and the pid? The PID seems to me to be a very bad source of any randomness. I just ran a test across 60 of our hosts. 43 of them shared their PID with at least one other machine.

7 13

[libvirt-users] Two Node Cluster
by Cobin Bluth 01 Jun '18

01 Jun '18

Hello Libvirt Users, I would like to setup a two node bare-metal cluster. I need to guidance on the network configuration. I have attached a small diagram, the same diagram can be seen here: https://i.imgur.com/SOk6a6G.png I would like to configure the following details: - Each node has a DHCP enabled guest network where VMs will run. (eg, *192.168.1.0/24 <http://192.168.1.0/24>* for Host1, and *192.168.2.0/24 <http://192.168.2.0/24>* for Host2) - Any guest in Host1 should be able to ping guests in Host2, and vice versa. - All guests have routes to reach the open internet (so that '*yum update*' will work "out-of-the-box") - Each node will be able to operate fully if the other physical node fails. (no central DHCP server, etc) - I would like to *add more physical nodes later* when I need the resources. This is what I have done so far: - Installed latest Ubuntu 18.04, with latest version of libvirt and supporting software from ubuntu's apt repo. - Each node can reach the other via its own eth0. - Each node has a working vxlan0, which can ping the other via its vxlan0, so it looks like the vxlan config is working. (I used *ip link add vxlan0 type vxlan...*) - Configured route on Host1 like so: *ip route add 192.168.2.0/24 <http://192.168.2.0/24> via 172.20.0.1* - Configured route on Host2 also: *ip route add 192.168.1.0/24 <http://192.168.1.0/24> via 172.20.0.2* - All guests on Host1 (and Host1) can ping eth0 and vxlan0 on Host2, and vice versa, yay. - Guests on Host1 *cannot* ping guests on Host2, I suspect because the the default NAT config of the libvirt network. So, at this point I started to search for tutorials or more information/documentation, but I am a little overwhelmed by the sheer amount of information, as well as a lot of "stale" information on blogs etc. I have learned that I can *virsh net-edit default*, and then change it to an "open" network:* <forward mode='open'/>* After doing this, the guests cannot reach outside their own network, nor reach the internet, so I assume that I would need to add some routes, or something else to get the network functioning like I want it. There is also *<forward mode="route"/>*, but I dont fully understand the scenarios where one would need an *open* or a *route* forward mode. I have also shied away from using openvswitch, and have opted for ifupdown2. (I have taken most of my inspiration from this blog post: https://joejulian.name/post/how-to-configure-linux-vxlans-with-multiple-uni… ) Some questions that I have for the mailing list, any help would be greatly appreciated: - Is my target configuration of a KVM cluster uncommon? Do you see drawbacks of this setup, or does it go against "typical convention"? - Would my scenario be better suited for an "*open*" network or a "*route*" network? - What would be the approach to complete this setup?

3 4

Re: [libvirt-users] VM I/O performance drops dramatically during storage migration with drive-mirror
by Kashyap Chamarthy 29 May '18

29 May '18

Cc the QEMU Block Layer mailing list (qemu-block(a)nongnu.org) who might have more insights here; and wrap long lines. On Mon, May 28, 2018 at 06:07:51PM +0800, Chunguang Li wrote: > Hi, everyone. > > Recently I am doing some tests on the VM storage+memory migration with > KVM/QEMU/libvirt. I use the following migrate command through virsh: > "virsh migrate --live --copy-storage-all --verbose vm1 > qemu+ssh://192.168.1.91/system tcp://192.168.1.91". I have checked the > libvirt debug output, and make sure that the drive-mirror + NBD > migration method is used. > > Inside the VM, I use an I/O benchmark (Iometer) to generate an oltp > workload. I record the I/O performance (IOPS) before/during/after > migration. When the migration begins, the IOPS dropped by 30%-40%. > This is reasonable, because the migration I/O competes with the > workload I/O. However, during almost the last period of migration > (which is 66s in my case), the IOPS dropped dramatically, from about > 170 to less than 10. I also show the figure of this experiment in the > attachment of this email. [The attachment should arrive on the 'libvirt-users' list archives; but it's not there yet -- https://www.redhat.com/archives/libvirt-users/2018-May/thread.html] > I want to figure out what results in this period with very low IOPS. > First, I added some printf()s in the QEMU code, and knew that, this > period occurs just before the memory migration phase. (BTW, the memory > migration is very fast, which is just about 5s.) So I think this > period should be the last phase of the "drive-mirror" process of QEMU. > So then I tried to read the code of "drive-mirror" in QEMU, but failed > to understand it very well. > > Does anybody know what may lead to this period with very low IOPS? > Thank you very much. > > Some details of this experiment: The VM disk image file is 30GB > (format = raw,cache=none,aio=native), and Iometer operates on an 10GB > file inside the VM. The oltp workload consists of 33% writes and 67% > reads (8KB request size, all random). The VM memory size is 4GB, most > of which should be zero pages, so the memory migration is very fast. > > -- > Chunguang Li, Ph.D. Candidate > Wuhan National Laboratory for Optoelectronics (WNLO) > Huazhong University of Science & Technology (HUST) > Wuhan, Hubei Prov., China -- /kashyap

3 4

[libvirt-users] Using libvirt to access Ceph RBDs with Xen
by thg 29 May '18

29 May '18

Hi everybody, my background: I'm doing Xen since 10++ years, many years with DRBD for high availability, since some time I'm using preferable GlusterFS with FUSE as replicated storage, where I place the image-files for the vms. In my current project we started (successfully) with Xen/GlusterFS too, but because the provider, where we placed the servers, uses widely CEPH, we decided to switch too. Unfortunately I'm new to CEPH, but with help of a technician, we have running a 3 node CEPH-cluster now, that seems to work fine. Hardware: - Xeons, 24 Cores, 256 GB RAM, 2x 240 GB system-SSDs RAID1, 4x 1,92 TB data-SSDs (no RAID) Software we are using: - CentOS 7.5.1804 - Kernel: 4.9.86-30.el7 @centos-virt-xen-48 - Xen: 4.8.3-5.el7 @centos-virt-xen-48 - libvirt-xen: 4.1.0-2.xen48.el7 @centos-virt-xen-48 - Ceph: 2:12.2.5-0.el7 @Ceph CEPH itself works, Xen too and even libvirt (I'm quite new in this too, normally I use the XL-tools to manage my VMs) ;-) I can manually map an RBD and start a PV-VM via libvirt, after modifying the XML-config and defined the VM with libvirt. That is what happens: # virsh define xml/testvm3.xml Domain testvm3 defined from xml/testvm3.xml # virsh start --console testvm3 error: Failed to start domain testvm3 error: internal error: libxenlight failed to create new domain 'testvm3' xml/testvm3.xml: -> ... <devices> <disk type='network' device='disk'> <source protocol='rbd' name='devel-pool/testvm3.rbd'> <host name="10.20.30.1" port="6789"/> <host name="10.20.30.2" port="6789"/> <host name="10.20.30.3" port="6789"/> </source> <auth username='libvirt'> <secret type='ceph' uuid='07f3a0fe-9178-xxxxxxxxx'/> </auth> <target dev='xvda' bus='xen'/> </disk> ... Logs: /var/log/libvirt/libxl/libxl-driver.log: -> ... 2018-05-27 11:24:36.666+0000: libxl: libxl_bootloader.c:634:bootloader_finished: bootloader failed - consult logfile /var/log/xen/bootloader.14.log 2018-05-27 11:24:36.666+0000: libxl: libxl_exec.c:118:libxl_report_child_exitstatus: bootloader [183216] exited with error status 1 2018-05-27 11:24:36.667+0000: libxl: libxl_create.c:1259:domcreate_rebuild_done: cannot (re-)build domain: -3 /var/log/xen/bootloader.14.log: -> Traceback (most recent call last): File "/usr/lib64/xen/bin/pygrub", line 896, in <module> part_offs = get_partition_offsets(file) File "/usr/lib64/xen/bin/pygrub", line 113, in get_partition_offsets image_type = identify_disk_image(file) File "/usr/lib64/xen/bin/pygrub", line 56, in identify_disk_image fd = os.open(file, os.O_RDONLY) OSError: [Errno 2] No such file or directory: 'rbd:devel-pool/testvm3.rbd:id=libvirt:key=AQBThwFbGFRYFxxxxxxxxxxxxxxx==:auth_supported=cephx\\;none:mon_host=10.20.30.1\\:6789\\;10.20.30.2\\:6789\\;10.20.30.3\\:6789' So the Xen-Python-script, that searches for the kernel/bootloader of the VM, does not find the imagefile. But if I grab the above output and use it with qemu, it does find the blockdevice: # qemu-img info "rbd:devel-pool/testvm3.rbd:id=libvirt:key=AQBThwFbGFRYFxxxxxxxxxxxxxxx==:auth_supported=cephx\\;none:mon_host=10.20.30.1\\:6789\\;10.20.30.2\\:6789\\;10.20.30.3\\:6789" -> image: rbd:devel-pool/testvm3.rbd:id=libvirt:key=AQBThwFbGFRYFxxxxxxxxxxxxxxx==:auth_supported=cephx\;none:mon_host=10.20.30.1\:6789\;10.20.30.2\:6789\;10.20.30.3\:6789 file format: raw virtual size: 16G (17179869184 bytes) disk size: unavailable Thus it seems, that CEPH-auth works and the VM-XML-config is interpreted correct by libvirt. But the next step fails, when libvirt should pass the blockdevice to Xen? Anybody out there who uses a similar configuration? Any hints how to debug this? Unfortunately I do not have anymore logs to look at. Thank you very much, -- Kind regards, thg

2 1

[libvirt-users] virsh migration job unexpectedly failed
by cokabug 26 May '18

26 May '18

Hi , when i have run virsh migrate command , the migration progress fiailed at [61 %]. command : virsh migrate --live --undefinesource --p2p --copy-storage-inc --verbose --persistent instance-00001959 qemu+tcp://host292/system result : Migration: [ 61 %]error: operation failed: migration job: unexpectedly failed os/kernel version: centos 6.7 / kernel 2.6.32 qemu-kvm version : 0.12.1.2 libvirt version : 0.10.2-54 the libvirt.log on target host: 2018-05-22 11:45:38.603+0000: 32314: warning : qemuDomainObjTaint:1459 : Domain id=30 name='instance-00001959' uuid=41975d30-2490-473e-be7b-9d2d08bf66d2 is tainted: high-privileges 2018-05-22 11:45:38.883+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:38.902+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:38.903+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:39.090+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:39.090+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:39.091+0000: 32314: warning : qemuDomainObjEnterMonitorInternal:1062 : This thread seems to be the async job owner; entering monitor without asking for a nested job is dangerous 2018-05-22 11:45:41.841+0000: 32310: error : qemuMonitorIO:615 : internal error End of file from monitor the qmeu-kvminstance-00001959.log: 2018-05-22 11:45:38.603+0000: starting up LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/home/ning.zhao USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -name instance-00001959 -S -M rhel6.6.0 -cpu Broadwell,+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+m$ nitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -enable-kvm -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 41975d30-2490-473e-be7b-9d2d08bf66d2 -smbios type=1,manufacturer=Red Hat Inc.,product=OpenStack Nova,version=2013.1.2-4.el6,serial=fcbe6c27-c9$ d-e811-ac36-a281f9ebc503,uuid=41975d30-2490-473e-be7b-9d2d08bf66d2 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00001959.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -n$ -kvm-pit-reinjection -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/41975d30-2490-473e-be7b-9d2d08bf66d2/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4$ drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=28,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:c0:8f:6b,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/var/lib/nova/instances/41975d30-2490-473e-b$ 7b-9d2d08bf66d2/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -vnc 127.0.0.1:2 -k en-us -vga cirrus -incoming tcp: 0.0.0.0:49152 -device virtio-bal$ oon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on Domain id=30 is tainted: high-privileges char device redirected to /dev/pts/2 Receiving block device images qemu: warning: error while loading state for instance 0x0 of device 'block' load of migration failed 2018-05-22 11:45:41.841+0000: shutting down i checked the log and disk file and they have right permissions, has anyone know about this ,i have google many times ,but i cant find the right debug way to resolve it. thanks

2 2