Re: [libvirt-users] virsh not detecting hugepage mount; disabled by config?

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Thanks a lot you two, yes hugetlbfs is not mounted, when libvirtd is started… It was a long night. Did put it to fstab and since I do not allocate them on regular boots, it won’t eat my ram (that was the main consideration for doing it like this). As a last means before going to sleep, I added <qemu:arg value='-numa'/> <qemu:arg value='node,memdev=mem42'/> <qemu:arg value='-object'/> <qemu:arg value='memory-backend-file,size=6G,mem-path=/var/lib/hugetlbfs,share=on,id=mem42'/> to the xml which worked too (surprise), but I have it working now in the intended way. Then I never got to the last part of the qemu.conf option comment so I had to do another 3 reboots. Well, reading helps. So the fstab entry reads now: hugetlbfs /var/lib/hugetlbfs/libvirt/qemu hugetlbfs relatime,rw,pagesize=1073741824 0 0 with /var/lib/hugetlbfs being the mount point in qemu.conf. At least I didn’t forget to create the mount point before rebooting. Best regards, Manuel pci > passthrough and hugetlbs. > > Things I had to do for hugetlbs: > 1) Made sure their support was enabled in kernel > 2) Passed parameters for their allocation to kernel. There are other > ways, but I use 1GB page size and I have to allocate them before memory > gets fragmented. Note, you do that with hugepages=3072. > 3) Passed hugetlbs settings to libvirt VM domain (XML). > > That is it. I don't mount them myself. I think (can't check now) they > are automounted to /dev/hugetlbs. By default, libvirtd runs as root, and > qemu as user qemu. Consuming hugetlb and releasing them should be > transparent. > > I suggest starting without softlevel=qemuvm into default runlevel, and > then check /proc/meminfo. Hugetlbs total and free should be > 0. Do the > same with qemuvm runlevel. Maybe this runlevel misses something. > > On 2016-12-14 00:05, Manuel Ullmann wrote: >> Hi, >> >> I’m struggling with virsh not detecting my hugepage mount. >> >> I have the following kernel command line: >> BOOT_IMAGE=/vmlinuz-4.8.13-gentoo root=/dev/mapper/gensd-gentoo ro quiet >> splash intel_iommu=on video=efifb:off,vesafb:off,simplefb:off >> splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 >> transparent_hugepage=never hugepages=3072 softlevel=qemuvm >> >> My startup script outputs the following: >> hugetlbfs /var/lib/hugetlbfs hugetlbfs >> rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 >> [2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 >> [2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs >> filesystem is not mounted or disabled by administrator config >> virsh was unsuccessful >> >> So hugetlbfs is definitively mounted, but virsh does either not detect >> it or some magical option, that I’ve not found in a day searching for >> it, disables it (the documentation could explain that better). Since the >> Ubuntu guys refer to the KVM_HUGEPAGES environment, I tried adding it to >> start-stop-daemon environment, but it seems non-relevant. 77 is the qemu >> user id, but I’m quite sure the permission issues were excluded (tried >> root permissions as well). Besides I had permission errors occasionally >> occasionally and don’t have them anymore. If you could provide a hint to helpful. >> Tried hugeadm as well, pointing the config to the correct destination. >> >> Thanks in advance, >> Manuel >> >> PS: >> The virsh script reads like this: >> #!/bin/sh >> cmdline="$(cat /proc/cmdline)" rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t >> hugetlbfs hugetlbfs /var/lib/hugetlbfs >> sysctl kernel.shmmax=6442450944 -i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.conf >> counter=0 ]; do >> sleep 1 >> counter=$(( ++counter )) >> done >> if [ $counter -gt 9 ]; then >> echo "libvirtd socket generation timed out" >> /var/log/virsh.log >> fi >> if pidof libvirtd &>/dev/null; then >> LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log start win10 >> if [ $? -gt 0 ]; then >> echo "virsh was unsuccessful" >> /var/log/virsh.log >> reboot >> fi >> else >> echo "libvirtd is not started" >> /var/log/virsh.log >> reboot >> fi >> fi >> >> reboot is more convenient, since this is a vfio-igd passthrough vm. >> >> >> _______________________________________________ >> libvirt-users mailing list >> libvirt-users(a)redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-users >>