[libvirt-users] virsh not detecting hugepage mount; disabled by config?
Hi, I’m struggling with virsh not detecting my hugepage mount. I have the following kernel command line: BOOT_IMAGE=/vmlinuz-4.8.13-gentoo root=/dev/mapper/gensd-gentoo ro quiet splash intel_iommu=on video=efifb:off,vesafb:off,simplefb:off splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 transparent_hugepage=never hugepages=3072 softlevel=qemuvm My startup script outputs the following: hugetlbfs /var/lib/hugetlbfs hugetlbfs rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 [2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 [2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs filesystem is not mounted or disabled by administrator config virsh was unsuccessful So hugetlbfs is definitively mounted, but virsh does either not detect it or some magical option, that I’ve not found in a day searching for it, disables it (the documentation could explain that better). Since the Ubuntu guys refer to the KVM_HUGEPAGES environment, I tried adding it to start-stop-daemon environment, but it seems non-relevant. 77 is the qemu user id, but I’m quite sure the permission issues were excluded (tried root permissions as well). Besides I had permission errors occasionally occasionally and don’t have them anymore. If you could provide a hint to the magical administrator configuration option, that would be helpful. Tried hugeadm as well, pointing the config to the correct destination. Thanks in advance, Manuel PS: The virsh script reads like this: #!/bin/sh cmdline="$(cat /proc/cmdline)" if [[ "${cmdline##* }" == "softlevel=qemuvm" ]]; then mount -o rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t hugetlbfs hugetlbfs /var/lib/hugetlbfs sysctl kernel.shmmax=6442450944 cat /proc/mounts | grep hugetlb >> /var/log/virsh.log sed -i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.conf counter=0 while [ "${counter}" -lt 10 && ! -S /var/run/libvirt/libvirt-sock ]; do sleep 1 counter=$(( ++counter )) done if [ $counter -gt 9 ]; then echo "libvirtd socket generation timed out" >> /var/log/virsh.log fi if pidof libvirtd &>/dev/null; then LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log start win10 if [ $? -gt 0 ]; then echo "virsh was unsuccessful" >> /var/log/virsh.log reboot fi else echo "libvirtd is not started" >> /var/log/virsh.log reboot fi fi reboot is more convenient, since this is a vfio-igd passthrough vm.
On Tue, Dec 13, 2016 at 11:05:11PM +0100, Manuel Ullmann wrote:
Hi,
I’m struggling with virsh not detecting my hugepage mount.
I have the following kernel command line: BOOT_IMAGE=/vmlinuz-4.8.13-gentoo root=/dev/mapper/gensd-gentoo ro quiet splash intel_iommu=on video=efifb:off,vesafb:off,simplefb:off splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 transparent_hugepage=never hugepages=3072 softlevel=qemuvm
My startup script outputs the following: hugetlbfs /var/lib/hugetlbfs hugetlbfs rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 [2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 [2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs filesystem is not mounted or disabled by administrator config virsh was unsuccessful
So hugetlbfs is definitively mounted, but virsh does either not detect it or some magical option, that I’ve not found in a day searching for it, disables it (the documentation could explain that better). Since the Ubuntu guys refer to the KVM_HUGEPAGES environment, I tried adding it to start-stop-daemon environment, but it seems non-relevant. 77 is the qemu user id, but I’m quite sure the permission issues were excluded (tried root permissions as well). Besides I had permission errors occasionally occasionally and don’t have them anymore. If you could provide a hint to the magical administrator configuration option, that would be helpful.
Nothing magical about that. If you have no hugepages configured in qemu.conf, libvirtd will *try* to find all the mounts if running as root. The only thing is that in the error message "or" should be changed to "and", I guess.
Tried hugeadm as well, pointing the config to the correct destination.
Thanks in advance, Manuel
PS: The virsh script reads like this: #!/bin/sh cmdline="$(cat /proc/cmdline)" if [[ "${cmdline##* }" == "softlevel=qemuvm" ]]; then mount -o rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t hugetlbfs hugetlbfs /var/lib/hugetlbfs
If libvirtd starts before this ^^
sysctl kernel.shmmax=6442450944 cat /proc/mounts | grep hugetlb >> /var/log/virsh.log sed -i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.conf
or this ^^ is done, then the settings are not reflected. That's the only thing I could glean from the quick look. I bet adding 'service libvirtd restart' here helps.
counter=0 while [ "${counter}" -lt 10 && ! -S /var/run/libvirt/libvirt-sock ]; do sleep 1 counter=$(( ++counter )) done if [ $counter -gt 9 ]; then echo "libvirtd socket generation timed out" >> /var/log/virsh.log fi if pidof libvirtd &>/dev/null; then LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log start win10 if [ $? -gt 0 ]; then echo "virsh was unsuccessful" >> /var/log/virsh.log reboot fi else echo "libvirtd is not started" >> /var/log/virsh.log reboot fi fi
reboot is more convenient, since this is a vfio-igd passthrough vm.
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
participants (2)
-
Manuel Ullmann -
Martin Kletzander