Re: [libvirt-users] virsh not detecting hugepage mount; disabled by config?

14 Dec 2016

      ...
Hi, > > I use gentoo with desktop+systemd profile to run Windows 10 VM with
...
...
the magical administrator configuration option, that would be
helpful. >> Tried hugeadm as well, pointing the config to the correct
destination. >> >> Thanks in advance, >> Manuel >> >> PS: >> The virsh
...
...
if [[ "${cmdline##* }" == "softlevel=qemuvm" ]]; then >>     mount -o
rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t >> hugetlbfs
hugetlbfs /var/lib/hugetlbfs >>     sysctl kernel.shmmax=6442450944
    cat /proc/mounts | grep hugetlb >> /var/log/virsh.log >>     sed
-i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.conf >>     counter=0
    while [ "${counter}" -lt 10 && ! -S /var/run/libvirt/libvirt-sock
]; do >>     sleep 1 >>     counter=$(( ++counter )) >>     done >>    
if [ $counter -gt 9 ]; then >>     echo "libvirtd socket generation
timed out" >> /var/log/virsh.log >>     fi >>     if pidof libvirtd
&>/dev/null; then >>     LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log
start win10 >>     if [ $? -gt 0 ]; then >>         echo "virsh was
unsuccessful" >> /var/log/virsh.log >>         reboot >>     fi >>    
else >>     echo "libvirtd is not started" >> /var/log/virsh.log >>    
reboot >>     fi >> fi >> >> reboot is more convenient, since this is a
vfio-igd passthrough vm. >> >> >>
_______________________________________________ >> libvirt-users mailing
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Thanks a lot you two,

yes hugetlbfs is not mounted, when libvirtd is started… It was a long
night. Did put it to fstab and since I do not allocate them on regular
boots, it won’t eat my ram (that was the main consideration for doing it
like this).
As a last means before going to sleep, I added
    <qemu:arg value='-numa'/>
    <qemu:arg value='node,memdev=mem42'/>
    <qemu:arg value='-object'/>
    <qemu:arg
value='memory-backend-file,size=6G,mem-path=/var/lib/hugetlbfs,share=on,id=mem42'/>
to the xml which worked too (surprise), but I have it working now in the
intended way.
Then I never got to the last part of the qemu.conf option comment so I
had to do another 3 reboots. Well, reading helps.
So the fstab entry reads now:
hugetlbfs   /var/lib/hugetlbfs/libvirt/qemu  hugetlbfs
relatime,rw,pagesize=1073741824  0 0
with /var/lib/hugetlbfs being the mount point in qemu.conf.
At least I didn’t forget to create the mount point before rebooting.

Best regards,
Manuel
pci > passthrough and hugetlbs. > > Things I had to do for hugetlbs: >
1) Made sure their support was enabled in kernel > 2) Passed parameters
for their allocation to kernel. There are other > ways, but I use 1GB
page size and I have to allocate them before memory > gets fragmented.
Note, you do that with hugepages=3072. > 3) Passed hugetlbs settings to
libvirt VM domain (XML). > > That is it. I don't mount them myself. I
think (can't check now) they > are automounted to /dev/hugetlbs. By
default, libvirtd runs as root, and > qemu as user qemu. Consuming
hugetlb and releasing them should be > transparent. > > I suggest
starting without softlevel=qemuvm into default runlevel, and > then
check /proc/meminfo. Hugetlbs total and free should be > 0. Do the >
same with qemuvm runlevel. Maybe this runlevel misses something. > > On
2016-12-14 00:05, Manuel Ullmann wrote: >> Hi, >> >> I’m struggling with
virsh not detecting my hugepage mount. >> >> I have the following kernel
command line: >> BOOT_IMAGE=/vmlinuz-4.8.13-gentoo
root=/dev/mapper/gensd-gentoo ro quiet >> splash intel_iommu=on
video=efifb:off,vesafb:off,simplefb:off >>
splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 >>
transparent_hugepage=never hugepages=3072 softlevel=qemuvm >> >> My
startup script outputs the following: >> hugetlbfs /var/lib/hugetlbfs
hugetlbfs >> rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 >>
[2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 >>
[2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs >>
filesystem is not mounted or disabled by administrator config >> virsh
was unsuccessful >> >> So hugetlbfs is definitively mounted, but virsh
does either not detect >> it or some magical option, that I’ve not found
in a day searching for >> it, disables it (the documentation could
explain that better). Since the >> Ubuntu guys refer to the
KVM_HUGEPAGES environment, I tried adding it to >> start-stop-daemon
environment, but it seems non-relevant. 77 is the qemu >> user id, but
I’m quite sure the permission issues were excluded (tried >> root
permissions as well). Besides I had permission errors occasionally >>
occasionally and don’t have them anymore. If you could provide a hint to
script reads like this: >> #!/bin/sh >> cmdline="$(cat /proc/cmdline)"
list >> libvirt-users@redhat.com >>
https://www.redhat.com/mailman/listinfo/libvirt-users >>

Manuel Ullmann

tags

participants (1)