8:42 p.m.
(Copied from my post on the Arch Linux forums:
https://bbs.archlinux.org/viewtopic.php?pid=1650650#p1650650)
I have a FreeNAS 10 KVM setup via libvirt on my Arch server and I'd like to
be able to test out the virtualization features in the nightly FreeNAS 10
builds but the problem is that I can't seem to get VT-x to correctly pass
through to the guest. I have followed the [u][url=
https://wiki.archlinux.org/index.php/KVM#Nested_virtualization]Nested
Virtualization[/url][/u] section of the KVM wiki and I'm sure it's
supported and enabled.
Here's proof
[code] [bran@nas ~]$ sudo systool -m kvm_intel -v | grep nested
nested = "Y"
[bran@nas ~]$ lscpu|grep Virtualization
Virtualization: VT-x[/code]
So the host/hardware isn't the problem, I believe the problem lies within
libvirt.
RedHat says to use [b]copy host CPU configuration[/b] or
[b]host-passthrough[/b], with the latter being preferred. If I use the
former I get the [b]CMT not supported[/b] error, but if I type in
[b]host-passthrough[/b] which according to [u][url=
https://bbs.archlinux.org/viewtopic.php?id=214539]this post[/url][/u]
should work. When I set [b]host-passthrough[/b] it allows the system to
boot up, but when I try to start a guest FreeNAS gives me the error that
VT-x instructions aren't available, I have no idea how to check them either
since the [b]proc[/b] pseudo-filesystem doesn't exist in BSD. I know for a
fact that this isn't a problem with the FreeNAS builds because I've been
testing them out for months on my Windows 10 desktop via VMware and nested
virtualization works without issue, so it must be an issue with
KVM/libvirt.
Can someone clue me in on what the issue is?
Here's the entire XML config for the FreeNAS VM
[code]<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made
using:
virsh edit FreeNAS_10
or other application using the libvirt API.
-->
<domain type='kvm'>
<name>FreeNAS_10</name>
<uuid>ea816b85-7685-495a-bc97-28a882f190d7</uuid>
<title>FreeNAS v10</title>
<description>Nightly Alpha Test Releases</description>
<memory unit='KiB'>6340608</memory>
<currentMemory unit='KiB'>6340608</currentMemory>
<vcpu placement='static'>4</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
<bootmenu enable='no'/>
</os>
<features>
<acpi/>
<apic/>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough'/>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/sbin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/mnt/storage/vm-storage/FreeNAS_Disk1.img'/>
<target dev='vdb' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x07'
function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/mnt/storage/vm-storage/FreeNAS_Disk2.img'/>
<target dev='vdc' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x08'
function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/mnt/storage/vm-storage/FreeNAS_Disk3.img'/>
<target dev='vdd' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x09'
function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='directsync'
io='native'/>
<source file='/var/lib/libvirt/images/FreeNAS_10.img'/>
<target dev='vde' bus='virtio'/>
<boot order='1'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x06'
function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='hda' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='0'
target='0' unit='0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x1'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x04'
function='0x0'/>
</controller>
<interface type='bridge'>
<mac address='52:54:00:24:5c:08'/>
<source bridge='vmbridge'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'/>
<address type='virtio-serial' controller='0' bus='0'
port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'
listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<video>
<model type='qxl' ram='65536' vram='65536'
vgamem='16384' heads='1'
primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x02'
function='0x0'/>
</video>
<redirdev bus='usb' type='spicevmc'>
<address type='usb' bus='0' port='1'/>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
<address type='usb' bus='0' port='2'/>
</redirdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x0a'
function='0x0'/>
</memballoon>
</devices>
</domain>[/code]
Thanks,
Brandon Golway
5:48 a.m.
On Mon, Aug 29, 2016 at 09:42:17PM -0400, Brandon Golway wrote:
(Copied from my post on the Arch Linux forums:
https://bbs.archlinux.org/viewtopic.php?pid=1650650#p1650650)
I have a FreeNAS 10 KVM setup via libvirt on my Arch server and I'd like to
be able to test out the virtualization features in the nightly FreeNAS 10
builds but the problem is that I can't seem to get VT-x to correctly pass
through to the guest. I have followed the [u][url=
https://wiki.archlinux.org/index.php/KVM#Nested_virtualization]Nested
Virtualization[/url][/u] section of the KVM wiki and I'm sure it's
supported and enabled.
Here's proof
[code] [bran@nas ~]$ sudo systool -m kvm_intel -v | grep nested
nested = "Y"
[bran@nas ~]$ lscpu|grep Virtualization
Virtualization: VT-x[/code]
So the host/hardware isn't the problem, I believe the problem lies within
libvirt.
RedHat says to use [b]copy host CPU configuration[/b] or
[b]host-passthrough[/b], with the latter being preferred. If I use the
former I get the [b]CMT not supported[/b] error, but if I type in
[b]host-passthrough[/b] which according to [u][url=
https://bbs.archlinux.org/viewtopic.php?id=214539]this post[/url][/u]
should work. When I set [b]host-passthrough[/b] it allows the system to
boot up, but when I try to start a guest FreeNAS gives me the error that
VT-x instructions aren't available, I have no idea how to check them either
since the [b]proc[/b] pseudo-filesystem doesn't exist in BSD. I know for a
fact that this isn't a problem with the FreeNAS builds because I've been
testing them out for months on my Windows 10 desktop via VMware and nested
virtualization works without issue, so it must be an issue with
KVM/libvirt.
Can someone clue me in on what the issue is?
Here's the entire XML config for the FreeNAS VM
[code]<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made
using:
virsh edit FreeNAS_10
or other application using the libvirt API.
-->
Just as I see you grabbed it from the place where it is stored, you
should rather instead do 'virsh dumpxml FreeNAS_10', or remove the
comment above. Otherwise it steers other users to believing that
mangling with internal XML files is safe.
<domain type='kvm'>
<name>FreeNAS_10</name>
<uuid>ea816b85-7685-495a-bc97-28a882f190d7</uuid>
<title>FreeNAS v10</title>
<description>Nightly Alpha Test Releases</description>
<memory unit='KiB'>6340608</memory>
<currentMemory unit='KiB'>6340608</currentMemory>
<vcpu placement='static'>4</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
<bootmenu enable='no'/>
</os>
<features>
<acpi/>
<apic/>
<vmport state='off'/>
</features>
<cpu mode='host-passthrough'/>
Although what you did should be enough for the nested virt to work, I
have just an idea to try out. I can't try it because I don't have
nested virt turned on for the module and don't want to reboot now. But
if you have new enough libvirt, you should be able to do:
<cpu mode='host-passthrough'>
<feature policy='require' name='vmx'/>
</cpu>
Also check that you are not missing the kvm_intel module in the L1 guest
kernel and other similar non-nested troubleshooting steps as well.
Hope that helps,
Martin
5:11 a.m.
On Wed, Aug 31, 2016 at 12:48:33PM +0200, Martin Kletzander wrote:
On Mon, Aug 29, 2016 at 09:42:17PM -0400, Brandon Golway wrote:
> (Copied from my post on the Arch Linux forums:
[...]
> Can someone clue me in on what the issue is?
>
> Here's the entire XML config for the FreeNAS VM
>
> [code]<!--
> WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
> OVERWRITTEN AND LOST. Changes to this xml configuration should be made
> using:
> virsh edit FreeNAS_10
> or other application using the libvirt API.
> -->
>
Just as I see you grabbed it from the place where it is stored, you
should rather instead do 'virsh dumpxml FreeNAS_10', or remove the
comment above. Otherwise it steers other users to believing that
mangling with internal XML files is safe.
Yeah, or use the `virt-xml` tool (comes as part of `virt-install`
package). To pass-through host CPU to the level-1 guest:
$ virt-xml FreeNAS_10 --edit --cpu host-passthrough,clearxml=yes
If you issue this while the guest is running, it'll take affect on next
reboot.
> <domain type='kvm'>
> <name>FreeNAS_10</name>
> <uuid>ea816b85-7685-495a-bc97-28a882f190d7</uuid>
> <title>FreeNAS v10</title>
> <description>Nightly Alpha Test Releases</description>
> <memory unit='KiB'>6340608</memory>
> <currentMemory unit='KiB'>6340608</currentMemory>
> <vcpu placement='static'>4</vcpu>
> <os>
> <type arch='x86_64'
machine='pc-i440fx-2.6'>hvm</type>
> <bootmenu enable='no'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <vmport state='off'/>
> </features>
> <cpu mode='host-passthrough'/>
Although what you did should be enough for the nested virt to work, I
have just an idea to try out. I can't try it because I don't have
nested virt turned on for the module and don't want to reboot now.
Assuming you're talking about the physical host, you don't need a reboot
of the host :-) You can simply:
$ sudo rmmod kvm-intel
$ echo "options kvm-intel nested=y" > /etc/modprobe.d/dist.conf
$ sudo modprobe kvm-intel
But
if you have new enough libvirt, you should be able to do:
<cpu mode='host-passthrough'>
<feature policy='require' name='vmx'/>
</cpu>
Also check that you are not missing the kvm_intel module in the L1 guest
kernel and other similar non-nested troubleshooting steps as well.
Yeah. If the L1 guest has the /dev/kvm character device, then he's good:
$ file /dev/kvm
/dev/kvm: character special (10/232)
--
/kashyap
8 a.m.
On Thu, Sep 01, 2016 at 12:11:17PM +0200, Kashyap Chamarthy wrote:
On Wed, Aug 31, 2016 at 12:48:33PM +0200, Martin Kletzander wrote:
> On Mon, Aug 29, 2016 at 09:42:17PM -0400, Brandon Golway wrote:
> > (Copied from my post on the Arch Linux forums:
[...]
> > Can someone clue me in on what the issue is?
> >
> > Here's the entire XML config for the FreeNAS VM
> >
> > [code]<!--
> > WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
> > OVERWRITTEN AND LOST. Changes to this xml configuration should be made
> > using:
> > virsh edit FreeNAS_10
> > or other application using the libvirt API.
> > -->
> >
>
> Just as I see you grabbed it from the place where it is stored, you
> should rather instead do 'virsh dumpxml FreeNAS_10', or remove the
> comment above. Otherwise it steers other users to believing that
> mangling with internal XML files is safe.
Yeah, or use the `virt-xml` tool (comes as part of `virt-install`
package). To pass-through host CPU to the level-1 guest:
$ virt-xml FreeNAS_10 --edit --cpu host-passthrough,clearxml=yes
If you issue this while the guest is running, it'll take affect on next
reboot.
> > <domain type='kvm'>
> > <name>FreeNAS_10</name>
> > <uuid>ea816b85-7685-495a-bc97-28a882f190d7</uuid>
> > <title>FreeNAS v10</title>
> > <description>Nightly Alpha Test Releases</description>
> > <memory unit='KiB'>6340608</memory>
> > <currentMemory unit='KiB'>6340608</currentMemory>
> > <vcpu placement='static'>4</vcpu>
> > <os>
> > <type arch='x86_64'
machine='pc-i440fx-2.6'>hvm</type>
> > <bootmenu enable='no'/>
> > </os>
> > <features>
> > <acpi/>
> > <apic/>
> > <vmport state='off'/>
> > </features>
> > <cpu mode='host-passthrough'/>
>
> Although what you did should be enough for the nested virt to work, I
> have just an idea to try out. I can't try it because I don't have
> nested virt turned on for the module and don't want to reboot now.
Assuming you're talking about the physical host, you don't need a reboot
of the host :-) You can simply:
$ sudo rmmod kvm-intel
$ echo "options kvm-intel nested=y" > /etc/modprobe.d/dist.conf
$ sudo modprobe kvm-intel
Unless:
$ zgrep -i kvm_intel /proc/config.gz
CONFIG_KVM_INTEL=y
Maybe I could switch to =m when I'm building the next kernel.
> But
> if you have new enough libvirt, you should be able to do:
>
> <cpu mode='host-passthrough'>
> <feature policy='require' name='vmx'/>
> </cpu>
>
> Also check that you are not missing the kvm_intel module in the L1 guest
> kernel and other similar non-nested troubleshooting steps as well.
Yeah. If the L1 guest has the /dev/kvm character device, then he's good:
$ file /dev/kvm
/dev/kvm: character special (10/232)
--
/kashyap
2:39 p.m.
New subject: [libvirt-users] snapshots handout
hi all & @Kayshap
does [this handout on
snapshots](https://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-hando... by
Kashyap Chamarthy (I believe from 2012) still apply to current versions of libvirt-bin
& qemu?
- ubuntu: 14.04
- qemu: 2.0.0
- libvirt-bin 1.2.2-0ubuntu13.1.17
if not, is there an updated version available somewhere?
Right now mainly interested in internal snapshots (which can be taken & removed on a
running machine, if I am not mistaken)
4:32 a.m.
New subject: [libvirt-users] snapshots handout
On Sat, Sep 03, 2016 at 02:39:11PM +0800, gunnar.wagner(a)netcologne.de wrote:
hi all & @Kayshap
does [this handout on
snapshots](https://kashyapc.fedorapeople.org/virt/lc-2012/snapshots-hando...
by Kashyap Chamarthy (I believe from 2012) still apply to current
versions of libvirt-bin & qemu?
Yes, as the URL indicates, it's from 2012. And, most of the content
there is valid. However, some things have changed, with recent
improvements in QEMU and libvirt. E.g. from the "blockcommit" section
in the doc, now (from QEMU 2.0) it is possible to merge down the current
active layer into base.
- ubuntu: 14.04
- qemu: 2.0.0
- libvirt-bin 1.2.2-0ubuntu13.1.17
if not, is there an updated version available somewhere?
Though I haven't updated that document to reflect current new
reality, that details mentioned there are still relevant.
However, Eric Blake has done an excellent talk on backing chains,
snapshots, and their management at KVM Forum 2015.
Here's the talk (slides are in the video description):
https://www.youtube.com/watch?v=etIGp12RHRE -- Backing Chain
management in QEMU and libvirt by Eric Blake
Right now mainly interested in internal snapshots (which can be
taken
& removed on a running machine, if I am not mistaken)
Yes, internal snapshots can be performed on a running guest (or
offline). Compare and contrast them with external snapshots, to see
what works best for your use case.
--
/kashyap
2878
days inactive
2887
days old
6 comments
4 participants
participants (4)
-
Brandon Golway -
gunnar.wagner＠netcologne.de -
Kashyap Chamarthy -
Martin Kletzander