[libvirt-users] Virtual gaming/workstation desktops using KVM VGA passthrough?
Dear List, Recently I read an article [1] about how to use KVM and VGA passthrough to create multiple GPU accelerated VM's for use in gaming or as virtual workstations: [1] http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ub... I'm planning on building a KVM workstation to do this using CentOS 7 or Ubuntu 14.04 LTS as the base virtualization host and then an Intel Xeon CPU and two Nvidia GeForce GTX 750's as GPU's for the virtual machines. I'm really new to this sort of GPU passthrough on KVM and am wondering if anyone has any advice or has done this before? Thank you!
Hi Alex, The first bit of advice that I was about to give was that you should have a look at a particular thread on the Arch Linux user forum. But as it turns out, the article that you found (which seems really well-written) already mentions that thread, and in fact even links to one of my comments on that thread, namely this comment: https://bbs.archlinux.org/viewtopic.php?pid=1313007#p1313007 With regard to CentOS 7 vs Ubuntu 14.04, keep in mind that VFIO (which is part of the Linux kernel) is relatively new and is still rapidly evolving. I'm not very familiar with CentOS, but if I'm not mistaken, CentOS 7 uses kernel 3.10 by default. You'll probably want to use a newer kernel version, which would have better VFIO support. Again, I'm not really familiar with CentOS, but my understanding is that many feel that updating the kernel in CentOS isn't a good idea. So with this in mind, you might be better off using a distro that ships with a newer kernel version, such as Fedora or Ubuntu, etc. In my experience, Ubuntu 14.04, which ships with kernel 3.13 and QEMU 2.0, works pretty well with VFIO. However, Ubuntu 14.10, which will ship with kernel 3.16 and QEMU 2.1, seems to work a little better, but is still pre-release and so has a few other bugs. If you go with Ubuntu 14.04, you might want to install a newer "mainline" kernel. But in any case, please do keep in mind that VFIO is still new and so you will almost assuredly encounter some bugs. So don't expect 100% stability. Hope the above is of some help. Regards, GizmoChicken On Fri, Oct 3, 2014 at 5:19 PM, Alex G.S. <alxgrtnstrngl@gmail.com> wrote:
Dear List,
Recently I read an article [1] about how to use KVM and VGA passthrough to create multiple GPU accelerated VM's for use in gaming or as virtual workstations:
[1] http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ub...
I'm planning on building a KVM workstation to do this using CentOS 7 or Ubuntu 14.04 LTS as the base virtualization host and then an Intel Xeon CPU and two Nvidia GeForce GTX 750's as GPU's for the virtual machines.
I'm really new to this sort of GPU passthrough on KVM and am wondering if anyone has any advice or has done this before?
Thank you!
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
GizmoChicken, Read that there's a problem with Nvidia GeForce GTX cards and both Xen and KVM, like it's not on a Nvidia whitelist, I'm a bit concerned. From the ArchLinux thread I've seen some success with Nvidia GeForce GTX cards, this is confusing. The bit I read here was: " That would mean you are stuck with the GTX for that, but Nvidia nerfs their cards to stop passthrough, attempting to force you to pay for a special card that allows it, costing in the neighborhood of like 2000$ for a cheap one. " http://www.reddit.com/r/linux/comments/23lpa6/question_where_are_we_with_vid... Do I need a Quadro for this? In that case it would be impossible to justify. Originally the GeForce GTX 750 Ti seemed like the ideal card for this type of deployment. Now I'm wondering if AMD is safer for the VDI guest. What should I do? CC'd Alex Williamson into the thread so we can get some verification here that Nvidia GeForce GTX are in fact able to do VGA passthrough with a stock Fedora 20 virtualization host using VFIO with a KVM hypervisor. Thanks. On Sat, Oct 4, 2014 at 5:13 PM, Gizmo Chicken <gizmochicken@gmail.com> wrote:
Hi Alex,
The first bit of advice that I was about to give was that you should have a look at a particular thread on the Arch Linux user forum. But as it turns out, the article that you found (which seems really well-written) already mentions that thread, and in fact even links to one of my comments on that thread, namely this comment: https://bbs.archlinux.org/viewtopic.php?pid=1313007#p1313007
With regard to CentOS 7 vs Ubuntu 14.04, keep in mind that VFIO (which is part of the Linux kernel) is relatively new and is still rapidly evolving.
I'm not very familiar with CentOS, but if I'm not mistaken, CentOS 7 uses kernel 3.10 by default. You'll probably want to use a newer kernel version, which would have better VFIO support. Again, I'm not really familiar with CentOS, but my understanding is that many feel that updating the kernel in CentOS isn't a good idea. So with this in mind, you might be better off using a distro that ships with a newer kernel version, such as Fedora or Ubuntu, etc.
In my experience, Ubuntu 14.04, which ships with kernel 3.13 and QEMU 2.0, works pretty well with VFIO. However, Ubuntu 14.10, which will ship with kernel 3.16 and QEMU 2.1, seems to work a little better, but is still pre-release and so has a few other bugs. If you go with Ubuntu 14.04, you might want to install a newer "mainline" kernel. But in any case, please do keep in mind that VFIO is still new and so you will almost assuredly encounter some bugs. So don't expect 100% stability.
Hope the above is of some help.
Regards, GizmoChicken
On Fri, Oct 3, 2014 at 5:19 PM, Alex G.S. <alxgrtnstrngl@gmail.com> wrote:
Dear List,
Recently I read an article [1] about how to use KVM and VGA passthrough to create multiple GPU accelerated VM's for use in gaming or as virtual workstations:
[1]
http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ub...
I'm planning on building a KVM workstation to do this using CentOS 7 or Ubuntu 14.04 LTS as the base virtualization host and then an Intel Xeon
CPU
and two Nvidia GeForce GTX 750's as GPU's for the virtual machines.
I'm really new to this sort of GPU passthrough on KVM and am wondering if anyone has any advice or has done this before?
Thank you!
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On Mon, 2014-10-06 at 22:59 -0400, Alex G.S. wrote:
GizmoChicken,
Read that there's a problem with Nvidia GeForce GTX cards and both Xen and KVM, like it's not on a Nvidia whitelist, I'm a bit concerned. From the ArchLinux thread I've seen some success with Nvidia GeForce GTX cards, this is confusing. The bit I read here was:
" That would mean you are stuck with the GTX for that, but Nvidia nerfs their cards to stop passthrough, attempting to force you to pay for a special card that allows it, costing in the neighborhood of like 2000$ for a cheap one. " http://www.reddit.com/r/linux/comments/23lpa6/question_where_are_we_with_vid...
Do I need a Quadro for this? In that case it would be impossible to justify. Originally the GeForce GTX 750 Ti seemed like the ideal card for this type of deployment. Now I'm wondering if AMD is safer for the VDI guest. What should I do?
CC'd Alex Williamson into the thread so we can get some verification here that Nvidia GeForce GTX are in fact able to do VGA passthrough with a stock Fedora 20 virtualization host using VFIO with a KVM hypervisor.
If you want to use a stock distro kernel then you need to use OVMF (UEFI) for the guest or get lucky and not use IGD on the host (and use a very new kernel and use a wrapper around qemu for the x-vga option). See: http://vfio.blogspot.com/2014/08/primary-graphics-assignment-without-vga.htm... http://vfio.blogspot.com/2014/09/libvirt-now-supports-ovmf.html http://vfio.blogspot.com/2014/09/ovmf-split-image-support.html Nvidia has been throwing us some curve balls with their driver and hypervisor detection, but there are still combinations that work. For the latest driver it's required to hide the KVM hypervisor signature and disable Hyper-V enlightenment. If you're looking for something supported in a VM, Quadro is the only game in town afaik. I have a GTX750 (not Ti) that works great on stock Fedora 20 kernel, upstream qemu + ovmf + libvirt, win8.1 guest (obligatory ymmv). Thanks, Alex
On Sat, Oct 4, 2014 at 5:13 PM, Gizmo Chicken <gizmochicken@gmail.com> wrote:
Hi Alex,
The first bit of advice that I was about to give was that you should have a look at a particular thread on the Arch Linux user forum. But as it turns out, the article that you found (which seems really well-written) already mentions that thread, and in fact even links to one of my comments on that thread, namely this comment: https://bbs.archlinux.org/viewtopic.php?pid=1313007#p1313007
With regard to CentOS 7 vs Ubuntu 14.04, keep in mind that VFIO (which is part of the Linux kernel) is relatively new and is still rapidly evolving.
I'm not very familiar with CentOS, but if I'm not mistaken, CentOS 7 uses kernel 3.10 by default. You'll probably want to use a newer kernel version, which would have better VFIO support. Again, I'm not really familiar with CentOS, but my understanding is that many feel that updating the kernel in CentOS isn't a good idea. So with this in mind, you might be better off using a distro that ships with a newer kernel version, such as Fedora or Ubuntu, etc.
In my experience, Ubuntu 14.04, which ships with kernel 3.13 and QEMU 2.0, works pretty well with VFIO. However, Ubuntu 14.10, which will ship with kernel 3.16 and QEMU 2.1, seems to work a little better, but is still pre-release and so has a few other bugs. If you go with Ubuntu 14.04, you might want to install a newer "mainline" kernel. But in any case, please do keep in mind that VFIO is still new and so you will almost assuredly encounter some bugs. So don't expect 100% stability.
Hope the above is of some help.
Regards, GizmoChicken
On Fri, Oct 3, 2014 at 5:19 PM, Alex G.S. <alxgrtnstrngl@gmail.com> wrote:
Dear List,
Recently I read an article [1] about how to use KVM and VGA passthrough to create multiple GPU accelerated VM's for use in gaming or as virtual workstations:
[1]
http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ub...
I'm planning on building a KVM workstation to do this using CentOS 7 or Ubuntu 14.04 LTS as the base virtualization host and then an Intel Xeon
CPU
and two Nvidia GeForce GTX 750's as GPU's for the virtual machines.
I'm really new to this sort of GPU passthrough on KVM and am wondering if anyone has any advice or has done this before?
Thank you!
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
Nvidia has been throwing us some curve balls with their driver and hypervisor detection, but there are still combinations that work.
Okay, thanks for the information. Would it better then to go with an AMD Radeon R7 or R9 for VGA passthrough to the Linux or Windows VM? Are they problem free and more predictable when it comes to passthrough? Best, AlexGS On Mon, Oct 6, 2014 at 11:46 PM, Alex Williamson <alex.williamson@redhat.com
wrote:
GizmoChicken,
Read that there's a problem with Nvidia GeForce GTX cards and both Xen and KVM, like it's not on a Nvidia whitelist, I'm a bit concerned. From the ArchLinux thread I've seen some success with Nvidia GeForce GTX cards,
On Mon, 2014-10-06 at 22:59 -0400, Alex G.S. wrote: this
is confusing. The bit I read here was:
" That would mean you are stuck with the GTX for that, but Nvidia nerfs their cards to stop passthrough, attempting to force you to pay for a special card that allows it, costing in the neighborhood of like 2000$ for a cheap one. "
http://www.reddit.com/r/linux/comments/23lpa6/question_where_are_we_with_vid...
Do I need a Quadro for this? In that case it would be impossible to justify. Originally the GeForce GTX 750 Ti seemed like the ideal card
for
this type of deployment. Now I'm wondering if AMD is safer for the VDI guest. What should I do?
CC'd Alex Williamson into the thread so we can get some verification here that Nvidia GeForce GTX are in fact able to do VGA passthrough with a stock Fedora 20 virtualization host using VFIO with a KVM hypervisor.
If you want to use a stock distro kernel then you need to use OVMF (UEFI) for the guest or get lucky and not use IGD on the host (and use a very new kernel and use a wrapper around qemu for the x-vga option). See:
http://vfio.blogspot.com/2014/08/primary-graphics-assignment-without-vga.htm... http://vfio.blogspot.com/2014/09/libvirt-now-supports-ovmf.html http://vfio.blogspot.com/2014/09/ovmf-split-image-support.html
Nvidia has been throwing us some curve balls with their driver and hypervisor detection, but there are still combinations that work. For the latest driver it's required to hide the KVM hypervisor signature and disable Hyper-V enlightenment. If you're looking for something supported in a VM, Quadro is the only game in town afaik. I have a GTX750 (not Ti) that works great on stock Fedora 20 kernel, upstream qemu + ovmf + libvirt, win8.1 guest (obligatory ymmv). Thanks,
Alex
On Sat, Oct 4, 2014 at 5:13 PM, Gizmo Chicken <gizmochicken@gmail.com> wrote:
Hi Alex,
The first bit of advice that I was about to give was that you should have a look at a particular thread on the Arch Linux user forum. But as it turns out, the article that you found (which seems really well-written) already mentions that thread, and in fact even links to one of my comments on that thread, namely this comment: https://bbs.archlinux.org/viewtopic.php?pid=1313007#p1313007
With regard to CentOS 7 vs Ubuntu 14.04, keep in mind that VFIO (which is part of the Linux kernel) is relatively new and is still rapidly evolving.
I'm not very familiar with CentOS, but if I'm not mistaken, CentOS 7 uses kernel 3.10 by default. You'll probably want to use a newer kernel version, which would have better VFIO support. Again, I'm not really familiar with CentOS, but my understanding is that many feel that updating the kernel in CentOS isn't a good idea. So with this in mind, you might be better off using a distro that ships with a newer kernel version, such as Fedora or Ubuntu, etc.
In my experience, Ubuntu 14.04, which ships with kernel 3.13 and QEMU 2.0, works pretty well with VFIO. However, Ubuntu 14.10, which will ship with kernel 3.16 and QEMU 2.1, seems to work a little better, but is still pre-release and so has a few other bugs. If you go with Ubuntu 14.04, you might want to install a newer "mainline" kernel. But in any case, please do keep in mind that VFIO is still new and so you will almost assuredly encounter some bugs. So don't expect 100% stability.
Hope the above is of some help.
Regards, GizmoChicken
Dear List,
Recently I read an article [1] about how to use KVM and VGA
On Fri, Oct 3, 2014 at 5:19 PM, Alex G.S. <alxgrtnstrngl@gmail.com> wrote: passthrough to
create multiple GPU accelerated VM's for use in gaming or as virtual workstations:
[1]
http://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ub...
I'm planning on building a KVM workstation to do this using CentOS 7
or
Ubuntu 14.04 LTS as the base virtualization host and then an Intel Xeon CPU and two Nvidia GeForce GTX 750's as GPU's for the virtual machines.
I'm really new to this sort of GPU passthrough on KVM and am wondering if anyone has any advice or has done this before?
Thank you!
_______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
participants (3)
-
Alex G.S. -
Alex Williamson -
Gizmo Chicken