[libvirt] PCI devices passthough to LXC containers using libvirt
Hi All Please let me know if anyone have given access to PCI devices for a LXC container. I have tried getting the xml from "virsh nodedev-dumpxml pci_device" and added to the libvirt xml file as shown below <device> <name>pci_0000_03_00_0</name> <parent>pci_0000_00_03_0</parent> <driver> <name>nvidia</name> </driver> <capability type='pci'> <domain>0</domain> <bus>3</bus> <slot>0</slot> <function>0</function> <product id='0x06fd' /> <vendor id='0x10de'>nVidia Corporation</vendor> </capability> </device> But it didn't work. I see the logs and it says couldn't get physical and virtual functions of these devices with error get_physical_function_linux:323 : Attempting to get SR IOV physical function for device with sysfs path '/sys/devices/pci0000:00/0000:00:00.0' 16:48:34.033: 13802: debug : get_sriov_function:270 : Attempting to resolve device path from device link '/sys/devices/pci0000:00/0000:00:00.0/physfn' 16:48:34.033: 13802: debug : get_sriov_function:274 : SR IOV function link '/sys/devices/pci0000:00/0000:00:00.0/physfn' does not exist 16:48:34.033: 13802: debug : get_virtual_functions_linux:348 : Attempting to get SR IOV virtual functions for devicewith sysfs path '/sys/devices/pci0000:00/0000:00:00.0' If anyone got some guidelines how to debug, please let me know. Thanks in advance Regards Devendra
On Wed, Jul 27, 2011 at 02:13:13PM -0700, Devendra K. Modium wrote:
Hi All
Please let me know if anyone have given access to PCI devices for a LXC container.
What are you actually trying to achieve as your end result ? PCI device assignment doesn't entirely make sense in the LXC world. Since the container shares a kernel with the "host" OS, there's nothing to really assign the PCI device to. If you're trying to use a PCI NIC, then is the bridging/NAT setup not sufficient ? If you're trying to use a block device, I've got a patch which will let libvirt's LXC code automatically mount any host block as a filesystem inside the container at startup. Your next mail about cgroups device ACL makes me think this is what you're trying todo... If neither of these are suitable, then we could think about how to support logical device assignment. ie assign a NIC, or assign a block device, rather than assigning the PCI device proividing it. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Hi Daniel, Thanks for the reply. I am trying to access GPU devices from inside the containers. (Only way I know)For this I need to add the GPU device numbers in ACL/(device whiteist) to get access to these devices from inside the container. I found that libvirt lxc driver currently allows set of devices while starting the container. I have browsed through the libvirt lxc code and I believe there is no elegant way currently where you can request the devices to be allowed inside LXC container using the usual libvirt xml file.CORRECT ME IF I AM WRONG. Please provide your suggestions if any. Thanks in advance Regards Devendra ----- Original Message ----- From: "Daniel P. Berrange" <berrange@redhat.com> To: "Devendra K. Modium" <dmodium@isi.edu> Cc: libvir-list@redhat.com Sent: Thursday, August 4, 2011 11:53:02 AM Subject: Re: [libvirt] PCI devices passthough to LXC containers using libvirt On Wed, Jul 27, 2011 at 02:13:13PM -0700, Devendra K. Modium wrote:
Hi All
Please let me know if anyone have given access to PCI devices for a LXC container.
What are you actually trying to achieve as your end result ? PCI device assignment doesn't entirely make sense in the LXC world. Since the container shares a kernel with the "host" OS, there's nothing to really assign the PCI device to. If you're trying to use a PCI NIC, then is the bridging/NAT setup not sufficient ? If you're trying to use a block device, I've got a patch which will let libvirt's LXC code automatically mount any host block as a filesystem inside the container at startup. Your next mail about cgroups device ACL makes me think this is what you're trying todo... If neither of these are suitable, then we could think about how to support logical device assignment. ie assign a NIC, or assign a block device, rather than assigning the PCI device proividing it. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On Thu, Aug 04, 2011 at 11:23:20AM -0700, Devendra K. Modium wrote:
Hi Daniel,
Thanks for the reply.
I am trying to access GPU devices from inside the containers.
(Only way I know)For this I need to add the GPU device numbers in ACL/(device whiteist) to get access to these devices from inside the container.
Ah ha, so you're not really wanting todo PCI device passthrough, but rather just want to be able to access something like /dev/video0 inside the container ? We don't currently have a way to enable that in LXC, but our host device passthrough was sort of anticipating this need. To support this in libvirt I think we'd need to define something new in the XML along the lines of <hostdev mode='capability' type='video'> <source name='video0'/> </hostdev>
I found that libvirt lxc driver currently allows set of devices while starting the container.
I have browsed through the libvirt lxc code and I believe there is no elegant way currently where you can request the devices to be allowed inside LXC container using the usual libvirt xml file.CORRECT ME IF I AM WRONG.
That is correct, we can't currently do that. Having said that, since LXC is not currently at all secure[1], you can in fact just modify the cgroups device ACL once inside the container Daniel [1] There is work going on upstream to introduce proper user/capability namespaces into the kernel which will plug the biggest missing piece of security. We also aim to integrate sVirt into LXC to enable use of MAC to plug the DAC security holes. -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
participants (2)
-
Daniel P. Berrange -
Devendra K. Modium