[libvirt] need custom /dev entries in LXC

Hi, We run nvidia devices inside libvirt-managed LXC containers. It used to be that simply doing: $ echo 'c 195:* rwm' > /sys/fs/cgroup/devices/libvirt/lxc Then, after booting the container, we would do: $ mknod -m 666 /dev/nvidia0 c 195 0 .... would be good enough to run our CUDA applications. But, according to: $ cat src/lxc/lxc_container.c The CAP_MKNOD capability is being dropped and only a specific set of devices is being created before booting the container. Is there any reason why this is not per-device configurable? Thanks, - Michael R. Hines

On Wed, May 22, 2013 at 06:22:12PM -0400, Michael R. Hines wrote:
Hi,
We run nvidia devices inside libvirt-managed LXC containers.
It used to be that simply doing:
$ echo 'c 195:* rwm' > /sys/fs/cgroup/devices/libvirt/lxc
Then, after booting the container, we would do:
$ mknod -m 666 /dev/nvidia0 c 195 0
.... would be good enough to run our CUDA applications.
But, according to:
$ cat src/lxc/lxc_container.c
The CAP_MKNOD capability is being dropped and only a specific set of devices is being created before booting the container.
Is there any reason why this is not per-device configurable?
With recent libvirt you can pass through arbitrary block and character devices explicitly, using the following XML: http://libvirt.org/formatdomain.html#elementsHostDevCaps As such there is never any need to change cgroups or use mknod as you describe. Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

On 06/03/2013 06:03 AM, Daniel P. Berrange wrote:
On Wed, May 22, 2013 at 06:22:12PM -0400, Michael R. Hines wrote:
Hi,
We run nvidia devices inside libvirt-managed LXC containers.
It used to be that simply doing:
$ echo 'c 195:* rwm' > /sys/fs/cgroup/devices/libvirt/lxc
Then, after booting the container, we would do:
$ mknod -m 666 /dev/nvidia0 c 195 0
.... would be good enough to run our CUDA applications.
But, according to:
$ cat src/lxc/lxc_container.c
The CAP_MKNOD capability is being dropped and only a specific set of devices is being created before booting the container.
Is there any reason why this is not per-device configurable? With recent libvirt you can pass through arbitrary block and character devices explicitly, using the following XML:
http://libvirt.org/formatdomain.html#elementsHostDevCaps
As such there is never any need to change cgroups or use mknod as you describe.
Daniel
Thanks for the response, Daniel. That's good news.
participants (2)
-
Daniel P. Berrange
-
Michael R. Hines