Further followups! We are correlating DEBUG-level output from libvirt with the libvirt 1.2.2 code to try to figure out what libvirt is doing under the hood. Even though we have the log level set to 1 (info) in our libvirtd.conf, we are not seeing the VIR_DEBUG() [1] statements being printed out. There are tons of other presumably-debug lines of output showing up in our log. We are sort of expecting to see the output from [1] in our logs somewhere, given the type of usb mounting we are trying to do? [1] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/lxc/lxc_controller.c;h=c0... On Wed, Apr 16, 2014 at 3:58 PM, Filip Maj <fil@saucelabs.com> wrote:
To follow up on this a little bit, tail'ing kern.log while trying to get our little container up doesn't yield anything with apparmor complaining, so, unless I'm looking in the wrong spots for apparmor logs (which I don't think so, as I see other apparmor-related log entries in kern.log), I am not entirely sure this is an apparmor issue at this point.
On Wed, Apr 16, 2014 at 3:25 PM, Filip Maj <fil@saucelabs.com> wrote:
Yeah, AppArmor is enabled, but I put everything (that I could find) into complain mode:
$ sudo apparmor_status apparmor module is loaded. 12 profiles are loaded. 3 profiles are in enforce mode. lxc-container-default lxc-container-default-with-mounting lxc-container-default-with-nesting 9 profiles are in complain mode. /sbin/dhclient /usr/bin/lxc-start /usr/lib/NetworkManager/nm-dhcp-client.action /usr/lib/connman/scripts/dhclient-script /usr/lib/libvirt/virt-aa-helper /usr/sbin/libvirtd /usr/sbin/ntpd /usr/sbin/rsyslogd /usr/sbin/tcpdump 3 processes have profiles defined. 0 processes are in enforce mode. 2 processes are in complain mode. /usr/sbin/libvirtd (30419) /usr/sbin/ntpd (3418) 1 processes are unconfined but have a profile defined. /usr/sbin/rsyslogd (626)
And still get issues. From libvirtd.log:
2014-04-16 22:19:10.855+0000: 30419: info : libvirt version: 1.2.2 2014-04-16 22:19:10.855+0000: 30419: error : virNetSocketReadWire:1446 : Cannot recv data: Connection reset by peer 2014-04-16 22:19:10.940+0000: 30420: error : virLXCProcessStart:1299 : internal error: guest failed to start: Unable to create device //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not permitted
2014-04-16 22:19:10.964+0000: 30420: warning : virLXCDomainReAttachHostUsbDevices:388 : Unable to find device 000.000 in list of active USB devices
Thanks in advance for any help, Daniel!
Cheers, Fil
On Tue, Apr 15, 2014 at 1:33 AM, Daniel P. Berrange <berrange@redhat.com>wrote:
On Fri, Apr 11, 2014 at 05:32:28PM -0700, Filip Maj wrote:
Hi!
First post, kind of a noobie. I've been working with LXC and libvirt for a few months now. Trying to do some interesting things with containers and Android devices :D Here's my entire domain definition:
<domain type='lxc'> <name>oshi32134</name> <uuid>xxxxx</uuid> <memory unit='KiB'>3145728</memory> <currentMemory unit='KiB'>3145728</currentMemory> <vcpu placement='static'>1</vcpu> <resource> <partition>/machine</partition> </resource> <os> <type arch='i686'>exe</type> <init>/sbin/init</init> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/lib/libvirt/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/some/valid/filesystem/location'/> <target dir='/'/> </filesystem> <filesystem type='mount' accessmode='passthrough'> <source dir='/another/valid/filesystem/location'/> <target dir='/mnt/android'/> </filesystem> <interface type='bridge'> <mac address='xx:xx:xx:xx:xx:xx'/> <source bridge='br1'/> </interface> <console type='pty'> <target type='lxc' port='0'/> </console> <hostdev mode='capabilities' type='misc'> <source> <char>/dev/kvm</char> </source> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x04e8'/> <product id='0x6860'/> </source> </hostdev> </devices> </domain>
Your config looks fine here.
Everything worked fine until I added the USB <hostdev> element. I'm essentially trying to get access to a physical Android device
connected to
the host from inside a container. When I go to start the container, I get an error about Operation not permitted. Here's the relevant bits from /var/log/libvirt/lxc/machine.log:
2014-04-11 22:46:40.491+0000: starting up PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/libvirt_lxc --name oshi32134 --console 24 --security=none --handshake 27 --background --veth vnet1 2014-04-11 22:46:40.597+0000: 685: info : libvirt version: 1.2.2 2014-04-11 22:46:40.597+0000: 685: error : virLXCControllerSetupHostdevSubsysUSB:1390 : Unable to create device //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not permitted Unable to create device //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not permitted
Do you have AppArmour enabled on the machine. That seems like the most likely thing that would result in libvirt getting that permission error.
Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|