6:50 p.m.
* Fischer, Anna (anna.fischer(a)hp.com) wrote:
So, when setting a breakpoint for the exit() call I'm getting a
bit closer to figuring where it kills my guest.
Thanks, this helps clarify what is happening.
Breakpoint 1, exit (status=1) at exit.c:99
99 {
Current language: auto
The current source language is "auto; currently c".
(gdb) bt
#0 exit (status=1) at exit.c:99
#1 0x0000000000470c6e in assigned_dev_pci_read_config (d=0x259c6f0, address=64, len=4)
assigned_dev_pci_read_config(..., 64, 4)
^^
This is a libvirt issue. When you use virt-manager it has libvirtd
fork/exec qemu-kvm. libvirtd will drop privileges and run qemu-kvm as
user qemu (or perhaps root if you've edited qemu.conf). Regardless of
the user, it clears capabilities. Reading PCI config space beyond just
the header requires CAP_SYS_ADMIN. The above is reading the first 4
bytes of device dependent config space, and the kernel is returning 0
because qemu doesn't have CAP_SYS_ADMIN.
Basically, this means that device assignment w/ libvirt will break
MSI/MSI-X because qemu will never be able to see that the host device
has those PCI capabilities. This, in turn, renders VF device assignment
useless (since a VF is required to support MSI and/or MSI-X).
Granting CAP_SYS_ADMIN for each qemu instance that does device assignment
would render the privilege reduction useless (CAP_SYS_ADMIN is the
kitchen sink catchall of the Linux capability system).
Hmmph...
5:04 a.m.
New subject: [libvirt] pci-stub error and MSI-X for KVM guest
On Thu, Jan 07, 2010 at 04:50:03PM -0800, Chris Wright wrote:
* Fischer, Anna (anna.fischer(a)hp.com) wrote:
> So, when setting a breakpoint for the exit() call I'm getting a bit closer to
figuring where it kills my guest.
Thanks, this helps clarify what is happening.
> Breakpoint 1, exit (status=1) at exit.c:99
> 99 {
> Current language: auto
> The current source language is "auto; currently c".
> (gdb) bt
> #0 exit (status=1) at exit.c:99
> #1 0x0000000000470c6e in assigned_dev_pci_read_config (d=0x259c6f0, address=64,
len=4)
assigned_dev_pci_read_config(..., 64, 4)
^^
This is a libvirt issue. When you use virt-manager it has libvirtd
fork/exec qemu-kvm. libvirtd will drop privileges and run qemu-kvm as
user qemu (or perhaps root if you've edited qemu.conf). Regardless of
the user, it clears capabilities. Reading PCI config space beyond just
the header requires CAP_SYS_ADMIN. The above is reading the first 4
bytes of device dependent config space, and the kernel is returning 0
because qemu doesn't have CAP_SYS_ADMIN.
Hmm, libvirt also chown()'s the files in /sys/bus/pci/devices/<DEVICE>/*
to 'qemu' (and sets SELinux context) so that the unprivileged QEMU process
can have full read/write access to them. I would have hoped that would
avoid the need to have any capabilities like CAP_SYS_ADMIN :-(
Basically, this means that device assignment w/ libvirt will break
MSI/MSI-X because qemu will never be able to see that the host device
has those PCI capabilities. This, in turn, renders VF device assignment
useless (since a VF is required to support MSI and/or MSI-X).
Granting CAP_SYS_ADMIN for each qemu instance that does device assignment
would render the privilege reduction useless (CAP_SYS_ADMIN is the
kitchen sink catchall of the Linux capability system).
Yeah that's pretty troublesome, even when libvirt runs QEMU as 'root', it
will
remove all capabilities. Why is the 'CAP_SYS_ADMIN' check there - is it a
mistakenly over-zealous permission check that could be removed, just relying
on access controls on the sysfs /sys/bus/pci/devices/<DEVICE>/config
file ?
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
12:03 p.m.
New subject: [libvirt] pci-stub error and MSI-X for KVM guest
* Daniel P. Berrange (berrange(a)redhat.com) wrote:
On Thu, Jan 07, 2010 at 04:50:03PM -0800, Chris Wright wrote:
> Basically, this means that device assignment w/ libvirt will break
> MSI/MSI-X because qemu will never be able to see that the host device
> has those PCI capabilities. This, in turn, renders VF device assignment
> useless (since a VF is required to support MSI and/or MSI-X).
>
> Granting CAP_SYS_ADMIN for each qemu instance that does device assignment
> would render the privilege reduction useless (CAP_SYS_ADMIN is the
> kitchen sink catchall of the Linux capability system).
Yeah that's pretty troublesome, even when libvirt runs QEMU as 'root', it
will
remove all capabilities. Why is the 'CAP_SYS_ADMIN' check there - is it a
mistakenly over-zealous permission check that could be removed, just relying
on access controls on the sysfs /sys/bus/pci/devices/<DEVICE>/config
file ?
Tools like lspci expect the ability to read a pci function's config
space header. In the past there have been devices that wedge when reading
areas of device dependent config space that are not implemented, so this
portion of config space is protected (hence the need for privileges to
do lspci -v).
thanks,
-chris
5462
days inactive
5462
days old
2 comments
2 participants
participants (2)
-
Chris Wright -
Daniel P. Berrange