[libvirt] patch for pci passthrough
Hi, This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79 I used the structure as already defined in src/domain_conf.h so analog to usb one now can add pci devices as example: lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01) <hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev> values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7 Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks) Regards Jason -- Collax GmbH . Burkheimer Str. 3 . 79111 Freiburg . Germany p: +49 (0) 89-990157-0 f: +49 (0) 89-990157-11 . http://www.collax.com
Hi Jason, On Tue, 2008-12-16 at 03:02 +0100, Jason Krieg wrote:
Hi,
This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79
I used the structure as already defined in src/domain_conf.h
so analog to usb one now can add pci devices
as example:
lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01)
<hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev>
values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7
Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks)
The patch looks good to me and makes perfect sense; good stuff. One thing that might be worth thinking about is whether we can automatically unload host drivers for a device when assigning; e.g. if you didn't do "rmmod e1000e" before assigning a NIC to a guest, then the guest would just fail to start. Another thing to think about for PCI device assignment is how we can allow a management tool to filter the devices returned from the device enumeration API to only those devices that could successfully be assigned to a guest. This gets quite tricky - if you don't have e.g. VT-d or its a conventional PCI device (in some cases) or if the device is in use or if its a graphics card (currently), then it's just not going to work. We clearly don't want to have a heap of logic in libvirt about what kind of devices can be assigned - and anyway, that logic would be ever changing - so, we perhaps want a ioctl to query KVM as to whether a device is assignable. Cheers, Mark.
On Fri, Dec 19, 2008 at 02:04:43PM +0000, Mark McLoughlin wrote:
Hi Jason,
On Tue, 2008-12-16 at 03:02 +0100, Jason Krieg wrote:
Hi,
This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79
I used the structure as already defined in src/domain_conf.h
so analog to usb one now can add pci devices
as example:
lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01)
<hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev>
values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7
Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks)
The patch looks good to me and makes perfect sense; good stuff.
One thing that might be worth thinking about is whether we can automatically unload host drivers for a device when assigning; e.g. if you didn't do "rmmod e1000e" before assigning a NIC to a guest, then the guest would just fail to start.
Unloading isn't correct because that impact alls devices associated with the device. You just want to unbind te specific device using the 'unbind' file in sysfs. I dont think we should do this automatically though - you wouldn't want to accidentally, automatically remotely unbind your network. I think the node device driver APIs could report what driver is associated with each device they know, and provide an API to bind and unbind the device to/from the driver. Its easy enough for a mgmt app to automatically / seemlessly invoke such an API, after validating that the PCI device isn't asociated with the ethernet device its talking to libvirt over .. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Fri, 2008-12-19 at 14:09 +0000, Daniel P. Berrange wrote:
On Fri, Dec 19, 2008 at 02:04:43PM +0000, Mark McLoughlin wrote:
Hi Jason,
On Tue, 2008-12-16 at 03:02 +0100, Jason Krieg wrote:
Hi,
This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79
I used the structure as already defined in src/domain_conf.h
so analog to usb one now can add pci devices
as example:
lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01)
<hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev>
values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7
Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks)
The patch looks good to me and makes perfect sense; good stuff.
One thing that might be worth thinking about is whether we can automatically unload host drivers for a device when assigning; e.g. if you didn't do "rmmod e1000e" before assigning a NIC to a guest, then the guest would just fail to start.
Unloading isn't correct because that impact alls devices associated with the device. You just want to unbind te specific device using the 'unbind' file in sysfs.
Yeah, and in 2.6.29 we have a pci-stub driver to prevent anything re-attaching to the device, so e.g. $> echo "8086 10f5" > /sys/bus/pci/drivers/pci-stub/new_id $> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/e1000e/unbind $> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/pci-stub/bind
I dont think we should do this automatically though - you wouldn't want to accidentally, automatically remotely unbind your network.
I think the node device driver APIs could report what driver is associated with each device they know, and provide an API to bind and unbind the device to/from the driver. Its easy enough for a mgmt app to automatically / seemlessly invoke such an API, after validating that the PCI device isn't asociated with the ethernet device its talking to libvirt over ..
The management app can do that validation when setting up the guest, but not always when starting the guest - e.g. thinking about autostarted guests ... Cheers, Mark.
On Fri, Dec 19, 2008 at 02:54:45PM +0000, Mark McLoughlin wrote:
On Fri, 2008-12-19 at 14:09 +0000, Daniel P. Berrange wrote:
On Fri, Dec 19, 2008 at 02:04:43PM +0000, Mark McLoughlin wrote:
Hi Jason,
On Tue, 2008-12-16 at 03:02 +0100, Jason Krieg wrote:
Hi,
This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79
I used the structure as already defined in src/domain_conf.h
so analog to usb one now can add pci devices
as example:
lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01)
<hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev>
values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7
Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks)
The patch looks good to me and makes perfect sense; good stuff.
One thing that might be worth thinking about is whether we can automatically unload host drivers for a device when assigning; e.g. if you didn't do "rmmod e1000e" before assigning a NIC to a guest, then the guest would just fail to start.
Unloading isn't correct because that impact alls devices associated with the device. You just want to unbind te specific device using the 'unbind' file in sysfs.
Yeah, and in 2.6.29 we have a pci-stub driver to prevent anything re-attaching to the device, so e.g.
$> echo "8086 10f5" > /sys/bus/pci/drivers/pci-stub/new_id $> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/e1000e/unbind $> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/pci-stub/bind
I dont think we should do this automatically though - you wouldn't want to accidentally, automatically remotely unbind your network.
I think the node device driver APIs could report what driver is associated with each device they know, and provide an API to bind and unbind the device to/from the driver. Its easy enough for a mgmt app to automatically / seemlessly invoke such an API, after validating that the PCI device isn't asociated with the ethernet device its talking to libvirt over ..
The management app can do that validation when setting up the guest, but not always when starting the guest - e.g. thinking about autostarted guests ...
Hmm, which suggests we might want someway to persist the 'unbind' across reboots, so when libvirt does autostart at boot, it'll replay all the unbinds. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Tue, Dec 16, 2008 at 03:02:11AM +0100, Jason Krieg wrote:
Hi,
This patch adds support for the -pcidevice host=bus:dev.func added in kvm-79
I used the structure as already defined in src/domain_conf.h
so analog to usb one now can add pci devices
as example:
lspci: 06:02.0 Network controller: Eicon Networks Corporation Diva Server 2FX (rev 01)
<hostdev mode='subsystem' type='pci'> <source> <address bus="0x06" slot="0x02" function="0x0"/> </source> </hostdev>
values are hex so for bus: 0 to ff, slot: 0 to 1f and function: 0 to 7
Hope this patch is useful it applies to libvirt-0.5.1 and cvs checkout from today (with some hunks)
Okay, applied to CVS along with some reformatting and various fixes. We also need to update the documentation, I'm doing this separately, basically the section elementsUSB of docs/formatdomain.html.in need to be augmented with the new example syntax. I guess one test should be added to the regression too for good measure, Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Jason Krieg -
Mark McLoughlin