[libvirt] Question about virtio-pci in Aarch64
Hi Cole && All, Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. Nowadays I find that the default "virtio" bus is "virtio-mmio" , not "virtio-pci". Since virtio-mmio do not has the host plugged function , something is wrong with the function in OpenStack Nova. Then I search and find some mail information as belows by Cole :-) 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html I see some efforts have been doing by you. Really thanks for your efforts for Aarch64. And I have some small questions. 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci for Aarch64? 2. If not , how can I change the xml file(generated by Virsh) for virtio-pci so that Qemu can recognize it ? Really big thanks for your help. Looking forwards to your response! Best Regards, Kevin Zhao
On 06/12/2016 10:29 AM, Kevin Zhao wrote:
Hi Cole && All, Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. Nowadays I find that the default "virtio" bus is "virtio-mmio" , not "virtio-pci". Since virtio-mmio do not has the host plugged function , something is wrong with the function in OpenStack Nova.
Then I search and find some mail information as belows by Cole :-) 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html
I see some efforts have been doing by you. Really thanks for your efforts for Aarch64. And I have some small questions. 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
2. If not , how can I change the xml file(generated by Virsh) for virtio-pci so that Qemu can recognize it ?
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like: <interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface> But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio' - Cole
Hi Cole, Thanks for your kindly help. On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote:
Hi Cole && All, Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. Nowadays I find that the default "virtio" bus is "virtio-mmio" , not "virtio-pci". Since virtio-mmio do not has the host plugged function , something is wrong with the function in OpenStack Nova.
Then I search and find some mail information as belows by Cole :-) 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html
I see some efforts have been doing by you. Really thanks for your efforts for Aarch64. And I have some small questions. 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
OK . Hope it will be true in the future.
2. If not , how can I change the xml file(generated by Virsh) for
virtio-pci so that Qemu can recognize it ?
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio'
In my opinion, that is to say, when I want to attach a block device to a
guest and hope it take effects soon,* the only way is using 'scsi' instead of 'virtio'* in Aarch64 just like what virt-manager does since scsi supports hot plugged. Am I right?
- Cole
On 06/12/2016 10:08 PM, Kevin Zhao wrote:
Hi Cole, Thanks for your kindly help.
On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com <mailto:crobinso@redhat.com>> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote: > Hi Cole && All, > Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! > I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. > Nowadays I find that the default "virtio" bus is "virtio-mmio" , not > "virtio-pci". Since virtio-mmio do not has the host plugged function , > something is wrong with the function in OpenStack Nova. > > Then I search and find some mail information as belows by Cole :-) > 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html > 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html > > I see some efforts have been doing by you. Really thanks for your > efforts for Aarch64. And I have some small questions. > 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci > for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
OK . Hope it will be true in the future.
> 2. If not , how can I change the xml file(generated by Virsh) for > virtio-pci so that Qemu can recognize it ? >
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio'
In my opinion, that is to say, when I want to attach a block device to a guest and hope it take effects soon,*the only way is using 'scsi' instead of 'virtio'* in Aarch64 just like what virt-manager does since scsi supports hot plugged. Am I right?
On x86 at least virtio-blk + pci supports hotplug, but I don't know if that combo works for aarch64 at the moment. However disk=scsi hotplug should currently work on aarch64 with controller=virtio-scsi, I've tested that before Thanks, Cole
Yeah I see. In virt-manager ,aarch64 can realize hotplug by virtio-scsi . Thanks Cole ~ On Mon, 13 Jun 2016 at 22:07 Cole Robinson <crobinso@redhat.com> wrote:
Hi Cole, Thanks for your kindly help.
On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com <mailto:crobinso@redhat.com>> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote: > Hi Cole && All, > Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! > I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. > Nowadays I find that the default "virtio" bus is "virtio-mmio" , not > "virtio-pci". Since virtio-mmio do not has the host plugged function , > something is wrong with the function in OpenStack Nova. > > Then I search and find some mail information as belows by Cole :-) > 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html > 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html > > I see some efforts have been doing by you. Really thanks for your > efforts for Aarch64. And I have some small questions. > 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci > for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But
On 06/12/2016 10:08 PM, Kevin Zhao wrote: the main
blocker is that most linux distributions currently don't work out of
the box
with virtio-pci, at least Fedora doesn't as far as I know. So
changing the
default now is pre-mature
OK . Hope it will be true in the future.
> 2. If not , how can I change the xml file(generated by
Virsh) for
> virtio-pci so that Qemu can recognize it ? >
The latest libvirt-1.3.5 release accepts device XML containing
<address
type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for
example
if you want to specify a virtio nic, but have it use virtio-pci
rather than
virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
But there isn't currently any switch to say 'always give me
virtio-pci instead
of virtio-mmio'
In my opinion, that is to say, when I want to attach a block device to a
guest
and hope it take effects soon,*the only way is using 'scsi' instead of 'virtio'* in Aarch64 just like what virt-manager does since scsi supports hot plugged. Am I right?
On x86 at least virtio-blk + pci supports hotplug, but I don't know if that combo works for aarch64 at the moment. However disk=scsi hotplug should currently work on aarch64 with controller=virtio-scsi, I've tested that before
Thanks, Cole
Hi Cole, Long time no see~ Thanks for giving help me about AArch64 of virtio-pci. You have rich experience about this, I am green hand and seeking for your help again :-) On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote:
Hi Cole && All, Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. Nowadays I find that the default "virtio" bus is "virtio-mmio" , not "virtio-pci". Since virtio-mmio do not has the host plugged function , something is wrong with the function in OpenStack Nova.
Then I search and find some mail information as belows by Cole :-) 1) https://www.redhat.com/archives/libvir-list/2015-December/ msg00217.html 2) https://www.redhat.com/archives/libvir-list/2016-March/ msg00167.html
I see some efforts have been doing by you. Really thanks for your efforts for Aarch64. And I have some small questions. 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
2. If not , how can I change the xml file(generated by Virsh) for
virtio-pci so that Qemu can recognize it ?
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/ After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ? 2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64? Big Big Thanks~
But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio'
- Cole
On 08/17/2016 07:49 AM, Kevin Zhao wrote:
Hi Cole, Long time no see~ Thanks for giving help me about AArch64 of virtio-pci. You have rich experience about this, I am green hand and seeking for your help again :-)
Hello again, happy to help :)
On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com <mailto:crobinso@redhat.com>> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote: > Hi Cole && All, > Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! > I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. > Nowadays I find that the default "virtio" bus is "virtio-mmio" , not > "virtio-pci". Since virtio-mmio do not has the host plugged function , > something is wrong with the function in OpenStack Nova. > > Then I search and find some mail information as belows by Cole :-) > 1) https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html <https://www.redhat.com/archives/libvir-list/2015-December/msg00217.html> > 2) https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html <https://www.redhat.com/archives/libvir-list/2016-March/msg00167.html> > > I see some efforts have been doing by you. Really thanks for your > efforts for Aarch64. And I have some small questions. > 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci > for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
> 2. If not , how can I change the xml file(generated by Virsh) for > virtio-pci so that Qemu can recognize it ? >
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/
After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device
My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ?
My guess is that the support isn't complete. Last I checked Fedora doesn't even work with aarch64 virtio-pci + uefi, because it requires some kernel changes that haven't been fully upstreamed yet. But that was several months ago... There may be a way to work around it nowadays but I don't personally know. You may want to test with either RHELSA if you have a copy, or linaro images.
2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64?
That should work fine in my testing with RHELSA, so I don't think it's a libvirt or qemu limitation. Probably the guest OS + UEFI. - Cole
Big Big Thanks~
But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio'
- Cole
On Wed, 2016-08-17 at 09:18 -0400, Cole Robinson wrote:
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/ After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ? My guess is that the support isn't complete. Last I checked Fedora doesn't even work with aarch64 virtio-pci + uefi, because it requires some kernel changes that haven't been fully upstreamed yet. But that was several months ago... There may be a way to work around it nowadays but I don't personally know. You may want to test with either RHELSA if you have a copy, or linaro images. 2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64? That should work fine in my testing with RHELSA, so I don't think it's a libvirt or qemu limitation. Probably the guest OS + UEFI.
Looks like Debian 8.5 simply doesn't include virtio-pci support: $ lsinitramfs /boot/initrd.img | grep virtio lib/modules/3.16.0-4-arm64/kernel/drivers/scsi/virtio_scsi.ko lib/modules/3.16.0-4-arm64/kernel/drivers/block/virtio_blk.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_ring.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_mmio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/net/virtio_net.ko $ modprobe virtio-pci modprobe: FATAL: Module virtio-pci not found. I think kernel 3.16.0 is simply too old. Unfortunately, I tried the same setup with an up-to-date Debian testing guest and I was still unable to boot it with virtio-pci disk and network. Support for this kind of setup is still far from being ready for mass consumption, I'm afraid. -- Andrea Bolognani / Red Hat / Virtualization
Hi Andrea, I'm using the Debian Jessie with kernel Linux u202154 4.4.0-135-arm64. But I am not sure whether the virtio-pci module has been built into the kernel. not as a module. So I am changing to Ubuntu and RHEL73. As far as I know, ubuntu Wily has the driver of virtio-pci. Thanks~ On 17 August 2016 at 21:51, Andrea Bolognani <abologna@redhat.com> wrote:
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/
After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device
My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ?
My guess is that the support isn't complete. Last I checked Fedora doesn't even work with aarch64 virtio-pci + uefi, because it requires some kernel changes that haven't been fully upstreamed yet. But that was several months ago... There may be a way to work around it nowadays but I don't
know. You may want to test with either RHELSA if you have a copy, or
On Wed, 2016-08-17 at 09:18 -0400, Cole Robinson wrote: pfifo_fast state personally linaro
images.
2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64?
That should work fine in my testing with RHELSA, so I don't think it's a libvirt or qemu limitation. Probably the guest OS + UEFI.
Looks like Debian 8.5 simply doesn't include virtio-pci support:
$ lsinitramfs /boot/initrd.img | grep virtio lib/modules/3.16.0-4-arm64/kernel/drivers/scsi/virtio_scsi.ko lib/modules/3.16.0-4-arm64/kernel/drivers/block/virtio_blk.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_ring.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_mmio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/net/virtio_net.ko
$ modprobe virtio-pci modprobe: FATAL: Module virtio-pci not found.
I think kernel 3.16.0 is simply too old.
Unfortunately, I tried the same setup with an up-to-date Debian testing guest and I was still unable to boot it with virtio-pci disk and network. Support for this kind of setup is still far from being ready for mass consumption, I'm afraid.
-- Andrea Bolognani / Red Hat / Virtualization
On Wed, Aug 17, 2016 at 03:51:47PM +0200, Andrea Bolognani wrote:
On Wed, 2016-08-17 at 09:18 -0400, Cole Robinson wrote:
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/ After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ? My guess is that the support isn't complete. Last I checked Fedora doesn't even work with aarch64 virtio-pci + uefi, because it requires some kernel changes that haven't been fully upstreamed yet. But that was several months ago... There may be a way to work around it nowadays but I don't personally know. You may want to test with either RHELSA if you have a copy, or linaro images. 2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64? That should work fine in my testing with RHELSA, so I don't think it's a libvirt or qemu limitation. Probably the guest OS + UEFI.
Looks like Debian 8.5 simply doesn't include virtio-pci support:
$ lsinitramfs /boot/initrd.img | grep virtio lib/modules/3.16.0-4-arm64/kernel/drivers/scsi/virtio_scsi.ko lib/modules/3.16.0-4-arm64/kernel/drivers/block/virtio_blk.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_ring.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio_mmio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/virtio/virtio.ko lib/modules/3.16.0-4-arm64/kernel/drivers/net/virtio_net.ko
$ modprobe virtio-pci modprobe: FATAL: Module virtio-pci not found.
I think kernel 3.16.0 is simply too old.
According to the changelog it has been enabled in 3.14.7-1 on all architectures: http://metadata.ftp-master.debian.org/changelogs//main/l/linux/stable_change... Cheers, -- Guido
On Fri, 2016-08-19 at 10:27 +0200, Guido Günther wrote:
I think kernel 3.16.0 is simply too old. According to the changelog it has been enabled in 3.14.7-1 on all architectures: http://metadata.ftp-master.debian.org/changelogs//main/l/linux/stable_change...
Are you referring to the * [armhf] Enable VIRTIO_BALLOON and VIRTIO_PCI (Closes: #750742) entry? That's armhf, not arm64. Or was there a different reference that I missed? Anyway, in my 8.5 arm64 guest, virtio-pci doesn't seem to be there at all: $ grep VIRTIO /boot/config-3.16.0-4-arm64 CONFIG_NET_9P_VIRTIO=m CONFIG_VIRTIO_BLK=m CONFIG_SCSI_VIRTIO=m CONFIG_VIRTIO_NET=m CONFIG_VIRTIO_CONSOLE=m CONFIG_HW_RANDOM_VIRTIO=m CONFIG_VIRTIO=m CONFIG_VIRTIO_BALLOON=m CONFIG_VIRTIO_MMIO=m # CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES is not set -- Andrea Bolognani / Red Hat / Virtualization
On Fri, Aug 19, 2016 at 11:52:55AM +0200, Andrea Bolognani wrote:
On Fri, 2016-08-19 at 10:27 +0200, Guido Günther wrote:
I think kernel 3.16.0 is simply too old. According to the changelog it has been enabled in 3.14.7-1 on all architectures: http://metadata.ftp-master.debian.org/changelogs//main/l/linux/stable_change...
Are you referring to the
* [armhf] Enable VIRTIO_BALLOON and VIRTIO_PCI (Closes: #750742)
entry? That's armhf, not arm64. Or was there a different reference that I missed?
I mixed up armhf and arm64 but …
Anyway, in my 8.5 arm64 guest, virtio-pci doesn't seem to be there at all:
$ grep VIRTIO /boot/config-3.16.0-4-arm64 CONFIG_NET_9P_VIRTIO=m CONFIG_VIRTIO_BLK=m CONFIG_SCSI_VIRTIO=m CONFIG_VIRTIO_NET=m CONFIG_VIRTIO_CONSOLE=m CONFIG_HW_RANDOM_VIRTIO=m CONFIG_VIRTIO=m CONFIG_VIRTIO_BALLOON=m CONFIG_VIRTIO_MMIO=m # CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES is not set
… 4.6.0 has it, including the version available in jessie-backports: $ grep VIRTIO_PCI boot/config-4.6.0-0.bpo.1-arm64 CONFIG_VIRTIO_PCI=m CONFIG_VIRTIO_PCI_LEGACY=y Cheers, -- Guido
On Fri, 2016-08-19 at 12:08 +0200, Guido Günther wrote:
I mixed up armhf and arm64 but … … 4.6.0 has it, including the version available in jessie-backports: $ grep VIRTIO_PCI boot/config-4.6.0-0.bpo.1-arm64 CONFIG_VIRTIO_PCI=m CONFIG_VIRTIO_PCI_LEGACY=y
Right, I didn't take backports into account :) -- Andrea Bolognani / Red Hat / Virtualization
Hi Cole, Greate ! Thanks for your help. On 17 August 2016 at 21:18, Cole Robinson <crobinso@redhat.com> wrote:
On 08/17/2016 07:49 AM, Kevin Zhao wrote:
Hi Cole, Long time no see~ Thanks for giving help me about AArch64 of virtio-pci. You have rich experience about this, I am green hand and seeking for your help again :-)
Hello again, happy to help :)
On 13 June 2016 at 06:21, Cole Robinson <crobinso@redhat.com <mailto:crobinso@redhat.com>> wrote:
On 06/12/2016 10:29 AM, Kevin Zhao wrote: > Hi Cole && All, > Nice meeting you in the libvirt mail-list. Greetings from Linaro !!! > I am Kevin Zhao from Linaro and working for OpenStack on Aarch64. > Nowadays I find that the default "virtio" bus is "virtio-mmio" , not > "virtio-pci". Since virtio-mmio do not has the host plugged function , > something is wrong with the function in OpenStack Nova. > > Then I search and find some mail information as belows by Cole :-) > 1) https://www.redhat.com/archives/libvir-list/2015- December/msg00217.html <https://www.redhat.com/archives/libvir-list/2015- December/msg00217.html> > 2) https://www.redhat.com/archives/libvir-list/2016- March/msg00167.html <https://www.redhat.com/archives/libvir-list/2016- March/msg00167.html> > > I see some efforts have been doing by you. Really thanks for your > efforts for Aarch64. And I have some small questions. > 1. Is Libvirt planning to replace the default virtio-mmio to virtio-pci > for Aarch64?
We have vague plans to maybe change the virtio-mmio default to virtio-pci at some future point, based on some new enough qemu machine type. But the main blocker is that most linux distributions currently don't work out of the box with virtio-pci, at least Fedora doesn't as far as I know. So changing the default now is pre-mature
> 2. If not , how can I change the xml file(generated by Virsh) for > virtio-pci so that Qemu can recognize it ? >
The latest libvirt-1.3.5 release accepts device XML containing <address type='pci'/> . The address block is an explicit request to libvirt to 'allocate a PCI address' rather than the default virtio-mmio. So for example if you want to specify a virtio nic, but have it use virtio-pci rather than virtio-mmio, you can do something like:
<interface type='network'> <source network='default'/> <model type='virtio'/> <address type='pci'/> </interface>
I have changed the libvirt to 1.3.5 now, also add the pci to net-device xml like: <address type='pci'/>,then use the virsh to boot the VM,the total xml file is: https://paste.fedoraproject.org/409534/71434141/
After booting, the eth0 device disappear(eth0 occur when the address is virtio-mmio), but I can find another net-device, also it can't work for dhcp: 2: enp2s1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:0d:25:26 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe0d:2526/64 scope link valid_lft forever preferred_lft forever Running lspci: 02:01.0 Ethernet controller: Red Hat, Inc Virtio network device
My question is: 1. I use Debian 8 AArch64 as the Guest OS, do you think the virtio-pci for net-device works is OK ?
My guess is that the support isn't complete. Last I checked Fedora doesn't even work with aarch64 virtio-pci + uefi, because it requires some kernel changes that haven't been fully upstreamed yet. But that was several months ago... There may be a way to work around it nowadays but I don't personally know. You may want to test with either RHELSA if you have a copy, or linaro images.
You are right ! Today I happen to receive a RHEL7.3 for AArch64. I have been using this as Guest OS. It has virtio-pci driver. In Libvirt 1.3.5, I use the xml : https://paste.fedoraproject.org/410158/21672147/ Virtio-pci disk works fine(RHEL73+UEFI). But using virsh attach-device to attach a device: <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/var/lib/libvirt/images/rhel2.qcow2"/> <target dev="vdb" bus="virtio"/> <address type='pci'/> </disk> After adding, I can't find vdb in the Guest OS. I need to rescan the pci by: echo 1 > /sys/bus/pci/rescan Then vdb appear. Do you have the same problem? Another question, when changing to Libvirt 2.1.0, I add the : <address type='pci'/> to a disk device or net device, it can't work, it seems that user need to assign the PCI manually ? It means that I should assign the slots and bus all by myself? Thanks~
2. If I change the disk address-type to pci(Libvirt pass the virtio pci parameters to Qemu for disk device), but I can't boot the VM. Does Qemu not support virtio pci for disk device in AArch64?
That should work fine in my testing with RHELSA, so I don't think it's a libvirt or qemu limitation. Probably the guest OS + UEFI.
- Cole
Big Big Thanks~
But there isn't currently any switch to say 'always give me virtio-pci instead of virtio-mmio'
- Cole
participants (4)
-
Andrea Bolognani -
Cole Robinson -
Guido Günther -
Kevin Zhao