[PATCH] docs: add zpci information to pci-addresses.rst
Add some information on how pci address work on s390x. Signed-off-by: Cornelia Huck <cohuck@redhat.com> --- docs/pci-addresses.rst | 63 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/pci-addresses.rst b/docs/pci-addresses.rst index 923783a151b0..9e241a24fcfb 100644 --- a/docs/pci-addresses.rst +++ b/docs/pci-addresses.rst @@ -184,3 +184,66 @@ guest OS rather than as ``0001:08:00.1``, which is the address of the device on the host. Of course, all the rules and behaviors described above still apply. + +zPCI addresses +============== + +For s390x machines, PCI addresses are handled yet differently. No +topology information is relayed in the PCI addresses; instead, the +``fid`` and ``uid`` elements of the ``zpci`` device convey information. +In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + +Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device -- 2.21.1
On 4/15/20 12:51 PM, Cornelia Huck wrote:
Add some information on how pci address work on s390x.
Signed-off-by: Cornelia Huck <cohuck@redhat.com> Conny, thanks for catching this "wacky case"... :) I took the liberty to remove "completely" because there needs to be room for pci multifunction support... :D
--- docs/pci-addresses.rst | 63 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)
diff --git a/docs/pci-addresses.rst b/docs/pci-addresses.rst index 923783a151b0..9e241a24fcfb 100644 --- a/docs/pci-addresses.rst +++ b/docs/pci-addresses.rst @@ -184,3 +184,66 @@ guest OS rather than as ``0001:08:00.1``, which is the address of the device on the host.
Of course, all the rules and behaviors described above still apply. + +zPCI addresses +============== + +For s390x machines, PCI addresses are handled yet differently. No +topology information is relayed in the PCI addresses; instead, the +``fid`` and ``uid`` elements of the ``zpci`` device convey information. Should it be mentioned that qemu uses the pci address internally to plug the device into its pci bus since the pci address as far as I know must still be properly provided or generated.
+In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + This explains that the uid is used by the guest to define the pci domain of the guest device. How about an addition for the fid? Something like:
The function id 'fid' defines the slot address of the pci card in the guest. It can be observed in the guest at /sys/bus/pci/slots. In the example given above the card would be at /sys/bus/pci/slots/00000000.
+Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device
and the card would be at /sys/bus/pci/slots/00000003. -- Mit freundlichen Grüßen/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
Sorry about the time travel. I will try to give up that super power... :( On 4/14/20 11:06 PM, Boris Fiuczynski wrote:
On 4/15/20 12:51 PM, Cornelia Huck wrote:
Add some information on how pci address work on s390x.
Signed-off-by: Cornelia Huck <cohuck@redhat.com> Conny, thanks for catching this "wacky case"... :) I took the liberty to remove "completely" because there needs to be room for pci multifunction support... :D
--- docs/pci-addresses.rst | 63 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)
diff --git a/docs/pci-addresses.rst b/docs/pci-addresses.rst index 923783a151b0..9e241a24fcfb 100644 --- a/docs/pci-addresses.rst +++ b/docs/pci-addresses.rst @@ -184,3 +184,66 @@ guest OS rather than as ``0001:08:00.1``, which is the address of the device on the host. Of course, all the rules and behaviors described above still apply. + +zPCI addresses +============== + +For s390x machines, PCI addresses are handled yet differently. No +topology information is relayed in the PCI addresses; instead, the +``fid`` and ``uid`` elements of the ``zpci`` device convey information. Should it be mentioned that qemu uses the pci address internally to plug the device into its pci bus since the pci address as far as I know must still be properly provided or generated.
+In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + This explains that the uid is used by the guest to define the pci domain of the guest device. How about an addition for the fid? Something like:
The function id 'fid' defines the slot address of the pci card in the guest. It can be observed in the guest at /sys/bus/pci/slots. In the example given above the card would be at /sys/bus/pci/slots/00000000.
+Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device
and the card would be at /sys/bus/pci/slots/00000003.
-- Mit freundlichen Grüßen/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On Tue, 14 Apr 2020 23:06:47 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
On 4/15/20 12:51 PM, Cornelia Huck wrote:
+zPCI addresses +============== + +For s390x machines, PCI addresses are handled yet differently. No +topology information is relayed in the PCI addresses; instead, the +``fid`` and ``uid`` elements of the ``zpci`` device convey information. Should it be mentioned that qemu uses the pci address internally to plug the device into its pci bus since the pci address as far as I know must still be properly provided or generated.
Not sure how much is autogenerated; however, as this document talks about how pci addresses in the xml translate to pci addresses in the guest, I think we can simply ignore it?
+In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + This explains that the uid is used by the guest to define the pci domain of the guest device. How about an addition for the fid? Something like:
The function id 'fid' defines the slot address of the pci card in the guest. It can be observed in the guest at /sys/bus/pci/slots. In the example given above the card would be at /sys/bus/pci/slots/00000000.
Hm, is it intentional that this does not show up in the actual pci address? But maybe I'm confused.
+Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device
and the card would be at /sys/bus/pci/slots/00000003.
Do you want to explain the fid/slot stuff in a patch on top? Despite your attempts at time travel, this patch has already been pushed :)
On 4/15/20 3:42 PM, Cornelia Huck wrote:
On Tue, 14 Apr 2020 23:06:47 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
On 4/15/20 12:51 PM, Cornelia Huck wrote:
+zPCI addresses +============== + +For s390x machines, PCI addresses are handled yet differently. No +topology information is relayed in the PCI addresses; instead, the +``fid`` and ``uid`` elements of the ``zpci`` device convey information. Should it be mentioned that qemu uses the pci address internally to plug the device into its pci bus since the pci address as far as I know must still be properly provided or generated.
Not sure how much is autogenerated; however, as this document talks about how pci addresses in the xml translate to pci addresses in the guest, I think we can simply ignore it? ok. I just wanted to explain why the pci address part still exists and why libvirt also will autogenerate all other required bridges.
+In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + This explains that the uid is used by the guest to define the pci domain of the guest device. How about an addition for the fid? Something like:
The function id 'fid' defines the slot address of the pci card in the guest. It can be observed in the guest at /sys/bus/pci/slots. In the example given above the card would be at /sys/bus/pci/slots/00000000.
Hm, is it intentional that this does not show up in the actual pci address? But maybe I'm confused.
Are you referring to the slot of the pci address? If yes, the pci slot does not provided the required address space for the function id. Also we once said that the pci bus in qemu would be like the pci bus of the CEC and the firmware does provide a a function id for every pci function. The zpci fid does one allow to do the same.
+Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device
and the card would be at /sys/bus/pci/slots/00000003.
Do you want to explain the fid/slot stuff in a patch on top? Despite your attempts at time travel, this patch has already been pushed :)
It seems to be a very pushy time... :( no matter how many breaks are produced... ;) 11 minutes from patch production time to commit time. Why even send it for review? :D Once I understand you confusion above I will provide a patch... -- Mit freundlichen Grüßen/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On Wed, 15 Apr 2020 16:23:46 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
On 4/15/20 3:42 PM, Cornelia Huck wrote:
On Tue, 14 Apr 2020 23:06:47 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
On 4/15/20 12:51 PM, Cornelia Huck wrote:
+In the simplest case, the following XML snippet + +:: + + <controller type='pci' index='0' model='pci-root'/> + <controller type='pci' index='1' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='1'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'> + <zpci uid='0x0002' fid='0x00000001'/> + </address> + </controller> + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'> + <zpci uid='0x0001' fid='0x00000000'/> + </address> + </interface> + +will result in the following in a Linux guest:: + + 0001:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device + +Note that the PCI bridge is not visible in the guest; s390x always has a flat +topology. + +Neither are any changes in the PCI address visible in the guest; replacing +the PCI address for the virtio-net device with + +:: + + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + +will result in the exactly same view in the guest, as the addresses there +are generated from the information provided via the ``zpci`` element (in +fact, from the ``uid``). + This explains that the uid is used by the guest to define the pci domain of the guest device. How about an addition for the fid? Something like:
The function id 'fid' defines the slot address of the pci card in the guest. It can be observed in the guest at /sys/bus/pci/slots. In the example given above the card would be at /sys/bus/pci/slots/00000000.
Hm, is it intentional that this does not show up in the actual pci address? But maybe I'm confused. Are you referring to the slot of the pci address?
Yes.
If yes, the pci slot does not provided the required address space for the function id. Also we once said that the pci bus in qemu would be like the pci bus of the CEC and the firmware does provide a a function id for every pci function. The zpci fid does one allow to do the same.
Ok, now I dimly remember something like that. Still confusing, but makes sense.
+Therefore, replacing the virtio-net device definition with the following XML +snippet + +:: + + <interface type='bridge'> + <mac address='02:ca:fe:fa:ce:04'/> + <source bridge='virbr0'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x3'> + <zpci uid='0x0007' fid='0x00000003'/> + </address> + </interface> + +will yield the following result in a Linux guest:: + + 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device
and the card would be at /sys/bus/pci/slots/00000003.
Do you want to explain the fid/slot stuff in a patch on top? Despite your attempts at time travel, this patch has already been pushed :)
It seems to be a very pushy time... :( no matter how many breaks are produced... ;) 11 minutes from patch production time to commit time. Why even send it for review? :D
Well, I don't have commit rights O:-)
Once I understand you confusion above I will provide a patch...
I'd say just go ahead.
On Wed, 2020-04-15 at 16:45 +0200, Cornelia Huck wrote:
On Wed, 15 Apr 2020 16:23:46 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
Once I understand you confusion above I will provide a patch...
I'd say just go ahead.
While I appreciate the sentiment, I would rather not have an excessive amount of detail added to this page. The idea behind it was to show users that they shouldn't expect the address in the domain XML to match the one in the guest OS, with a few real-life examples to illustrate the point. So, I don't think the details of how exactly zPCI IDs translate to guest-visible PCI addresses is in scope. It would be great information to have in some other document, though! Is there something like that in qemu.git, or in the QEMU wiki? Those are the places where I would expect it to live, since it's not really tied to libvirt... -- Andrea Bolognani / Red Hat / Virtualization
On Wed, 15 Apr 2020 18:38:06 +0200 Andrea Bolognani <abologna@redhat.com> wrote:
On Wed, 2020-04-15 at 16:45 +0200, Cornelia Huck wrote:
On Wed, 15 Apr 2020 16:23:46 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
Once I understand you confusion above I will provide a patch...
I'd say just go ahead.
While I appreciate the sentiment, I would rather not have an excessive amount of detail added to this page.
The idea behind it was to show users that they shouldn't expect the address in the domain XML to match the one in the guest OS, with a few real-life examples to illustrate the point. So, I don't think the details of how exactly zPCI IDs translate to guest-visible PCI addresses is in scope.
It would be great information to have in some other document, though! Is there something like that in qemu.git, or in the QEMU wiki? Those are the places where I would expect it to live, since it's not really tied to libvirt...
I don't think there's much in QEMU documentation yet; someone(tm) wrote https://virtualpenguins.blogspot.com/2018/02/notes-on-pci-on-s390x.html, which might provide a starting point.
On 4/15/20 6:38 PM, Andrea Bolognani wrote:
On Wed, 2020-04-15 at 16:45 +0200, Cornelia Huck wrote:
On Wed, 15 Apr 2020 16:23:46 +0200 Boris Fiuczynski <fiuczy@linux.ibm.com> wrote:
Once I understand you confusion above I will provide a patch...
I'd say just go ahead.
While I appreciate the sentiment, I would rather not have an excessive amount of detail added to this page.
The idea behind it was to show users that they shouldn't expect the address in the domain XML to match the one in the guest OS, with a few real-life examples to illustrate the point. So, I don't think the details of how exactly zPCI IDs translate to guest-visible PCI addresses is in scope.
It would be great information to have in some other document, though! Is there something like that in qemu.git, or in the QEMU wiki? Those are the places where I would expect it to live, since it's not really tied to libvirt...
I disagree because fid is a parameter of the pci address just like domain, bus, slot, function and uid. Besides the fact that one of the first sentences in the document is "When discussing PCI addresses, it's important to understand the relationship between the addresses that can be seen in the domain XML and those that are visible inside the guest OS." as a document reader/user of libvirt I would not expect to search around in other documentation for fid if all other parameters are correlated here. So how about a short explanatory sentence for fid like: "The slot for the PCI device in the guest OS is defined by the fid (function id)." -- Mit freundlichen Grüßen/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On Thu, 2020-04-16 at 11:46 +0200, Boris Fiuczynski wrote:
On 4/15/20 6:38 PM, Andrea Bolognani wrote:
The idea behind it was to show users that they shouldn't expect the address in the domain XML to match the one in the guest OS, with a few real-life examples to illustrate the point. So, I don't think the details of how exactly zPCI IDs translate to guest-visible PCI addresses is in scope.
It would be great information to have in some other document, though! Is there something like that in qemu.git, or in the QEMU wiki? Those are the places where I would expect it to live, since it's not really tied to libvirt...
I disagree because fid is a parameter of the pci address just like domain, bus, slot, function and uid. Besides the fact that one of the first sentences in the document is "When discussing PCI addresses, it's important to understand the relationship between the addresses that can be seen in the domain XML and those that are visible inside the guest OS."
You're right, the current language is not really explaining the purpose of the document correctly - it's making it sound like it's the complete guide to all things PCI addresses, which it's definitely not intended to be. What about Looking at the configuration for a guest, it would be reasonable to expect that each PCI device would show up in the guest OS with a PCI address that matches the one present in the corresponding ``<address>`` element of the domain XML, but that's not guaranteed to happen and will in fact not be the case in all but the simplest scenarios. ?
as a document reader/user of libvirt I would not expect to search around in other documentation for fid if all other parameters are correlated here.
So how about a short explanatory sentence for fid like: "The slot for the PCI device in the guest OS is defined by the fid (function id)."
The document already contains the following sentence: [...] the addresses there are generated from the information provided via the zpci element (in fact, from the uid). I'm not sure what "slot" means in the sentence you suggest. It doesn't seem to be the same as slot in domain:bus:slot.function, because in the second zPCI example that was removed <zpci uid='0x0007' fid='0x00000003'/> would result in 0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device with the fid nowhere to be seen. Can you explain what "slot" means in this context? Anyway, we can tweak the existing sentence to read something like [...] the addresses there are generated from the information provided via the zpci element: the uid is used as PCI domain, and the fid is used as [your explanation here]. Does that sound good? -- Andrea Bolognani / Red Hat / Virtualization
On 4/16/20 2:21 PM, Andrea Bolognani wrote:
On Thu, 2020-04-16 at 11:46 +0200, Boris Fiuczynski wrote:
On 4/15/20 6:38 PM, Andrea Bolognani wrote:
The idea behind it was to show users that they shouldn't expect the address in the domain XML to match the one in the guest OS, with a few real-life examples to illustrate the point. So, I don't think the details of how exactly zPCI IDs translate to guest-visible PCI addresses is in scope.
It would be great information to have in some other document, though! Is there something like that in qemu.git, or in the QEMU wiki? Those are the places where I would expect it to live, since it's not really tied to libvirt...
I disagree because fid is a parameter of the pci address just like domain, bus, slot, function and uid. Besides the fact that one of the first sentences in the document is "When discussing PCI addresses, it's important to understand the relationship between the addresses that can be seen in the domain XML and those that are visible inside the guest OS."
You're right, the current language is not really explaining the purpose of the document correctly - it's making it sound like it's the complete guide to all things PCI addresses, which it's definitely not intended to be.
What about
Looking at the configuration for a guest, it would be reasonable to expect that each PCI device would show up in the guest OS with a PCI address that matches the one present in the corresponding ``<address>`` element of the domain XML, but that's not guaranteed to happen and will in fact not be the case in all but the simplest scenarios.
? Yes, it does bring the documents intention across better.
as a document reader/user of libvirt I would not expect to search around in other documentation for fid if all other parameters are correlated here.
So how about a short explanatory sentence for fid like: "The slot for the PCI device in the guest OS is defined by the fid (function id)."
The document already contains the following sentence:
[...] the addresses there are generated from the information provided via the zpci element (in fact, from the uid).
I'm not sure what "slot" means in the sentence you suggest. It doesn't seem to be the same as slot in domain:bus:slot.function, because in the second zPCI example that was removed
<zpci uid='0x0007' fid='0x00000003'/>
would result in
0007:00:00.0 Ethernet controller: Red Hat, Inc. Virtio network device I added the sentence "Also fid does not define slot or function of the
PCI address." following the "Note...topology." sentence.
with the fid nowhere to be seen. Can you explain what "slot" means in this context? I guess that you now ask to explain it makes it most likely that it should be explained here, correct?
The fid (function id) is not used within the PCI address which the guest OS generates for the PCI device. Only the uid is used in the PCI address to define the PCI domain. PCI bus, slot and function address elements are always 0. This might change in the future BUT it would/will still be completely unrelated to the PCI attributes bus, slot and function of the address element. The fid is used in the sysfs (/sys/bus/pci/slots/...) to represent the PCI card such e.g. you can power on or off the PCI card. This is what results in guest OS. root@qemus390x:~# ls /sys/bus/pci/slots/ 00000003 root@qemus390x:~# cat /sys/bus/pci/devices/0007\:00\:00.0/function_id 0x00000003
Anyway, we can tweak the existing sentence to read something like
[...] the addresses there are generated from the information provided via the zpci element: the uid is used as PCI domain, and the fid is used as [your explanation here].
[my explanation] the fid is used as the PCI devices slot in the sysfs. Does that together with the additional sentence above make more "wacky" sense?
Does that sound good?
Sure. Since I started editing in the pci-addresses file already, I would try to also include these changes if you agree to them. If you want I can also include your changes in the document introduction. -- Mit freundlichen Grüßen/Kind regards Boris Fiuczynski IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
On a Wednesday in 2020, Cornelia Huck wrote:
Add some information on how pci address work on s390x.
Signed-off-by: Cornelia Huck <cohuck@redhat.com> --- docs/pci-addresses.rst | 63 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)
diff --git a/docs/pci-addresses.rst b/docs/pci-addresses.rst index 923783a151b0..9e241a24fcfb 100644 --- a/docs/pci-addresses.rst +++ b/docs/pci-addresses.rst @@ -184,3 +184,66 @@ guest OS rather than as ``0001:08:00.1``, which is the address of the device on the host.
Of course, all the rules and behaviors described above still apply. +
I added one extra blank line to match what we use elsehwere in this file for this level of headings.
+zPCI addresses +============== +
Reviewed-by: Ján Tomko <jtomko@redhat.com> Jano
On Wed, 2020-04-15 at 12:51 +0200, Cornelia Huck wrote:
Add some information on how pci address work on s390x.
Signed-off-by: Cornelia Huck <cohuck@redhat.com> --- docs/pci-addresses.rst | 63 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+)
There are a few comments that I didn't have a chance to raise before the patch was pushed, so I just posted a follow-up series addressing them: https://www.redhat.com/archives/libvir-list/2020-April/msg00749.html Please take a look and let me know whether you're okay with them. -- Andrea Bolognani / Red Hat / Virtualization
participants (4)
-
Andrea Bolognani -
Boris Fiuczynski -
Cornelia Huck -
Ján Tomko