10:25 a.m.
I’m using virt-install to spin up VMs. At times I have a need to spin up
VMs which have 100s of interfaces. I ran into the PCI issue mentioned in
this previous thread based on how virt-install assigns PCI addresses to
interfaces:
https://www.redhat.com/archives/libvirt-users/2018-December/msg00064.html
Using the info mention there I was able to part hand / part sw re-write an
XML where I would remove PCI bus references above the value of 8 and
re-address the per interface PCI info to use the function field (I don’t
need hot pluggable).
But the process I’ve built is brittle.
Wondering what options i might have to better deal with this PCI issue.
Is there a way to tell virt-install, when building the info it passes to
qemu, to use the function field during the PCI assignment process so as to
support many more interfaces before hitting the PCI bus == 8 issue?
If not, is there a way with one of the virt command line tools to create
the XML (with the PCI addresses specified) so that I can process that XML
and re-write the PCI addressing values? Right now the only way I’ve been
able to get that detailed XML file is to 1) virt-install and let the VM
begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
destroy/undefine that VM, 4) modify the XML and then 5) virsh create
./modified.xml. Is there a cleaner way to do this?
Thanks
3:57 p.m.
On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415(a)gmail.com> wrote:
If not, is there a way with one of the virt command line tools to
create
the XML (with the PCI addresses specified) so that I can process that XML
and re-write the PCI addressing values? Right now the only way I’ve been
able to get that detailed XML file is to 1) virt-install and let the VM
begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
destroy/undefine that VM, 4) modify the XML and then 5) virsh create
./modified.xml. Is there a cleaner way to do this?
It won't do everything you want, but check out virt-install --print-step
--dry-run. This should at least allow you to get at the XML for the
intermediate steps and remove the brittleness of defining, destroying,
undefining the VM.
Cheers,
- Peter
5:59 p.m.
Thanks. Maybe I'm missing something but when I try virt-install --dry-run
--print-xml it generates this snippet of XML:
<interface type="bridge">
<source bridge="ge_0_0_0_77"/>
<mac address="52:54:00:01:00:4e"/>
<model type="virtio"/>
</interface>
but what i need (and get from virsh dumpxml) has this extra PCI line.
<interface type='bridge'>
<mac address='52:54:00:01:00:4e'/>
<source bridge='ge_0_0_0_77'/>
<target dev='vnet78'/>
<model type='virtio'/>
<alias name='net78'/>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x0a'
function='0x6' />
</interface>
Did I miss something? Do you see PCI in the XML when you do the
virt-install --dryrun approach?
Thanks
On Wed, Dec 26, 2018 at 2:57 PM Peter Crowther <peter.crowther(a)melandra.com>
wrote:
On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415(a)gmail.com>
wrote:
> If not, is there a way with one of the virt command line tools to create
> the XML (with the PCI addresses specified) so that I can process that XML
> and re-write the PCI addressing values? Right now the only way I’ve been
> able to get that detailed XML file is to 1) virt-install and let the VM
> begin the boot process and then do a 2) virsh dumpxml and then 3) virsh
> destroy/undefine that VM, 4) modify the XML and then 5) virsh create
> ./modified.xml. Is there a cleaner way to do this?
>
It won't do everything you want, but check out virt-install --print-step
--dry-run. This should at least allow you to get at the XML for the
intermediate steps and remove the brittleness of defining, destroying,
undefining the VM.
Cheers,
- Peter
10:29 a.m.
On 12/26/18 6:59 PM, b f31415 wrote:
Thanks. Maybe I'm missing something but when I try virt-install
--dry-run --print-xml it generates this snippet of XML:
<interface type="bridge">
<source bridge="ge_0_0_0_77"/>
<mac address="52:54:00:01:00:4e"/>
<model type="virtio"/>
</interface>
but what i need (and get from virsh dumpxml) has this extra PCI line.
<interface type='bridge'>
<mac address='52:54:00:01:00:4e'/>
<source bridge='ge_0_0_0_77'/>
<target dev='vnet78'/>
<model type='virtio'/>
<alias name='net78'/>
<address type='pci' domain='0x0000' bus='0x01'
slot='0x0a' function='0x6' />
</interface>
Did I miss something? Do you see PCI in the XML when you do the
virt-install --dryrun approach?
virt-install never explicitly specifies a PCI address in the XML it
creates and sends to libvirt, and what you're seeing as the output from
--print-xml is the XML before it has been sent to libvirt.
Additionally, in response to a question in the original mail, there is
no way to tell virt-install (or libvirt itself) to use non-0 functions
for devices (except to manually specify the entire PCI address).
Since 99 44/100% of users (probably much higher, but that number has
such a nice ring to anyone who recalls the old Ivory Soap commercials
:-) never need more devices than are possible using only function 0,
there has been no effort made to automate assignment of non-0 functions.
So if you need this many devices, you should probably consider just
using virt-install once to generate proto-template XML, then "virsh
define" that template to get an XML that has an <address...> element for
every device that needs it, write a script that condenses all the device
PCI addresses into less slots by using non-0 functions (and saves that
as the "final" template), and finally modify your installation to do a
series of virsh commands using that template XML as a start.
I know it sounds awkward, but there is really nothing in libvirt or
virt-install to automate assignment of PCI devices to non-0 functions.
On Wed, Dec 26, 2018 at 2:57 PM Peter Crowther
<peter.crowther(a)melandra.com <mailto:peter.crowther@melandra.com>> wrote:
On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415(a)gmail.com
<mailto:bf31415@gmail.com>> wrote:
If not, is there a way with one of the virt command line tools
to create the XML (with the PCI addresses specified) so that I
can process that XML and re-write the PCI addressing values?
Right now the only way I’ve been able to get that detailed XML
file is to 1) virt-install and let the VM begin the boot process
and then do a 2) virsh dumpxml and then 3) virsh
destroy/undefine that VM, 4) modify the XML and then 5) virsh
create ./modified.xml. Is there a cleaner way to do this?
It won't do everything you want, but check out virt-install
--print-step --dry-run. This should at least allow you to get at
the XML for the intermediate steps and remove the brittleness of
defining, destroying, undefining the VM.
Cheers,
- Peter
_______________________________________________
libvirt-users mailing list
libvirt-users(a)redhat.com
https://www.redhat.com/mailman/listinfo/libvirt-users
12:24 p.m.
Thanks-- i get this eco-system setup isn't mainstream (yet). just wanted
to double check to see if there was some pre-existing way to use the
function aspect.
B
On Wed, Jan 2, 2019 at 9:29 AM Laine Stump <laine(a)redhat.com> wrote:
On 12/26/18 6:59 PM, b f31415 wrote:
> Thanks. Maybe I'm missing something but when I try virt-install
> --dry-run --print-xml it generates this snippet of XML:
>
> <interface type="bridge">
>
> <source bridge="ge_0_0_0_77"/>
>
> <mac address="52:54:00:01:00:4e"/>
>
> <model type="virtio"/>
>
> </interface>
>
>
> but what i need (and get from virsh dumpxml) has this extra PCI line.
>
> <interface type='bridge'>
>
> <mac address='52:54:00:01:00:4e'/>
>
> <source bridge='ge_0_0_0_77'/>
>
> <target dev='vnet78'/>
>
> <model type='virtio'/>
>
> <alias name='net78'/>
>
> <address type='pci' domain='0x0000' bus='0x01'
slot='0x0a'
function='0x6' />
>
> </interface>
>
>
> Did I miss something? Do you see PCI in the XML when you do the
> virt-install --dryrun approach?
virt-install never explicitly specifies a PCI address in the XML it
creates and sends to libvirt, and what you're seeing as the output from
--print-xml is the XML before it has been sent to libvirt.
Additionally, in response to a question in the original mail, there is
no way to tell virt-install (or libvirt itself) to use non-0 functions
for devices (except to manually specify the entire PCI address).
Since 99 44/100% of users (probably much higher, but that number has
such a nice ring to anyone who recalls the old Ivory Soap commercials
:-) never need more devices than are possible using only function 0,
there has been no effort made to automate assignment of non-0 functions.
So if you need this many devices, you should probably consider just
using virt-install once to generate proto-template XML, then "virsh
define" that template to get an XML that has an <address...> element for
every device that needs it, write a script that condenses all the device
PCI addresses into less slots by using non-0 functions (and saves that
as the "final" template), and finally modify your installation to do a
series of virsh commands using that template XML as a start.
I know it sounds awkward, but there is really nothing in libvirt or
virt-install to automate assignment of PCI devices to non-0 functions.
> On Wed, Dec 26, 2018 at 2:57 PM Peter Crowther
> <peter.crowther(a)melandra.com <mailto:peter.crowther@melandra.com>>
wrote:
>
> On Wed, 26 Dec 2018 at 16:26, b f31415 <bf31415(a)gmail.com
> <mailto:bf31415@gmail.com>> wrote:
>
> If not, is there a way with one of the virt command line tools
> to create the XML (with the PCI addresses specified) so that I
> can process that XML and re-write the PCI addressing values?
> Right now the only way I’ve been able to get that detailed XML
> file is to 1) virt-install and let the VM begin the boot process
> and then do a 2) virsh dumpxml and then 3) virsh
> destroy/undefine that VM, 4) modify the XML and then 5) virsh
> create ./modified.xml. Is there a cleaner way to do this?
>
>
> It won't do everything you want, but check out virt-install
> --print-step --dry-run. This should at least allow you to get at
> the XML for the intermediate steps and remove the brittleness of
> defining, destroying, undefining the VM.
>
> Cheers,
>
> - Peter
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users(a)redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>
2032
days inactive
2039
days old
4 comments
3 participants
participants (3)
-
b f31415 -
Laine Stump -
Peter Crowther