4:35 a.m.
Hi all,
I see in libvirt you are supporting attach/detach devices to existing xml domain using
attachDeviceFlags and detachDeviceFlags APIs.
Now we are adding some qemu command to the xml domain related to some interfaces using
alias names before starting the VM, but we will face an issue with hot plug such devices,
so I have two question here:
1. Is it applicable to set the alias names for interfaces because I saw it's
ignored when I add it to xml domain before starting the VM?
2. Is there a way or API to attach qemu commands to running domain as you are doing in
attaching the device using attachDeviceFlags?
Example of my xml
<domain type='kvm' id='5'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<devices>
<interface type='vhostuser'>
<mac address='fa:16:3e:ac:12:4c'/>
<source type='unix'
path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5' mode='server'/>
<target dev='tapbbb6bbe9-eb'/>
<model type='virtio'/>
<driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
</devices>
<qemu:commandline>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.page-per-vq=on'/>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.host_mtu=8942'/>
</qemu:commandline>
</domain>
Regards,
Waleed Mousa
Software Engineer,
Nvidia <https://www.nvidia.com/en-me/geforce/>
5:42 a.m.
On Tue, Mar 30, 2021 at 09:35:48 +0000, Waleed Musa wrote:
Hi all,
I see in libvirt you are supporting attach/detach devices to existing xml domain using
attachDeviceFlags and detachDeviceFlags APIs.
Now we are adding some qemu command to the xml domain related to some interfaces using
alias names before starting the VM, but we will face an issue with hot plug such devices,
so I have two question here:
1. Is it applicable to set the alias names for interfaces because I saw it's
ignored when I add it to xml domain before starting the VM?
User specified aliases are possible with 'ua-' prefix:
https://www.libvirt.org/formatdomain.html#devices
2. Is there a way or API to attach qemu commands to running domain
as you are doing in attaching the device using attachDeviceFlags?
No. For VMs which use <qemu:commandline> we don't formally provide
support. I suggest that if you have a good use case for the attributes
below, you send patches adding the functionality to libvirt if you want
to have it on device hotplug.
Example of my xml
<domain type='kvm' id='5'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<devices>
<interface type='vhostuser'>
<mac address='fa:16:3e:ac:12:4c'/>
<source type='unix'
path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5' mode='server'/>
<target dev='tapbbb6bbe9-eb'/>
<model type='virtio'/>
<driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
</devices>
<qemu:commandline>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.page-per-vq=on'/>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.host_mtu=8942'/>
</qemu:commandline>
</domain>
Regards,
Waleed Mousa
Software Engineer,
Nvidia <https://www.nvidia.com/en-me/geforce/>
6:49 a.m.
________________________________
From: Peter Krempa <pkrempa(a)redhat.com>
Sent: Tuesday, March 30, 2021 01:42 PM
To: Waleed Musa <waleedm(a)nvidia.com>
Cc: libvir-list(a)redhat.com <libvir-list(a)redhat.com>; Moshe Levi
<moshele(a)nvidia.com>; Adrian Chiris <adrianc(a)nvidia.com>
Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
External email: Use caution opening links or attachments
On Tue, Mar 30, 2021 at 09:35:48 +0000, Waleed Musa wrote:
Hi all,
I see in libvirt you are supporting attach/detach devices to existing xml domain using
attachDeviceFlags and detachDeviceFlags APIs.
Now we are adding some qemu command to the xml domain related to some interfaces using
alias names before starting the VM, but we will face an issue with hot plug such devices,
so I have two question here:
1. Is it applicable to set the alias names for interfaces because I saw it's
ignored when I add it to xml domain before starting the VM?
User specified aliases are possible with 'ua-' prefix:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.lib...
Thanks, I'll try it
2. Is there a way or API to attach qemu commands to running domain
as you are doing in attaching the device using attachDeviceFlags?
No. For VMs which use <qemu:commandline> we don't formally provide
support. I suggest that if you have a good use case for the attributes
below, you send patches adding the functionality to libvirt if you want
to have it on device hotplug.
We can't implement hot-plug
Example of my xml
<domain type='kvm' id='5'
xmlns:qemu='https://nam11.safelinks.protection.outlook.com/?url=http%...
<devices>
<interface type='vhostuser'>
<mac address='fa:16:3e:ac:12:4c'/>
<source type='unix'
path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5' mode='server'/>
<target dev='tapbbb6bbe9-eb'/>
<model type='virtio'/>
<driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
</devices>
<qemu:commandline>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.page-per-vq=on'/>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.host_mtu=8942'/>
</qemu:commandline>
</domain>
Regards,
Waleed Mousa
Software Engineer,
Nvidia <https://www.nvidia.com/en-me/geforce/>
6:57 a.m.
On Tue, Mar 30, 2021 at 11:49:51 +0000, Waleed Musa wrote:
Your reply email looks very broken, but I think I can find the relevant
parts:
________________________________
From: Peter Krempa <pkrempa(a)redhat.com>
Sent: Tuesday, March 30, 2021 01:42 PM
To: Waleed Musa <waleedm(a)nvidia.com>
Cc: libvir-list(a)redhat.com <libvir-list(a)redhat.com>; Moshe Levi
<moshele(a)nvidia.com>; Adrian Chiris <adrianc(a)nvidia.com>
Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
External email: Use caution opening links or attachments
On Tue, Mar 30, 2021 at 09:35:48 +0000, Waleed Musa wrote:
[...]
> 2. Is there a way or API to attach qemu commands to running
domain as you are doing in attaching the device using attachDeviceFlags?
No. For VMs which use <qemu:commandline> we don't formally provide
support. I suggest that if you have a good use case for the attributes
below, you send patches adding the functionality to libvirt if you want
to have it on device hotplug.
> We can't implement hot-plug
Can you elaborate why? If the options are justifiably useful and qemu
upstream supports them there's nothing preventing that.
6:09 a.m.
On 3/30/21 11:35 AM, Waleed Musa wrote:
Hi all,
I see in libvirt you are supporting attach/detach devices to existing
xml domain using *attachDeviceFlags *and *detachDeviceFlags *APIs.
Now we are adding some qemu command to the xml domain related to some
interfaces using alias names before starting the VM, but we will face an
issue with hot plug such devices, so I have two question here:
1. Is it applicable to set the alias names for interfaces because I saw
it's ignored when I add it to xml domain before starting the VM?
2. Is there a way or API to attach qemu commands to running domain as
you are doing in attaching the device using *attachDeviceFlags?*
*Example of my xml*
*<domain type='kvm' id='5'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
*
* <devices>
*
<interface type='vhostuser'>
<mac address='fa:16:3e:ac:12:4c'/>
<source type='unix'
path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5' mode='server'/>
<target dev='tapbbb6bbe9-eb'/>
<model type='virtio'/>
<driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
</interface>
* </devices>
*
* <qemu:commandline>*
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.page-per-vq=on'/>
<qemu:arg value='-set'/>
<qemu:arg value='device.net0.host_mtu=8942'/>
* </qemu:commandline>*
*</domain>*
Is this perhaps related to the following bug?
"interface type='vhostuser' libvirtError: Cannot set interface MTU"
https://bugzilla.redhat.com/show_bug.cgi?id=1940559
Are you trying to work around it? I am discussing with Moshe how libvirt
can help, but honestly, I don't like the solution I proposed.
Long story short, the interface is in different container (among with
OVS bridge) and thus when we query ovs-vsctl it connects to the system
one and doesn't find that interface. What I proposed was to allow
specifying path to ovs db.socket but this would need to be done o per
domain basis.
What I particularly don't like about this solution is that while it may
fix this one use case, it opens the gate for whole lot of other
requests. For instance, consider if other interfaces of other types live
in different network namespace. "Hey, I want to use eth0 from that
namespace, but run QEMU in another one".
Also, we don't really like exposing paths in libvirt XML unless necessary.
Michal
6:36 a.m.
On Wed, Mar 31, 2021 at 01:09:32PM +0200, Michal Privoznik wrote:
On 3/30/21 11:35 AM, Waleed Musa wrote:
> Hi all,
>
> I see in libvirt you are supporting attach/detach devices to existing
> xml domain using *attachDeviceFlags *and *detachDeviceFlags *APIs.
> Now we are adding some qemu command to the xml domain related to some
> interfaces using alias names before starting the VM, but we will face an
> issue with hot plug such devices, so I have two question here:
>
> 1. Is it applicable to set the alias names for interfaces because I saw
> it's ignored when I add it to xml domain before starting the VM?
> 2. Is there a way or API to attach qemu commands to running domain as
> you are doing in attaching the device using *attachDeviceFlags?*
>
> *Example of my xml*
> *<domain type='kvm' id='5'
> xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> *
> * <devices>
> *
> <interface type='vhostuser'>
> <mac address='fa:16:3e:ac:12:4c'/>
> <source type='unix'
> path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5' mode='server'/>
> <target dev='tapbbb6bbe9-eb'/>
> <model type='virtio'/>
> <driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
> <alias name='net0'/>
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
> function='0x0'/>
> </interface>
> * </devices>
> *
> * <qemu:commandline>*
> <qemu:arg value='-set'/>
> <qemu:arg value='device.net0.page-per-vq=on'/>
> <qemu:arg value='-set'/>
> <qemu:arg value='device.net0.host_mtu=8942'/>
> * </qemu:commandline>*
> *</domain>*
Is this perhaps related to the following bug?
"interface type='vhostuser' libvirtError: Cannot set interface MTU"
https://bugzilla.redhat.com/show_bug.cgi?id=1940559
Are you trying to work around it? I am discussing with Moshe how libvirt can
help, but honestly, I don't like the solution I proposed.
Long story short, the interface is in different container (among with OVS
bridge) and thus when we query ovs-vsctl it connects to the system one and
doesn't find that interface. What I proposed was to allow specifying path to
ovs db.socket but this would need to be done o per domain basis.
If it is possible to specify a ovs db.socket path in the XML, then
why can't this path simply be bind mounted to the right location
in the first place.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
7:11 a.m.
On 3/31/21 1:36 PM, Daniel P. Berrangé wrote:
On Wed, Mar 31, 2021 at 01:09:32PM +0200, Michal Privoznik wrote:
> On 3/30/21 11:35 AM, Waleed Musa wrote:
>> Hi all,
>>
>> I see in libvirt you are supporting attach/detach devices to existing
>> xml domain using *attachDeviceFlags *and *detachDeviceFlags *APIs.
>> Now we are adding some qemu command to the xml domain related to some
>> interfaces using alias names before starting the VM, but we will face an
>> issue with hot plug such devices, so I have two question here:
>>
>> 1. Is it applicable to set the alias names for interfaces because I saw
>> it's ignored when I add it to xml domain before starting the VM?
>> 2. Is there a way or API to attach qemu commands to running domain as
>> you are doing in attaching the device using *attachDeviceFlags?*
>>
>> *Example of my xml*
>> *<domain type='kvm' id='5'
>> xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
>> *
>> * <devices>
>> *
>> <interface type='vhostuser'>
>> <mac address='fa:16:3e:ac:12:4c'/>
>> <source type='unix'
>> path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5'
mode='server'/>
>> <target dev='tapbbb6bbe9-eb'/>
>> <model type='virtio'/>
>> <driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
>> <alias name='net0'/>
>> <address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
>> function='0x0'/>
>> </interface>
>> * </devices>
>> *
>> * <qemu:commandline>*
>> <qemu:arg value='-set'/>
>> <qemu:arg value='device.net0.page-per-vq=on'/>
>> <qemu:arg value='-set'/>
>> <qemu:arg value='device.net0.host_mtu=8942'/>
>> * </qemu:commandline>*
>> *</domain>*
>
> Is this perhaps related to the following bug?
>
> "interface type='vhostuser' libvirtError: Cannot set interface
MTU"
> https://bugzilla.redhat.com/show_bug.cgi?id=1940559
>
> Are you trying to work around it? I am discussing with Moshe how libvirt can
> help, but honestly, I don't like the solution I proposed.
>
> Long story short, the interface is in different container (among with OVS
> bridge) and thus when we query ovs-vsctl it connects to the system one and
> doesn't find that interface. What I proposed was to allow specifying path to
> ovs db.socket but this would need to be done o per domain basis.
If it is possible to specify a ovs db.socket path in the XML, then
why can't this path simply be bind mounted to the right location
in the first place.
Because then you'd override the path for system wide OVS. I mean, by
default the socket is under /var/run/openvswitch/db.sock and if you have
another OVS running inside a container you can expose it under
/prefix/var/run/openvswitch/db.sock. ovs-vsctl allows this by --db=$path
argument.
Michal
7:23 a.m.
On Wed, Mar 31, 2021 at 02:11:40PM +0200, Michal Privoznik wrote:
On 3/31/21 1:36 PM, Daniel P. Berrangé wrote:
> On Wed, Mar 31, 2021 at 01:09:32PM +0200, Michal Privoznik wrote:
> > On 3/30/21 11:35 AM, Waleed Musa wrote:
> > > Hi all,
> > >
> > > I see in libvirt you are supporting attach/detach devices to existing
> > > xml domain using *attachDeviceFlags *and *detachDeviceFlags *APIs.
> > > Now we are adding some qemu command to the xml domain related to some
> > > interfaces using alias names before starting the VM, but we will face an
> > > issue with hot plug such devices, so I have two question here:
> > >
> > > 1. Is it applicable to set the alias names for interfaces because I saw
> > > it's ignored when I add it to xml domain before starting the VM?
> > > 2. Is there a way or API to attach qemu commands to running domain as
> > > you are doing in attaching the device using *attachDeviceFlags?*
> > >
> > > *Example of my xml*
> > > *<domain type='kvm' id='5'
> > > xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
> > > *
> > > * <devices>
> > > *
> > > <interface type='vhostuser'>
> > > <mac address='fa:16:3e:ac:12:4c'/>
> > > <source type='unix'
> > > path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5'
mode='server'/>
> > > <target dev='tapbbb6bbe9-eb'/>
> > > <model type='virtio'/>
> > > <driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
> > > <alias name='net0'/>
> > > <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
> > > function='0x0'/>
> > > </interface>
> > > * </devices>
> > > *
> > > * <qemu:commandline>*
> > > <qemu:arg value='-set'/>
> > > <qemu:arg value='device.net0.page-per-vq=on'/>
> > > <qemu:arg value='-set'/>
> > > <qemu:arg value='device.net0.host_mtu=8942'/>
> > > * </qemu:commandline>*
> > > *</domain>*
> >
> > Is this perhaps related to the following bug?
> >
> > "interface type='vhostuser' libvirtError: Cannot set interface
MTU"
> > https://bugzilla.redhat.com/show_bug.cgi?id=1940559
> >
> > Are you trying to work around it? I am discussing with Moshe how libvirt can
> > help, but honestly, I don't like the solution I proposed.
> >
> > Long story short, the interface is in different container (among with OVS
> > bridge) and thus when we query ovs-vsctl it connects to the system one and
> > doesn't find that interface. What I proposed was to allow specifying path
to
> > ovs db.socket but this would need to be done o per domain basis.
>
> If it is possible to specify a ovs db.socket path in the XML, then
> why can't this path simply be bind mounted to the right location
> in the first place.
Because then you'd override the path for system wide OVS. I mean, by default
the socket is under /var/run/openvswitch/db.sock and if you have another OVS
running inside a container you can expose it under
/prefix/var/run/openvswitch/db.sock. ovs-vsctl allows this by --db=$path
argument.
Oh, right I was mis-understanding the scenario.
I've always thought if we wanted to support use of resources from
other namespaces, then that would involve use of a namespace="$PID"
attribute to identify the namespace.
<source type='unix' namespace="523532"
path='/var/lib/vhost_sockets/sock7f9a971a-cf3' mode='server'/>
IOW, libvirt could enter the namespace, and then talk to the OVS
db.sock at its normal location.
It would be bit strange having namespace refer to a different
NS for the OVS, but using the path=/var/lib/..... from the
current namespace, but as long as we define the semantics
clearly its not the end of the world.
Regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
9:01 a.m.
-----Original Message-----
From: Daniel P. Berrangé <berrange(a)redhat.com>
Sent: Wednesday, March 31, 2021 3:23 PM
To: Michal Privoznik <mprivozn(a)redhat.com>
Cc: Waleed Musa <waleedm(a)nvidia.com>; libvir-list(a)redhat.com; Moshe
Levi <moshele(a)nvidia.com>; Adrian Chiris <adrianc(a)nvidia.com>
Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
External email: Use caution opening links or attachments
On Wed, Mar 31, 2021 at 02:11:40PM +0200, Michal Privoznik wrote:
> On 3/31/21 1:36 PM, Daniel P. Berrangé wrote:
> > On Wed, Mar 31, 2021 at 01:09:32PM +0200, Michal Privoznik wrote:
> > > On 3/30/21 11:35 AM, Waleed Musa wrote:
> > > > Hi all,
> > > >
> > > > I see in libvirt you are supporting attach/detach devices to
> > > > existing xml domain using *attachDeviceFlags *and
*detachDeviceFlags *APIs.
> > > > Now we are adding some qemu command to the xml domain related
to
> > > > some interfaces using alias names before starting the VM, but we
> > > > will face an issue with hot plug such devices, so I have two
question
here:
> > > >
> > > > 1. Is it applicable to set the alias names for interfaces because I
saw
> > > > it's ignored when I add it to xml domain before starting the
VM?
> > > > 2. Is there a way or API to attach qemu commands to running domain
as
> > > > you are doing in attaching the device using
> > > > *attachDeviceFlags?*
> > > >
> > > > *Example of my xml*
> > > > *<domain type='kvm' id='5'
> > > > xmlns:qemu='https://nam11.safelinks.protection.outlook.com/?url=
> > > >
http%3A%2F%2Flibvirt.org%2Fschemas%2Fdomain%2Fqemu%2F1.0&da
t
> > > >
a=04%7C01%7Cmoshele%40nvidia.com%7C310b722c5d9b4dbef69e08d8f43fc
> > > >
7c5%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527902060468
> > > >
241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2lu
MzIi
> > > >
LCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mGgrFOieiNdozuu
U
> > > > 1tYThVFou2E4caoR0ECW1jljRfk%3D&reserved=0'>
> > > > *
> > > > * <devices>
> > > > *
> > > > <interface type='vhostuser'>
> > > > <mac address='fa:16:3e:ac:12:4c'/>
> > > > <source type='unix'
> > > > path='/var/lib/vhost_sockets/sockbbb6bbe9-eb5'
mode='server'/>
> > > > <target dev='tapbbb6bbe9-eb'/>
> > > > <model type='virtio'/>
> > > > <driver queues='4' rx_queue_size='512'
tx_queue_size='512'/>
> > > > <alias name='net0'/>
> > > > <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
> > > > function='0x0'/>
> > > > </interface>
> > > > * </devices>
> > > > *
> > > > * <qemu:commandline>*
> > > > <qemu:arg value='-set'/>
> > > > <qemu:arg value='device.net0.page-per-vq=on'/>
> > > > <qemu:arg value='-set'/>
> > > > <qemu:arg value='device.net0.host_mtu=8942'/>
> > > > * </qemu:commandline>*
> > > > *</domain>*
> > >
> > > Is this perhaps related to the following bug?
> > >
> > > "interface type='vhostuser' libvirtError: Cannot set
interface MTU"
> > >
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > >
bugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1940559&data=04%7C0
1
> > >
%7Cmoshele%40nvidia.com%7C310b722c5d9b4dbef69e08d8f43fc7c5%7C430
83
> > >
d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637527902060468241%7CUnkn
own
> > >
%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
Ww
> > >
iLCJXVCI6Mn0%3D%7C1000&sdata=g8nFl0ygr2JdDz1juugnbC0dpfpIrKX
NG
> > > exEL6oMnDc%3D&reserved=0
> > >
> > > Are you trying to work around it? I am discussing with Moshe how
> > > libvirt can help, but honestly, I don't like the solution I proposed.
> > >
> > > Long story short, the interface is in different container (among
> > > with OVS
> > > bridge) and thus when we query ovs-vsctl it connects to the system
> > > one and doesn't find that interface. What I proposed was to allow
> > > specifying path to ovs db.socket but this would need to be done o per
domain basis.
> >
> > If it is possible to specify a ovs db.socket path in the XML, then
> > why can't this path simply be bind mounted to the right location in
> > the first place.
>
> Because then you'd override the path for system wide OVS. I mean, by
> default the socket is under /var/run/openvswitch/db.sock and if you
> have another OVS running inside a container you can expose it under
> /prefix/var/run/openvswitch/db.sock. ovs-vsctl allows this by
> --db=$path argument.
Oh, right I was mis-understanding the scenario.
I've always thought if we wanted to support use of resources from other
namespaces, then that would involve use of a namespace="$PID"
attribute to identify the namespace.
<source type='unix' namespace="523532"
path='/var/lib/vhost_sockets/sock7f9a971a-cf3' mode='server'/>
IOW, libvirt could enter the namespace, and then talk to the OVS db.sock at
its normal location.
It would be bit strange having namespace refer to a different NS for the
OVS, but using the path=/var/lib/..... from the current namespace, but as
long as we define the semantics clearly its not the end of the world.
Just to
better explain the use case here so we have user space vdpa solution which requires the
following qemu flags to work:
1. page-per-vq=on
2. host_mtu
I understand the adding page-per-vq is feature requested to I am putting that aside.
Regarding the host_mtu we want to use the libvirt xml to set mtu like in [1] so we can set
mtu in hot plug.
Basically we just want that the libvirt xml mtu will just set the qemu host_mtu.
Th problem is the this using mtu in libvirt xml will query ovs (it assume that if ovs
installed on the host it needs to set it mtu).
In our case we have OVS running on the host for switching and OVS in container to connect
the VF netdevice with vhostuser to qemu to just forward traffic from VF to virtio using
vdpa)
1. One option is to allow to disable the set of mtu in ovs so it will just add the qemu
host_mtu flag (we can set the mtu on ovs by openstack)
2. Second option is to change how we query ovs to add --db flag so we can change the
query to be on the ovs in the container
Both option are fine with me.
[1] -
<interface type='vhostuser'>
<mac address='fa:16:3e:92:6d:79'/>
<source type='unix'
path='/var/lib/vhost_sockets/sock7f9a971a-cf3' mode='server'/>
<model type='virtio'/>
<driver rx_queue_size='512' tx_queue_size='512'/>
<mtu size='8942'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
</interface>
Regards,
Daniel
--
|:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fberr
ange.com%2F&data=04%7C01%7Cmoshele%40nvidia.com%7C310b722c
5d9b4dbef69e08d8f43fc7c5%7C43083d15727340c1b7db39efd9ccc17a%7C0%7
C0%7C637527902060468241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&am
p;sdata=SRttkiVBWP9rgqQBzgDsRQSKcUL1T%2B3Z0f96i4%2B4tm8%3D&
reserved=0 -o-
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
w.flickr.com%2Fphotos%2Fdberrange&data=04%7C01%7Cmoshele%40
nvidia.com%7C310b722c5d9b4dbef69e08d8f43fc7c5%7C43083d15727340c1b7
db39efd9ccc17a%7C0%7C0%7C637527902060468241%7CUnknown%7CTWFpb
GZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
6Mn0%3D%7C1000&sdata=v8hkQ6F4lwKoECekJNt4WHgpReHo8GzMPz
YbW8wv5V0%3D&reserved=0 :|
|:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flibvir
t.org%2F&data=04%7C01%7Cmoshele%40nvidia.com%7C310b722c5d9b
4dbef69e08d8f43fc7c5%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%
7C637527902060468241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sda
ta=y%2F%2BTU%2FG6qz5kdYeA8KVzZEMzhH6zwp3%2F639ylpIIT1Y%3D&am
p;reserved=0 -o-
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffsto
p138.berrange.com%2F&data=04%7C01%7Cmoshele%40nvidia.com%7C
310b722c5d9b4dbef69e08d8f43fc7c5%7C43083d15727340c1b7db39efd9ccc17
a%7C0%7C0%7C637527902060468241%7CUnknown%7CTWFpbGZsb3d8eyJWI
joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1
000&sdata=WtiEpRFS83Hp1Ya9pzjvjGsVDD5SyBPdpHRHKPOop24%3D&
amp;reserved=0 :|
|:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fenta
ngle-
photo.org%2F&data=04%7C01%7Cmoshele%40nvidia.com%7C310b722c
5d9b4dbef69e08d8f43fc7c5%7C43083d15727340c1b7db39efd9ccc17a%7C0%7
C0%7C637527902060468241%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4w
LjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&am
p;sdata=Xok16UY9Qt2QY31nHgu01hR0ykuduqwoJeRR5K3NnUM%3D&r
eserved=0 -o-
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
w.instagram.com%2Fdberrange&data=04%7C01%7Cmoshele%40nvidia.
com%7C310b722c5d9b4dbef69e08d8f43fc7c5%7C43083d15727340c1b7db39e
fd9ccc17a%7C0%7C0%7C637527902060468241%7CUnknown%7CTWFpbGZsb3
d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0
%3D%7C1000&sdata=GtgEbUuRNH5WPXi%2F5PkE74aEbh8N7mEwJ8ntE
te3%2FfM%3D&reserved=0 :|
9:48 a.m.
On 3/31/21 4:01 PM, Moshe Levi wrote:
> -----Original Message-----
> From: Daniel P. Berrangé <berrange(a)redhat.com>
> Sent: Wednesday, March 31, 2021 3:23 PM
> To: Michal Privoznik <mprivozn(a)redhat.com>
> Cc: Waleed Musa <waleedm(a)nvidia.com>; libvir-list(a)redhat.com; Moshe
> Levi <moshele(a)nvidia.com>; Adrian Chiris <adrianc(a)nvidia.com>
> Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
>
> I've always thought if we wanted to support use of resources from other
> namespaces, then that would involve use of a namespace="$PID"
> attribute to identify the namespace.
>
> <source type='unix' namespace="523532"
> path='/var/lib/vhost_sockets/sock7f9a971a-cf3' mode='server'/>
>
> IOW, libvirt could enter the namespace, and then talk to the OVS db.sock at
> its normal location.
>
> It would be bit strange having namespace refer to a different NS for the
> OVS, but using the path=/var/lib/..... from the current namespace, but as
> long as we define the semantics clearly its not the end of the world.
Just to better explain the use case here so we have user space vdpa solution which
requires the following qemu flags to work:
1. page-per-vq=on
2. host_mtu
I understand the adding page-per-vq is feature requested to I am putting that aside.
Correct.
Regarding the host_mtu we want to use the libvirt xml to set mtu like in [1] so we can
set mtu in hot plug.
Basically we just want that the libvirt xml mtu will just set the qemu host_mtu.
Th problem is the this using mtu in libvirt xml will query ovs (it assume that if ovs
installed on the host it needs to set it mtu).
In our case we have OVS running on the host for switching and OVS in container to
connect the VF netdevice with vhostuser to qemu to just forward traffic from VF to virtio
using vdpa)
1. One option is to allow to disable the set of mtu in ovs so it will just add the qemu
host_mtu flag (we can set the mtu on ovs by openstack)
Don't you need to set the MTU also on the TAP device? Or will it be
inherited from the OVS bridge?
2. Second option is to change how we query ovs to add --db flag so
we can change the query to be on the ovs in the container
Both option are fine with me.
Michal
9:54 a.m.
-----Original Message-----
From: Michal Privoznik <mprivozn(a)redhat.com>
Sent: Wednesday, March 31, 2021 5:49 PM
To: Moshe Levi <moshele(a)nvidia.com>; Daniel P. Berrangé
<berrange(a)redhat.com>
Cc: Waleed Musa <waleedm(a)nvidia.com>; libvir-list(a)redhat.com; Adrian
Chiris <adrianc(a)nvidia.com>
Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
External email: Use caution opening links or attachments
On 3/31/21 4:01 PM, Moshe Levi wrote:
>
>
>> -----Original Message-----
>> From: Daniel P. Berrangé <berrange(a)redhat.com>
>> Sent: Wednesday, March 31, 2021 3:23 PM
>> To: Michal Privoznik <mprivozn(a)redhat.com>
>> Cc: Waleed Musa <waleedm(a)nvidia.com>; libvir-list(a)redhat.com;
Moshe
>> Levi <moshele(a)nvidia.com>; Adrian Chiris <adrianc(a)nvidia.com>
>> Subject: Re: [Libvirt] Attach qemu commands to an running xml domain
>>
>> I've always thought if we wanted to support use of resources from
>> other namespaces, then that would involve use of a namespace="$PID"
>> attribute to identify the namespace.
>>
>> <source type='unix' namespace="523532"
>> path='/var/lib/vhost_sockets/sock7f9a971a-cf3'
mode='server'/>
>>
>> IOW, libvirt could enter the namespace, and then talk to the OVS
>> db.sock at its normal location.
>>
>> It would be bit strange having namespace refer to a different NS for
>> the OVS, but using the path=/var/lib/..... from the current
>> namespace, but as long as we define the semantics clearly its not the end
of the world.
> Just to better explain the use case here so we have user space vdpa
solution which requires the following qemu flags to work:
> 1. page-per-vq=on
> 2. host_mtu
>
> I understand the adding page-per-vq is feature requested to I am putting
that aside.
Correct.
>
> Regarding the host_mtu we want to use the libvirt xml to set mtu like in [1]
so we can set mtu in hot plug.
> Basically we just want that the libvirt xml mtu will just set the qemu
host_mtu.
> Th problem is the this using mtu in libvirt xml will query ovs (it assume that
if ovs installed on the host it needs to set it mtu).
> In our case we have OVS running on the host for switching and OVS in
> container to connect the VF netdevice with vhostuser to qemu to just
> forward traffic from VF to virtio using vdpa) 1. One option is to
> allow to disable the set of mtu in ovs so it will just add the qemu
> host_mtu flag (we can set the mtu on ovs by openstack)
Don't you need to set the MTU also on the TAP device? Or will it be inherited
from the OVS bridge?
In our case it MTU of the VF nerdevice and we set it by OVS.
> 2. Second option is to change how we query ovs to add --db flag so we
> can change the query to be on the ovs in the container
>
> Both option are fine with me.
>
Michal
1332
days inactive
1333
days old
10 comments
5 participants
participants (5)
-
Daniel P. Berrangé -
Michal Privoznik -
Moshe Levi -
Peter Krempa -
Waleed Musa