[libvirt] [Question] vhost-user hotplug: multiqueue hotplug seems doesn't pass mq and vectors

Hi ALL: Recently I review the vhost-user hotplug codes, when attach a vhost-user nic with multiqueue, it seems that it doesn't pass mq=on and vectors num to qemu. Is there any special consideration or just a small mistake? Parts of codes: qemuDomainAttachNetDevice(vhostfdSize is 0 when type is vhost-user) qemuBuildNicDevStr: if (usingVirtio && vhostfdSize > 1) { if (net->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { /* ccw provides a one to one relation of fds to queues and * does not support the vectors option */ virBufferAddLit(&buf, ",mq=on"); } else { /* As advised at http://www.linux-kvm.org/page/Multiqueue * we should add vectors=2*N+2 where N is the vhostfdSize */ virBufferAsprintf(&buf, ",mq=on,vectors=%zu", 2 * vhostfdSize + 2); } } Thanks. Haifeng Gao

On 30.11.2016 08:41, Gaohaifeng (A) wrote:
Hi ALL: Recently I review the vhost-user hotplug codes, when attach a vhost-user nic with multiqueue, it seems that it doesn't pass mq=on and vectors num to qemu. Is there any special consideration or just a small mistake? Parts of codes:
qemuDomainAttachNetDevice(vhostfdSize is 0 when type is vhost-user) qemuBuildNicDevStr: if (usingVirtio && vhostfdSize > 1) { if (net->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { /* ccw provides a one to one relation of fds to queues and * does not support the vectors option */ virBufferAddLit(&buf, ",mq=on"); } else { /* As advised at http://www.linux-kvm.org/page/Multiqueue * we should add vectors=2*N+2 where N is the vhostfdSize */ virBufferAsprintf(&buf, ",mq=on,vectors=%zu", 2 * vhostfdSize + 2); } }
Ah, probably the latter. Frankly, I don't have idea whether we should pass vectors= too. Do you? Michal

I think we should pass it. Two reasons: 1. in none hotplug, we will pass it. We can see from libvirt function qemuBuildVhostuserCommandLine, it will pass queue num as vhostfdSize 2. If we don't pass vectors, qemu will use default value. Now default value is 3, (qemu codes: DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, 3)) So that qemu init msix table with 3 vectors(qemu function: msix_init_exclusive_bar), this will cause Guest can only use 3 interrupts at most. Actually, it use Two, one for config one for other virtqueues.
On 30.11.2016 08:41, Gaohaifeng (A) wrote:
Hi ALL: Recently I review the vhost-user hotplug codes, when attach a vhost-user nic with multiqueue, it seems that it doesn't pass mq=on and vectors num to qemu. Is there any special consideration or just a small mistake? Parts of codes:
qemuDomainAttachNetDevice(vhostfdSize is 0 when type is vhost-user) qemuBuildNicDevStr: if (usingVirtio && vhostfdSize > 1) { if (net->info.type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_CCW) { /* ccw provides a one to one relation of fds to queues and * does not support the vectors option */ virBufferAddLit(&buf, ",mq=on"); } else { /* As advised at http://www.linux-kvm.org/page/Multiqueue * we should add vectors=2*N+2 where N is the vhostfdSize */ virBufferAsprintf(&buf, ",mq=on,vectors=%zu", 2 * vhostfdSize + 2); } }
Ah, probably the latter. Frankly, I don't have idea whether we should pass vectors= too. Do you?
Michal

On 30.11.2016 10:02, Gaohaifeng (A) wrote:
I think we should pass it. Two reasons: 1. in none hotplug, we will pass it. We can see from libvirt function qemuBuildVhostuserCommandLine, it will pass queue num as vhostfdSize 2. If we don't pass vectors, qemu will use default value. Now default value is 3, (qemu codes: DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, 3)) So that qemu init msix table with 3 vectors(qemu function: msix_init_exclusive_bar), this will cause Guest can only use 3 interrupts at most. Actually, it use Two, one for config one for other virtqueues.
I see. Do you want to provide a patch or should I do that? Michal

OK. I will try to provide a patch. On 30.11.2016 10:02, Gaohaifeng (A) wrote:
I think we should pass it. Two reasons: 1. in none hotplug, we will pass it. We can see from libvirt function qemuBuildVhostuserCommandLine, it will pass queue num as vhostfdSize 2. If we don't pass vectors, qemu will use default value. Now default value is 3, (qemu codes: DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, 3)) So that qemu init msix table with 3 vectors(qemu function: msix_init_exclusive_bar), this will cause Guest can only use 3 interrupts at most. Actually, it use Two, one for config one for other virtqueues.
I see. Do you want to provide a patch or should I do that? Michal
participants (2)
-
Gaohaifeng (A)
-
Michal Privoznik