Re: [libvirt] Handling large amount of VF's with intelligent passthrough

Hi, I've run into a rather interesting problem recently where a weird interaction between libnl3 and libvirt caused some difficult-to-debug issues. From libvirt's side, the issue was that a netlink response was much larger than the pagesize and truncated by libnl3. When virNetDevLinkDump() calls virNetlinkCommand(), nl_recv() is supposed to return a rather large structure with information for all the Virtual Functions. When called in a system where the number of PCIe Virtual Functions are more than 30 for a given Physical Function, the netlink response is larger than 4k, meaning that a message is truncated. Unfortunately libnl3 truncates this silently, meaning that a cryptic error pops up much later in virNetDevParseVfConfig(), "missing IFLA_VF_INFO in netlink response". Aside from the error propagation (which might be fixable in libnl3), there still remains the need to enable libvirt to function in cases like this. This can be done in two ways, both in virNetlinkCommand().

1. Message peeking can be enabled. In theory this slows down any netlink messages by doing a two stage query: query the buffer size, then allocate the receive buffer and receive the message. This is a reliability/performance tradeoff, I guess. This is as simple as adding: nl_socket_enable_msg_peek(nlhandle); 2. The receive buffer size can also be made larger: nl_socket_set_msg_buf_size(nlhandle, ARBITRARY_BUFFER_SIZE); This does not incur a performance penalty, but until libnl3 can propagate the truncation error, this merely postpones the error for future generations... Jan -- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list