Re: [libvirt] question about rdma migration
On Tue, Mar 22, 2016 at 16:50:40 +0000, Roy Shterman wrote:
On 3/22/2016 3:58 PM, Jiri Denemark wrote:
> On Tue, Mar 22, 2016 at 13:16:49 +0000, Roy Shterman wrote:
>> OK, I will describe it better,
>>
>> I'm developing iSER transport option inside libiscsi, as part of that
I'm planning to implement it also in QEMU block layer.
>>
>> iSER RDMA components (QP, CQ, MR) need to lock some amount of memory
(predictable amount),
>>
>> So, needed memory to be locked is (num of libiscsi-iser devices in VM)*(constant
amount per device),
> Well, it should be easy enough to compute and automatically set the
> limit from libvirt then. Unless the "constant amount per device" changes
> anytime someone touches the code in QEMU or libiscsi.
Yes, it should be constant
Perfect.
>> For now, I'm using locked option in libvirt, although I
don't really need to lock all VM memory.
> For testing you could perhaps try playing with hooks and prlimit, but
> forcing libvirt to lock all memory is still easier.
tried with setrlimit inside QEMU without any success.
Yeah, QEMU is not allowed to increase the limit if started by libvirt.
I'm not familiar with libvirt code, where do you suggest I will
add the calculation and setting?
The code which calculates how much memory needs to be locked should go
in qemuDomainGetMemLockLimitBytes. In addition to that you should modify
qemuDomainRequiresMemLock to return true in case the domain has iscsi
devices that will use the new transport option. In other words, the code
which sets memory locking limit is:
if (qemuDomainRequiresMemLock(def))
virCommandSetMaxMemLock(cmd, qemuDomainGetMemLockLimitBytes(def));
Jirka