3:50 a.m.
OpenVZ has several parameters for memory management. All of them can be
configured independetly.
Patch allow to get configuration of memory from container config and
then calculate total usage of memory.
It is open question how to manage memory?
6:15 a.m.
On Wed, Aug 13, 2008 at 12:50:33PM +0400, Evgeniy Sokolov wrote:
OpenVZ has several parameters for memory management. All of them can
be
configured independetly.
Patch allow to get configuration of memory from container config and
then calculate total usage of memory.
It is open question how to manage memory?
Basically the 'memory' as shown by libvirt should correspond to
the 'Total' seen inside the guest when running 'free'.
eg
# free
total used free shared buffers cached
Mem: 524288 34676 489612 0 0 0
-/+ buffers/cache: 34676 489612
Swap: 0 0 0
In this case 'memory' is 524288 kb.
This looks to correspond to OpenVZs 'PRIVVMPAGES' pages parameter.
+openvzFreeUbParam(struct openvz_ub_param *ub)
+{
+ if (ub == NULL)
+ return;
+#define FREE_P(x) if (ub->x != NULL) VIR_FREE(ub->x);
This is not required. VIR_FREE is happy to accept a NULL
and will do nothing. Just call VIR_FREE directly without
testing for != NULL.
+/* calculate guaranteed memory for container
+*/
+int
+openvzUb2Memory(struct openvz_ub_param *ub, unsigned long *memory)
+{
+ unsigned long lowmem, mem, allmem;
+
+ if (ub->oomguarpages == NULL ||
+ ub->kmemsize == NULL ||
+ ub->tcpsndbuf == NULL ||
+ ub->tcprcvbuf == NULL ||
+ ub->othersockbuf == NULL ||
+ ub->dgramrcvbuf == NULL ||
+ ub->lockedpages == NULL)
+ return -1;
+
+ lowmem = ub->kmemsize->limit +
+ ub->tcpsndbuf->limit +
+ ub->tcprcvbuf->limit +
+ ub->othersockbuf->limit +
+ ub->dgramrcvbuf->limit +
+ ub->lockedpages->limit * get_pagesize();
+
+ mem = ub->oomguarpages->barrier * get_pagesize();
+ allmem = mem + lowmem; //in bytes
+ *memory = allmem / 1024; //in kb
Summing all the different memory limits together is wrong - this will
make it impossible to implement a 'setMemory' operation because you've
now no way of deciding which parameter to set.
In my testing the total memory inside a OpenVZ container seems to
correspond to the 'lockedpages' parameter, so we should just return
that value.
Since you convert to 'bytes' before converting back into kb, you'll
potentially overflow a 'long' on 32-bit platforms. You should use
'unsigned long long' instead to be safe.
+int
+openvzUb2MaxMemory(struct openvz_ub_param *ub, unsigned long *memory)
+{
+ unsigned long lowmem, mem, allmem;
+
+ if (ub->privvmpages == NULL ||
+ ub->kmemsize == NULL ||
+ ub->tcpsndbuf == NULL ||
+ ub->tcprcvbuf == NULL ||
+ ub->othersockbuf == NULL ||
+ ub->dgramrcvbuf == NULL ||
+ ub->lockedpages == NULL)
+ return -1;
+
+ lowmem = ub->kmemsize->limit +
+ ub->tcpsndbuf->limit +
+ ub->tcprcvbuf->limit +
+ ub->othersockbuf->limit +
+ ub->dgramrcvbuf->limit +
+ ub->lockedpages->limit * get_pagesize();
+
+ mem = ub->privvmpages->barrier * get_pagesize();
+ allmem = mem + lowmem; //in bytes
+ *memory = allmem / 1024; //in kb
Likewise this should simply use the lockedpages parameter I believe.
What is the difference between a 'barrier' and a 'limit' in openvz
config ? Would it make sense to map 'memory' to the 'barrier' and
'maxmemory' to the 'limit' ?
@@ -262,6 +264,10 @@ static int openvzDomainGetInfo(virDomain
}
}
+ openvzGetMemory(dom->conn, strtoI(dom->name), &(info->memory)) ;
+ openvzGetMaxMemory(dom->conn, strtoI(dom->name), &(info->maxMem)) ;
Can't you use 'vm->vpsid' and avoid the strtoI() call ?
It seems it would be more efficient to call openvzReadUbParams()
and openvzUb2Memory() & openvzUb2MaxMemory directly, rather than
calling out to openvzGetMemory() and openvzGetMaxMemory() since
it would avoid reading the config file twice - you can also get
the nrVirtCPUs at the same time.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:08 a.m.
On Wed, Aug 13, 2008 at 12:50:33PM +0400, Evgeniy Sokolov wrote:
> OpenVZ has several parameters for memory management. All of them can be
> configured independetly.
>
> Patch allow to get configuration of memory from container config and
> then calculate total usage of memory.
>
> It is open question how to manage memory?
Basically the 'memory' as shown by libvirt should correspond to
the 'Total' seen inside the guest when running 'free'.
eg
# free
total used free shared buffers cached
Mem: 524288 34676 489612 0 0 0
-/+ buffers/cache: 34676 489612
Swap: 0 0 0
In this case 'memory' is 524288 kb.
This looks to correspond to OpenVZs 'PRIVVMPAGES' pages parameter.
It may
be configured what will be shown in /proc/meminfo inside container.
Memory from node, total memory or privvmpages. Privvmpages is default.
> +openvzFreeUbParam(struct openvz_ub_param *ub)
> +{
> + if (ub == NULL)
> + return;
> +#define FREE_P(x) if (ub->x != NULL) VIR_FREE(ub->x);
This is not required. VIR_FREE is happy to accept a NULL
and will do nothing. Just call VIR_FREE directly without
testing for != NULL.
> +/* calculate guaranteed memory for container
> +*/
> +int
> +openvzUb2Memory(struct openvz_ub_param *ub, unsigned long *memory)
> +{
> + unsigned long lowmem, mem, allmem;
> +
> + if (ub->oomguarpages == NULL ||
> + ub->kmemsize == NULL ||
> + ub->tcpsndbuf == NULL ||
> + ub->tcprcvbuf == NULL ||
> + ub->othersockbuf == NULL ||
> + ub->dgramrcvbuf == NULL ||
> + ub->lockedpages == NULL)
> + return -1;
> +
> + lowmem = ub->kmemsize->limit +
> + ub->tcpsndbuf->limit +
> + ub->tcprcvbuf->limit +
> + ub->othersockbuf->limit +
> + ub->dgramrcvbuf->limit +
> + ub->lockedpages->limit * get_pagesize();
> +
> + mem = ub->oomguarpages->barrier * get_pagesize();
> + allmem = mem + lowmem; //in bytes
> + *memory = allmem / 1024; //in kb
Summing all the different memory limits together is wrong - this will
make it impossible to implement a 'setMemory' operation because you've
now no way of deciding which parameter to set.
It is correct calcuation of total
memory.
Because of non-obvious implenetation of 'setMemory', I have asked 'how
to manage memory' in begining.
I propouse to set some portion of 'memory' to each limit.
In my testing the total memory inside a OpenVZ container seems to
correspond to the 'lockedpages' parameter, so we should just return
that value.
How did you get total memory?
Most big part of memory is oomguarpages (by default). Lockedpages is
about %1.
Since you convert to 'bytes' before converting back into kb, you'll
potentially overflow a 'long' on 32-bit platforms. You should use
'unsigned long long' instead to be safe.
yes, you right.
> +int
> +openvzUb2MaxMemory(struct openvz_ub_param *ub, unsigned long *memory)
> +{
> + unsigned long lowmem, mem, allmem;
> +
> + if (ub->privvmpages == NULL ||
> + ub->kmemsize == NULL ||
> + ub->tcpsndbuf == NULL ||
> + ub->tcprcvbuf == NULL ||
> + ub->othersockbuf == NULL ||
> + ub->dgramrcvbuf == NULL ||
> + ub->lockedpages == NULL)
> + return -1;
> +
> + lowmem = ub->kmemsize->limit +
> + ub->tcpsndbuf->limit +
> + ub->tcprcvbuf->limit +
> + ub->othersockbuf->limit +
> + ub->dgramrcvbuf->limit +
> + ub->lockedpages->limit * get_pagesize();
> +
> + mem = ub->privvmpages->barrier * get_pagesize();
> + allmem = mem + lowmem; //in bytes
> + *memory = allmem / 1024; //in kb
Likewise this should simply use the lockedpages parameter I believe.
What is the difference between a 'barrier' and a 'limit' in openvz
config ? Would it make sense to map 'memory' to the 'barrier' and
'maxmemory' to the 'limit' ?
Barrier & limit are soft and hard
limits.
Barrier is guarantee availability of resources. When resource is grown
up to barrier, it can grow more resurce within grace period, but not
more than limit.
Memory for application:
oomguarpages & vmguarpages - barrier has matter. It is guarantee number
of pages. Limits should be always very big - 2147483647.
When the whole node is in out of memory, kernel may don't allow to
allocate memory or kill container process.
Container may allocate more memory if memory on whole node is available,
but no more than privvmpages.
>
> @@ -262,6 +264,10 @@ static int openvzDomainGetInfo(virDomain
> }
> }
>
> + openvzGetMemory(dom->conn, strtoI(dom->name), &(info->memory)) ;
> + openvzGetMaxMemory(dom->conn, strtoI(dom->name), &(info->maxMem))
;
Can't you use 'vm->vpsid' and avoid the strtoI() call ?
vm->vpsid = -1 when VM is stoped.
It seems it would be more efficient to call openvzReadUbParams()
and openvzUb2Memory() & openvzUb2MaxMemory directly, rather than
calling out to openvzGetMemory() and openvzGetMaxMemory() since
it would avoid reading the config file twice - you can also get
the nrVirtCPUs at the same time.
Yes, we can read memory once. But get number of
CPUs is little complicated.
Regards,
Daniel
9:46 a.m.
On Thu, Aug 14, 2008 at 06:08:31PM +0400, Evgeniy Sokolov wrote:
>On Wed, Aug 13, 2008 at 12:50:33PM +0400, Evgeniy Sokolov wrote:
>>OpenVZ has several parameters for memory management. All of them can be
>>configured independetly.
>
>Summing all the different memory limits together is wrong - this will
>make it impossible to implement a 'setMemory' operation because you've
>now no way of deciding which parameter to set.
It is correct calcuation of total memory.
Because of non-obvious implenetation of 'setMemory', I have asked 'how
to manage memory' in begining.
I propouse to set some portion of 'memory' to each limit.
I don't think that will be lead to good behaviour - ultimtely I think
we'll need to expose more (all?) of these memory settings directly as
tunable resources. THus for the 'memory' setting we which pick the
value which provides the primary memory size as available to apps
within the container.
>
>In my testing the total memory inside a OpenVZ container seems to
>correspond to the 'lockedpages' parameter, so we should just return
>that value.
How did you get total memory?
Most big part of memory is oomguarpages (by default). Lockedpages is
about %1.
I've just found & read these descriptions...
http://wiki.openvz.org/Oomguarpages#oomguarpages
http://wiki.openvz.org/Oomguarpages#privvmpages
http://wiki.openvz.org/Vmguarpages#vmguarpages
To summarize
"Oomguarpages parameter accounts the total amount of memory and
swap space used by the processes of a particular container.
"Privvmpages parameter accounts allocated (but, possibly, not
used yet) memory. The accounted value is an estimation how
much memory will be really consumed when the container's
applications start to use the allocated memory. Consumed
memory is accounted into oomguarpages parameter
"vmguarpages parameter controls how much memory is available to
the Virtual Environment (i.e. how much memory its applications
can allocate by malloc(3) or other standard Linux memory
allocation mechanisms"
"The memory allocation guarantee (vmguarpages) is a primary tool
for controlling the memory available to containers, because it
allows administrators to provide Service Level Agreements"
So from those I'd think that 'vmguarpages' maps best into libvirt's
concept of 'memory'. Though perhaps 'oomguardpages' could be set for
'memory' and 'vmguarpages' used for 'max memory'
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
10:52 a.m.
On Thu, Aug 14, 2008 at 06:08:31PM +0400, Evgeniy Sokolov wrote:
>> On Wed, Aug 13, 2008 at 12:50:33PM +0400, Evgeniy Sokolov wrote:
>>> OpenVZ has several parameters for memory management. All of them can be
>>> configured independetly.
>> Summing all the different memory limits together is wrong - this will
>> make it impossible to implement a 'setMemory' operation because
you've
>> now no way of deciding which parameter to set.
> It is correct calcuation of total memory.
> Because of non-obvious implenetation of 'setMemory', I have asked 'how
> to manage memory' in begining.
> I propouse to set some portion of 'memory' to each limit.
I don't think that will be lead to good behaviour - ultimtely I think
we'll need to expose more (all?) of these memory settings directly as
tunable resources.
Good idea! 5 setting for low memory + application
THus for the 'memory' setting we which pick the
value which provides the primary memory size as available to apps
within the container.
>> In my testing the total memory inside a OpenVZ container seems to
>> correspond to the 'lockedpages' parameter, so we should just return
>> that value.
> How did you get total memory?
> Most big part of memory is oomguarpages (by default). Lockedpages is
> about %1.
I've just found & read these descriptions...
http://wiki.openvz.org/Oomguarpages#oomguarpages
http://wiki.openvz.org/Oomguarpages#privvmpages
http://wiki.openvz.org/Vmguarpages#vmguarpages
To summarize
"Oomguarpages parameter accounts the total amount of memory and
swap space used by the processes of a particular container.
"Privvmpages parameter accounts allocated (but, possibly, not
used yet) memory. The accounted value is an estimation how
much memory will be really consumed when the container's
applications start to use the allocated memory. Consumed
memory is accounted into oomguarpages parameter
"vmguarpages parameter controls how much memory is available to
the Virtual Environment (i.e. how much memory its applications
can allocate by malloc(3) or other standard Linux memory
allocation mechanisms"
"The memory allocation guarantee (vmguarpages) is a primary tool
for controlling the memory available to containers, because it
allows administrators to provide Service Level Agreements"
So from those I'd think that 'vmguarpages' maps best into libvirt's
concept of 'memory'. Though perhaps 'oomguardpages' could be set for
'memory' and 'vmguarpages' used for 'max memory'
Oomguarpages & Privvmpages & vmguarpages are settings for apps memory.
We may
1 set 'memory' to 'privvmpages' - then free will show correct value.
2 set 'memory' to 'vmguarpages' or 'oomguardpages' and
'maxmemory' to 'privvmpages'.
What do 'MaxMemory' mean for other libvirt drivers?
Bad point of it, for example:
We can set many of Oomguarpages & Privvmpages & vmguarpages but
application can not start because of 'kmemsize' is over.
Daniel
5944
days inactive
5945
days old
4 comments
2 participants
participants (2)
-
Daniel P. Berrange -
Evgeniy Sokolov