[lkml]   [2014]   [Jan]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [RFC 0/4] memcg: Low-limit reclaim
On Wed 29-01-14 11:08:46, Greg Thelen wrote:
> The series looks useful. We (Google) have been using something similar.
> In practice such a low_limit (or memory guarantee), doesn't nest very
> well.
> Example:
> - parent_memcg: limit 500, low_limit 500, usage 500
> 1 privately charged non-reclaimable page (e.g. mlock, slab)
> - child_memcg: limit 500, low_limit 500, usage 499

I am not sure this is a good example. Your setup basically say that no
single page should be reclaimed. I can imagine this might be useful in
some cases and I would like to allow it but it sounds too extreme (e.g.
a load which would start trashing heavily once the reclaim starts and it
makes more sense to start it again rather than crowl - think about some
mathematical simulation which might diverge).

> If a streaming file cache workload (e.g. sha1sum) starts gobbling up
> page cache it will lead to an oom kill instead of reclaiming.

Does it make any sense to protect all of such memory although it is
easily reclaimable?

> One could
> argue that this is working as intended because child_memcg was promised
> 500 but can only get 499. So child_memcg is oom killed rather than
> being forced to operate below its promised low limit.
> This has led to various internal workarounds like:
> - don't charge any memory to interior tree nodes (e.g. parent_memcg);
> only charge memory to cgroup leafs. This gets tricky when dealing
> with reparented memory inherited to parent from child during cgroup
> deletion.

Do those need any protection at all?

> - don't set low_limit on non leafs (e.g. do not set low limit on
> parent_memcg). This constrains the cgroup layout a bit. Some
> customers want to purchase $MEM and setup their workload with a few
> child cgroups. A system daemon hands out $MEM by setting low_limit
> for top-level containers (e.g. parent_memcg). Thereafter such
> customers are able to partition their workload with sub memcg below
> child_memcg. Example:
> parent_memcg
> \
> child_memcg
> / \
> server backup

I think that the low_limit makes sense where you actually want to
protect something from reclaim. And backup sounds like a bad fit for

> Thereafter customers often want some weak isolation between server and
> backup. To avoid undesired oom kills the server/backup isolation is
> provided with a softer memory guarantee (e.g. soft_limit). The soft
> limit acts like the low_limit until priority becomes desperate.

Johannes was already suggesting that the low_limit should allow for a
weaker semantic as well. I am not very much inclined to that but I can
leave with a knob which would say oom_on_lowlimit (on by default but
allowed to be set to 0). We would fallback to the full reclaim if
no groups turn out to be reclaimable.
Michal Hocko

 \ /
  Last update: 2014-01-30 13:41    [W:0.192 / U:0.688 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site