[lkml]   [2008]   [Apr]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [-mm] Add an owner to the mm_struct (v8)
Paul Menage wrote:
> On Fri, Apr 4, 2008 at 2:25 AM, Balbir Singh <> wrote:
>> >> For other controllers,
>> >> they'll need to monitor exit() callbacks to know when the leader is dead :( (sigh).
>> >
>> > That sounds like a nightmare ...
>> >
>> Yes, it would be, but worth the trouble. Is it really critical to move a dead
>> cgroup leader to init_css_set in cgroup_exit()?
> It struck me that this whole group leader optimization is broken as it
> stands since there could (in strange configurations) be multiple
> thread groups sharing the same mm.
> I wonder if we can't just delay the exit_mm() call of a group leader
> until all its threads have exited?

Not sure about this one, I suspect keeping the group_leader around is an
optimization, changing exit_mm() for the group_leader, not sure how that will
impact functionality or standards. It might even break some applications.

Repeating my question earlier

Can we delay setting task->cgroups = &init_css_set for the group_leader, until
all threads have exited? If the user is unable to remove a cgroup node, it will
be due a valid reason, the group_leader is still around, since the threads are
still around. The user in that case should wait for notify_on_release.

>> > As long as we find someone to pass the mm to quickly, it shouldn't be
>> > too bad - I think we're already optimized for that case. Generally the
>> > group leader's first child will be the new owner, and any subsequent
>> > times the owner exits, they're unlikely to have any children so
>> > they'll go straight to the sibling check and pass the mm to the
>> > parent's first child.
>> >
>> > Unless they all exit in strict sibling order and hence pass the mm
>> > along the chain one by one, we should be fine. And if that exit
>> > ordering does turn out to be common, then simply walking the child and
>> > sibling lists in reverse order to find a victim will minimize the
>> > amount of passing.
>> >
>> Finding the next mm might not be all that bad, but doing it each time a task
>> exits, can be an overhead, specially for large multi threaded programs.
> Right, but we only have that overhead if we actually end up passing
> the mm from one to another each time they exit. It would be
> interesting to know what order the threads in a large multi-threaded
> process exit typically (when the main process exits and all the
> threads die).
> I guess it's likely to be one of:
> - in thread creation order (i.e. in order of parent->children list),
> in which case we should try to throw the mm to the parent's last child
> - in reverse creation order, in which case we should try to throw the
> mm to the parent's first child
> - in random order depending on which threads the scheduler runs first
> (in which case we can expect that a small fraction of the threads will
> have to throw the mm whichever end we start from)
>> This can
>> get severe if the new mm->owner belongs to a different cgroup, in which case we
>> need to use callbacks as well.
>> If half the threads belonged to a different cgroup and the new mm->owner kept
>> switching between cgroups, the overhead would be really high, with the callbacks
>> and the mm->owner changing frequently.
> To me, it seems that setting up a *virtual address space* cgroup
> hierarchy and then putting half your threads in one group and half in
> the another is asking for trouble. We need to not break in that
> situation, but I'm not sure it's a case to optimize for.

That could potentially happen, if the virtual address space cgroup and cpu
control cgroup were bound together in the same hierarchy by the sysadmin.

I measured the overhead of removing the delay_group_leader optimization and
found a 4% impact on throughput (with volanomark, that is one of the
multi-threaded benchmarks I know of).

Warm Regards,
Balbir Singh
Linux Technology Center

 \ /
  Last update: 2008-04-05 16:51    [W:0.060 / U:5.004 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site