Messages in this thread | | | Date | Fri, 16 Aug 2013 12:34:38 +0200 | From | Peter Zijlstra <> | Subject | Re: [PATCH 6/6] sched, fair: Rework and comment the group_imb code |
| |
On Fri, Aug 16, 2013 at 12:12:27PM +0200, Peter Zijlstra wrote: > +/* > + * Group imbalance indicates (and tries to solve) the problem where balancing > + * groups is inadequate due to tsk_cpus_allowed() constraints. > + * > + * Imagine a situation of two groups of 4 cpus each and 4 tasks each with a > + * cpumask covering 1 cpu of the first group and 3 cpus of the second group. > + * Something like: > + * > + * { 0 1 2 3 } { 4 5 6 7 } > + * * * * *
And yes, people think this is a reasonable thing to do :/
> + * > + * If we were to balance group-wise we'd place two tasks in the first group and > + * two tasks in the second group. Clearly this is undesired as it will overload > + * cpu 3 and leave one of the cpus in the second group unused. > + * > + * The current solution to this issue is detecting the skew in the first group > + * by noticing it has a cpu that is overloaded while the remaining cpus are > + * idle -- or rather, there's a distinct imbalance in the cpus; see > + * sg_imbalanced(). > + * > + * When this is so detected; this group becomes a candidate for busiest; see > + * update_sd_pick_busiest(). And calculcate_imbalance() and > + * find_busiest_group() avoid some of the usual balance conditional to allow it > + * to create an effective group imbalance. > + * > + * This is a somewhat tricky proposition since the next run might not find the > + * group imbalance and decide the groups need to be balanced again. A most > + * subtle and fragile situation. > + */
One of the things that I know about which can (and does) go wrong with this is that we typically pull 'all' tasks to the balance cpu only to then let future (lower) load-balance passes spread that out again.
But until its spread out, we have this spike in the task distribution that will trigger the group_imb condition.
| |