Messages in this thread Patch in this message | | | Subject | [RFC] optimize the locking in the rebalance_domains() | From | Vlad Zolotarov <> | Date | Sun, 22 Jul 2012 19:23:55 +0300 |
| |
Ingo, we've noticed that rebalance_domains() will try to take a lock every time it's called (every jiffy) if SD_SERIALIZE is set (which is a default configuration). This is done regardless the fact that maybe there hasn't passed enough time since the last rebalancing in which case there is no need to take a lock the first place.
The above creates a heavy false sharing problem on the "balancing" spin-lock on large SMP systems: try_lock() is implemented with an (atomic) xchng instruction which invalidates the cache line "balancing" belongs to and therefore creates an intensive cross-NUMA-nodes traffic.
The below patch will minimize the above phenomena to the time slots it's really needed, namely when the "interval" has really passed.
Pls., comment.
thanks, vlad
--- kernel/sched/fair.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c099cc6..6777d38 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4689,6 +4689,9 @@ static void rebalance_domains(int cpu, enum cpu_idle_type idle) interval = msecs_to_jiffies(interval); interval = clamp(interval, 1UL, max_load_balance_interval); + if (!time_after_eq(jiffies, sd->last_balance + interval)) + goto out; + need_serialize = sd->flags & SD_SERIALIZE; if (need_serialize) { @@ -4696,16 +4699,15 @@ static void rebalance_domains(int cpu, enum cpu_idle_type idle) goto out; } - if (time_after_eq(jiffies, sd->last_balance + interval)) { - if (load_balance(cpu, rq, sd, idle, &balance)) { - /* - * We've pulled tasks over so either we're no - * longer idle. - */ - idle = CPU_NOT_IDLE; - } - sd->last_balance = jiffies; + if (load_balance(cpu, rq, sd, idle, &balance)) { + /* + * We've pulled tasks over so either we're no + * longer idle. + */ + idle = CPU_NOT_IDLE; } + sd->last_balance = jiffies; + if (need_serialize) spin_unlock(&balancing); out: -- 1.7.9.5
| |