lkml.org 
[lkml]   [2021]   [Oct]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[tip: sched/core] sched: Improve wake_up_all_idle_cpus() take #2
The following commit has been merged into the sched/core branch of tip:

Commit-ID: 96611c26dc351c33f73b48756a9feacc109e5bab
Gitweb: https://git.kernel.org/tip/96611c26dc351c33f73b48756a9feacc109e5bab
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Mon, 18 Oct 2021 16:41:05 +02:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 22 Oct 2021 15:32:46 +02:00

sched: Improve wake_up_all_idle_cpus() take #2

As reported by syzbot and experienced by Pavel, using cpus_read_lock()
in wake_up_all_idle_cpus() generates lock inversion (against mmap_sem
and possibly others).

Instead, shrink the preempt disable region by iterating all CPUs and
checking the online status for each individual CPU while having
preemption disabled.

Fixes: 8850cb663b5c ("sched: Simplify wake_up_*idle*()")
Reported-by: syzbot+d5b23b18d2f4feae8a67@syzkaller.appspotmail.com
Reported-by: Pavel Machek <pavel@ucw.cz>
Reported-by: Qian Cai <quic_qiancai@quicinc.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Qian Cai <quic_qiancai@quicinc.com>
---
kernel/smp.c | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index ad0b68a..01a7c17 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1170,14 +1170,12 @@ void wake_up_all_idle_cpus(void)
{
int cpu;

- cpus_read_lock();
- for_each_online_cpu(cpu) {
- if (cpu == raw_smp_processor_id())
- continue;
-
- wake_up_if_idle(cpu);
+ for_each_possible_cpu(cpu) {
+ preempt_disable();
+ if (cpu != smp_processor_id() && cpu_online(cpu))
+ wake_up_if_idle(cpu);
+ preempt_enable();
}
- cpus_read_unlock();
}
EXPORT_SYMBOL_GPL(wake_up_all_idle_cpus);

\
 
 \ /
  Last update: 2021-10-22 17:43    [W:0.246 / U:0.104 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site