Messages in this thread Patch in this message | | | Date | Fri, 22 Oct 2021 15:42:00 -0000 | From | "tip-bot2 for Peter Zijlstra" <> | Subject | [tip: sched/core] sched: Improve wake_up_all_idle_cpus() take #2 |
| |
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 96611c26dc351c33f73b48756a9feacc109e5bab Gitweb: https://git.kernel.org/tip/96611c26dc351c33f73b48756a9feacc109e5bab Author: Peter Zijlstra <peterz@infradead.org> AuthorDate: Mon, 18 Oct 2021 16:41:05 +02:00 Committer: Peter Zijlstra <peterz@infradead.org> CommitterDate: Fri, 22 Oct 2021 15:32:46 +02:00
sched: Improve wake_up_all_idle_cpus() take #2
As reported by syzbot and experienced by Pavel, using cpus_read_lock() in wake_up_all_idle_cpus() generates lock inversion (against mmap_sem and possibly others).
Instead, shrink the preempt disable region by iterating all CPUs and checking the online status for each individual CPU while having preemption disabled.
Fixes: 8850cb663b5c ("sched: Simplify wake_up_*idle*()") Reported-by: syzbot+d5b23b18d2f4feae8a67@syzkaller.appspotmail.com Reported-by: Pavel Machek <pavel@ucw.cz> Reported-by: Qian Cai <quic_qiancai@quicinc.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Qian Cai <quic_qiancai@quicinc.com> --- kernel/smp.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/kernel/smp.c b/kernel/smp.c index ad0b68a..01a7c17 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -1170,14 +1170,12 @@ void wake_up_all_idle_cpus(void) { int cpu; - cpus_read_lock(); - for_each_online_cpu(cpu) { - if (cpu == raw_smp_processor_id()) - continue; - - wake_up_if_idle(cpu); + for_each_possible_cpu(cpu) { + preempt_disable(); + if (cpu != smp_processor_id() && cpu_online(cpu)) + wake_up_if_idle(cpu); + preempt_enable(); } - cpus_read_unlock(); } EXPORT_SYMBOL_GPL(wake_up_all_idle_cpus);
| |