Messages in this thread | | | From | Valentin Schneider <> | Subject | Re: [PATCH 3/3] sched: Use cpu_dying() to fix balance_push vs hotplug-rollback | Date | Wed, 21 Apr 2021 10:32:43 +0100 |
| |
On 20/04/21 16:58, Peter Zijlstra wrote: > On Tue, Apr 20, 2021 at 04:39:04PM +0200, Peter Zijlstra wrote: >> On Tue, Apr 20, 2021 at 04:20:56PM +0200, Peter Zijlstra wrote: >> > On Tue, Apr 20, 2021 at 10:46:33AM +0100, Vincent Donnefort wrote: >> > >> > > Found the issue: >> > > >> > > $ cat hotplug/states: >> > > 219: sched:active >> > > 220: online >> > > >> > > CPU0: >> > > >> > > $ echo 219 > hotplug/fail >> > > $ echo 0 > online >> > > >> > > => cpu_active = 1 cpu_dying = 1 >> > > >> > > which means that later on, for another CPU hotunplug, in >> > > __balance_push_cpu_stop(), the fallback rq for a kthread can select that >> > > CPU0, but __migrate_task() would fail and we end-up in an infinite loop, >> > > trying to migrate that task to CPU0. >> > > >> > > The problem is that for a failure in sched:active, as "online" has no callback, >> > > there will be no call to cpuhp_invoke_callback(). Hence, the cpu_dying bit would >> > > not be reset. >> > >> > Urgh! Good find. > >> I seem to have triggered the BUG() in select_fallback_rq() with your recipie. >> Have cpu0 fail on sched:active, then offline all other CPUs. >> >> Now lemme add that patch. > > (which obviously didn't actually build) seems to fix it. >
Moving the cpu_dying_mask update from cpuhp_invoke_callback() to cpuhp_{set, reset}_state() means we lose an update in cpuhp_issue_call(), but AFAICT that wasn't required (this doesn't actually change a CPU's hotplug state, rather executes some newly installed/removed callbacks whose state maps below the CPU's current hp state).
Actually staring at it some more, it might have caused bugs: if a cpuhp_setup_state() fails, we can end up in cpuhp_rollback_install() which will end up calling cpuhp_invoke_callback(bringup=false) and mess with the dying mask.
FWIW:
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
| |