Messages in this thread | | | Date | Tue, 23 Oct 2018 23:06:54 +0530 | From | Srikar Dronamraju <> | Subject | Re: [PATCH] sched/numa: fix choosing isolated CPUs when task_numa_migrate() |
| |
Hi Yi Wang,
> When trying to migrate to a CPU in task_numa_migrate(), we invoke > task_numa_find_cpu() to choose a spot, in which function we skip > the CPU which is not in cpus_allowed, but forgot to concern the > isolated CPUs, and this may cause the task would run on the isolcpus. > > This patch fixes this issue by checking the load_balance_mask. > > Signed-off-by: Yi Wang <wang.yi59@zte.com.cn> > Reviewed-by: Yi Liu <liu.yi24@zte.com.cn> > --- > kernel/sched/fair.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) >
I had proposed something similar http://lkml.kernel.org/r/1491326848-5748-1-git-send-email-srikar@linux.vnet.ibm.com
but at that time, Peter felt we should fix it at different level. http://lkml.kernel.org/r/20170406073659.y6ubqriyshax4v4m@hirez.programming.kicks-ass.net
and I think he is right.
I have system with 32 cpus, have passed isolcpus as a kernel parameter.
$ grep -o "isolcpus=[,,1-9]*" /proc/cmdline isolcpus=1,5,9,13 $ grep -i cpus_allowed /proc/$$/status Cpus_allowed: ffffdddd Cpus_allowed_list: 0,2-4,6-8,10-12,14-31
So for a task running on top_cpuset will not have isolcpus as part of the cpus_allowed.
However if a said task were to call sched_setaffinity, then there is every likely hood of the cpus being passed being a mix of isolcpus and nonisolcpus.
For example perf bench numa mem --no-data_rand_walk -p 4 -t 8 -G 0 -P 3072 -T 0 -l 50 -c -s 1000
$ for i in $(pgrep -f perf); do grep -i cpus_allowed_list /proc/$i/task/*/status ; done | head -n 10 Cpus_allowed_list: 0,2-4,6-8,10-12,14-31 /proc/2107/task/2107/status:Cpus_allowed_list: 0-31 /proc/2107/task/2196/status:Cpus_allowed_list: 0-31 /proc/2107/task/2197/status:Cpus_allowed_list: 0-31 /proc/2107/task/2198/status:Cpus_allowed_list: 0-31 /proc/2107/task/2199/status:Cpus_allowed_list: 0-31 /proc/2107/task/2200/status:Cpus_allowed_list: 0-31 /proc/2107/task/2201/status:Cpus_allowed_list: 0-31 /proc/2107/task/2202/status:Cpus_allowed_list: 0-31 /proc/2107/task/2203/status:Cpus_allowed_list: 0-31
So the cpus_allowed has a mix of isolcpus and nonisolcpus.
While this patch fixes the problem, there is a risk of missing other places like update_numa_stats also iterates and accounts for tasks stats running on isolcpus.
I will send a patch with a slightly different approach. Request you to review and verify the same.
-- Thanks and Regards Srikar Dronamraju
| |