lkml.org 
[lkml]   [2022]   [Jul]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH 1/3] sched/core: Fix the bug that task won't enqueue into core tree when update cookie
On Mon, Jul 04, 2022 at 10:53:20AM +0200, Peter Zijlstra wrote:
> On Tue, Jun 28, 2022 at 03:57:23PM +0800, Cruz Zhao wrote:
> > In function sched_core_update_cookie(), a task will enqueue into the
> > core tree only when it enqueued before, that is, if an uncookied task
> > is cookied, it will not enqueue into the core tree until it enqueue
> > again, which will result in unnecessary force idle.
> >
> > Here follows the scenario:
> > CPU x and CPU y are a pair of SMT siblings.
> > 1. Start task a running on CPU x without sleeping, and task b and
> > task c running on CPU y without sleeping.
> > 2. We create a cookie and share it to task a and task b, and then
> > we create another cookie and share it to task c.
> > 3. Simpling core_forceidle_sum of task a and b from /proc/PID/sched
> >
> > And we will find out that core_forceidle_sum of task a takes 30%
> > time of the sampling period, which shouldn't happen as task a and b
> > have the same cookie.
> >
> > Then we migrate task a to CPU x', migrate task b and c to CPU y', where
> > CPU x' and CPU y' are a pair of SMT siblings, and sampling again, we
> > will found out that core_forceidle_sum of task a and b are almost zero.
> >
> > To solve this problem, we enqueue the task into the core tree if it's
> > on rq.
> >
> > Fixes: 6e33cad0af49("sched: Trivial core scheduling cookie management")
> > Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
> > ---
> > kernel/sched/core_sched.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/sched/core_sched.c b/kernel/sched/core_sched.c
> > index 38a2cec..ba2466c 100644
> > --- a/kernel/sched/core_sched.c
> > +++ b/kernel/sched/core_sched.c
> > @@ -75,7 +75,7 @@ static unsigned long sched_core_update_cookie(struct task_struct *p,
> > old_cookie = p->core_cookie;
> > p->core_cookie = cookie;
> >
> > - if (enqueued)
> > + if (task_on_rq_queued(p))
> > sched_core_enqueue(rq, p);
> >
> > /*
>
> Yeah; I suppose that's true. However if we want to consider the
> asymmetric case, we should be complete and also consider the case where
> we clear the cookie.
>
> And if you remove the second use of @enqueued, having that variable is
> rather redudant, which then leaves me with something like this.

Can you please confirm the below works for you so I can queue it?

> ---
> Subject: sched/core: Fix the bug that task won't enqueue into core tree when update cookie
> From: Cruz Zhao <CruzZhao@linux.alibaba.com>
> Date: Tue, 28 Jun 2022 15:57:23 +0800
>
> From: Cruz Zhao <CruzZhao@linux.alibaba.com>
>
> In function sched_core_update_cookie(), a task will enqueue into the
> core tree only when it enqueued before, that is, if an uncookied task
> is cookied, it will not enqueue into the core tree until it enqueue
> again, which will result in unnecessary force idle.
>
> Here follows the scenario:
> CPU x and CPU y are a pair of SMT siblings.
> 1. Start task a running on CPU x without sleeping, and task b and
> task c running on CPU y without sleeping.
> 2. We create a cookie and share it to task a and task b, and then
> we create another cookie and share it to task c.
> 3. Simpling core_forceidle_sum of task a and b from /proc/PID/sched
>
> And we will find out that core_forceidle_sum of task a takes 30%
> time of the sampling period, which shouldn't happen as task a and b
> have the same cookie.
>
> Then we migrate task a to CPU x', migrate task b and c to CPU y', where
> CPU x' and CPU y' are a pair of SMT siblings, and sampling again, we
> will found out that core_forceidle_sum of task a and b are almost zero.
>
> To solve this problem, we enqueue the task into the core tree if it's
> on rq.
>
> Fixes: 6e33cad0af49("sched: Trivial core scheduling cookie management")
> Signed-off-by: Cruz Zhao <CruzZhao@linux.alibaba.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Link: https://lkml.kernel.org/r/1656403045-100840-2-git-send-email-CruzZhao@linux.alibaba.com
> ---
> kernel/sched/core_sched.c | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> --- a/kernel/sched/core_sched.c
> +++ b/kernel/sched/core_sched.c
> @@ -56,7 +56,6 @@ static unsigned long sched_core_update_c
> unsigned long old_cookie;
> struct rq_flags rf;
> struct rq *rq;
> - bool enqueued;
>
> rq = task_rq_lock(p, &rf);
>
> @@ -68,14 +67,16 @@ static unsigned long sched_core_update_c
> */
> SCHED_WARN_ON((p->core_cookie || cookie) && !sched_core_enabled(rq));
>
> - enqueued = sched_core_enqueued(p);
> - if (enqueued)
> + if (sched_core_enqueued(p))
> sched_core_dequeue(rq, p, DEQUEUE_SAVE);
>
> old_cookie = p->core_cookie;
> p->core_cookie = cookie;
>
> - if (enqueued)
> + /*
> + * Consider the cases: !prev_cookie and !cookie.
> + */
> + if (cookie && task_on_rq_queued(p))
> sched_core_enqueue(rq, p);
>
> /*

\
 
 \ /
  Last update: 2022-07-14 13:37    [W:0.097 / U:0.068 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site