Messages in this thread | | | From | Vincent Guittot <> | Date | Fri, 25 Feb 2022 14:15:04 +0100 | Subject | Re: [RFC PATCH 0/5] introduce sched-idle balancing |
| |
On Fri, 25 Feb 2022 at 11:46, Abel Wu <wuyun.abel@bytedance.com> wrote: > > On 2/25/22 4:29 PM, Vincent Guittot Wrote: > > On Fri, 25 Feb 2022 at 07:46, Abel Wu <wuyun.abel@bytedance.com> wrote: > >> > >> Hi Peter, > >> > >> On 2/24/22 11:20 PM, Peter Zijlstra Wrote: > >>> On Thu, Feb 17, 2022 at 11:43:56PM +0800, Abel Wu wrote: > >>>> Current load balancing is mainly based on cpu capacity > >>>> and task util, which makes sense in the POV of overall > >>>> throughput. While there still might be some improvement > >>>> can be done by reducing number of overloaded cfs rqs if > >>>> sched-idle or idle rq exists. > >>> > >>> I'm much confused, there is an explicit new-idle balancer and a periodic > >>> idle balancer already there. > >> > >> The two balancers are triggered on the rqs that have no tasks on them, > >> and load_balance() seems don't show a preference for non-idle tasks so > > > > The load balance will happen at the idle pace if a sched_idle task is > > running on the cpu so you will have an ILB on each cpu that run a > > sched-idle task > > I'm afraid I don't quite follow you, since sched-idle balancer doesn't > touch the ILB part, can you elaborate on this? Thanks.
I was referring to your sentence " The two balancers are triggered on the rqs that have no tasks on them". When there is only sched-idle tasks on a rq, the load_balance behave like the Idle Load Balance when there is no task i.e. as often
> > > > >> there might be possibility that only idle tasks are pulled during load > >> balance while overloaded rqs (rq->cfs.h_nr_running > 1) exist. As a > > > > There is a LB_MIN feature (disable by default) that filters task with > > very low load ( < 16) which includes sched-idle task which has a max > > load of 3
but we could easily change this like if !sched_idle_cpus then LB can migrate only cfs tasks otherwise can migrate sched_idle task as well. Instead of creating another side channel
> > This feature might not that friendly to the situation that only > sched-idle tasks are running in the system. And this situation > can last more than half a day in our co-location systems in which > the training/batch tasks are placed under idle groups or directly > assigned to SCHED_IDLE. > > > > >> result the normal tasks, mostly latency-critical ones in our case, on > >> that overloaded rq still suffer waiting for each other. I observed this > >> through perf sched. > >> > >> IOW the main difference from the POV of load_balance() between the > >> latency-critical tasks and the idle ones is load. > >> > >> The sched-idle balancer is triggered on the sched-idle rqs periodically > >> and the newly-idle ones. It does a 'fast' pull of non-idle tasks from > >> the overloaded rqs to the sched-idle/idle ones to let the non-idle tasks > >> make full use of cpu resources. > >> > >> The sched-idle balancer only focuses on non-idle tasks' performance, so > >> it can introduce overall load imbalance, and that's why I put it before > >> load_balance(). > > > > According to the very low weight of a sched-idle task, I don't expect > > much imbalance because of sched-idle tasks. But this also depends of > > the number of sched-idle task. > > > > > >> > >> Best Regards, > >> Abel
| |