Messages in this thread | | | From | Vincent Guittot <> | Date | Tue, 31 Jul 2018 14:17:27 +0200 | Subject | Re: [PATCHv4 12/12] sched/core: Disable SD_PREFER_SIBLING on asymmetric cpu capacity domains |
| |
On Fri, 6 Jul 2018 at 16:31, Morten Rasmussen <morten.rasmussen@arm.com> wrote: > > On Fri, Jul 06, 2018 at 12:18:17PM +0200, Vincent Guittot wrote: > > On Wed, 4 Jul 2018 at 12:18, Morten Rasmussen <morten.rasmussen@arm.com> wrote: > > > > > > The 'prefer sibling' sched_domain flag is intended to encourage > > > spreading tasks to sibling sched_domain to take advantage of more caches > > > and core for SMT systems. It has recently been changed to be on all > > > non-NUMA topology level. However, spreading across domains with cpu > > > capacity asymmetry isn't desirable, e.g. spreading from high capacity to > > > low capacity cpus even if high capacity cpus aren't overutilized might > > > give access to more cache but the cpu will be slower and possibly lead > > > to worse overall throughput. > > > > > > To prevent this, we need to remove SD_PREFER_SIBLING on the sched_domain > > > level immediately below SD_ASYM_CPUCAPACITY. > > > > This makes sense. Nevertheless, this patch also raises a scheduling > > problem and break the 1 task per CPU policy that is enforced by > > SD_PREFER_SIBLING. > > Scheduling one task per cpu when n_task == n_cpus on asymmetric > topologies is generally broken already and this patch set doesn't fix > that problem. > > SD_PREFER_SIBLING might seem to help in very specific cases: > n_litte_cpus == n_big_cpus. In that case the little group might > classified as overloaded. It doesn't guarantee that anything gets pulled > as the grp_load/grp_capacity in the imbalance calculation on some system > still says the little cpus are more loaded than the bigs despite one of > them being idle. That depends on the little cpu capacities. > > On systems where n_little_cpus != n_big_cpus SD_PREFER_SIBLING is broken > as it assumes the group_weight to be the same. This is the case on Juno > and several other platforms. > > IMHO, SD_PREFER_SIBLING isn't the solution to this problem. It might
I agree but this patchset creates a regression in the scheduling behavior
> help for a limited subset of topologies/capacities but the right > solution is to change the imbalance calculation. As the name says, it is
Yes that what does the prototype that I came with.
> meant to spread tasks and does so unconditionally. For asymmetric > systems we would like to consider cpu capacity before migrating tasks. > > > When running the tests of your cover letter, 1 long > > running task is often co scheduled on a big core whereas short pinned > > tasks are still running and a little core is idle which is not an > > optimal scheduling decision > > This can easily happen with SD_PREFER_SIBLING enabled too so I wouldn't > say that this patch breaks anything that isn't broken already. In fact > we this happening with and without this patch applied.
At least for the use case above, this doesn't happen when SD_PREFER_SIBLING is set
> > Morten
| |