Messages in this thread | | | From | Vincent Guittot <> | Date | Wed, 30 Nov 2022 15:29:59 +0100 | Subject | Re: [PATCH v1] Revert "cpufreq: schedutil: Move max CPU capacity to sugov_policy" |
| |
On Wed, 30 Nov 2022 at 15:04, Lukasz Luba <lukasz.luba@arm.com> wrote: > > Hi Vincent, > > On 11/30/22 10:42, Vincent Guittot wrote: > > Hi All > > > > Just for the log and because it took me a while to figure out the root > > cause of the problem: This patch also creates a regression for > > snapdragon845 based systems and probably on any QC chipsets that use a > > LUT to update the OPP table at boot. The behavior is the same as > > described by Sam with a staled value in sugov_policy.max field. > > Thanks for sharing this info and apologies that you spent cycles > on it. > > I have checked that whole setup code (capacity + cpufreq policy and > governor). It looks like to have a proper capacity of CPUs, we need > to wait till the last policy is created. It's due to the arch_topology.c > mechanism which is only triggered after all CPUs' got the policy. > Unfortunately, this leads to a chicken & egg situation for this > schedutil setup of max capacity. > > I have experimented with this code, which triggers an update in > the schedutil, when all CPUs got the policy and sugov gov > (with trace_printk() to mach the output below)
Your proposal below looks similar to what is done in arch_topology.c. arch_topology.c triggers a rebuild of sched_domain and removes its cpufreq notifier cb once it has visited all CPUs, could it also trigger an update of CPU's policy with cpufreq_update_policy() ?
At this point you will be sure that the normalization has happened and the max capacity will not change.
I don't know if it's a global problem or only for systems using arch_topology
> > -------------------------8<----------------------------------------- > diff --git a/kernel/sched/cpufreq_schedutil.c > b/kernel/sched/cpufreq_schedutil.c > index 9161d1136d01..f1913a857218 100644 > --- a/kernel/sched/cpufreq_schedutil.c > +++ b/kernel/sched/cpufreq_schedutil.c > @@ -59,6 +59,7 @@ struct sugov_cpu { > }; > > static DEFINE_PER_CPU(struct sugov_cpu, sugov_cpu); > +static cpumask_var_t cpus_to_visit; > > /************************ Governor internals ***********************/ > > @@ -783,6 +784,22 @@ static int sugov_start(struct cpufreq_policy *policy) > > cpufreq_add_update_util_hook(cpu, &sg_cpu->update_util, > uu); > } > + > + cpumask_andnot(cpus_to_visit, cpus_to_visit, policy->related_cpus); > + > + if (cpumask_empty(cpus_to_visit)) { > + trace_printk("schedutil the visit cpu mask is empty now\n"); > + for_each_possible_cpu(cpu) { > + struct sugov_cpu *sg_cpu = &per_cpu(sugov_cpu, cpu); > + struct sugov_policy *sg_policy = sg_cpu->sg_policy; > + > + sg_policy->max = arch_scale_cpu_capacity(cpu); > + > + trace_printk("SCHEDUTIL: NEW CPU%u > cpu_capacity=%lu\n", > + cpu, sg_policy->max); > + } > + } > + > return 0; > } > > @@ -800,6 +817,8 @@ static void sugov_stop(struct cpufreq_policy *policy) > irq_work_sync(&sg_policy->irq_work); > kthread_cancel_work_sync(&sg_policy->work); > } > + > + cpumask_or(cpus_to_visit, cpus_to_visit, policy->related_cpus); > } > > static void sugov_limits(struct cpufreq_policy *policy) > @@ -829,6 +848,11 @@ struct cpufreq_governor schedutil_gov = { > #ifdef CONFIG_CPU_FREQ_DEFAULT_GOV_SCHEDUTIL > struct cpufreq_governor *cpufreq_default_governor(void) > { > + if (!alloc_cpumask_var(&cpus_to_visit, GFP_KERNEL)) > + return NULL; > + > + cpumask_copy(cpus_to_visit, cpu_possible_mask); > + > return &schedutil_gov; > } > #endif > > ---------------------------------->8--------------------------------- > > > That simple approach fixes the issue. I have also tested it with > governor change a few times and setting back the schedutil. > > ------------------------------------------- > kworker/u12:1-48 [004] ..... 2.208847: sugov_start: > schedutil the visit cpu mask is empty now > kworker/u12:1-48 [004] ..... 2.208854: sugov_start: > SCHEDUTIL: NEW CPU0 cpu_capacity=381 > kworker/u12:1-48 [004] ..... 2.208857: sugov_start: > SCHEDUTIL: NEW CPU1 cpu_capacity=381 > kworker/u12:1-48 [004] ..... 2.208860: sugov_start: > SCHEDUTIL: NEW CPU2 cpu_capacity=381 > kworker/u12:1-48 [004] ..... 2.208862: sugov_start: > SCHEDUTIL: NEW CPU3 cpu_capacity=381 > kworker/u12:1-48 [004] ..... 2.208864: sugov_start: > SCHEDUTIL: NEW CPU4 cpu_capacity=1024 > kworker/u12:1-48 [004] ..... 2.208866: sugov_start: > SCHEDUTIL: NEW CPU5 cpu_capacity=1024 > bash-615 [005] ..... 35.317113: sugov_start: > schedutil the visit cpu mask is empty now > bash-615 [005] ..... 35.317120: sugov_start: > SCHEDUTIL: NEW CPU0 cpu_capacity=381 > bash-615 [005] ..... 35.317123: sugov_start: > SCHEDUTIL: NEW CPU1 cpu_capacity=381 > bash-615 [005] ..... 35.317125: sugov_start: > SCHEDUTIL: NEW CPU2 cpu_capacity=381 > bash-615 [005] ..... 35.317127: sugov_start: > SCHEDUTIL: NEW CPU3 cpu_capacity=381 > bash-615 [005] ..... 35.317129: sugov_start: > SCHEDUTIL: NEW CPU4 cpu_capacity=1024 > bash-615 [005] ..... 35.317131: sugov_start: > SCHEDUTIL: NEW CPU5 cpu_capacity=1024 > bash-623 [003] ..... 57.633328: sugov_start: > schedutil the visit cpu mask is empty now > bash-623 [003] ..... 57.633336: sugov_start: > SCHEDUTIL: NEW CPU0 cpu_capacity=381 > bash-623 [003] ..... 57.633339: sugov_start: > SCHEDUTIL: NEW CPU1 cpu_capacity=381 > bash-623 [003] ..... 57.633340: sugov_start: > SCHEDUTIL: NEW CPU2 cpu_capacity=381 > bash-623 [003] ..... 57.633342: sugov_start: > SCHEDUTIL: NEW CPU3 cpu_capacity=381 > bash-623 [003] ..... 57.633343: sugov_start: > SCHEDUTIL: NEW CPU4 cpu_capacity=1024 > bash-623 [003] ..... 57.633344: sugov_start: > SCHEDUTIL: NEW CPU5 cpu_capacity=1024 > ---------------------------------------------------- > > It should work. > > Regards, > Lukasz
| |