Messages in this thread | ![/](/images/icornerl.gif) | | Subject | Re: [PATCH] timers/nohz: Update nohz load even if tick already stopped | From | Scott Wood <> | Date | Tue, 05 Nov 2019 01:30:58 -0600 |
| |
On Tue, 2019-11-05 at 00:43 +0100, Thomas Gleixner wrote: > On Mon, 4 Nov 2019, Thomas Gleixner wrote: > > On Fri, 1 Nov 2019, Scott Wood wrote: > > > On Wed, 2019-10-30 at 14:31 +0100, Peter Zijlstra wrote: > > > > Oh argh! that's a bit radical of the remote tick. The normal tick > > > > runs > > > > just fine on idle CPUs, so lets mirror that. > > > > > > > > How's this then? > > > > .... > > > > > Needs to be tick_nohz_tick_stopped_cpu(cpu) > > > > > > After fixing that, I get: > > > > > > [ 7.439068] WARNING: CPU: 20 PID: 7 at > > > /home/root/linux/kernel/sched/core.c:3681 > > > sched_tick_remote+0x132/0x150 > > > > So I'm going to apply Scotts patch if nobody comes up with a better idea > > until tomorrow. > > As Peter pointed out to me privately we should rather go and analyze the > real thing instead of just applying duct tape. > > /me drops the patch again.
The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.
While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().
Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?
As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.
-Scott
| ![\](/images/icornerr.gif) |