lkml.org 
[lkml]   [2021]   [Jul]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Date
SubjectRe: [PATCH v3 1/1] psi: stop relying on timer_pending for poll_work rescheduling
On Wed, Jul 7, 2021 at 6:39 AM Johannes Weiner <hannes@cmpxchg.org> wrote:
>
> This looks good to me now code wise. Just a comment on the comments:
>
> On Tue, Jul 06, 2021 at 07:39:33PM -0700, Suren Baghdasaryan wrote:
> > @@ -559,18 +560,14 @@ static u64 update_triggers(struct psi_group *group, u64 now)
> > return now + group->poll_min_period;
> > }
> >
> > -/* Schedule polling if it's not already scheduled. */
> > -static void psi_schedule_poll_work(struct psi_group *group, unsigned long delay)
> > +/* Schedule polling if it's not already scheduled or forced. */
> > +static void psi_schedule_poll_work(struct psi_group *group, unsigned long delay,
> > + bool force)
> > {
> > struct task_struct *task;
> >
> > - /*
> > - * Do not reschedule if already scheduled.
> > - * Possible race with a timer scheduled after this check but before
> > - * mod_timer below can be tolerated because group->polling_next_update
> > - * will keep updates on schedule.
> > - */
> > - if (timer_pending(&group->poll_timer))
> > + /* xchg should be called even when !force to set poll_scheduled */
> > + if (atomic_xchg(&group->poll_scheduled, 1) && !force)
> > return;
>
> This explains what the code does, but not why. It would be good to
> explain the ordering with poll_work, here or there. But both sides
> should mention each other.

How about this:

/*
* atomic_xchg should be called even when !force to always set poll_scheduled
* and to provide a memory barrier (see the comment inside psi_poll_work).
*/

>
> > @@ -595,6 +595,28 @@ static void psi_poll_work(struct psi_group *group)
> >
> > now = sched_clock();
> >
> > + if (now > group->polling_until) {
> > + /*
> > + * We are either about to start or might stop polling if no
> > + * state change was recorded. Resetting poll_scheduled leaves
> > + * a small window for psi_group_change to sneak in and schedule
> > + * an immegiate poll_work before we get to rescheduling. One
> > + * potential extra wakeup at the end of the polling window
> > + * should be negligible and polling_next_update still keeps
> > + * updates correctly on schedule.
> > + */
> > + atomic_set(&group->poll_scheduled, 0);
> > + /*
> > + * Ensure that operations of clearing group->poll_scheduled and
> > + * obtaining changed_states are not reordered.
> > + */
> > + smp_mb();
>
> Same here, it would be good to explain that this is ordering the
> scheduler with the timer such that no events are missed. Feel free to
> reuse my race diagram from the other thread - those are better at
> conveying the situation than freeform text.

I tried to make your diagram a bit less abstract by using the actual
names. How about this?

/*
* We need to enforce ordering between poll_scheduled and psi_group_cpu.times
* reads and writes in psi_poll_work and psi_group_change functions.
Otherwise we
* might fail to reschedule the timer when monitored states change:
*
* psi_poll_work:
* poll_scheduled = 0
* smp_mb()
* changed_states = collect_percpu_times()
* if changed_states && xchg(poll_scheduled, 1) == 0
* mod_timer()
*
* psi_group_change:
* record_times()
* smp_mb()
* if xchg(poll_scheduled, 1) == 0
* mod_timer()
*
* atomic_xchg in psi_schedule_poll_work implements an implicit memory
barrier but
* we need an explicit one here.
*/
If we remove smp_mb barriers then there are the following possible
reordering cases:

Case1: reordering in psi_poll_work
psi_poll_work psi_group_change
changed_states = collect_percpu_times()
record_times()
if xchg(poll_scheduled,
1) == 0 <-- false
mod_timer()
poll_scheduled = 0
if changed_states && xchg(poll_scheduled, 1) == 0 <-- changed_states is false
mod_timer()

Case2: reordering in psi_group_change
psi_poll_work psi_group_change
if xchg(poll_scheduled,
1) == 0 <-- false
mod_timer()
poll_scheduled = 0
changed_states = collect_percpu_times()
record_times()
if changed_states && xchg(poll_scheduled, 1) == 0 <-- changed_states is false
mod_timer()

In both cases mod_timer() is not called, poll update is missed. But
describing this all in the comments would be an overkill IMHO.
WDYT?

>
> Thanks

\
 
 \ /
  Last update: 2021-07-08 00:44    [W:0.059 / U:0.516 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site