Messages in this thread | | | Date | Thu, 10 Jan 2019 06:19:38 -0800 | From | "Paul E. McKenney" <> | Subject | Re: Possible use of RCU while in extended QS: idle vs RCU read-side in interrupt vs rcu_eqs_exit |
| |
On Thu, Jan 10, 2019 at 01:30:05AM -0500, Mathieu Desnoyers wrote: > ----- On Jan 9, 2019, at 8:13 PM, paulmck paulmck@linux.ibm.com wrote: > > > On Wed, Jan 09, 2019 at 08:38:51PM -0500, Mathieu Desnoyers wrote: > >> Hi Paul, > >> > >> I've had a user report that trace_sched_waking() appears to be > >> invoked while !rcu_is_watching() in some situation, so I started > >> digging into the scheduler idle code. > >> > >> It appears that interrupts are re-enabled before rcu_eqs_exit() is > >> invoked when exiting idle code from the scheduler. > >> > >> I wonder what happens if an interrupt handler (including scheduler code) > >> happens to issue a RCU read-side critical section before rcu_eqs_exit() > >> is called ? Is there some code on interrupt entry that ensures rcu eqs > >> state is exited in such scenario ? > > > > Interrupt handlers are supposed to invoke irq_enter(), which will in > > turn invoke rcu_irq_enter(), which should take care of things. > > > > However, there are cases where a given architecture knows that a given > > interrupt handler does not contain RCU readers, and in this case, the > > architecture might omit the rcu_irq_enter() or maybe even the whole > > irq_enter(). And then it is all fun and games until someone adds an > > RCU read-side critical section. ;-) > > Even if an irq handler does not contain any RCU read-side critical > section, won't it end by possibly invoking the scheduler before > returning ? Considering that the scheduler has tracepoints which > use RCU, this might be related to the issue that has been brought > to my attention.
Most interrupt handlers just return, but yes, scheduler state is often checked during return from interrupt. But in that case, the interrupt handler needs to have invoked irq_enter().
> Do you have examples of such interrupt handlers which do not invoke > rcu_irq_enter() ?
Mostly examples of lightweight interrupts handlers that used to not invoke irq_enter() and thus not rcu_irq_enter(), but which later started using RCU readers. Which means that they are no longer examples that do not invoke rcu_irq_enter(). ;-)
Some of them just invoked rcu_irq_enter(), others had to do the full irq_enter() call (which in turn invokes rcu_irq_enter()).
These interrupt handlers were very light-weight. Page-table walkers, hardware events, and the like. Take an interrupt, look at a hardware register, update a data structure, maybe write to a hardware register, return from interrupt.
If there is only one such tracepoint, one approach is to use _rcuidle, that is, instead of trace_blarvitz(), trace_blarvitz_rcuidle(). This can add overhead, so this might not be appropriate for any of the scheduler's fastpaths. Which brings me back to the interrupt handler invoking either irq_enter() or rcu_irq_enter(). Or moving the tracepoints to a nearby region of code that RCU is already watching.
So, is it reasonably to add the rcu_irq_enter()? If you do change this, please test with CONFIG_RCU_EQS_DEBUG=y.
Thanx, Paul
| |