Messages in this thread Patch in this message | | | From | "Paul E. McKenney" <> | Subject | [PATCH tip/core/rcu 08/40] rcu: Make call_rcu() do deferred NOCB wakeups | Date | Wed, 12 Apr 2017 10:39:53 -0700 |
| |
If a CONFIG_RCU_NOCB_CPUS kernel invokes call_rcu() with interrupts disabled, wakeups must be deferred in order to avoid self-deadlock in the cases where the disabled interrupts are due to scheduler locks being held. In this case, a flag is set and is checked on entry to extended quiescent states (usermode, idle), on exit from the RCU_SOFTIRQ handler, and when the CPU in question goes offline. However, a given CPU could avoid all of those states for a considerable length of time.
This commit therefore allows a subsequent invocation of call_rcu() with interrupts enabled to do the wakeup. It also makes the wakeup function clear the deferred-wakeup flag.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> --- kernel/rcu/tree_plugin.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 0a62a8f1caac..5a149091d262 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1768,6 +1768,7 @@ static void wake_nocb_leader(struct rcu_data *rdp, bool force) return; if (READ_ONCE(rdp_leader->nocb_leader_sleep) || force) { /* Prior smp_mb__after_atomic() orders against prior enqueue. */ + WRITE_ONCE(rdp->nocb_defer_wakeup, RCU_NOGP_WAKE_NOT); WRITE_ONCE(rdp_leader->nocb_leader_sleep, false); swake_up(&rdp_leader->nocb_wq); } @@ -1853,7 +1854,9 @@ static void __call_rcu_nocb_enqueue(struct rcu_data *rdp, return; } len = atomic_long_read(&rdp->nocb_q_count); - if (old_rhpp == &rdp->nocb_head) { + if (old_rhpp == &rdp->nocb_head || + (rcu_nocb_need_deferred_wakeup(rdp) && + !irqs_disabled_flags(flags))) { if (!irqs_disabled_flags(flags)) { /* ... if queue was empty ... */ wake_nocb_leader(rdp, false); -- 2.5.2
| |