Messages in this thread | | | Subject | Re: [PATCH v9 3/7] srcu: Fix invoke_rcu_callbacks() segcb length adjustment | From | Neeraj Upadhyay <> | Date | Wed, 4 Nov 2020 19:07:53 +0530 |
| |
On 11/3/2020 7:55 PM, Joel Fernandes (Google) wrote: > With earlier patches, the negative counting of the unsegmented list > cannot be used to adjust the segmented one. To fix this, sample the > unsegmented length in advance, and use it after CB execution to adjust > the segmented list's length. > > Reviewed-by: Frederic Weisbecker <frederic@kernel.org> > Suggested-by: Frederic Weisbecker <frederic@kernel.org> > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org> > ---
Reviewed-by: Neeraj Upadhyay <neeraju@codeaurora.org>
> kernel/rcu/srcutree.c | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c > index 0f23d20d485a..79b7081143a7 100644 > --- a/kernel/rcu/srcutree.c > +++ b/kernel/rcu/srcutree.c > @@ -1160,6 +1160,7 @@ static void srcu_advance_state(struct srcu_struct *ssp) > */ > static void srcu_invoke_callbacks(struct work_struct *work) > { > + long len; > bool more; > struct rcu_cblist ready_cbs; > struct rcu_head *rhp; > @@ -1182,6 +1183,7 @@ static void srcu_invoke_callbacks(struct work_struct *work) > /* We are on the job! Extract and invoke ready callbacks. */ > sdp->srcu_cblist_invoking = true; > rcu_segcblist_extract_done_cbs(&sdp->srcu_cblist, &ready_cbs); > + len = ready_cbs.len; > spin_unlock_irq_rcu_node(sdp); > rhp = rcu_cblist_dequeue(&ready_cbs); > for (; rhp != NULL; rhp = rcu_cblist_dequeue(&ready_cbs)) { > @@ -1190,13 +1192,14 @@ static void srcu_invoke_callbacks(struct work_struct *work) > rhp->func(rhp); > local_bh_enable(); > } > + WARN_ON_ONCE(ready_cbs.len); > > /* > * Update counts, accelerate new callbacks, and if needed, > * schedule another round of callback invocation. > */ > spin_lock_irq_rcu_node(sdp); > - rcu_segcblist_insert_count(&sdp->srcu_cblist, &ready_cbs); > + rcu_segcblist_add_len(&sdp->srcu_cblist, -len); > (void)rcu_segcblist_accelerate(&sdp->srcu_cblist, > rcu_seq_snap(&ssp->srcu_gp_seq)); > sdp->srcu_cblist_invoking = false; >
-- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
| |