lkml.org 
[lkml]   [2020]   [Oct]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH 14/16] rcu: Locally accelerate callbacks as long as offloading isn't complete
    Date
    The local callbacks processing checks if some callbacks need
    acceleration. Keep that behaviour under nocb lock protection when
    rcu_core() executes concurrently with GP/CB kthreads.

    Inspired-by: Paul E. McKenney <paulmck@kernel.org>
    Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
    Cc: Paul E. McKenney <paulmck@kernel.org>
    Cc: Josh Triplett <josh@joshtriplett.org>
    Cc: Steven Rostedt <rostedt@goodmis.org>
    Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
    Cc: Lai Jiangshan <jiangshanlai@gmail.com>
    Cc: Joel Fernandes <joel@joelfernandes.org>
    Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
    ---
    kernel/rcu/tree.c | 7 +++----
    1 file changed, 3 insertions(+), 4 deletions(-)

    diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
    index 45fad6977bea..4af5fed11885 100644
    --- a/kernel/rcu/tree.c
    +++ b/kernel/rcu/tree.c
    @@ -2673,7 +2673,6 @@ static __latent_entropy void rcu_core(void)
    unsigned long flags;
    struct rcu_data *rdp = raw_cpu_ptr(&rcu_data);
    struct rcu_node *rnp = rdp->mynode;
    - const bool offloaded = rcu_segcblist_is_offloaded(&rdp->cblist);
    const bool do_batch = !rcu_segcblist_completely_offloaded(&rdp->cblist);

    if (cpu_is_offline(smp_processor_id()))
    @@ -2694,11 +2693,11 @@ static __latent_entropy void rcu_core(void)

    /* No grace period and unregistered callbacks? */
    if (!rcu_gp_in_progress() &&
    - rcu_segcblist_is_enabled(&rdp->cblist) && !offloaded) {
    - local_irq_save(flags);
    + rcu_segcblist_is_enabled(&rdp->cblist) && do_batch) {
    + rcu_nocb_lock_irqsave(rdp, flags);
    if (!rcu_segcblist_restempty(&rdp->cblist, RCU_NEXT_READY_TAIL))
    rcu_accelerate_cbs_unlocked(rnp, rdp);
    - local_irq_restore(flags);
    + rcu_nocb_unlock_irqrestore(rdp, flags);
    }

    rcu_check_gp_start_stall(rnp, rdp, rcu_jiffies_till_stall_check());
    --
    2.25.1
    \
     
     \ /
      Last update: 2020-10-23 16:48    [W:3.227 / U:0.196 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site