lkml.org 
[lkml]   [2022]   [Sep]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH rcu 03/10] rcu: Add QS check in rcu_exp_handler() for non-preemptible kernels
    On Wed, Aug 31, 2022 at 11:07:58AM -0700, Paul E. McKenney wrote:
    > From: Zqiang <qiang1.zhang@intel.com>
    >
    > Kernels built with CONFIG_PREEMPTION=n and CONFIG_PREEMPT_COUNT=y maintain
    > preempt_count() state. Because such kernels map __rcu_read_lock()
    > and __rcu_read_unlock() to preempt_disable() and preempt_enable(),
    > respectively, this allows the expedited grace period's !CONFIG_PREEMPT_RCU
    > version of the rcu_exp_handler() IPI handler function to use
    > preempt_count() to detect quiescent states.
    >
    > This preempt_count() usage might seem to risk failures due to
    > use of implicit RCU readers in portions of the kernel under #ifndef
    > CONFIG_PREEMPTION, except that rcu_core() already disallows such implicit
    > RCU readers. The moral of this story is that you must use explicit
    > read-side markings such as rcu_read_lock() or preempt_disable() even if
    > the code knows that this kernel does not support preemption.
    >
    > This commit therefore adds a preempt_count()-based check for a quiescent
    > state in the !CONFIG_PREEMPT_RCU version of the rcu_exp_handler()
    > function for kernels built with CONFIG_PREEMPT_COUNT=y, reporting an
    > immediate quiescent state when the interrupted code had both preemption
    > and softirqs enabled.
    >
    > This change results in about a 2% reduction in expedited grace-period
    > latency in kernels built with both CONFIG_PREEMPT_RCU=n and
    > CONFIG_PREEMPT_COUNT=y.
    >
    > Signed-off-by: Zqiang <qiang1.zhang@intel.com>
    > Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
    > Link: https://lore.kernel.org/all/20220622103549.2840087-1-qiang1.zhang@intel.com/
    > ---
    > kernel/rcu/tree_exp.h | 4 +++-
    > 1 file changed, 3 insertions(+), 1 deletion(-)
    >
    > diff --git a/kernel/rcu/tree_exp.h b/kernel/rcu/tree_exp.h
    > index be667583a5547..b07998159d1fa 100644
    > --- a/kernel/rcu/tree_exp.h
    > +++ b/kernel/rcu/tree_exp.h
    > @@ -828,11 +828,13 @@ static void rcu_exp_handler(void *unused)
    > {
    > struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
    > struct rcu_node *rnp = rdp->mynode;
    > + bool preempt_bh_enabled = !(preempt_count() & (PREEMPT_MASK |
    > SOFTIRQ_MASK));

    I don't know if nested hardirqs still exist. I only heard old rumours
    about broken drivers. Should we take care of them?

    Also are we sure that all callers of flush_smp_call_function_queue()
    are QS?

    Let's see we know that rcu_exp_handler() can either be executed from:

    * hardirqs

    Or from process context, expected to be RCU QS states at least in idle
    as the comment above flush_smp_call_function_queue() in idle says
    (but I'd rather check all the in-process callers before stating all
    of them are in QS)

    * idle (in which case preemption is disabled unfortunately so the current
    test won't help)
    * stop_machine
    _ When CPU is dead and out of RCU (rcutree_dead_cpu() called)
    so that should be a QS.
    _ When CPU is migrating (is it a QS then?)

    If we check further that all non-IRQ callers of flush_smp_call_function_queue()
    are always quiescent states then we could deduce that !in_hardirq() means we are in
    a quiescent state, whether preemption is disabled or not.

    In any case for the current patch, perhaps a more robust test against nested
    hardirqs would be:

    unsigned long cnt = preempt_count();
    bool preempt_bh_enabled = (!cnt || cnt == HARDIRQ_OFFSET)

    Thanks.

    >
    > if (!(READ_ONCE(rnp->expmask) & rdp->grpmask) ||
    > __this_cpu_read(rcu_data.cpu_no_qs.b.exp))
    > return;
    > - if (rcu_is_cpu_rrupt_from_idle()) {
    > + if (rcu_is_cpu_rrupt_from_idle() ||
    > + (IS_ENABLED(CONFIG_PREEMPT_COUNT) && preempt_bh_enabled)) {
    > rcu_report_exp_rdp(this_cpu_ptr(&rcu_data));
    > return;
    > }
    > --
    > 2.31.1.189.g2e36527f23
    >

    \
     
     \ /
      Last update: 2022-09-07 14:10    [W:2.526 / U:0.040 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site