Messages in this thread | | | Date | Wed, 19 Jan 2022 18:35:42 +0100 | Subject | Re: [PATCH v2 1/7] entry: add arch_in_rcu_eqs() | From | Christian Borntraeger <> |
| |
Am 19.01.22 um 11:58 schrieb Mark Rutland: > All architectures have an interruptible RCU extended quiescent state > (EQS) as part of their idle sequences, where interrupts can occur > without RCU watching. Entry code must account for this and wake RCU as > necessary; the common entry code deals with this in irqentry_enter() by > treating any interrupt from an idle thread as potentially having > occurred with an EQS and waking RCU for the duration of the interrupt > via rcu_irq_enter() .. rcu_irq_exit(). > > Some architectures may have other interruptible EQSs which require > similar treatment. For example, on s390 is it necessary to enable > interrupts around guest entry in the middle of a period where core KVM > code has entered an EQS. > > So that architectueres can wake RCU in these cases, this patch adds a ^ > new arch_in_rcu_eqs() hook to the common entry code which is checked in > addition to the existing is_idle_thread() check, with RCU woken if > either returns true. A default implementation is provided which always > returns false, which suffices for most architectures. > > As no architectures currently implement arch_in_rcu_eqs(), there should > be no functional change as a result of this patch alone. A subsequent > patch will add an s390 implementation to fix a latent bug with missing > RCU wakeups. > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > Cc: Andy Lutomirski <luto@kernel.org> > Cc: Christian Borntraeger <borntraeger@linux.ibm.com> > Cc: Heiko Carstens <hca@linux.ibm.com> > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Paul E. McKenney <paulmck@kernel.org> > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: Sven Schnelle <svens@linux.ibm.com> > Cc: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com> > --- > include/linux/entry-common.h | 16 ++++++++++++++++ > kernel/entry/common.c | 3 ++- > 2 files changed, 18 insertions(+), 1 deletion(-) > > diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h > index 2e2b8d6140ed4..f1b91a13a15a6 100644 > --- a/include/linux/entry-common.h > +++ b/include/linux/entry-common.h > @@ -99,6 +99,22 @@ static inline __must_check int arch_syscall_enter_tracehook(struct pt_regs *regs > } > #endif > > +/** > + * arch_in_rcu_eqs - Architecture specific check for RCU extended quiescent > + * states. > + * > + * Returns: true if the CPU is potentially in an RCU EQS, false otherwise. > + * > + * Architectures only need to define this if threads other than the idle thread > + * may have an interruptible EQS. This does not need to handle idle threads. It > + * is safe to over-estimate at the cost of redundant RCU management work. > + * > + * Invoked from irqentry_enter() > + */ > +#ifndef arch_in_rcu_eqs > +static __always_inline bool arch_in_rcu_eqs(void) { return false; } > +#endif > + > /** > * enter_from_user_mode - Establish state when coming from user mode > * > diff --git a/kernel/entry/common.c b/kernel/entry/common.c > index d5a61d565ad5d..b13d4e0b0b643 100644 > --- a/kernel/entry/common.c > +++ b/kernel/entry/common.c > @@ -349,7 +349,8 @@ noinstr irqentry_state_t irqentry_enter(struct pt_regs *regs) > * TINY_RCU does not support EQS, so let the compiler eliminate > * this part when enabled. > */ > - if (!IS_ENABLED(CONFIG_TINY_RCU) && is_idle_task(current)) { > + if (!IS_ENABLED(CONFIG_TINY_RCU) && > + (is_idle_task(current) || arch_in_rcu_eqs())) { > /* > * If RCU is not watching then the same careful > * sequence vs. lockdep and tracing is required
| |