Messages in this thread | | | From | Lai Jiangshan <> | Subject | [PATCH 00/10] rcu/x86: Use per-cpu rcu preempt count | Date | Thu, 28 Mar 2024 15:53:08 +0800 |
| |
From: Lai Jiangshan <jiangshan.ljs@antgroup.com>
X86 can access percpu data in a single instruction.
Use per-cpu rcu preempt count and make it able to be inlined.
patch 1-7: prepare patch 8-10: implement PCPU_RCU_PREEMPT_COUNT
Cc: "Paul E. McKenney" <paulmck@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Frederic Weisbecker <frederic@kernel.org>
Lai Jiangshan (10): lib: Use rcu_preempt_depth() to replace current->rcu_read_lock_nesting rcu: Move rcu_preempt_depth_set() to rcupdate.h rcu: Reorder tree_exp.h after tree_plugin.h rcu: Add macros set_rcu_preempt_special() and clear_rcu_preempt_special() rcu: Make rcu_read_unlock_special() global rcu: Rename marco __LINUX_RCU_H to __KERNEL_RCU_H sched/core: Add rcu_preempt_switch() rcu: Implement PCPU_RCU_PREEMPT_COUNT framework x86/rcu: Add rcu_preempt_count x86/rcu: Add THUNK rcu_read_unlock_special_thunk
arch/x86/Kconfig | 1 + arch/x86/entry/thunk_64.S | 5 ++ arch/x86/include/asm/current.h | 3 + arch/x86/include/asm/rcu_preempt.h | 109 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/common.c | 7 +- include/linux/rcupdate.h | 36 ++++++++++ kernel/rcu/Kconfig | 8 +++ kernel/rcu/rcu.h | 15 +++- kernel/rcu/tree.c | 2 +- kernel/rcu/tree_exp.h | 2 +- kernel/rcu/tree_plugin.h | 41 +++++++---- kernel/sched/core.c | 2 + lib/locking-selftest.c | 6 +- 13 files changed, 211 insertions(+), 26 deletions(-) create mode 100644 arch/x86/include/asm/rcu_preempt.h
-- 2.19.1.6.gb485710b
| |