| From | Michel Lespinasse <> | Subject | [PATCH v2 14/35] mm: add speculative_page_walk_begin() and speculative_page_walk_end() | Date | Fri, 28 Jan 2022 05:09:45 -0800 |
| |
Speculative page faults will use these to protect against races with page table reclamation.
This could always be handled by disabling local IRQs as the fast GUP code does; however speculative page faults do not need to protect against races with THP page splitting, so a weaker rcu read lock is sufficient in the MMU_GATHER_RCU_TABLE_FREE case.
Signed-off-by: Michel Lespinasse <michel@lespinasse.org> --- mm/memory.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
diff --git a/mm/memory.c b/mm/memory.c index aa24cd8c06e9..663952d14bad 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2725,6 +2725,28 @@ int apply_to_existing_page_range(struct mm_struct *mm, unsigned long addr, } EXPORT_SYMBOL_GPL(apply_to_existing_page_range); +#ifdef CONFIG_SPECULATIVE_PAGE_FAULT + +/* + * speculative_page_walk_begin() ... speculative_page_walk_end() protects + * against races with page table reclamation. + * + * This is similar to what fast GUP does, but fast GUP also needs to + * protect against races with THP page splitting, so it always needs + * to disable interrupts. + * Speculative page faults only need to protect against page table reclamation, + * so rcu_read_lock() is sufficient in the MMU_GATHER_RCU_TABLE_FREE case. + */ +#ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE +#define speculative_page_walk_begin() rcu_read_lock() +#define speculative_page_walk_end() rcu_read_unlock() +#else +#define speculative_page_walk_begin() local_irq_disable() +#define speculative_page_walk_end() local_irq_enable() +#endif + +#endif /* CONFIG_SPECULATIVE_PAGE_FAULT */ + /* * handle_pte_fault chooses page fault handler according to an entry which was * read non-atomically. Before making any commitment, on those architectures -- 2.20.1
|