| Date | Thu, 25 Oct 2012 14:16:22 +0200 | From | Peter Zijlstra <> | Subject | [PATCH 05/31] x86/mm: Reduce tlb flushes from ptep_set_access_flags() |
| |
From: Rik van Riel <riel@redhat.com>
If ptep_set_access_flags() is invoked to upgrade access permissions on a PTE, there is no security or data integrity reason to do a remote TLB flush.
Lazily letting another CPU incur a spurious page fault occasionally is (much!) cheaper than aggressively flushing everybody else's TLB.
Signed-off-by: Rik van Riel <riel@redhat.com> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Signed-off-by: Ingo Molnar <mingo@kernel.org> --- arch/x86/mm/pgtable.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-)
Index: tip/arch/x86/mm/pgtable.c =================================================================== --- tip.orig/arch/x86/mm/pgtable.c +++ tip/arch/x86/mm/pgtable.c @@ -306,11 +306,26 @@ int ptep_set_access_flags(struct vm_area pte_t entry, int dirty) { int changed = !pte_same(*ptep, entry); + /* + * If the page used to be inaccessible (_PAGE_PROTNONE), or + * this call upgrades the access permissions on the same page, + * it is safe to skip the remote TLB flush. + */ + bool flush_remote = false; + if (!pte_accessible(*ptep)) + flush_remote = false; + else if (pte_pfn(*ptep) != pte_pfn(entry) || + (pte_write(*ptep) && !pte_write(entry)) || + (pte_exec(*ptep) && !pte_exec(entry))) + flush_remote = true; if (changed && dirty) { *ptep = entry; pte_update_defer(vma->vm_mm, address, ptep); - flush_tlb_page(vma, address); + if (flush_remote) + flush_tlb_page(vma, address); + else + __flush_tlb_one(address); } return changed;
|