Messages in this thread Patch in this message | | | From | Nadav Amit <> | Subject | [PATCH] x86/mm/tlb: ignore f->new_tlb_gen when zero | Date | Thu, 7 Jul 2022 17:30:53 -0700 |
| |
From: Nadav Amit <namit@vmware.com>
Commit aa44284960d5 ("x86/mm/tlb: Avoid reading mm_tlb_gen when possible") introduced an optimization of skipping the flush if the TLB generation that is flushed (as provided in flush_tlb_info) was already flushed.
However, arch_tlbbatch_flush() does not provide any generation in flush_tlb_info. As a result, try_to_unmap_one() would not perform any TLB flushes.
Fix it by checking whether f->new_tlb_gen is nonzero. Zero value is anyhow is an invalid generation value.
In addition, add the missing unlikely() and jump to get tracing right.
Fixes: aa44284960d5 ("x86/mm/tlb: Avoid reading mm_tlb_gen when possible") Reported-by: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Andy Lutomirski <luto@kernel.org> Signed-off-by: Nadav Amit <namit@vmware.com> --- arch/x86/mm/tlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index d9314cc8b81f..d81b4084bb8a 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -771,14 +771,14 @@ static void flush_tlb_func(void *info) return; } - if (f->new_tlb_gen <= local_tlb_gen) { + if (unlikely(f->new_tlb_gen != 0 && f->new_tlb_gen <= local_tlb_gen)) { /* * The TLB is already up to date in respect to f->new_tlb_gen. * While the core might be still behind mm_tlb_gen, checking * mm_tlb_gen unnecessarily would have negative caching effects * so avoid it. */ - return; + goto done; } /* -- 2.25.1
| |