Messages in this thread | | | Date | Tue, 5 Dec 2017 23:05:57 +0100 | From | Peter Zijlstra <> | Subject | Re: [patch 53/60] x86/mm: Use/Fix PCID to optimize user/kernel switches |
| |
On Tue, Dec 05, 2017 at 01:46:36PM -0800, Andy Lutomirski wrote: > On Mon, Dec 4, 2017 at 6:07 AM, Thomas Gleixner <tglx@linutronix.de> wrote: > > We can use PCID to retain the TLBs across CR3 switches; including > > those now part of the user/kernel switch. This increases performance > > of kernel entry/exit at the cost of more expensive/complicated TLB > > flushing. > > > > Now that we have two address spaces, one for kernel and one for user > > space, we need two PCIDs per mm. We use the top PCID bit to indicate a > > user PCID (just like we use the PFN LSB for the PGD). Since we do TLB > > invalidation from kernel space, the existing code will only invalidate > > the kernel PCID, we augment that by marking the corresponding user > > PCID invalid, and upon switching back to userspace, use a flushing CR3 > > write for the switch. > > > > In order to access the user_pcid_flush_mask we use PER_CPU storage, > > which means the previously established SWAPGS vs CR3 ordering is now > > mandatory and required. > > > > Having to do this memory access does require additional registers, > > most sites have a functioning stack and we can spill one (RAX), sites > > without functional stack need to otherwise provide the second scratch > > register. > > > > Note: PCID is generally available on Intel Sandybridge and later CPUs. > > Note: Up until this point TLB flushing was broken in this series. > > I haven't checked that hard which patch introduces this bug, but it > seems that, with this applied, nothing propagates > non-mm-switch-related flushes to usermode. Shouldn't > flush_tlb_func_common() contain a call to invalidate_user_asid() near > the bottom? Alternatively, it could be in local_flush_tlb() and > __flush_tlb_single() (or whatever the hell the flush-one-usermode-TLB > function ends up being called).
__native_flush_tlb_single() has the invalidate_user_asid() __native_flush_tlb() has the invalidate_user_asid().
Which should be exactly that last option you mention.
> Also, on a somewhat related note, __flush_tlb_single() is called from > both flush_tlb_func_common() and do_kernel_range_flush. That sounds > wrong.
Fixed that in the patches I send out earlier today.
| |