Messages in this thread | | | Date | Fri, 27 Jul 2018 14:33:32 +0100 | From | Will Deacon <> | Subject | Re: [PATCH v3] locking/rwsem: Exit read lock slowpath if queue empty & no writer |
| |
On Tue, Jul 24, 2018 at 03:10:25PM -0400, Waiman Long wrote: > It was discovered that a constant stream of readers with occassional > writers pounding on a rwsem may cause many of the readers to enter the > slowpath unnecessarily thus increasing latency and lowering performance. > > In the current code, a reader entering the slowpath critical section > will unconditionally set the WAITING_BIAS, if not set yet, and clear > its active count even if no one is in the wait queue and no writer > is present. This causes some incoming readers to observe the presence > of waiters in the wait queue and hence have to go into the slowpath > themselves. > > With sufficient numbers of readers and a relatively short lock hold time, > the WAITING_BIAS may be repeatedly turned on and off and a substantial > portion of the readers will go into the slowpath sustaining a rather > long queue in the wait queue spinlock and repeated WAITING_BIAS on/off > cycle until the logjam is broken opportunistically. > > To avoid this situation from happening, an additional check is added to > detect the special case that the reader in the critical section is the > only one in the wait queue and no writer is present. When that happens, > it can just exit the slowpath and return immediately as its active count > has already been set in the lock. Other incoming readers won't observe > the presence of waiters and so will not be forced into the slowpath. > > The issue was found in a customer site where they had an application > that pounded on the pread64 syscalls heavily on an XFS filesystem. The > application was run in a recent 4-socket boxes with a lot of CPUs. They > saw significant spinlock contention in the rwsem_down_read_failed() call. > With this patch applied, the system CPU usage went down from 85% to 57%, > and the spinlock contention in the pread64 syscalls was gone. > > v3: Revise the commit log and comment again. > v2: Add customer testing results and remove wording that may cause > confusion. > > Signed-off-by: Waiman Long <longman@redhat.com> > --- > kernel/locking/rwsem-xadd.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-)
My nits with the commit message have been addressed, so:
Acked-by: Will Deacon <will.deacon@arm.com>
Thanks!
Will
| |