lkml.org 
[lkml]   [2022]   [Oct]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[PATCH v3 5/5] locking/rwsem: Update handoff lock events tracking
Date
With the new direct rwsem lock handoff, the corresponding handoff lock
events are updated to also track the number of secondary lock handoffs
in rwsem_down_read_slowpath() to see how prevalent those handoff
events are. The number of primary lock handoffs in the unlock paths is
(rwsem_handoff_read + rwsem_handoff_write - rwsem_handoff_rslow).

After running a 96-thread rwsem microbenchmark with equal number
of readers and writers on a 2-socket 96-thread system for 40s, the
following handoff stats were obtained:

rwsem_handoff_read=189
rwsem_handoff_rslow=1
rwsem_handoff_write=6678
rwsem_handoff_wspin=6681

The number of primary handoffs was 6866, whereas there was only one
secondary handoff for this test run.

Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/locking/lock_events_list.h | 6 ++++--
kernel/locking/rwsem.c | 9 +++++----
2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/kernel/locking/lock_events_list.h b/kernel/locking/lock_events_list.h
index 97fb6f3f840a..04d101767c2c 100644
--- a/kernel/locking/lock_events_list.h
+++ b/kernel/locking/lock_events_list.h
@@ -63,7 +63,9 @@ LOCK_EVENT(rwsem_rlock) /* # of read locks acquired */
LOCK_EVENT(rwsem_rlock_steal) /* # of read locks by lock stealing */
LOCK_EVENT(rwsem_rlock_fast) /* # of fast read locks acquired */
LOCK_EVENT(rwsem_rlock_fail) /* # of failed read lock acquisitions */
-LOCK_EVENT(rwsem_rlock_handoff) /* # of read lock handoffs */
LOCK_EVENT(rwsem_wlock) /* # of write locks acquired */
LOCK_EVENT(rwsem_wlock_fail) /* # of failed write lock acquisitions */
-LOCK_EVENT(rwsem_wlock_handoff) /* # of write lock handoffs */
+LOCK_EVENT(rwsem_handoff_read) /* # of read lock handoffs */
+LOCK_EVENT(rwsem_handoff_write) /* # of write lock handoffs */
+LOCK_EVENT(rwsem_handoff_rslow) /* # of handoffs in read slowpath */
+LOCK_EVENT(rwsem_handoff_wspin) /* # of handoff spins in write slowpath */
diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c
index 287606aee0e6..46aea1994bf8 100644
--- a/kernel/locking/rwsem.c
+++ b/kernel/locking/rwsem.c
@@ -472,10 +472,8 @@ static void rwsem_mark_wake(struct rw_semaphore *sem,
* force the issue.
*/
if (time_after(jiffies, waiter->timeout)) {
- if (!(oldcount & RWSEM_FLAG_HANDOFF)) {
+ if (!(oldcount & RWSEM_FLAG_HANDOFF))
adjustment -= RWSEM_FLAG_HANDOFF;
- lockevent_inc(rwsem_rlock_handoff);
- }
WRITE_ONCE(waiter->handoff_state, HANDOFF_REQUESTED);
}

@@ -680,7 +678,6 @@ static inline bool rwsem_try_write_lock(struct rw_semaphore *sem,
*/
if (new & RWSEM_FLAG_HANDOFF) {
WRITE_ONCE(first->handoff_state, HANDOFF_REQUESTED);
- lockevent_inc(rwsem_wlock_handoff);
return false;
}

@@ -1019,10 +1016,12 @@ static void rwsem_handoff(struct rw_semaphore *sem, long adj,
wake_type = RWSEM_WAKE_ANY;
adj += RWSEM_WRITER_LOCKED;
atomic_long_set(&sem->owner, (long)waiter->task);
+ lockevent_inc(rwsem_handoff_write);
} else {
wake_type = RWSEM_WAKE_READ_OWNED;
adj += RWSEM_READER_BIAS;
__rwsem_set_reader_owned(sem, waiter->task);
+ lockevent_inc(rwsem_handoff_read);
}
atomic_long_add(adj, &sem->count);
rwsem_mark_wake(sem, wake_type, wake_q);
@@ -1128,6 +1127,7 @@ rwsem_down_read_slowpath(struct rw_semaphore *sem, long count, unsigned int stat
if (rwsem_first_waiter(sem)->type == RWSEM_WAITING_FOR_READ)
adjustment = 0;
rwsem_handoff(sem, adjustment, &wake_q);
+ lockevent_inc(rwsem_handoff_rslow);

if (!adjustment) {
raw_spin_unlock_irq(&sem->wait_lock);
@@ -1257,6 +1257,7 @@ rwsem_down_write_slowpath(struct rw_semaphore *sem, int state)
preempt_disable();
rwsem_spin_on_owner(sem);
preempt_enable();
+ lockevent_inc(rwsem_handoff_wspin);
}
if (READ_ONCE(waiter.handoff_state) == HANDOFF_GRANTED)
goto skip_sleep;
--
2.31.1
\
 
 \ /
  Last update: 2022-10-17 23:25    [W:0.172 / U:0.160 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site