Messages in this thread Patch in this message | | | Subject | [PATCH v5 4/6] rwsem/wake: check lock before do atomic update | From | Tim Chen <> | Date | Tue, 24 Sep 2013 15:22:40 -0700 |
| |
Atomic update lock and roll back will cause cache bouncing in large machine. A lock status pre-read can relieve this problem
Suggested-by: Davidlohr bueso <davidlohr.bueso@hp.com> Suggested-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Alex Shi <alex.shi@intel.com> --- lib/rwsem.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)
diff --git a/lib/rwsem.c b/lib/rwsem.c index a8055cf..1d6e6e8 100644 --- a/lib/rwsem.c +++ b/lib/rwsem.c @@ -64,7 +64,7 @@ __rwsem_do_wake(struct rw_semaphore *sem, enum rwsem_wake_type wake_type) struct rwsem_waiter *waiter; struct task_struct *tsk; struct list_head *next; - long oldcount, woken, loop, adjustment; + long woken, loop, adjustment; waiter = list_entry(sem->wait_list.next, struct rwsem_waiter, list); if (waiter->type == RWSEM_WAITING_FOR_WRITE) { @@ -86,6 +86,12 @@ __rwsem_do_wake(struct rw_semaphore *sem, enum rwsem_wake_type wake_type) if (wake_type != RWSEM_WAKE_READ_OWNED) { adjustment = RWSEM_ACTIVE_READ_BIAS; while (1) { + long oldcount; + + /* A writer stole the lock. */ + if (unlikely(sem->count < RWSEM_WAITING_BIAS)) + return sem; + oldcount = rwsem_atomic_update(adjustment, sem) - adjustment; if (likely(oldcount >= RWSEM_WAITING_BIAS)) -- 1.7.4.4
| |