Messages in this thread Patch in this message | | | Date | Mon, 15 Nov 2021 17:29:10 -0500 | Subject | Re: [PATCH v4] locking/rwsem: Make handoff bit handling more consistent | From | Waiman Long <> |
| |
On 11/15/21 11:01, Peter Zijlstra wrote: > On Sun, Nov 14, 2021 at 10:38:57PM -0500, Waiman Long wrote: >> On 11/12/21 07:10, Peter Zijlstra wrote: >>> Argh, rwsem_mark_wake() doesn't clear HANDOFF when list_empty(), and >>> write_slowpath() is *far* too clever about all of this. >> rwsem_mark_wake() does clear the HANDOFF flag if it was set. > Argh, yeah, I got confused by the whole !woken case, but that case won't > ever hit list_empty() either. Perhaps that stuff could use a bit of a > reflow too. I think your modification already have included the rewrite for that part. > >>>> @@ -1098,7 +1110,7 @@ rwsem_down_write_slowpath(struct rw_semaphore *sem, int state) >>>> * In this case, we attempt to acquire the lock again >>>> * without sleeping. >>>> */ >>>> - if (wstate == WRITER_HANDOFF) { >>>> + if (waiter.handoff_set) { >>> I'm thinking this wants to be something like: >>> >>> if (rwsem_first_waiter(sem) == &waiter && waiter.handoff_set) { >> handoff_set flag is only set when the waiter becomes the first. > Yes, but a random waiter can wake up and see it be set and also start > spinning.
The handoff_set flag can only be true for a first waiter. A random waiter in the middle of a wait queue will never has this flag set.
This flag is set in two places in rwsem_try_write_lock():
1)
if (has_handoff && !first) return false; new = count;
if (count & RWSEM_LOCK_MASK) { /* * Only the first waiter can inherit a previously set * handoff bit. */ waiter->handoff_set = has_handoff;
handoff_set can only be set to true here if first is also true. In that case, it will also return false immediately afterward.
2)
if (new & RWSEM_FLAG_HANDOFF) { waiter->handoff_set = true; lockevent_inc(rwsem_wlock_handoff); return false; }
Again, only first waiter will have a chance of setting the handoff bit and have handoff_set set to true.
>>>> enum owner_state owner_state; >>>> preempt_disable(); >>> @@ -575,6 +610,11 @@ static inline bool rwsem_try_write_lock( >>> return false; >>> } >>> + /* >>> + * Have rwsem_try_write_lock() fully imply rwsem_del_waiter() on >>> + * success. >>> + */ >>> + list_del(&waiter->list); >>> rwsem_set_owner(sem); >>> return true; >>> } >>> @@ -1128,16 +1153,14 @@ rwsem_down_write_slowpath(struct rw_sema >>> raw_spin_lock_irq(&sem->wait_lock); >>> } >>> __set_current_state(TASK_RUNNING); >>> - list_del(&waiter.list); >> + rwsem_del_waiter(sem, &waiters); ? > I tried that, but then we get an extra atomic in this path. As is I made > try_write_lock() do the full del_waiter, see the hunk above.
You are right. I missed your change in rwsem_try_write_lock().
Thanks, Longman
| |