lkml.org 
[lkml]   [2021]   [Dec]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[PATCH v2 9/9] refcount: Optimize __refcount_add_not_zero(.i=1)
Allow the same off-by-one on the target range that refcount_inc()
already has by only testing the new value for overflow when the
increment is not constant-1.

Improves code-gen, for the common case from:

a887: 41 8b 14 24 mov (%r12),%edx
a88b: 83 fa ff cmp $0xffffffff,%edx
a88e: 74 1f je a8af <ring_buffer_get+0x3f>
a890: 8d 4a 01 lea 0x1(%rdx),%ecx
a893: 89 d0 mov %edx,%eax
a895: f0 41 0f b1 0c 24 lock cmpxchg %ecx,(%r12)
a89b: 75 35 jne a8d2 <ring_buffer_get+0x62>
a89d: 83 c2 02 add $0x2,%edx
a8a0: 09 ca or %ecx,%edx
a8a2: 78 19 js a8bd <ring_buffer_get+0x4d>

to:

a887: 41 8b 04 24 mov (%r12),%eax
a88b: 83 f8 ff cmp $0xffffffff,%eax
a88e: 74 1a je a8aa <ring_buffer_get+0x3a>
a890: 8d 50 01 lea 0x1(%rax),%edx
a893: f0 41 0f b1 14 24 lock cmpxchg %edx,(%r12)
a899: 75 f0 jne a88b <ring_buffer_get+0x1b>
a89b: 85 d2 test %edx,%edx
a89d: 78 19 js a8b8 <ring_buffer_get+0x48>

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
include/linux/refcount.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

--- a/include/linux/refcount.h
+++ b/include/linux/refcount.h
@@ -161,7 +161,7 @@ static inline __must_check bool __refcou
if (oldp)
*oldp = old;

- if (unlikely(old < 0 || old + i < 0))
+ if (unlikely(old < 0 || (!(__builtin_constant_p(i) && i == 1) && old + i < 0)))
refcount_warn_saturate(r, REFCOUNT_ADD_NOT_ZERO_OVF);

return old;

\
 
 \ /
  Last update: 2021-12-10 17:28    [W:0.367 / U:0.172 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site