lkml.org 
[lkml]   [2022]   [Jun]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH] kernfs: Change kernfs_rwsem to a per-cpu rwsem.
From
Hello Tejun,

On 25/6/22 2:12 pm, Tejun Heo wrote:
> On Mon, Jun 20, 2022 at 01:26:34PM +1000, Imran Khan wrote:
>> On large systems when few hundred CPUs simulateously acquire kernfs_rwsem,
>> for reading we see performance degradation due to bouncing of cache line
>> that contains kernfs_rwsem. Changing kernfs_rwsem into a per-fs, per-cpu
>> rwsem can fix this degradation.
> ...
>> Moreover this run of 200 applications take more than 32 secs to finish on
>> this system.
>>
>> After changing kernfs_rwsem to a per-cpu rwsem, I can see that contention
>> for kernfs_rwsem is no longer visible in perf data and the test execution
>> time has reduced to almost half (17 secs or less from 32 secs or more).
>>
>> The overhead involving write operations with per-cpu rwsem will be higher
>> but frequency of creation/deletion of kernfs files is much less than
>> frequency at which kernfs (cgroup, sysfs) files are read.
>
> The problem with percpu_rwsem is that write locking requires going
> through a RCU grace period, which can easily add two or more digit
> millisec latencies. I'm pretty sure there are code paths which are
> pretty heavy on write - e.g. during boot, depending on the machine
> configuration, we could be write-acquiring the rwsem hundreds of
> thousands of times and we'd be constantly doing RCU grace periods.
>
> So, I don't think kernfs_rwsem is a good candidate for percpu rwsem.
> There likely are plenty of cases where write path isn't cold enough.
>

Thanks for your feedback. I will continue with approach of hashed kernfs_rwsem then.

Thanks
-- Imran

\
 
 \ /
  Last update: 2022-06-25 11:16    [W:0.070 / U:0.012 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site