lkml.org 
[lkml]   [2023]   [Nov]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH V1 1/1] sched/numa: Fix mm numa_scan_seq based unconditional scan
From
On 11/1/2023 4:01 PM, Peter Zijlstra wrote:
> On Wed, Nov 01, 2023 at 09:21:01AM +0000, Mel Gorman wrote:
>> On Fri, Oct 20, 2023 at 09:27:46PM +0530, Raghavendra K T wrote:
>>> Since commit fc137c0ddab2 ("sched/numa: enhance vma scanning logic")
>>>
>>> NUMA Balancing allows updating PTEs to trap NUMA hinting faults if the
>>> task had previously accessed VMA. However unconditional scan of VMAs are
>>> allowed during initial phase of VMA creation until process's
>>> mm numa_scan_seq reaches 2 even though current task had not accessed VMA.
>>>
>>> Rationale:
>>> - Without initial scan subsequent PTE update may never happen.
>>> - Give fair opportunity to all the VMAs to be scanned and subsequently
>>> understand the access pattern of all the VMAs.
>>>
>>> But it has a corner case where, if a VMA is created after some time,
>>> process's mm numa_scan_seq could be already greater than 2.
>>>
>>> For e.g., values of mm numa_scan_seq when VMAs are created by running
>>> mmtest autonuma benchmark briefly looks like:
>>> start_seq=0 : 459
>>> start_seq=2 : 138
>>> start_seq=3 : 144
>>> start_seq=4 : 8
>>> start_seq=8 : 1
>>> start_seq=9 : 1
>>> This results in no unconditional PTE updates for those VMAs created after
>>> some time.
>>>
>>> Fix:
>>> - Note down the initial value of mm numa_scan_seq in per VMA start_seq.
>>> - Allow unconditional scan till start_seq + 2.
>>>
>>> Result:
>>> SUT: AMD EPYC Milan with 2 NUMA nodes 256 cpus.
>>> base kernel: upstream 6.6-rc6 with Mels patches [1] applied.
>>>
>>> kernbench
>>> ========== base patched %gain
>>> Amean elsp-128 165.09 ( 0.00%) 164.78 * 0.19%*
>>>
>>> Duration User 41404.28 41375.08
>>> Duration System 9862.22 9768.48
>>> Duration Elapsed 519.87 518.72
>>>
>>> Ops NUMA PTE updates 1041416.00 831536.00
>>> Ops NUMA hint faults 263296.00 220966.00
>>> Ops NUMA pages migrated 258021.00 212769.00
>>> Ops AutoNUMA cost 1328.67 1114.69
>>>
>>> autonumabench
>>>
>>> NUMA01_THREADLOCAL
>>> ==================
>>> Amean elsp-NUMA01_THREADLOCAL 81.79 (0.00%) 67.74 * 17.18%*
>>>
>>> Duration User 54832.73 47379.67
>>> Duration System 75.00 185.75
>>> Duration Elapsed 576.72 476.09
>>>
>>> Ops NUMA PTE updates 394429.00 11121044.00
>>> Ops NUMA hint faults 1001.00 8906404.00
>>> Ops NUMA pages migrated 288.00 2998694.00
>>> Ops AutoNUMA cost 7.77 44666.84
>>>
>>> Link: https://lore.kernel.org/linux-mm/ZSXF3AFZgIld1meX@gmail.com/T/
>>>
>>> Signed-off-by: Raghavendra K T <raghavendra.kt@amd.com>
>>
>> Acked-by: Mel Gorman <mgorman@suse.de>
>
> Thanks, will queue for the next merge window (6.8 I think that is) once
> 6.7-rc1 comes around.

Thank you Mel, PeterZ.

Meanwhile, I will check if extending #history (PeterZ) on this changed
baseline, as well as implications of extending #bits for PIDS (Ingo)
suggested (especially larger machine) helps and come back if I find
anything interesting.

Thanks and Regards
- Raghu

\
 
 \ /
  Last update: 2023-11-02 06:23    [W:0.052 / U:0.384 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site