Messages in this thread | | | Date | Thu, 12 May 2022 20:45:45 +0800 | From | Aaron Lu <> | Subject | Re: [mm/page_alloc] f26b3fa046: netperf.Throughput_Mbps -18.0% regression |
| |
On Thu, May 12, 2022 at 11:17:04AM +0800, ying.huang@intel.com wrote: > On Wed, 2022-05-11 at 19:04 +0800, Aaron Lu wrote: > > On Wed, May 11, 2022 at 09:58:23AM +0800, ying.huang@intel.com wrote: > > > On Tue, 2022-05-10 at 11:05 -0700, Linus Torvalds wrote: > > > > [ Adding locking people in case they have any input ] > > > > > > > > On Mon, May 9, 2022 at 11:23 PM ying.huang@intel.com > > > > <ying.huang@intel.com> wrote: > > > > > > > > > > > > > > > > > Can you point me to the regression report? I would like to take a look, > > > > > > thanks. > > > > > > > > > > https://lore.kernel.org/all/1425108604.10337.84.camel@linux.intel.com/ > > > > > > > > Hmm. > > > > > > > > That explanation looks believable, except that our qspinlocks > > > > shouldn't be spinning on the lock itself, but spinning on the mcs node > > > > it inserts into the lock. > > > > > > The referenced regression report is very old (in Feb 2015 for 3.16- > > > 3.17). The ticket spinlock was still used at that time. I believe that > > > things become much better after we used qspinlock. We can test that. > > > > 'will-it-scale/page_fault1 process mode' can greatly stress both zone > > lock and LRU lock when nr_process = nr_cpu with thp disabled. So I run > > it to see if it still makes a difference with qspinlock. > > https://github.com/antonblanchard/will-it-scale/blob/master/tests/page_fault1.c > > > > The result on an Icelake 2 sockets server with a total of 48cores/96cpus: > > > > tbox_group/testcase/rootfs/kconfig/compiler/nr_task/mode/test/thp_enabled/cpufreq_governor/ucode: > > lkp-icl-2sp4/will-it-scale/debian-10.4-x86_64-20200603.cgz/x86_64-rhel-8.3/gcc-11/100%/process/page_fault1/never/performance/0xd000331 > > > > commit: > > v5.18-rc4 > > 731a704c0d8760cfd641af4bf57167d8c68f9b99 > > > > v5.18-rc4 731a704c0d8760cfd641af4bf57 > > ---------------- --------------------------- > > %stddev %change %stddev > > \ | \ > > 12323894 -26.0% 9125299 will-it-scale.128.processes > > > > 22.33 ± 4% -22.3 0.00 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.release_pages.tlb_flush_mmu > > 9.80 -9.2 0.57 ± 3% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__pagevec_lru_add.folio_add_lru > > 36.25 +6.7 42.94 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist > > 4.28 ± 10% +34.6 38.93 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages > > 75.05 +7.8 82.83 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath > > > > commit 731a704c0d8760cfd641af4bf57 moves zone's lock back to above > > free_area by reverting commit a368ab67aa55("mm: move zone lock to a > > different cache line than order-0 free page lists") on top of v5.18-rc4. > > > > The interpretation of the above result is: after the revert, performance > > dropped 26%, zone lock increased 41% from 40% to 81%, the overall lock > > contention increased 7.8% from 75% to 82.83%. So it appears it still > > makes a difference with qspinlock. > > The performance impact is larger than what I have thought before. The > test may be too stressing to be realistic? Can you run the test with > less process number to check?
What about nr_process=16 and 1?
When nr_process=16, zone lock contention increased about 21% from 6% to 27%, performance dropped 17.8%, overall lock contention increased 14.3%:
commit: v5.18-rc4 731a704c0d8760cfd641af4bf57167d8c68f9b99
v5.18-rc4 731a704c0d8760cfd641af4bf57 ---------------- --------------------------- \ | \ 7179264 -17.8% 5898330 will-it-scale.16.processes
4.01 ± 11% -3.5 0.54 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.release_pages.tlb_flush_mmu 4.53 ± 4% -3.0 1.58 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.folio_lruvec_lock_irqsave.__pagevec_lru_add.folio_add_lru 4.61 ± 13% +10.3 14.90 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.rmqueue_bulk.rmqueue.get_page_from_freelist 1.58 ± 14% +11.0 12.62 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_pcppages_bulk.free_unref_page_list.release_pages 15.42 ± 2% +14.3 29.75 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
When running single process, there is no measuable performance change and lock contention change.
| |