Messages in this thread | | | Date | Fri, 11 Aug 2023 17:48:06 +0800 | From | Aaron Lu <> | Subject | Re: [RFC PATCH 2/4] sched/fair: Make tg->load_avg per node |
| |
On Wed, Aug 02, 2023 at 01:28:36PM +0200, Peter Zijlstra wrote: > On Wed, Jul 19, 2023 at 09:45:00PM +0800, Aaron Lu wrote: > > On Wed, Jul 19, 2023 at 01:53:58PM +0200, Peter Zijlstra wrote: > > > On Tue, Jul 18, 2023 at 09:41:18PM +0800, Aaron Lu wrote: > > > > +#if defined(CONFIG_FAIR_GROUP_SCHED) && defined(CONFIG_SMP) > > > > +static inline long tg_load_avg(struct task_group *tg) > > > > +{ > > > > + long load_avg = 0; > > > > + int i; > > > > + > > > > + /* > > > > + * The only path that can give us a root_task_group > > > > + * here is from print_cfs_rq() thus unlikely. > > > > + */ > > > > + if (unlikely(tg == &root_task_group)) > > > > + return 0; > > > > + > > > > + for_each_node(i) > > > > + load_avg += atomic_long_read(&tg->node_info[i]->load_avg); > > > > + > > > > + return load_avg; > > > > +} > > > > +#endif > > > > > > So I was working on something else numa and noticed that for_each_node() > > > (and most of the nodemask stuff) is quite moronic, afaict we should do > > > something like the below. > > > > > > I now see Mike added the nr_node_ids thing fairly recent, but given > > > distros have NODES_SHIFT=10 and actual machines typically only have <=4 > > > nodes, this would save a factor of 256 scanning. > > More complete nodemask patch here: > > https://lkml.kernel.org/r/20230802112458.230221601%40infradead.org
Thanks for the update.
I incorperated this numa change and collected some data and found that with the newly proposed approach to rate limit updates to tg->load_avg to at most once per ms, the cost of accessing tg->load_avg is dropped so much that adding other optimizations doesn't make much difference.
So I was thinking maybe I just need that one ratelimit patch to reduce the cost of accessing tg->load_avg. The detailed data is here: https://lore.kernel.org/lkml/20230811092811.GA399195@ziqianlu-dell/
| |