Messages in this thread | | | Date | Tue, 28 Jul 2020 12:39:07 +0200 | From | peterz@infradea ... | Subject | Re: [PATCH] sched: Make select_idle_sibling search domain configurable |
| |
On Tue, Jul 28, 2020 at 12:01:31AM -0700, Xi Wang wrote: > The scope of select_idle_sibling idle cpu search is LLC. This > becomes a problem for the AMD CCX architecture, as the sd_llc is only > 4 cores. On a many core machine, the range of search is too small to > reach a satisfactory level of statistical multiplexing / efficient > utilization of short idle time slices. > > With this patch idle sibling search is detached from LLC and it > becomes run time configurable. To reduce search and migration > overheads, a presearch domain is added. The presearch domain will be > searched first before the "main search" domain, e.g.: > > sysctl_sched_wake_idle_domain == 2 ("MC" domain) > sysctl_sched_wake_idle_presearch_domain == 1 ("DIE" domain) > > Presearch will go through 4 cores of a CCX. If no idle cpu is found > during presearch, full search will go through the remaining cores of > a cpu socket.
*groan*, this is horrific :-(
It is also in direct conflict with people wanting to make it smaller.
On top of that, a domain number is a terrible terrible interface. They aren't even available without SCHED_DEBUG on.
What is the inter-L3 latency? Going by this that had better be awesome. And if this Infinity Fabric stuff if highly effective in effectively merging L3s -- analogous to what Intel does with it's cache slices, then should we not change the AMD topology setup instead of this 'thing'?
Also, this commit:
051f3ca02e46 ("sched/topology: Introduce NUMA identity node sched domain")
seems to suggest L3 is actually bigger. Suravee, can you please comment?
| |