Messages in this thread Patch in this message | | | From | K Prateek Nayak <> | Subject | [PATCH 2/2] drivers: base: cacheinfo: Update cpu_map_populated during CPU Hotplug | Date | Mon, 8 May 2023 14:11:15 +0530 |
| |
Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online which let the arch specific functions handle (at least on x86) populating the shared_cpu_map. However, with the changes in the aforementioned commit, populate_cache_leaves() is not called when a CPU comes online as a result of hotplug since last_level_cache_is_valid() returns true as the cacheinfo data is not discarded. The CPU coming online is not present in shared_cpu_map, however, it will not be added since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in populate_cache_leaves() when cacheinfo is first populated for x86)
This can lead to inconsistencies in the shared_cpu_map when an offlined CPU comes online again. Example below depicts the inconsistency in the shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on a 3rd Generation EPYC processor:
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8
# cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 136
# cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 9-15,136-143
Clear the flag when the CPU is removed from shared_cpu_map when cache_shared_cpu_map_remove() is called during CPU hotplug. This will allow cache_shared_cpu_map_setup() to add the CPU coming back online in the shared_cpu_map. Set the flag again when the shared_cpu_map is setup. Following are results of performing the same test as described above with the changes:
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online
# for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143
# cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 8,136
# cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 8-15,136-143
Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs") Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com> --- drivers/base/cacheinfo.c | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/drivers/base/cacheinfo.c b/drivers/base/cacheinfo.c index d1ae443fd7a0..cbae8be1fe52 100644 --- a/drivers/base/cacheinfo.c +++ b/drivers/base/cacheinfo.c @@ -410,11 +410,14 @@ static int cache_shared_cpu_map_setup(unsigned int cpu) coherency_max_size = this_leaf->coherency_line_size; } + /* shared_cpu_map is now populated for the cpu */ + this_cpu_ci->cpu_map_populated = true; return 0; } static void cache_shared_cpu_map_remove(unsigned int cpu) { + struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu); struct cacheinfo *this_leaf, *sib_leaf; unsigned int sibling, index, sib_index; @@ -447,6 +450,9 @@ static void cache_shared_cpu_map_remove(unsigned int cpu) } } } + + /* cpu is no longer populated in the shared map */ + this_cpu_ci->cpu_map_populated = false; } static void free_cache_attributes(unsigned int cpu) -- 2.34.1
| |