Messages in this thread | | | From | alexander.antonov@linux ... | Subject | [PATCH 00/11] UPI topology discovery for SKX/ICX/SPR | Date | Thu, 17 Nov 2022 12:28:22 +0000 |
| |
From: Alexander Antonov <alexander.antonov@linux.intel.com>
Starting with the Intel Xeon Scalable family, Intel processors use a coherent interconnect for scaling to multiple sockets, known as Intel Ultra Path Interconnect (Intel UPI). The Intel UPI technology provides a cache-coherent socket-to-socket external communication interface between processors.
Unfortunately, users have little idea on how sockets are connected to each other. This patchset extends the current mapping implementation by adding sysfs attributes for UPI PMON blocks in the same manner as was done for IIO blocks. These attributes reveal the topology of the system by showing which how sockets and UPI controllers are connected to each other.
Example for a 4-socket Sapphire Rapids server:
# tail /sys/devices/uncore_upi_*/die* ==> /sys/devices/uncore_upi_0/die0 <== upi_1,die_1
==> /sys/devices/uncore_upi_0/die1 <== upi_0,die_3
==> /sys/devices/uncore_upi_0/die2 <== upi_1,die_3
==> /sys/devices/uncore_upi_0/die3 <== upi_0,die_1
==> /sys/devices/uncore_upi_1/die0 <== upi_1,die_2
==> /sys/devices/uncore_upi_1/die1 <== upi_0,die_0
==> /sys/devices/uncore_upi_1/die2 <== upi_1,die_0
==> /sys/devices/uncore_upi_1/die3 <== upi_0,die_2
==> /sys/devices/uncore_upi_2/die0 <== upi_2,die_3
==> /sys/devices/uncore_upi_2/die1 <== upi_2,die_2
==> /sys/devices/uncore_upi_2/die2 <== upi_2,die_1
==> /sys/devices/uncore_upi_2/die3 <== upi_2,die_0
The following output: # cat /sys/devices/uncore_upi_X/dieY upi_Z,die_W means that UPI link number X on die Y is connected to UPI link Z on die W.
Changes are splitted into two separate parts: 1. kernel part - adds and fills new sysfs attributes for UPI PMON blocks 2. perf-tool part - enables UPI topology info and per-link UPI metrics for 'perf iostat' mode. See example below.
The 'perf iostat list' will be extend to show IIO and UPI mappings, for example:
# perf iostat list S0-uncore_iio_0<0000:00> S1-uncore_iio_0<0000:80> S0-uncore_iio_1<0000:17> S1-uncore_iio_1<0000:85> S0-uncore_iio_2<0000:3a> S1-uncore_iio_2<0000:ae> S0-uncore_iio_3<0000:5d> S1-uncore_iio_3<0000:d7> UPI Link 0 on Socket 0 -> UPI Link 1 on Socket 1 UPI Link 0 on Socket 1 -> UPI Link 1 on Socket 0 UPI Link 1 on Socket 0 -> UPI Link 0 on Socket 1 UPI Link 1 on Socket 1 -> UPI Link 0 on Socket 0
The 'perf iostat upi' mode will be added to show per-link UPI metrics: # perf iostat upi link Outgoing Data(GB) Outgoing Non-Data(GB) UPI Link 0 on Socket 0 -> UPI Link 1 on Socket 1 0 0 UPI Link 0 on Socket 1 -> UPI Link 1 on Socket 0 0 0 UPI Link 1 on Socket 0 -> UPI Link 0 on Socket 1 0 0 UPI Link 1 on Socket 1 -> UPI Link 0 on Socket 0 0 0
1.437023089 seconds time elapsed
Alexander Antonov (11): perf/x86/intel/uncore: Generalize IIO topology support perf/x86/intel/uncore: Introduce UPI topology type perf/x86/intel/uncore: Clear attr_update properly perf/x86/intel/uncore: Disable I/O stacks to PMU mapping on ICX-D perf/x86/intel/uncore: Generalize get_topology() for SKX PMUs perf/x86/intel/uncore: Enable UPI topology discovery for Skylake Server perf/x86/intel/uncore: Get UPI NodeID and GroupID perf/x86/intel/uncore: Enable UPI topology discovery for Icelake Server perf/x86/intel/uncore: Enable UPI topology discovery for Sapphire Rapids perf/x86/intel/uncore: Update sysfs-devices-mapping file perf/x86/intel/uncore: Make set_mapping() procedure void
.../ABI/testing/sysfs-devices-mapping | 30 +- arch/x86/events/intel/uncore.h | 24 +- arch/x86/events/intel/uncore_snbep.c | 492 +++++++++++++++--- 3 files changed, 480 insertions(+), 66 deletions(-)
base-commit: ee6050c8af96bba2f81e8b0793a1fc2f998fcd20 -- 2.25.1
| |