Messages in this thread | | | Subject | Re: [PATCH 11/11] EDAC/ghes: Create one memory controller per physical memory array | From | John Garry <> | Date | Tue, 17 Mar 2020 16:34:32 +0000 |
| |
On 16/03/2020 09:51, Borislav Petkov wrote: > On Fri, Mar 06, 2020 at 04:13:18PM +0100, Robert Richter wrote: >> The ghes driver only creates one memory controller for the whole >> system. This does not reflect memory topology especially in multi-node >> systems. E.g. a Marvell ThunderX2 system shows: >> >> /sys/devices/system/edac/mc/mc0/dimm0 >> /sys/devices/system/edac/mc/mc0/dimm1 >> /sys/devices/system/edac/mc/mc0/dimm2 >> /sys/devices/system/edac/mc/mc0/dimm3 >> /sys/devices/system/edac/mc/mc0/dimm4 >> /sys/devices/system/edac/mc/mc0/dimm5 >> /sys/devices/system/edac/mc/mc0/dimm6 >> /sys/devices/system/edac/mc/mc0/dimm7 >> /sys/devices/system/edac/mc/mc0/dimm8 >> /sys/devices/system/edac/mc/mc0/dimm9 >> /sys/devices/system/edac/mc/mc0/dimm10 >> /sys/devices/system/edac/mc/mc0/dimm11 >> /sys/devices/system/edac/mc/mc0/dimm12 >> /sys/devices/system/edac/mc/mc0/dimm13 >> /sys/devices/system/edac/mc/mc0/dimm14 >> /sys/devices/system/edac/mc/mc0/dimm15 >> >> The DIMMs 9-15 are located on the 2nd node of the system. On >> comparable x86 systems there is one memory controller per node. The >> ghes driver should also group DIMMs depending on the topology and >> create one MC per node. >> >> There are several options to detect the topology. ARM64 systems >> retrieve the (NUMA) node information from the ACPI SRAT table (see >> acpi_table_parse_srat()). The node id is later stored in the physical >> address page. The pfn_to_nid() macro could be used for a DIMM after >> determining its physical address. The drawback of this approach is >> that there are too many subsystems involved it depends on. It could >> easily break and makes the implementation complex. E.g. pfn_to_nid() >> can only be reliable used on mapped address ranges which is not always >> granted, there are various firmware instances involved which could be >> broken, or results may vary depending on NUMA settings. >> >> Another approach that was suggested by James' is to use the DIMM's >> physical memory array handle to group DIMMs [1]. The advantage is to >> only use the information on memory devices from the SMBIOS table that >> contains a reference to the physical memory array it belongs too. This >> information is mandatory same as the use of DIMM handle references by >> GHES to provide the DIMM location of an error. There is only a single >> table to parse which eases implementation. This patch uses this >> approach for DIMM grouping. >> >> Modify the DMI decoder to also detect the physical memory array a DIMM >> is linked to and create one memory controller per array to group >> DIMMs. With the change DIMMs are grouped, e.g. a ThunderX2 system >> shows one MC per node now: >> >> # grep . /sys/devices/system/edac/mc/mc*/dimm*/dimm_label >> /sys/devices/system/edac/mc/mc0/dimm0/dimm_label:N0 DIMM_A0 >> /sys/devices/system/edac/mc/mc0/dimm1/dimm_label:N0 DIMM_B0 >> /sys/devices/system/edac/mc/mc0/dimm2/dimm_label:N0 DIMM_C0 >> /sys/devices/system/edac/mc/mc0/dimm3/dimm_label:N0 DIMM_D0 >> /sys/devices/system/edac/mc/mc0/dimm4/dimm_label:N0 DIMM_E0 >> /sys/devices/system/edac/mc/mc0/dimm5/dimm_label:N0 DIMM_F0 >> /sys/devices/system/edac/mc/mc0/dimm6/dimm_label:N0 DIMM_G0 >> /sys/devices/system/edac/mc/mc0/dimm7/dimm_label:N0 DIMM_H0 >> /sys/devices/system/edac/mc/mc1/dimm0/dimm_label:N1 DIMM_I0 >> /sys/devices/system/edac/mc/mc1/dimm1/dimm_label:N1 DIMM_J0 >> /sys/devices/system/edac/mc/mc1/dimm2/dimm_label:N1 DIMM_K0 >> /sys/devices/system/edac/mc/mc1/dimm3/dimm_label:N1 DIMM_L0 >> /sys/devices/system/edac/mc/mc1/dimm4/dimm_label:N1 DIMM_M0 >> /sys/devices/system/edac/mc/mc1/dimm5/dimm_label:N1 DIMM_N0 >> /sys/devices/system/edac/mc/mc1/dimm6/dimm_label:N1 DIMM_O0 >> /sys/devices/system/edac/mc/mc1/dimm7/dimm_label:N1 DIMM_P0 >> >> [1] https://lkml.kernel.org/r/f878201f-f8fd-0f2a-5072-ba60c64eefaf@arm.com >> >> Suggested-by: James Morse <james.morse@arm.com> >> Signed-off-by: Robert Richter <rrichter@marvell.com> >> --- >> drivers/edac/ghes_edac.c | 137 ++++++++++++++++++++++++++++++--------- >> 1 file changed, 107 insertions(+), 30 deletions(-) > > This is all fine and good but that change affects the one x86 platform > we support so the whole patchset should be tested there too. Adding > Toshi. > > As a matter of fact, the final version of this set should be tested on > all platforms which are using this thing. Adding John Garry too who > reported issues with this driver recently on his platform.
Adding other RAS-centric guys for H.
Cheers, John
> > Thx. >
| |