Messages in this thread | | | Date | Thu, 9 Nov 2023 09:46:53 -0800 | Subject | Re: [PATCH v7 13/24] x86/resctrl: Queue mon_event_read() instead of sending an IPI | From | Reinette Chatre <> |
| |
Hi James,
On 10/25/2023 11:03 AM, James Morse wrote: > Intel is blessed with an abundance of monitors, one per RMID, that can be > read from any CPU in the domain. MPAMs monitors reside in the MMIO MSC, > the number implemented is up to the manufacturer. This means when there are > fewer monitors than needed, they need to be allocated and freed. > > MPAM's CSU monitors are used to back the 'llc_occupancy' monitor file. The > CSU counter is allowed to return 'not ready' for a small number of > micro-seconds after programming. To allow one CSU hardware monitor to be > used for multiple control or monitor groups, the CPU accessing the > monitor needs to be able to block when configuring and reading the > counter. > > Worse, the domain may be broken up into slices, and the MMIO accesses > for each slice may need performing from different CPUs. > > These two details mean MPAMs monitor code needs to be able to sleep, and > IPI another CPU in the domain to read from a resource that has been sliced. > > mon_event_read() already invokes mon_event_count() via IPI, which means > this isn't possible. On systems using nohz-full, some CPUs need to be > interrupted to run kernel work as they otherwise stay in user-space > running realtime workloads. Interrupting these CPUs should be avoided, > and scheduling work on them may never complete. > > Change mon_event_read() to pick a housekeeping CPU, (one that is not using > nohz_full) and schedule mon_event_count() and wait. If all the CPUs > in a domain are using nohz-full, then an IPI is used as the fallback. > > This function is only used in response to a user-space filesystem request > (not the timing sensitive overflow code). > > This allows MPAM to hide the slice behaviour from resctrl, and to keep > the monitor-allocation in monitor.c. When the IPI fallback is used on > machines where MPAM needs to make an access on multiple CPUs, the counter > read will always fail. > > Tested-by: Shaopeng Tan <tan.shaopeng@fujitsu.com> > Tested-by: Peter Newman <peternewman@google.com> > Reviewed-by: Shaopeng Tan <tan.shaopeng@fujitsu.com> > Reviewed-by: Peter Newman <peternewman@google.com> > Signed-off-by: James Morse <james.morse@arm.com> > ---
Reviewed-by: Reinette Chatre <reinette.chatre@intel.com>
Reinette
| |