Messages in this thread | | | From | Peter Newman <> | Date | Thu, 6 Jul 2023 12:22:03 +0200 | Subject | Re: [RFC PATCH 2/2] resctrl2: Arch x86 modules for most of the legacy control/monitor functions |
| |
Hi Tony,
On Wed, Jul 5, 2023 at 6:46 AM Luck, Tony <tony.luck@intel.com> wrote: > The mbm_poll() code that makes sure that counters don't wrap is > doing all the expensive wrmsr(QM_EVTSEL);rdmsr(QM_COUNT) > once per second to give you the data you want.
I was doing that in the soft RMID series I posted earlier because it simplified things, but then I had some realizations about how much error +/- 1 second on the sampling point could result in[1]. We usually measure the bandwidth rate with a 5-second window, so a reading that's up to one second old would mean a 20% error in the bandwidth calculation.
> But existing resctrl > filesystem doesn't let you do a bulk read. I have some ideas on how > to provide something better. One question: do you really need that > snapshot to be system-wide? Or can you live with separate L3-scoped > snapshots that aren't tightly synchronized with each other?
The most demanding use case is responding to a bandwidth saturation emergency, where the admin wants a snapshot of all jobs' bandwidth rates to identify which one(s) to kill. Jobs aren't confined to L3 domains, so I expect it's more important to get a tight snapshot for an RMID-at-a-time. Going domain-at-a-time could create more races where a high-bandwidth job migrates at the right time and avoids detection.
One workaround we've been using to cut down on IPIs is providing a usermode threshold knob for limiting how old a reported event count can be, but this is more useful for soft RMIDs where reading a single CPU's counters makes all RMIDs stored counts current, and IPIs resulting from userspace reads of different groups in the same domain are all redundant.
We could probably work with second-old event counts if we knew the exact time they were recorded so we'd know the exact time delta for the bandwidth count delta, but that seems like it would be awkward in the current interface.
-Peter
[1] https://lore.kernel.org/lkml/CALPaoCi_N8JHbP4zFD=ijBh5-=4Q0Ec-LrspYNGGnj4G6A6U0g@mail.gmail.com/
| |