Messages in this thread | | | From | Carl Worth <> | Subject | Initial testing of MPAM patches | Date | Fri, 18 Aug 2023 07:13:52 -0700 |
| |
Hi James,
I'm just getting the chance to start testing your MPAM patches on an Ampere implementation. Specifically, I was working with your mpam/snapshot/v6.3 branch [1], an earlier version of which you posted to the list [2].
I have a few initial comments/questions. These are mostly from a black-box point of view, (without yet having looked at the code much). Later on, when I do dig into the code, I can follow up on some of these issues within the context of the relevant patches.
1. Is there a way to query the MPAM PARTID for a particular resctrl group?
I see a file named "id" in the directory for each resource group, but I get "Operation not supported" if I try to cat it. Meanwhile, in the code it looks like PARTIDs are being XORed with some random number. Is there a deliberate attempt to obscure the PARTID?
I don't know how much an end user will care about PARTID values, (so it's nice that the driver manages these implicitly), but for me, while debugging this stuff, it would be nice to be able to query them.
2. Top-level resctrl resource group is not being made to take effect
When I poke at the schemata of the top-level resource group, I can see that it is associated with PARTID 1. That is, if I do something like:
echo L3:1=face > /sys/fs/resctrl/schemata
I can see the value 0xface showing up in the cache portion bitmap registers associated with PARTID 1. So far, so good.
But meanwhile, I am not seeing the MPAM0_EL1 system register being modified to associate the various cores in this resource group with PARTID 1.
In contrast, for any lower-level resource group I create, I do see MPAM0_EL1 being set with PARTID values for the cores that I put into the 'cpus' node.
So it appears that PARTID 1 and its top-level resource group will have no effect currently. Am I seeing that correctly?
3. Is there special treatment of cpu 0?
When I put cpu 0 into a resource group I see both MPAM2_EL2 as well as MPAM0_EL1 for cpu 0 being set to the group's corresponding PARTID. But when I set any cpu other than 0, only MPAM0_EL1 is being set for that cpu. Is this the desired behavior?
I know that PARTID 0 is treated as reserved by the code, but is cpu 0 given any special treatment?
4. The current schemata allows for cache portion, but not cache capacity
The schemata file allows me to specify a bitmask to control cache-portion partitioning. But I don't see any mechanism (nor backing code) to control cache capacity partitioning.
Is this due to a limitation in mapping MPAM to the current resctrl interface? Or would it be easy to extend the schemata to include a cache capacity field as well?
I see that Amit has proposed patches recentl for extending the schemata to add priority partitioning control. So I'd like to do something similar for capacity partitioning control as well.
5. Linked-list corruption with missing cache entries in PPTT
At one point, I tried booting with the MPAM ACPI table populated for my L3 cache, but without the necessary entries in the PPTT ACPI table. The driver fell over with linked-list corruption, halting Linux boot. I'll follow up this report with more details.
6. No support for memory-side caches
MPAM allows for controlling memory-side caches, and the ACPI specification allows them to be described. But the current code appears to ignore any such caches, and won't expose them via resctrl. I'm very interested to know what work would need to be done to allow a memory-side cache to be supported.
7. Cache portion configuration for incorrect PARTID is applied
I've seen a case where, when trying to use a PARTID other than 1, (that is, a resource group other than the top-level), the configuration I've set in that resource group does not take effect. Instead, the configuration for PARTID 1 takes effect.
Querying the controlling MPAM registers does not obviously explain the buggy behavior. Things look correct. I'll be examining this case more closely to see what's happening.
So, that's what I've encountered on an initial look. I didn't call out all the things that are obviously working well, but there's a lot there. So that hasn't gone unnoticed.
Thanks again for this series, and I'm looking forward to engaging on it more going forward.
-Carl
[1] https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/log/?h=mpam/snapshot/v6.3 [2] https://lore.kernel.org/lkml/20210728170637.25610-1-james.morse@arm.com/
| |