Messages in this thread | | | Date | Thu, 31 Mar 2022 20:19:02 +0800 | From | Leo Yan <> | Subject | Re: [PATCH v4 2/4] perf arm-spe: Use SPE data source for neoverse cores |
| |
Hi Ali,
On Tue, Mar 29, 2022 at 02:32:14PM +0000, Ali Saidi wrote:
[...]
> > I still think we should consider to extend the memory levels to > > demonstrate clear momory hierarchy on Arm archs, I personally like the > > definitions for "PEER_CORE", "LCL_CLSTR", "PEER_CLSTR" and "SYS_CACHE", > > though these cache levels are not precise like L1/L2/L3 levels, they can > > help us to map very well for the cache topology on Arm archs and without > > any confusion. We could take this as an enhancement if you don't want > > to bother the current patch set's upstreaming. > > I'd like to do this in a separate patch, but I have one other proposal. The > Neoverse cores L2 is strictly inclusive of the L1, so even if it's in the L1, > it's also in the L2. Given that the Graviton systems and afaik the Ampere > systems don't have any cache between the L2 and the SLC, thus anything from > PEER_CORE, LCL_CLSTR, or PEER_CLSTR would hit in the L2, perhaps we > should just set L2 for these cases? German, are you good with this for now?
If we use a single cache level (no matterh it's L2 or ANY_CACHE) for these data sources, it's hard for users to understand what's the cost for the memory operations. So here I suggested for these new cache levels is not only about cache level, it's more about the information telling the memory operation's cost.
[...]
> > Alternatively, I think it's good to pick up the patch series "perf c2c: > > Sort cacheline with all loads" [1], rather than relying on HITM tag, the > > patch series extends a new option "-d all" for perf c2c, so it displays > > the suspecious false sharing cache lines based on load/store ops and > > thread infos. The main reason for holding on th patch set is due to we > > cannot verify it with Arm SPE at that time point, as the time being Arm > > SPE trace data was absent both store ops and data source packets. > > Looking at examples I don't, at least from my system, data-source isn't set for > stores, only for loads.
Ouch ... If data source is not set for store operation, then all store samples will absent cache level info. Or should we set ANY_CACHE as cache level for store operations?
> > I perfer to set PERF_MEM_SNOOP_HIT flag in this patch set and we can > > upstream the patch series "perf c2c: Sort cacheline with all loads" > > (only needs upstreaming patches 01, 02, 03, 10, 11, the rest patches > > have been merged in the mainline kernel). > > > > If this is fine for you, I can respin the patch series for "perf c2c". > > Or any other thoughts? > > I think this is a nice option to have in the tool-box, but from my point of > view, I'd like someone who is familiar with c2c output on x86 to come to an > arm64 system and be able to zero in on a ping-ponging line like they would > otherwise. Highlighting a line that is moving between cores frequently which is > likely in the exclusive state by tagging it an HITM accomplishes this and will > make it easier to find these cases. Your approach also has innaccurancies and > wouldn't be able to differentiate between core X accessing a line a lot followed > by core Y acessing a line alot vs the cores ping-ponging. Yes, I agree that we > will "overcount" HITM, but I don't think this is particularly bad and it does > specifically highlight the core-2-core transfers that are likely a performance > issue easily and it will result in easier identification of areas of false or > true sharing and improve performance.
I don't want to block this patch set by this part, and either I don't want to introduce any confusion for later users, especially I think users who in later use this tool but it's hard for them to be aware any assumptions in this discussion thread. So two options would be fine for me:
Option 1: if you and Arm mates can confirm that inaccuracy caused by setting HITM is low (e.g. 2%-3% inaccuracy that introduced by directly set HITM), I think this could be acceptable. Otherwise, please consider option 2.
Option 2: by default we set PERF_MEM_SNOOP_HIT flag since now actually we have no info to support HITM. Then use a new patch to add an extra option (say '--coarse-hitm') for 'perf c2c' tool, a user can explictly specify this option for 'perf c2c' command; when a user specifies this option it means that the user understands and accepts inaccuracy by forcing to use PERF_MEM_SNOOP_HITM flag. I think you could refer to the option '--stitch-lbr' for adding an option for 'perf c2c' tool.
Thanks, Leo
| |