Messages in this thread | | | Date | Fri, 2 Dec 2022 02:26:08 +0900 | Subject | Re: [PATCH 0/3] KVM: arm64: Handle CCSIDR associativity mismatches | From | Akihiko Odaki <> |
| |
On 2022/12/01 20:06, Marc Zyngier wrote: > On Thu, 01 Dec 2022 10:49:11 +0000, > Akihiko Odaki <akihiko.odaki@daynix.com> wrote: > > Thanks for looking into this. > >> M2 MacBook Air has mismatched CCSIDR associativity bits, which makes the >> bits a KVM vCPU sees inconsistent when migrating. > > Can you describe the actual discrepancy? Is that an issue between the > two core types? In which case, nothing says that these two cluster > should have the same cache topology.
Yes, the processor has big.LITTLE configuration.
On the processor, the valid CSSELR values are 0 (L1D), 1 (L1I), 3 (L2D). For each CSSELR values, each cluster has: - 0x700FE03A, 0x203FE01A, 0x70FFE07B - 0x701FE03A, 0x203FE02A, 0x73FFE07B
> >> It also makes QEMU fail restoring the vCPU registers because QEMU saves >> and restores all of the registers including CCSIDRs, and if the vCPU >> migrated among physical CPUs between saving and restoring, it tries to >> restore CCSIDR values that mismatch with the current physical CPU, which >> causes EFAULT. > > Well, QEMU will have plenty of other problems, starting with MIDRs, > which always reflect the physical one. In general, KVM isn't well > geared for VMs spanning multiple CPU types. It is improving, but there > is a long way to go.
On M2 MacBook Air, I have seen no other difference in standard ID registers and CCSIDRs are exceptions. Perhaps Apple designed this way so that macOS's Hypervisor can freely migrate vCPU, but I can't assure that without more analysis. This is still enough to migrate vCPU running Linux at least.
> >> Trap CCSIDRs if there are CCSIDR value msimatches, and override the >> associativity bits when handling the trap. > > TBH, I'd rather we stop reporting this stuff altogether. > > There is nothing a correctly written arm64 guest should do with any of > this (this is only useful for set/way CMOs, which non-secure SW should > never issue). It would be a lot better to expose a virtual topology > (one set, one way, one level). It would also save us from the CCSIDRX > silliness. > > The only complexity would be to still accept different topologies from > userspace so that we can restore a VM saved before this virtual > topology.
Another (minor) concern is that trapping relevant registers may cost too much. Currently KVM traps CSSELR and CCSIDR accesses with HCR_TID2, but HCR_TID2 also affects CTR_EL0. Although I'm not sure if the register is referred frequently, Arm introduced FEAT_EVT to trap CSSELR and CSSIDR but not CTR_EL0 so there may be some case where trapping CTR_EL0 is not tolerated. Perhaps Arm worried that a userspace application may read CTR_EL0 frequently.
If you think the concern on VM restoration you mentioned and the trapping overhead is tolerable, I'll write a new, much smaller patch accordingly.
Regards, Akihiko Odaki
> > Do you mind having a look at this? > > Thanks, > > M. >
| |