Messages in this thread | | | Date | Fri, 18 Aug 2023 10:18:50 -0700 | From | Nicolin Chen <> | Subject | Re: [PATCH v2] iommu/arm-smmu-v3: Add a user-configurable tlb_invalidate_threshold |
| |
On Fri, Aug 18, 2023 at 05:11:19PM +0100, Will Deacon wrote:
> On Thu, Aug 17, 2023 at 11:36:18AM -0700, Nicolin Chen wrote: > > On Wed, Aug 16, 2023 at 01:43:50PM -0700, Nicolin Chen wrote: > > > > > When receiving an __arm_smmu_tlb_inv_range() call with a large size, there > > > could be a long latency at this function call: one part is coming from a > > > large software overhead in the routine of building commands, and the other > > > part is coming from CMDQ hardware consuming the large number of commands. > > > This latency could be significantly large on an SMMU that does not support > > > range invalidation commands, i.e. no ARM_SMMU_FEAT_RANGE_INV. > > > > > > One way to optimize this is to replace a large number of VA invalidation > > > commands with one single per-asid invalidation command, when the requested > > > size reaches a threshold. This threshold can be configurable depending on > > > the SMMU implementaion. > > > > I'm rethinking about this size-based threshold, since what really > > affects the latency is the number of the invalidation commands in > > the request. So having an npages-based threshold might be optimal, > > though the idea and implementation would be similar. > > On the CPU side, we just have: > > #define MAX_TLBI_OPS PTRS_PER_PTE > > in asm/tlbflush.h > > Can we start off with something similar for the SMMU? I'm not massively > keen on exposing this as a knob to userspace, because I don't think most > people will have a clue about how to tune it.
Yes! I was hesitating about an arbitrary threshold setup that actually fits our situation better. Now it makes sense. Thanks for the input!
What I would do was to pick a number based on our test results. Yet, it seems that 1024 was chosen at the beginning to fix a softlock bug, and it changed to PTRS_PER_PTE for a better perf, IIUIC. Should we use the similar setup for SMMU? I found it is stored in data->bits_per_level, so perhaps pass it back to the driver via "struct io_pgtable_cfg".
Nicolin
| |