Messages in this thread | | | Date | Mon, 22 May 2023 13:02:19 -0400 | Subject | Re: [PATCH v2 3/5] iommu/amd: Introduce Disable IRTE Caching Support | From | Alejandro Jimenez <> |
| |
Hi Suravee,
On 5/18/2023 8:55 PM, Suravee Suthikulpanit wrote: > An Interrupt Remapping Table (IRT) stores interrupt remapping configuration > for each device. In a normal operation, the AMD IOMMU caches the table > to optimize subsequent data accesses. This requires the IOMMU driver to > invalidate IRT whenever it updates the table. The invalidation process > includes issuing an INVALIDATE_INTERRUPT_TABLE command following by > a COMPLETION_WAIT command. > > However, there are cases in which the IRT is updated at a high rate. > For example, for IOMMU AVIC, the IRTE[IsRun] bit is updated on every > vcpu scheduling (i.e. amd_iommu_update_ga()). On system with large > amount of vcpus and VFIO PCI pass-through devices, the invalidation > process could potentially become a performance bottleneck. > > Introducing a new kernel boot option: > > amd_iommu=irtcachedis > > which disables IRTE caching by setting the IRTCachedis bit in each IOMMU > Control register, and bypass the IRT invalidation process. > > Co-developed-by: Alejandro Jimenez <alejandro.j.jimenez@oracle.com> > [Awaiting sign-off by Alejandro] Signed-off-by: Alejandro Jimenez <alejandro.j.jimenez@oracle.com
I sanity tested the kdump mechanism, and confirmed that CONTROL_IRTCACHEDIS is set appropriately based on the irtcachedis kernel parameter.
Also, I have not observed any errors during multiple rounds of testing with large vCPU counts and VFIO passthrough devices under interrupt-intensive workload that causes heavy vCPU scheduling activity and exercises the relevant code path.
Thank you, Alejandro > Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> > --- > .../admin-guide/kernel-parameters.txt | 1 + > drivers/iommu/amd/amd_iommu_types.h | 4 +++ > drivers/iommu/amd/init.c | 36 +++++++++++++++++++ > 3 files changed, 41 insertions(+) > >
| |