Messages in this thread | | | Date | Fri, 17 Feb 2023 09:40:01 -0500 | Subject | Re: [PATCH v6 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs | From | Matthew Rosato <> |
| |
On 2/15/23 7:03 AM, Niklas Schnelle wrote: > In some virtualized environments, including s390 paged memory guests, > IOTLB flushes are used to update IOMMU shadow tables. Due to this, they > are much more expensive than in typical bare metal environments or > non-paged s390 guests. In addition they may parallelize more poorly in > virtualized environments. This changes the trade off for flushing IOVAs > such that minimizing the number of IOTLB flushes trumps any benefit of > cheaper queuing operations or increased paralellism. > > In this scenario per-CPU flush queues pose several problems. Firstly > per-CPU memory is often quite limited prohibiting larger queues. > Secondly collecting IOVAs per-CPU but flushing via a global timeout > reduces the number of IOVAs flushed for each timeout especially on s390 > where PCI interrupts may not be bound to a specific CPU. > > Let's introduce a single flush queue mode that reuses the same queue > logic but only allocates a single global queue. This mode can be > selected as a flag bit in a new dma_iommu_options struct which can be > modified from its defaults by IOMMU drivers implementing a new > ops.tune_dma_iommu() callback. As a first user the s390 IOMMU driver > selects the single queue mode if IOTLB flushes are needed on map which > indicates shadow table use. With the unchanged small FQ size and > timeouts this setting is worse than per-CPU queues but a follow up patch > will make the FQ size and timeout variable. Together this allows the > common IOVA flushing code to more closely resemble the global flush > behavior used on s390's previous internal DMA API implementation. > > Link: https://lore.kernel.org/linux-iommu/3e402947-61f9-b7e8-1414-fde006257b6f@arm.com/ > Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com> #s390
| |