Messages in this thread | | | Date | Tue, 17 Oct 2023 13:41:50 -0300 | From | Jason Gunthorpe <> | Subject | Re: [PATCH v8 0/5] Share sva domains with all devices bound to a mm |
| |
On Tue, Oct 17, 2023 at 08:47:57AM +0800, Tina Zhang wrote: > This series is to share sva(shared virtual addressing) domains with all > devices bound to one mm. > > Problem > ------- > In the current iommu core code, sva domain is allocated per IOMMU group, > when device driver is binding a process address space to a device (which is > handled in iommu_sva_bind_device()). If one than more device is bound to > the same process address space, there must be more than one sva domain > instance, with each device having one. In other words, the sva domain > doesn't share between those devices bound to the same process address > space, and that leads to two problems: > 1) device driver has to duplicate sva domains with enqcmd, as those sva > domains have the same PASID and are relevant to one virtual address space. > This makes the sva domain handling complex in device drivers. > 2) IOMMU driver cannot get sufficient info of the IOMMUs that have > devices behind them bound to the same virtual address space, when handling > mmu_notifier_ops callbacks. As a result, IOMMU IOTLB invalidation is > performed per device instead of per IOMMU, and that may lead to > superfluous IOTLB invalidation issue, especially in a virtualization > environment where all devices may be behind one virtual IOMMU. > > Solution > -------- > This patch-set tries to fix those two problems by allowing sharing sva > domains with all devices bound to a mm. To achieve this, a new structure > pointer is introduced to mm to replace the old PASID field, which can keep > the info of PASID as well as the corresponding shared sva domains. > Besides, function iommu_sva_bind_device() is updated to ensure a new sva > domain can only be allocated when the old ones cannot work for the IOMMU. > With these changes, a device driver can expect one sva domain could work > for per PASID instance(e.g., enqcmd PASID instance), and therefore may get > rid of handling sva domain duplication. Besides, IOMMU driver (e.g., intel > vt-d driver) can get sufficient info (e.g., the info of the IOMMUs having > their devices bound to one virtual address space) when handling > mmu_notifier_ops callbacks, to remove the redundant IOTLB invalidations. > > Arguably there shouldn't be more than one sva_domain with the same PASID, > and in any sane configuration there should be only 1 type of IOMMU driver > that needs only 1 SVA domain. However, in reality, IOMMUs on one platform > may not be identical to each other. Thus, attaching a sva domain that has > been successfully bound to device A behind a IOMMU A, to device B behind > IOMMU B may get failed due to the difference between IOMMU A and IOMMU > B. In this case, a new sva domain with the same PASID needs to be > allocated to work with IOMMU B. That's why we need a list to keep sva > domains of one PASID. For the platform where IOMMUs are compatible to each > other, there should be one sva domain in the list. > > v8: > - CC more people > - CC iommu@lists.linux.dev mailing list. > When sending version 7, some issue happened in my CC list and that caused > version 7 wasn't sent to iommu@lists.linux.dev. > - Rebase to v6.6-rc6 and make a few format changes.
You should based it on Joerg's tree so he can take it without conflcits.
The conflicts are trivial though (Take Michael's version and switch mm->pasid with mm_get_enqcmd_pasid(mm))
It looks fine, please lets get it in this cycle, the ARM and AMD SVA series depend on it.
Jason
| |