Messages in this thread | | | From | "Zhang, Tina" <> | Subject | RE: [PATCH v8 0/5] Share sva domains with all devices bound to a mm | Date | Wed, 18 Oct 2023 04:43:41 +0000 |
| |
Hi,
> -----Original Message----- > From: Jason Gunthorpe <jgg@ziepe.ca> > Sent: Wednesday, October 18, 2023 12:42 AM > To: Zhang, Tina <tina.zhang@intel.com> > Cc: iommu@lists.linux.dev; linux-kernel@vger.kernel.org; David Woodhouse > <dwmw2@infradead.org>; Lu Baolu <baolu.lu@linux.intel.com>; Joerg > Roedel <joro@8bytes.org>; Will Deacon <will@kernel.org>; Robin Murphy > <robin.murphy@arm.com>; Tian, Kevin <kevin.tian@intel.com>; Nicolin Chen > <nicolinc@nvidia.com>; Michael Shavit <mshavit@google.com>; Vasant > Hegde <vasant.hegde@amd.com> > Subject: Re: [PATCH v8 0/5] Share sva domains with all devices bound to a > mm > > On Tue, Oct 17, 2023 at 08:47:57AM +0800, Tina Zhang wrote: > > This series is to share sva(shared virtual addressing) domains with > > all devices bound to one mm. > > > > Problem > > ------- > > In the current iommu core code, sva domain is allocated per IOMMU > > group, when device driver is binding a process address space to a > > device (which is handled in iommu_sva_bind_device()). If one than more > > device is bound to the same process address space, there must be more > > than one sva domain instance, with each device having one. In other > > words, the sva domain doesn't share between those devices bound to the > > same process address space, and that leads to two problems: > > 1) device driver has to duplicate sva domains with enqcmd, as those > > sva domains have the same PASID and are relevant to one virtual address > space. > > This makes the sva domain handling complex in device drivers. > > 2) IOMMU driver cannot get sufficient info of the IOMMUs that have > > devices behind them bound to the same virtual address space, when > > handling mmu_notifier_ops callbacks. As a result, IOMMU IOTLB > > invalidation is performed per device instead of per IOMMU, and that > > may lead to superfluous IOTLB invalidation issue, especially in a > > virtualization environment where all devices may be behind one virtual > IOMMU. > > > > Solution > > -------- > > This patch-set tries to fix those two problems by allowing sharing sva > > domains with all devices bound to a mm. To achieve this, a new > > structure pointer is introduced to mm to replace the old PASID field, > > which can keep the info of PASID as well as the corresponding shared sva > domains. > > Besides, function iommu_sva_bind_device() is updated to ensure a new > > sva domain can only be allocated when the old ones cannot work for the > IOMMU. > > With these changes, a device driver can expect one sva domain could > > work for per PASID instance(e.g., enqcmd PASID instance), and > > therefore may get rid of handling sva domain duplication. Besides, > > IOMMU driver (e.g., intel vt-d driver) can get sufficient info (e.g., > > the info of the IOMMUs having their devices bound to one virtual > > address space) when handling mmu_notifier_ops callbacks, to remove the > redundant IOTLB invalidations. > > > > Arguably there shouldn't be more than one sva_domain with the same > > PASID, and in any sane configuration there should be only 1 type of > > IOMMU driver that needs only 1 SVA domain. However, in reality, IOMMUs > > on one platform may not be identical to each other. Thus, attaching a > > sva domain that has been successfully bound to device A behind a IOMMU > > A, to device B behind IOMMU B may get failed due to the difference > > between IOMMU A and IOMMU B. In this case, a new sva domain with the > > same PASID needs to be allocated to work with IOMMU B. That's why we > > need a list to keep sva domains of one PASID. For the platform where > > IOMMUs are compatible to each other, there should be one sva domain in > the list. > > > > v8: > > - CC more people > > - CC iommu@lists.linux.dev mailing list. > > When sending version 7, some issue happened in my CC list and that > caused > > version 7 wasn't sent to iommu@lists.linux.dev. > > - Rebase to v6.6-rc6 and make a few format changes. > > You should based it on Joerg's tree so he can take it without conflcits. > > The conflicts are trivial though (Take Michael's version and switch > mm->pasid with mm_get_enqcmd_pasid(mm)) > > It looks fine, please lets get it in this cycle, the ARM and AMD SVA series > depend on it. The V9 will be based on the next branch of Joerg's tree.
Like Baolu mentioned, besides ARM and AMD SVA series, we also have a VT-d series waiting for it.
Regards, -Tina > > Jason
| |