Messages in this thread | | | Date | Thu, 16 Mar 2023 16:17:00 +0800 | Subject | Re: [PATCH v2 5/5] iommu/vt-d: Move PRI handling to IOPF feature path | From | Baolu Lu <> |
| |
On 2023/3/16 15:17, Tian, Kevin wrote: >> From: Lu Baolu <baolu.lu@linux.intel.com> >> Sent: Thursday, March 9, 2023 10:57 AM >> >> @@ -4689,17 +4704,21 @@ static int intel_iommu_disable_iopf(struct device >> *dev) >> { >> struct device_domain_info *info = dev_iommu_priv_get(dev); >> struct intel_iommu *iommu = info->iommu; >> - int ret; >> >> - ret = iommu_unregister_device_fault_handler(dev); >> - if (ret) >> - return ret; >> + if (!info->pri_enabled) >> + return -EINVAL; >> >> - ret = iopf_queue_remove_device(iommu->iopf_queue, dev); >> - if (ret) >> - iommu_register_device_fault_handler(dev, >> iommu_queue_iopf, dev); >> + pci_disable_pri(to_pci_dev(dev)); >> + info->pri_enabled = 0; >> >> - return ret; >> + /* >> + * With pri_enabled checked, unregistering fault handler and >> + * removing device from iopf queue should never fail. >> + */ >> + iommu_unregister_device_fault_handler(dev); >> + iopf_queue_remove_device(iommu->iopf_queue, dev); >> + >> + return 0; >> } > > PCIe spec says that clearing the enable bit doesn't mean in-fly > page requests are completed: > -- > Enable (E) - This field, when set, indicates that the Page Request > Interface is allowed to make page requests. If this field is Clear, > the Page Request Interface is not allowed to issue page requests. > If both this field and the Stopped field are Clear, then the Page > Request Interface will not issue new page requests, but has > outstanding page requests that have been transmitted or are > queued for transmission
Yes. So the iommu driver should drain the in-fly PRQs.
The Intel VT-d implementation drains the PRQs when any PASID is unbound from the iommu domain (see intel_svm_drain_prq()) before reuse. Before disabling iopf, the device driver should unbind pasid and disable sva, so when it comes here, the PRQ should have been drained.
Perhaps I can add below comments to make this clear:
/* * PCIe spec states that by clearing PRI enable bit, the Page * Request Interface will not issue new page requests, but has * outstanding page requests that have been transmitted or are * queued for transmission. This is supposed to be called after * the device driver has stopped DMA, all PASIDs have been * unbound and the outstanding PRQs have been drained. */
Best regards, baolu
| |