Messages in this thread | | | Date | Wed, 27 Sep 2023 17:16:23 +0100 | Subject | Re: [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing | From | Robin Murphy <> |
| |
On 27/09/2023 4:40 pm, Jason Gunthorpe wrote: > On Wed, Sep 27, 2023 at 05:24:20PM +0200, Niklas Schnelle wrote: > >> Ok, another update. On trying it out again this problem actually also >> occurs when applying this v12 on top of v6.6-rc3 too. Also I guess >> unlike my prior thinking it probably doesn't occur with >> iommu.forcedac=1 since that still allows IOVAs below 4 GiB and we might >> be the only ones who don't support those. From my point of view this >> sounds like a mlx5_core issue they really should call >> dma_set_mask_and_coherent() before their first call to >> dma_alloc_coherent() not after. So I guess I'll send a v13 of this >> series rebased on iommu/core and with an additional mlx5 patch and then >> let's hope we can get that merged in a way that doesn't leave us with >> broken ConnectX VFs for too long. > > Yes, OK. It definitely sounds wrong that mlx5 is doing dma allocations before > setting it's dma_set_mask_and_coherent(). Please link to this thread > and we can get Leon or Saeed to ack it for Joerg. > > (though wondering why s390 is the only case that ever hit this?)
Probably because most systems happen to be able to satisfy the allocation within the default 32-bit mask - the whole bottom 4GB of IOVA space being reserved is pretty atypical.
TBH it makes me wonder the opposite - how this ever worked on s390 before? And I think the answer to that is "by pure chance", since upon inspection the existing s390_pci_dma_ops implementation appears to pay absolutely no attention to the device's DMA masks whatsoever :(
Robin.
| |