lkml.org 
[lkml]   [2022]   [May]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: [PATCH v6 00/21] Userspace P2PDMA with O_DIRECT NVMe devices
    Date
    On 4/7/22 08:46, Logan Gunthorpe wrote:
    > Hi,
    >
    > This patchset continues my work to add userspace P2PDMA access using
    > O_DIRECT NVMe devices. This posting contains some minor fixes and a
    > rebase onto v5.18-rc1 which contains cleanup from Christoph around
    > free_zone_device_page() that helps to enable this patchset. The
    > previous posting was here[1].
    >
    > The patchset enables userspace P2PDMA by allowing userspace to mmap()
    > allocated chunks of the CMB. The resulting VMA can be passed only
    > to O_DIRECT IO on NVMe backed files or block devices. A flag is added
    > to GUP() in Patch <>, then Patches <> through <> wire this flag up based
    > on whether the block queue indicates P2PDMA support. Patches <>
    > through <> enable the CMB to be mapped into userspace by mmaping
    > the nvme char device.
    >
    > This is relatively straightforward, however the one significant
    > problem is that, presently, pci_p2pdma_map_sg() requires a homogeneous
    > SGL with all P2PDMA pages or all regular pages. Enhancing GUP to
    > support enforcing this rule would require a huge hack that I don't
    > expect would be all that pallatable. So the first 13 patches add
    > support for P2PDMA pages to dma_map_sg[table]() to the dma-direct
    > and dma-iommu implementations. Thus systems without an IOMMU plus
    > Intel and AMD IOMMUs are supported. (Other IOMMU implementations would
    > then be unsupported, notably ARM and PowerPC but support would be added
    > when they convert to dma-iommu).
    >
    > dma_map_sgtable() is preferred when dealing with P2PDMA memory as it
    > will return -EREMOTEIO when the DMA device cannot map specific P2PDMA
    > pages based on the existing rules in calc_map_type_and_dist().
    >
    > The other issue is dma_unmap_sg() needs a flag to determine whether a
    > given dma_addr_t was mapped regularly or as a PCI bus address. To allow
    > this, a third flag is added to the page_link field in struct
    > scatterlist. This effectively means support for P2PDMA will now depend
    > on CONFIG_64BIT.
    >
    > Feedback welcome.
    >
    > This series is based on v5.18-rc1. A git branch is available here:
    >
    > https://github.com/sbates130272/linux-p2pmem/ p2pdma_user_cmb_v6
    >
    > Thanks,
    >
    > Logan
    >
    > [1] lkml.kernel.org/r/20220128002614.6136-1-logang@deltatee.com
    >
    > --


    Do you have any plans to re-spin this ?

    -ck


    \
     
     \ /
      Last update: 2022-05-17 00:33    [W:4.141 / U:0.156 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site