lkml.org 
[lkml]   [2019]   [Feb]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA
    On Thu, Feb 07, 2019 at 09:24:05AM -0800, Matthew Wilcox wrote:
    > On Thu, Feb 07, 2019 at 11:25:35AM -0500, Doug Ledford wrote:
    > > * Really though, as I said in my email to Tom Talpey, this entire
    > > situation is simply screaming that we are doing DAX networking wrong.
    > > We shouldn't be writing the networking code once in every single
    > > application that wants to do this. If we had a memory segment that we
    > > shared from server to client(s), and in that memory segment we
    > > implemented a clustered filesystem, then applications would simply mmap
    > > local files and be done with it. If the file needed to move, the kernel
    > > would update the mmap in the application, done. If you ask me, it is
    > > the attempt to do this the wrong way that is resulting in all this
    > > heartache. That said, for today, my recommendation would be to require
    > > ODP hardware for XFS filesystem with the DAX option, but allow ext2
    > > filesystems to mount DAX filesystems on non-ODP hardware, and go in and
    > > modify the ext2 filesystem so that on DAX mounts, it disables hole punch
    > > and ftrunctate any time they would result in the forced removal of an
    > > established mmap.
    >
    > I agree that something's wrong, but I think the fundamental problem is
    > that there's no concept in RDMA of having an STag for storage rather
    > than for memory.
    >
    > Imagine if we could associate an STag with a file descriptor on the
    > server. The client could then perform an RDMA to that STag. On the
    > server, we'd need lots of smarts in the card and in the OS to know how
    > to treat that packet on arrival -- depending on what the file descriptor
    > referred to, it might only have to write into the page cache, or it
    > might set up an NVMe DMA, or it might resolve the underlying physical
    > address and DMA directly to an NV-DIMM.

    I think you just described ODP MRs.

    Jason

    \
     
     \ /
      Last update: 2019-02-07 18:27    [W:2.162 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site