lkml.org 
[lkml]   [2019]   [Feb]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [LSF/MM TOPIC] Discuss least bad options for resolving longterm-GUP usage by RDMA
    On Thu, Feb 07, 2019 at 03:54:58PM -0800, Dan Williams wrote:
    > On Thu, Feb 7, 2019 at 9:17 AM Jason Gunthorpe <jgg@ziepe.ca> wrote:
    > >
    > > Insisting to run RDMA & DAX without ODP and building an elaborate
    > > revoke mechanism to support non-ODP HW is inherently baroque.
    > >
    > > Use the HW that supports ODP.
    > >
    > > Since no HW can do disable of a MR, the escalation path is SIGKILL
    > > which makes it a non-production toy.
    > >
    > > What you keep missing is that for people doing this - the RDMA is a
    > > critical compoment of the system, you can't just say the kernel will
    > > randomly degrade/kill RDMA processes - that is a 'toy' configuration
    > > that is not production worthy.
    > >
    > > Especially since this revoke idea is basically a DOS engine for the
    > > RDMA protocol if another process can do actions to trigger revoke. Now
    > > we have a new class of security problems. (again, screams non
    > > production toy)
    > >
    > > The only production worthy way is to have the FS be a partner in
    > > making this work without requiring revoke, so the critical RDMA
    > > traffic can operate safely.
    > >
    > > Otherwise we need to stick to ODP.
    >
    > Thanks for this it clears a lot of things up for me...
    >
    > ...but this statement:
    >
    > > The only production worthy way is to have the FS be a partner in
    > > making this work without requiring revoke, so the critical RDMA
    > > traffic can operate safely.
    >
    > ...belies a path forward. Just swap out "FS be a partner" with "system
    > administrator be a partner". In other words, If the RDMA stack can't
    > tolerate an MR being disabled then the administrator needs to actively
    > disable the paths that would trigger it. Turn off reflink, don't
    > truncate, avoid any future FS feature that might generate unwanted
    > lease breaks. We would need to make sure that lease notifications
    > include the information to identify the lease breaker to debug escapes
    > that might happen, but it is a solution that can be qualified to not
    > lease break. In any event, this lets end users pick their filesystem
    > (modulo RDMA incompatible features), provides an enumeration of lease
    > break sources in the kernel, and opens up FS-DAX to a wider array of
    > RDMA adapters. In general this is what Linux has historically done,
    > give end users technology freedom.

    To back off the details of this thread a bit...

    The details of limitations imposed and how they would be tracked within the
    kernel would be a great thing to discuss face to face. Hence the reason for my
    proposal as a topic.

    Ira

    \
     
     \ /
      Last update: 2019-02-08 02:44    [W:4.887 / U:0.028 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site