lkml.org 
[lkml]   [2021]   [Jun]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [PATCH v2 rdma-next] RDMA/mlx5: Enable Relaxed Ordering by default for kernel ULPs
    On Thu, Jun 24, 2021 at 02:06:46AM +0300, Max Gurtovoy wrote:
    >
    > On 6/9/2021 2:05 PM, Leon Romanovsky wrote:
    > > From: Avihai Horon <avihaih@nvidia.com>
    > >
    > > Relaxed Ordering is a capability that can only benefit users that support
    > > it. All kernel ULPs should support Relaxed Ordering, as they are designed
    > > to read data only after observing the CQE and use the DMA API correctly.
    > >
    > > Hence, implicitly enable Relaxed Ordering by default for kernel ULPs.
    > >
    > > Signed-off-by: Avihai Horon <avihaih@nvidia.com>
    > > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
    > > ---
    > > Changelog:
    > > v2:
    > > * Dropped IB/core patch and set RO implicitly in mlx5 exactly like in
    > > eth side of mlx5 driver.
    > > v1: https://lore.kernel.org/lkml/cover.1621505111.git.leonro@nvidia.com
    > > * Enabled by default RO in IB/core instead of changing all users
    > > v0: https://lore.kernel.org/lkml/20210405052404.213889-1-leon@kernel.org
    > > ---
    > > drivers/infiniband/hw/mlx5/mr.c | 10 ++++++----
    > > drivers/infiniband/hw/mlx5/wr.c | 5 ++++-
    > > 2 files changed, 10 insertions(+), 5 deletions(-)
    > >
    > > diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
    > > index 3363cde85b14..2182e76ae734 100644
    > > --- a/drivers/infiniband/hw/mlx5/mr.c
    > > +++ b/drivers/infiniband/hw/mlx5/mr.c
    > > @@ -69,6 +69,7 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
    > > struct ib_pd *pd)
    > > {
    > > struct mlx5_ib_dev *dev = to_mdev(pd->device);
    > > + bool ro_pci_enabled = pcie_relaxed_ordering_enabled(dev->mdev->pdev);
    > > MLX5_SET(mkc, mkc, a, !!(acc & IB_ACCESS_REMOTE_ATOMIC));
    > > MLX5_SET(mkc, mkc, rw, !!(acc & IB_ACCESS_REMOTE_WRITE));
    > > @@ -78,10 +79,10 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
    > > if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write))
    > > MLX5_SET(mkc, mkc, relaxed_ordering_write,
    > > - !!(acc & IB_ACCESS_RELAXED_ORDERING));
    > > + acc & IB_ACCESS_RELAXED_ORDERING && ro_pci_enabled);
    > > if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read))
    > > MLX5_SET(mkc, mkc, relaxed_ordering_read,
    > > - !!(acc & IB_ACCESS_RELAXED_ORDERING));
    > > + acc & IB_ACCESS_RELAXED_ORDERING && ro_pci_enabled);
    >
    > Jason,
    >
    > If it's still possible to add small change, it will be nice to avoid
    > calculating "acc & IB_ACCESS_RELAXED_ORDERING && ro_pci_enabled" twice.

    The patch is part of for-next now, so feel free to send followup patch.

    Thanks

    diff --git a/drivers/infiniband/hw/mlx5/mr.c b/drivers/infiniband/hw/mlx5/mr.c
    index c1e70c99b70c..c4f246c90c4d 100644
    --- a/drivers/infiniband/hw/mlx5/mr.c
    +++ b/drivers/infiniband/hw/mlx5/mr.c
    @@ -69,7 +69,8 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
    struct ib_pd *pd)
    {
    struct mlx5_ib_dev *dev = to_mdev(pd->device);
    - bool ro_pci_enabled = pcie_relaxed_ordering_enabled(dev->mdev->pdev);
    + bool ro_pci_enabled = acc & IB_ACCESS_RELAXED_ORDERING &&
    + pcie_relaxed_ordering_enabled(dev->mdev->pdev);

    MLX5_SET(mkc, mkc, a, !!(acc & IB_ACCESS_REMOTE_ATOMIC));
    MLX5_SET(mkc, mkc, rw, !!(acc & IB_ACCESS_REMOTE_WRITE));
    @@ -78,11 +79,9 @@ static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
    MLX5_SET(mkc, mkc, lr, 1);

    if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_write))
    - MLX5_SET(mkc, mkc, relaxed_ordering_write,
    - (acc & IB_ACCESS_RELAXED_ORDERING) && ro_pci_enabled);
    + MLX5_SET(mkc, mkc, relaxed_ordering_write, ro_pci_enabled);
    if (MLX5_CAP_GEN(dev->mdev, relaxed_ordering_read))
    - MLX5_SET(mkc, mkc, relaxed_ordering_read,
    - (acc & IB_ACCESS_RELAXED_ORDERING) && ro_pci_enabled);
    + MLX5_SET(mkc, mkc, relaxed_ordering_read, ro_pci_enabled);

    MLX5_SET(mkc, mkc, pd, to_mpd(pd)->pdn);
    MLX5_SET(mkc, mkc, qpn, 0xffffff);
    (END)

    >
    >

    \
     
     \ /
      Last update: 2021-06-24 08:39    [W:7.447 / U:0.760 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site