Messages in this thread | | | Subject | Re: [PATCH rdma-next 00/10] Enable relaxed ordering for ULPs | From | Max Gurtovoy <> | Date | Sun, 11 Apr 2021 13:09:03 +0300 |
| |
On 4/6/2021 2:53 PM, Jason Gunthorpe wrote: > On Tue, Apr 06, 2021 at 08:09:43AM +0300, Leon Romanovsky wrote: >> On Tue, Apr 06, 2021 at 10:37:38AM +0800, Honggang LI wrote: >>> On Mon, Apr 05, 2021 at 08:23:54AM +0300, Leon Romanovsky wrote: >>>> From: Leon Romanovsky <leonro@nvidia.com> >>>> >>>> From Avihai, >>>> >>>> Relaxed Ordering is a PCIe mechanism that relaxes the strict ordering >>>> imposed on PCI transactions, and thus, can improve performance. >>>> >>>> Until now, relaxed ordering could be set only by user space applications >>>> for user MRs. The following patch series enables relaxed ordering for the >>>> kernel ULPs as well. Relaxed ordering is an optional capability, and as >>>> such, it is ignored by vendors that don't support it. >>>> >>>> The following test results show the performance improvement achieved >>> Did you test this patchset with CPU does not support relaxed ordering? >> I don't think so, the CPUs that don't support RO are Intel's fourth/fifth-generation >> and they are not interesting from performance point of view. >> >>> We observed significantly performance degradation when run perftest with >>> relaxed ordering enabled over old CPU. >>> >>> https://github.com/linux-rdma/perftest/issues/116 >> The perftest is slightly different, but you pointed to the valid point. >> We forgot to call pcie_relaxed_ordering_enabled() before setting RO bit >> and arguably this was needed to be done in perftest too. > No, the PCI device should not have the RO bit set in this situation. > It is something mlx5_core needs to do. We can't push this into > applications.
pcie_relaxed_ordering_enabled is called in drivers/net/ethernet/mellanox/mlx5/core/en_common.c so probably need to move it to
mlx5_core in this series.
> > There should be no performance difference from asking for > IBV_ACCESS_RELAXED_ORDERING when RO is disabled at the PCI config and > not asking for it at all. > > Either the platform has working relaxed ordering that gives a > performance gain and the RO config spec bit should be set, or it > doesn't and the bit should be clear.
is this the case today ?
> > This is not something to decide in userspace, or in RDMA. At worst it > becomes another platform specific PCI tunable people have to set. > > I thought the old haswell systems were quirked to disable RO globally > anyhow? > > Jason
| |