lkml.org 
[lkml]   [2022]   [Jul]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH net-next v2 0/6] net/smc: Introduce virtually contiguous buffers for SMC-R
From


On 14.07.22 11:43, Wen Gu wrote:
> On long-running enterprise production servers, high-order contiguous
> memory pages are usually very rare and in most cases we can only get
> fragmented pages.
>
> When replacing TCP with SMC-R in such production scenarios, attempting
> to allocate high-order physically contiguous sndbufs and RMBs may result
> in frequent memory compaction, which will cause unexpected hung issue
> and further stability risks.
>
> So this patch set is aimed to allow SMC-R link group to use virtually
> contiguous sndbufs and RMBs to avoid potential issues mentioned above.
> Whether to use physically or virtually contiguous buffers can be set
> by sysctl smcr_buf_type.
>
> Note that using virtually contiguous buffers will bring an acceptable
> performance regression, which can be mainly divided into two parts:
>
> 1) regression in data path, which is brought by additional address
> translation of sndbuf by RNIC in Tx. But in general, translating
> address through MTT is fast. According to qperf test, this part
> regression is basically less than 10% in latency and bandwidth.
> (see patch 5/6 for details)
>
> 2) regression in buffer initialization and destruction path, which is
> brought by additional MR operations of sndbufs. But thanks to link
> group buffer reuse mechanism, the impact of this kind of regression
> decreases as times of buffer reuse increases.
>
> Patch set overview:
> - Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
> operation, which will reduce overhead on the data path, especially
> when using virtually contiguous buffers;
> - Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
> of buffers in new created link group;
> - Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
> including buffer creation, destruction, MR operation and access;
> - patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
>
> v1->v2:
> - Patch 5/6 fixes build issue on 32bit;
> - Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
>
> Guangguan Wang (2):
> net/smc: remove redundant dma sync ops
> net/smc: optimize for smc_sndbuf_sync_sg_for_device and
> smc_rmb_sync_sg_for_cpu
>
> Wen Gu (4):
> net/smc: Introduce a sysctl for setting SMC-R buffer type
> net/smc: Use sysctl-specified types of buffers in new link group
> net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
> net/smc: Extend SMC-R link group netlink attribute
>
> Documentation/networking/smc-sysctl.rst | 13 ++
> include/net/netns/smc.h | 1 +
> include/uapi/linux/smc.h | 1 +
> net/smc/af_smc.c | 68 +++++++--
> net/smc/smc_clc.c | 8 +-
> net/smc/smc_clc.h | 2 +-
> net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
> net/smc/smc_core.h | 20 ++-
> net/smc/smc_ib.c | 44 +++++-
> net/smc/smc_ib.h | 2 +
> net/smc/smc_llc.c | 33 +++--
> net/smc/smc_rx.c | 92 +++++++++---
> net/smc/smc_sysctl.c | 11 ++
> net/smc/smc_tx.c | 10 +-
> 14 files changed, 404 insertions(+), 147 deletions(-)
>
It looks good for us. Thank you!
Acked-by: Wenjia Zhang <wenjia@linux.ibm.com>

\
 
 \ /
  Last update: 2022-07-20 19:40    [W:0.194 / U:0.636 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site