Messages in this thread | | | Date | Sun, 10 Mar 2024 21:13:18 -0700 | Subject | Re: [PATCH v3] RDMA/cm: add timeout to cm_destroy_id wait | From | Manjunath Patil <> |
| |
On 3/10/24 4:17 AM, Leon Romanovsky wrote: > On Fri, Mar 08, 2024 at 10:33:23PM -0800, Manjunath Patil wrote: >> Add timeout to cm_destroy_id, so that userspace can trigger any data >> collection that would help in analyzing the cause of delay in destroying >> the cm_id. >> >> New noinline function helps dtrace/ebpf programs to hook on to it. >> Existing functionality isn't changed except triggering a probe-able new >> function at every timeout interval. >> >> We have seen cases where CM messages stuck with MAD layer (either due to >> software bug or faulty HCA), leading to cm_id getting stuck in the >> following call stack. This patch helps in resolving such issues faster. >> >> kernel: ... INFO: task XXXX:56778 blocked for more than 120 seconds. >> ... >> Call Trace: >> __schedule+0x2bc/0x895 >> schedule+0x36/0x7c >> schedule_timeout+0x1f6/0x31f >> ? __slab_free+0x19c/0x2ba >> wait_for_completion+0x12b/0x18a >> ? wake_up_q+0x80/0x73 >> cm_destroy_id+0x345/0x610 [ib_cm] >> ib_destroy_cm_id+0x10/0x20 [ib_cm] >> rdma_destroy_id+0xa8/0x300 [rdma_cm] >> ucma_destroy_id+0x13e/0x190 [rdma_ucm] >> ucma_write+0xe0/0x160 [rdma_ucm] >> __vfs_write+0x3a/0x16d >> vfs_write+0xb2/0x1a1 >> ? syscall_trace_enter+0x1ce/0x2b8 >> SyS_write+0x5c/0xd3 >> do_syscall_64+0x79/0x1b9 >> entry_SYSCALL_64_after_hwframe+0x16d/0x0 >> >> Orabug: 36280065 > > Not related to the upstream. > >> >> Signed-off-by: Manjunath Patil <manjunath.b.patil@oracle.com> >> --- >> v3: >> - added #define >> >> v2: >> - removed sysctl related code >> >> drivers/infiniband/core/cm.c | 20 +++++++++++++++++++- >> 1 file changed, 19 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c >> index ff58058aeadc..793103cf8152 100644 >> --- a/drivers/infiniband/core/cm.c >> +++ b/drivers/infiniband/core/cm.c >> @@ -34,6 +34,7 @@ MODULE_AUTHOR("Sean Hefty"); >> MODULE_DESCRIPTION("InfiniBand CM"); >> MODULE_LICENSE("Dual BSD/GPL"); >> >> +#define CM_DESTORY_ID_WAIT_TIMEOUT 10000 /* msecs */ > > CM_DESTORY_ID_WAIT_TIMEOUT -> CM_DESTROY_ID_WAIT_TIMEOUT > > Fixed and applied. Thank you fixing and applying. Sorry for Orabug and not being careful about the typo.
-Manjunath > > Thanks
| |