lkml.org 
[lkml]   [2021]   [Nov]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
From
Subject[PATCH 05/12] habanalabs: fix race condition in multi CS completion
Date
From: Dani Liberman <dliberman@habana.ai>

Race example scenario:
1. User have 2 threads that waits on multi CS:
- thread_0 waits on QID 0 and uses multi CS context 0.
- thread_1 waits on QID 1 and uses multi CS context 1.
2. thread_1 got completion and release multi CS context 1.
3. CS related to multi CS of thread_0 starts executing
complete_multi_cs function, the first iteration of the loop
completes the multi CS of thread_0, hence multi CS context 0
is released.
4. thread_1 waits on QID 1 and uses multi CS context 0.
5. thread_0 waits on QID 0 and uses multi CS context 1.
6. The second iterattion of the loop (from step 3) starts, which
means, start checking multi CS context 1:
- multi CS contetxt is being used by thread_0 waiting on QID 0.
- The fence of the CS (still CS from step 3) has QID map the same
as the multi CS context 1.
- multi CS context 1 (thread_0) gets completion on CS that triggered
already thread_0 (with multi CS context 0) and is no longer
being waited on.

Fixed by exiting the loop in complete_multi_cs after getting completion

Signed-off-by: Dani Liberman <dliberman@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
---
drivers/misc/habanalabs/common/command_submission.c | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/drivers/misc/habanalabs/common/command_submission.c b/drivers/misc/habanalabs/common/command_submission.c
index c1fd4ba14c60..4e893364a3cc 100644
--- a/drivers/misc/habanalabs/common/command_submission.c
+++ b/drivers/misc/habanalabs/common/command_submission.c
@@ -545,6 +545,13 @@ static void complete_multi_cs(struct hl_device *hdev, struct hl_cs *cs)
* mcs fences.
*/
fence->mcs_handling_done = true;
+ /*
+ * Since CS (and its related fence) can be associated with only one
+ * multi CS context, once it triggered multi CS completion no need to
+ * continue checking other multi CS contexts.
+ */
+ spin_unlock(&mcs_compl->lock);
+ break;
}

spin_unlock(&mcs_compl->lock);
--
2.25.1
\
 
 \ /
  Last update: 2021-11-28 20:37    [W:0.977 / U:0.136 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site