lkml.org 
[lkml]   [2023]   [Jul]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH v2] refscale: Fix use of uninitalized wait_queue_head_t
From
On 7/7/23 15:54, Paul E. McKenney wrote:
> On Fri, Jul 07, 2023 at 01:53:55PM -0400, Waiman Long wrote:
>> It was found that running the refscale test might crash the kernel once
>> in a while with the following error:
>>
>> [ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
>> [ 8569.952900] #PF: supervisor read access in kernel mode
>> [ 8569.952902] #PF: error_code(0x0000) - not-present page
>> [ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
>> [ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
>> [ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
>> [ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
>> :
>> [ 8569.952940] Call Trace:
>> [ 8569.952941] <TASK>
>> [ 8569.952944] ref_scale_reader+0x380/0x4a0 [refscale]
>> [ 8569.952959] kthread+0x10e/0x130
>> [ 8569.952966] ret_from_fork+0x1f/0x30
>> [ 8569.952973] </TASK>
>>
>> This is likely caused by the fact that init_waitqueue_head() is
>> called after the ref_scale_reader kthread is created. The kthread
>> can potentially try to use the waitqueue head before it is properly
>> initialized. The crash happened at
>>
>> static inline void __add_wait_queue(...)
>> {
>> :
>> if (!(wq->flags & WQ_FLAG_PRIORITY)) <=== Crash here
>>
>> The offset of flags from list_head entry in wait_queue_entry is -0x18. If
>> reader_tasks[i].wq.head.next is NULL as allocated reader_task structure
>> is zero initialized, the instruction will try to access address
>> 0xffffffffffffffe8 which is the fault address listed above.
>>
>> Fix this by initializing the waitqueue head first before kthread
>> creation.
>>
>> Fixes: 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")
>> Signed-off-by: Waiman Long <longman@redhat.com>
>> Reviewed-by: Qiuxu Zhuo <qiuxu.zhuo@intel.com>
>> Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>
> Queued and pushed, thank you all!
>
> As always, I could not resist wordsmithing the commit log, please see
> below.

Thanks for the wordsmithing. I don't mind at all.

Cheers,
Longman

> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> commit 933d3bf8f96d7cedf78081030e004d23aee2b56c
> Author: Waiman Long <longman@redhat.com>
> Date: Fri Jul 7 13:53:55 2023 -0400
>
> refscale: Fix uninitalized use of wait_queue_head_t
>
> Running the refscale test occasionally crashes the kernel with the
> following error:
>
> [ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
> [ 8569.952900] #PF: supervisor read access in kernel mode
> [ 8569.952902] #PF: error_code(0x0000) - not-present page
> [ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
> [ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
> [ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
> [ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
> :
> [ 8569.952940] Call Trace:
> [ 8569.952941] <TASK>
> [ 8569.952944] ref_scale_reader+0x380/0x4a0 [refscale]
> [ 8569.952959] kthread+0x10e/0x130
> [ 8569.952966] ret_from_fork+0x1f/0x30
> [ 8569.952973] </TASK>
>
> The likely cause is that init_waitqueue_head() is called after the call to
> the torture_create_kthread() function that creates the ref_scale_reader
> kthread. Although this init_waitqueue_head() call will very likely
> complete before this kthread is created and starts running, it is
> possible that the calling kthread will be delayed between the calls to
> torture_create_kthread() and init_waitqueue_head(). In this case, the
> new kthread will use the waitqueue head before it is properly initialized,
> which is not good for the kernel's health and well-being.
>
> The above crash happened here:
>
> static inline void __add_wait_queue(...)
> {
> :
> if (!(wq->flags & WQ_FLAG_PRIORITY)) <=== Crash here
>
> The offset of flags from list_head entry in wait_queue_entry is
> -0x18. If reader_tasks[i].wq.head.next is NULL as allocated reader_task
> structure is zero initialized, the instruction will try to access address
> 0xffffffffffffffe8, which is exactly the fault address listed above.
>
> This commit therefore invokes init_waitqueue_head() before creating
> the kthread.
>
> Fixes: 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")
> Signed-off-by: Waiman Long <longman@redhat.com>
> Reviewed-by: Qiuxu Zhuo <qiuxu.zhuo@intel.com>
> Reviewed-by: Davidlohr Bueso <dave@stgolabs.net>
> Acked-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
>
> diff --git a/kernel/rcu/refscale.c b/kernel/rcu/refscale.c
> index 1970ce5f22d4..71d138573856 100644
> --- a/kernel/rcu/refscale.c
> +++ b/kernel/rcu/refscale.c
> @@ -1107,12 +1107,11 @@ ref_scale_init(void)
> VERBOSE_SCALEOUT("Starting %d reader threads", nreaders);
>
> for (i = 0; i < nreaders; i++) {
> + init_waitqueue_head(&reader_tasks[i].wq);
> firsterr = torture_create_kthread(ref_scale_reader, (void *)i,
> reader_tasks[i].task);
> if (torture_init_error(firsterr))
> goto unwind;
> -
> - init_waitqueue_head(&(reader_tasks[i].wq));
> }
>
> // Main Task
>

\
 
 \ /
  Last update: 2023-07-07 22:19    [W:0.027 / U:2.360 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site