lkml.org 
[lkml]   [2023]   [May]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRE: [PATCH V1] block: Fix null pointer dereference issue on struct io_cq
Date
Hi,

-----Original Message-----
From: Damien Le Moal <dlemoal@kernel.org>
Sent: Wednesday, May 17, 2023 3:02 PM
To: Christoph Hellwig <hch@infradead.org>
Cc: Yu Kuai <yukuai1@huaweicloud.com>; Pradeep Pragallapati (QUIC) <quic_pragalla@quicinc.com>; axboe@kernel.dk; linux-block@vger.kernel.org; linux-kernel@vger.kernel.org; yukuai (C) <yukuai3@huawei.com>
Subject: Re: [PATCH V1] block: Fix null pointer dereference issue on struct io_cq

On 5/17/23 18:21, Christoph Hellwig wrote:
> On Wed, May 17, 2023 at 06:20:19PM +0900, Damien Le Moal wrote:
>> twice for the same icq. The missing rcu lock in ioc_exit_icqs()
>> already was in itself a bug, and the missing flag check is another.
>
> spinlocks imply a rcu critical section, no need to duplicate it.

Right. And I misread the code. As Yu said, given that ioc_exit_icqs() iterates the list of icqs under ioc->lock and the ioc is removed from that list under the same lock, ioc_exit_icqs() should never see an icq that went through ioc_destroy_icq()...
Very weird.

This weird can be possible
1. updating icq_hint which is annotated as __rcu type without RCU-protected context in ioc_destroy_icq().
Moreover, this was taken care in else part of ioc_release_fn() by rcu_read_lock/unlock() but missed in if statement which can lead to this weird.

2. extracting icq from hlist/list elements are done using rcu locks protected in ioc_clear_queue() but same was not at ioc_exit_icqs().

So, far we have seen 10+ instances of this crash on 6.1 kernel during stability testing (Involves IO, reboots, device suspend/resume, and few more).
With the V1 patch, we didn't observe the issue for at least 48hrs+ of stability testing.


--
Damien Le Moal
Western Digital Research

\
 
 \ /
  Last update: 2023-05-18 11:29    [W:0.057 / U:1.384 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site