Messages in this thread | | | Subject | Re: [PATCH V1] block: Fix use-after-free while iterating over requests | From | Hannes Reinecke <> | Date | Mon, 30 Nov 2020 08:04:33 +0100 |
| |
On 11/26/20 5:49 PM, John Garry wrote: > On 26/11/2020 16:27, Bart Van Assche wrote: >> On 11/26/20 7:02 AM, Pradeep P V K wrote: >>> Observes below crash while accessing (use-after-free) request queue >>> member of struct request. >>> >>> 191.784789: <2> Unable to handle kernel paging request at virtual >>> address ffffff81429a4440 >>> ... >>> 191.786174: <2> CPU: 3 PID: 213 Comm: kworker/3:1H Tainted: G S >>> O 5.4.61-qgki-debug-ge45de39 #1 >>> ... >>> 191.786226: <2> Workqueue: kblockd blk_mq_timeout_work >>> 191.786242: <2> pstate: 20c00005 (nzCv daif +PAN +UAO) >>> 191.786261: <2> pc : bt_for_each+0x114/0x1a4 >>> 191.786274: <2> lr : bt_for_each+0xe0/0x1a4 >>> ... >>> 191.786494: <2> Call trace: >>> 191.786507: <2> bt_for_each+0x114/0x1a4 >>> 191.786519: <2> blk_mq_queue_tag_busy_iter+0x60/0xd4 >>> 191.786532: <2> blk_mq_timeout_work+0x54/0xe8 >>> 191.786549: <2> process_one_work+0x2cc/0x568 >>> 191.786562: <2> worker_thread+0x28c/0x518 >>> 191.786577: <2> kthread+0x160/0x170 >>> 191.786594: <2> ret_from_fork+0x10/0x18 >>> 191.786615: <2> Code: 0b080148 f9404929 f8685921 b4fffe01 (f9400028) >>> 191.786630: <2> ---[ end trace 0f1f51d79ab3f955 ]--- >>> 191.786643: <2> Kernel panic - not syncing: Fatal exception >>> >>> Fix this by updating the freed request with NULL. >>> This could avoid accessing the already free request from other >>> contexts while iterating over the requests. >>> >>> Signed-off-by: Pradeep P V K <ppvk@codeaurora.org> >>> --- >>> block/blk-mq.c | 1 + >>> block/blk-mq.h | 1 + >>> 2 files changed, 2 insertions(+) >>> >>> diff --git a/block/blk-mq.c b/block/blk-mq.c >>> index 55bcee5..9996cb1 100644 >>> --- a/block/blk-mq.c >>> +++ b/block/blk-mq.c >>> @@ -492,6 +492,7 @@ static void __blk_mq_free_request(struct request >>> *rq) >>> blk_crypto_free_request(rq); >>> blk_pm_mark_last_busy(rq); >>> + hctx->tags->rqs[rq->tag] = NULL; >>> rq->mq_hctx = NULL; >>> if (rq->tag != BLK_MQ_NO_TAG) >>> blk_mq_put_tag(hctx->tags, ctx, rq->tag); >>> diff --git a/block/blk-mq.h b/block/blk-mq.h >>> index a52703c..8747bf1 100644 >>> --- a/block/blk-mq.h >>> +++ b/block/blk-mq.h >>> @@ -224,6 +224,7 @@ static inline int __blk_mq_active_requests(struct >>> blk_mq_hw_ctx *hctx) >>> static inline void __blk_mq_put_driver_tag(struct blk_mq_hw_ctx *hctx, >>> struct request *rq) >>> { >>> + hctx->tags->rqs[rq->tag] = NULL; >>> blk_mq_put_tag(hctx->tags, rq->mq_ctx, rq->tag); >>> rq->tag = BLK_MQ_NO_TAG; >> >> Is this perhaps a block driver bug instead of a block layer core bug? If >> this would be a block layer core bug, it would have been reported before. > > Isn't this the same issue which as been reported many times: > > https://lore.kernel.org/linux-block/20200820180335.3109216-1-ming.lei@redhat.com/ > > > https://lore.kernel.org/linux-block/8376443a-ec1b-0cef-8244-ed584b96fa96@huawei.com/ > > > But I never saw a crash, just kasan report. > And if that above were a concern, I would have thought one would need to use a WRITE_ONCE() here; otherwise we might have a race condition where other CPUs still see the old value, no?
Cheers,
Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer
| |