lkml.org 
[lkml]   [2022]   [Apr]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
SubjectRe: [PATCH] f2fs: avoid deadlock in gc thread under low memory
From
On 4/14/22 01:00, Jaegeuk Kim wrote:
> On 04/13, Rokudo Yan wrote:
>> There is a potential deadlock in gc thread may happen
>> under low memory as below:
>>
>> gc_thread_func
>> -f2fs_gc
>> -do_garbage_collect
>> -gc_data_segment
>> -move_data_block
>> -set_page_writeback(fio.encrypted_page);
>> -f2fs_submit_page_write
>> as f2fs_submit_page_write try to do io merge when possible, so the
>> encrypted_page is marked PG_writeback but may not submit to block
>> layer immediately, if system enter low memory when gc thread try
>> to move next data block, it may do direct reclaim and enter fs layer
>> as below:
>> -move_data_block
>> -f2fs_grab_cache_page(index=?, for_write=false)
>> -grab_cache_page
>> -find_or_create_page
>> -pagecache_get_page
>> -__page_cache_alloc -- __GFP_FS is set
>> -alloc_pages_node
>> -__alloc_pages
>> -__alloc_pages_slowpath
>> -__alloc_pages_direct_reclaim
>> -__perform_reclaim
>> -try_to_free_pages
>> -do_try_to_free_pages
>> -shrink_zones
>> -mem_cgroup_soft_limit_reclaim
>> -mem_cgroup_soft_reclaim
>> -mem_cgroup_shrink_node
>> -shrink_node_memcg
>> -shrink_list
>> -shrink_inactive_list
>> -shrink_page_list
>> -wait_on_page_writeback -- the page is marked
>> writeback during previous move_data_block call
>>
>> the gc thread wait for the encrypted_page writeback complete,
>> but as gc thread held sbi->gc_lock, the writeback & sync thread
>> may blocked waiting for sbi->gc_lock, so the bio contain the
>> encrypted_page may nerver submit to block layer and complete the
>> writeback, which cause deadlock. To avoid this deadlock condition,
>> we mark the gc thread with PF_MEMALLOC_NOFS flag, then it will nerver
>> enter fs layer when try to alloc cache page during move_data_block.
>>
>> Signed-off-by: Rokudo Yan <wu-yan@tcl.com>
>> ---
>> fs/f2fs/gc.c | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
>> index e020804f7b07..cc71f77b98c8 100644
>> --- a/fs/f2fs/gc.c
>> +++ b/fs/f2fs/gc.c
>> @@ -38,6 +38,12 @@ static int gc_thread_func(void *data)
>>
>> wait_ms = gc_th->min_sleep_time;
>>
>> + /*
>> + * Make sure that no allocations from gc thread will ever
>> + * recurse to the fs layer to avoid deadlock as it will
>> + * hold sbi->gc_lock during garbage collection
>> + */
>> + memalloc_nofs_save();
>
> I think this cannot cover all the f2fs_gc() call cases. Can we just avoid by:
>
> --- a/fs/f2fs/gc.c
> +++ b/fs/f2fs/gc.c
> @@ -1233,7 +1233,7 @@ static int move_data_block(struct inode *inode, block_t bidx,
> CURSEG_ALL_DATA_ATGC : CURSEG_COLD_DATA;
>
> /* do not read out */
> - page = f2fs_grab_cache_page(inode->i_mapping, bidx, false);
> + page = f2fs_grab_cache_page(inode->i_mapping, bidx, true);
> if (!page)
> return -ENOMEM;
>
> Thanks,
>
>> set_freezable();
>> do {
>> bool sync_mode, foreground = false;
>> --
>> 2.25.1

Hi, Jaegeuk

I'm not sure if any other case may trigger the issue, but the stack
traces I have caught so far are all the same as below:

f2fs_gc-253:12 D 226966.808196 572 302561 150976 0x1200840 0x0 572
237207473347056
<ffffff889d88668c> __switch_to+0x134/0x150
<ffffff889e764b6c> __schedule+0xd5c/0x1100
<ffffff889e76554c> io_schedule+0x90/0xc0
<ffffff889d9fb880> wait_on_page_bit+0x194/0x208
<ffffff889da167b4> shrink_page_list+0x62c/0xe74
<ffffff889da1d354> shrink_inactive_list+0x2c0/0x698
<ffffff889da181f4> shrink_node_memcg+0x3dc/0x97c
<ffffff889da17d44> mem_cgroup_shrink_node+0x144/0x218
<ffffff889da6610c> mem_cgroup_soft_limit_reclaim+0x188/0x47c
<ffffff889da17a40> do_try_to_free_pages+0x204/0x3a0
<ffffff889da176c8> try_to_free_pages+0x35c/0x4d0
<ffffff889da05d60> __alloc_pages_nodemask+0x7a4/0x10d0
<ffffff889d9fc82c> pagecache_get_page+0x184/0x2ec
<ffffff889dbf8860> do_garbage_collect+0xfe0/0x2828
<ffffff889dbf7434> f2fs_gc+0x4a0/0x8ec
<ffffff889dbf6bf4> gc_thread_func+0x240/0x4d4
<ffffff889d8de9b0> kthread+0x17c/0x18c
<ffffff889d88567c> ret_from_fork+0x10/0x18

Thanks
yanwu

\
 
 \ /
  Last update: 2022-04-14 03:55    [W:0.134 / U:0.700 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site