lkml.org 
[lkml]   [2021]   [Apr]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
SubjectRe: [PATCH v2] ext4: Fix bug on in ext4_es_cache_extent as ext4_split_extent_at failed
From
Date


On 2021/4/21 17:21, Jan Kara wrote:
> On Tue 20-04-21 22:25:18, Ye Bin wrote:
>> We got follow bug_on when run fsstress with injecting IO fault:
>> [130747.323114] kernel BUG at fs/ext4/extents_status.c:762!
>> [130747.323117] Internal error: Oops - BUG: 0 [#1] SMP
> Which BUG_ON is that please? I don't see any relevant one near that line in
> current upstream kernel...
ext4_es_cache_extent:
......
if (!len)
return;

BUG_ON(end < lblk); --> BUG_ON at here.
......

>> ......
>> [130747.334329] Call trace:
>> [130747.334553] ext4_es_cache_extent+0x150/0x168 [ext4]
>> [130747.334975] ext4_cache_extents+0x64/0xe8 [ext4]
>> [130747.335368] ext4_find_extent+0x300/0x330 [ext4]
>> [130747.335759] ext4_ext_map_blocks+0x74/0x1178 [ext4]
>> [130747.336179] ext4_map_blocks+0x2f4/0x5f0 [ext4]
>> [130747.336567] ext4_mpage_readpages+0x4a8/0x7a8 [ext4]
>> [130747.336995] ext4_readpage+0x54/0x100 [ext4]
>> [130747.337359] generic_file_buffered_read+0x410/0xae8
>> [130747.337767] generic_file_read_iter+0x114/0x190
>> [130747.338152] ext4_file_read_iter+0x5c/0x140 [ext4]
>> [130747.338556] __vfs_read+0x11c/0x188
>> [130747.338851] vfs_read+0x94/0x150
>> [130747.339110] ksys_read+0x74/0xf0
>>
>> If call ext4_ext_insert_extent failed but new extent already inserted, we just
>> update "ex->ee_len = orig_ex.ee_len", this will lead to extent overlap, then
>> cause bug on when cache extent.
>> If call ext4_ext_insert_extent failed don't update ex->ee_len with old value.
>> Maybe there will lead to block leak, but it can be fixed by fsck later.
> Thanks for the analysis and the patch but the fact is that the extent tree
> is going to be corrupted one way or the other. And I think other disk
> corruptions can lead to similar problematic extent layout anyway. So I
> rather think we need to make the code more robust to not take the kernel
> down when seeing corrupted extent tree - and we've done work to make code
> more robust in this sence in the past. Can you reproduce the issue with the
> current upstream kernel? If yes, we are probably still missing extent tree
> consistency checks on some path and we need to add them...
>
> Honza
I will try to reproduce this issue with the current upstream kernel.
Actually, the reason of this
issue is obvious, as i reproduced this issue with kernel-4.19.95 and got
follow information:
We inject IO fault when runing fsstress, JBD detect IO error then
trigger JBD abort. At the same time,
if ext4_ext_insert_extent already insert new extent then call
ext4_ext_dirty to dirty metadata , but
JBD already aborted , ext4_ext_dirty will return error.
In ext4_ext_dirty function call ext4_ext_check_inode check extent if
ok, if not, trigger BUG_ON and
also print extent detail information. I caught follow call trace and
extent information:
=================================================================================

[ 7597.436071] inode 1808 ext4_ext_check_inode
[ 7597.436074] inode[1808]: [4] lblk=76 len=5 orig_len=32773 prev=0
[ 7597.436077] inode[1808]: [3] lblk=81 len=25 orig_len=25 prev=81
[ 7597.436080] inode[1808]: [2] lblk=106 len=130 orig_len=32898 prev=106
[ 7597.436084] inode[1808]: [1] lblk=228 len=8 orig_len=32776 prev=236
[ 7597.436181] ------------[ cut here ]------------
[ 7597.436183] kernel BUG at fs/ext4/extents.c:221!
[ 7597.436216] invalid opcode: 0000 [#1] SMP KASAN
[ 7597.436219] Buffer I/O error on dev sda, logical block 131072, lost
sync page write
[ 7597.436224] CPU: 7 PID: 5157 Comm: kworker/u16:1 Not tainted
4.19.95-00020-gb095ded9163d-dirty #186
[ 7597.436230] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS ?-20190727_073836-buildvm-ppc64le-16.ppc.fedoraproject.org-3.fc31
04/01/2014
[ 7597.436242] Workqueue: writeback wb_workfn (flush-8:16)
[ 7597.436248] RIP: 0010:__ext4_ext_dirty.cold+0x2ee/0x308
[ 7597.436251] Code: ff 48 89 df e8 c7 e1 d0 ff e9 48 ff ff ff 48 c7 c7
80 d1 f7 ae e8 87 45 84 ff 48 83 05 ee 3b ca 07 01 48 83 05 c6 3c ca 07
01 <0f> 04
[ 7597.436253] RSP: 0018:ffff88838bcaf050 EFLAGS: 00010202
[ 7597.436256] RAX: 0000000000000000 RBX: ffff888364b67783 RCX:
0000000000000000
[ 7597.436258] RDX: 0000000000000001 RSI: 0000000000000008 RDI:
ffffed1071795dfd
[ 7597.436259] RBP: 000000000000003b R08: 0000000000000000 R09:
0000000000000001
[ 7597.436261] R10: ffffed10744fbd6e R11: ffff8883a27deb77 R12:
dffffc0000000000
[ 7597.436262] R13: dffffc0000000000 R14: ffff888364b673f0 R15:
ffff888364b6730c
[ 7597.436264] FS: 0000000000000000(0000) GS:ffff8883a27c0000(0000)
knlGS:0000000000000000
[ 7597.436266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7597.436267] CR2: 00007fca7a249000 CR3: 000000039ce75000 CR4:
00000000000006e0
[ 7597.436270] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 7597.436271] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 7597.436275] JBD2: Error -5 detected when updating journal superblock
for sda-8.
[ 7597.436277] Call Trace:
[ 7597.436294] ext4_split_extent_at+0x785/0xc80
[ 7597.466767] ext4_split_extent.isra.0+0x2af/0x550
[ 7597.472503] ext4_ext_convert_to_initialized+0x4b4/0x2510
[ 7597.536999] ext4_ext_map_blocks+0x37ef/0x50c0
[ 7597.542467] ext4_map_blocks+0x685/0x1910
[ 7597.544107] ext4_writepages+0x1809/0x32f0
[ 7597.547546] do_writepages+0x7f/0x1b0
[ 7597.548619] __writeback_single_inode+0xc6/0xc40
[ 7597.549203] writeback_sb_inodes+0x49f/0xd60
[ 7597.550748] wb_writeback+0x252/0x9d0
[ 7597.552335] wb_workfn+0x309/0xe60
[ 7597.557029] process_one_work+0x70b/0x1610
[ 7597.557566] worker_thread+0x5a9/0x1060
[ 7597.559091] kthread+0x35e/0x430
[ 7597.560601] ret_from_fork+0x1f/0x30
=================================================================================

Obviously, the follow extent is overlap:
[ 7597.436080] inode[1808]: [2] lblk=106 len=130 orig_len=32898 prev=106
[ 7597.436084] inode[1808]: [1] lblk=228 len=8 orig_len=32776 prev=236
>> Signed-off-by: Ye Bin <yebin10@huawei.com>
>> ---
>> fs/ext4/extents.c | 5 ++++-
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
>> index 77c84d6f1af6..6161db9c17c9 100644
>> --- a/fs/ext4/extents.c
>> +++ b/fs/ext4/extents.c
>> @@ -3245,8 +3245,11 @@ static int ext4_split_extent_at(handle_t *handle,
>> err = ext4_zeroout_es(inode, &zero_ex);
>>
>> goto out;
>> - } else if (err)
>> + } else if (err == -EROFS) {
>> + return err;
>> + } else {
>> goto fix_extent_len;
>> + }
>>
>> out:
>> ext4_ext_show_leaf(inode, path);
>> --
>> 2.25.4
>>

\
 
 \ /
  Last update: 2021-04-21 14:39    [W:0.057 / U:0.280 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site