lkml.org 
[lkml]   [2018]   [Oct]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: KASAN: use-after-scope in ext4_group_desc_csum
    On Fri, Oct 5, 2018 at 3:05 PM, Jan Glauber <Jan.Glauber@cavium.com> wrote:
    > On Fri, Oct 05, 2018 at 01:13:52PM +0200, Dmitry Vyukov wrote:
    >> On Fri, Oct 5, 2018 at 12:16 PM, Jan Glauber <Jan.Glauber@cavium.com> wrote:
    >> > Hi,
    >> >
    >> > I'm getting below warning when I enable CONFIG_KASAN_EXTRA=y on a arm64 ThunderX2 system.
    >> > As far as I can tell this is present since KASAN_EXTRA was introduced (4.16).
    >> >
    >> > [ 64.547333] ==================================================================
    >> > [ 64.561933] BUG: KASAN: use-after-scope in ext4_es_lookup_extent+0x130/0x980
    >> > [ 64.576105] Write of size 4 at addr ffff80222d81f0ec by task exe/4075
    >> >
    >> > [ 64.592044] CPU: 102 PID: 4075 Comm: exe Not tainted 4.19.0-rc6-jang+ #29
    >> > [ 64.605690] Hardware name: To be filled by O.E.M. Saber/To be filled by O.E.M., BIOS 0ACKL018 03/30/2018
    >> > [ 64.624750] Call trace:
    >> > [ 64.629666] dump_backtrace+0x0/0x360
    >> > [ 64.637024] show_stack+0x24/0x30
    >> > [ 64.643687] dump_stack+0x12c/0x1b4
    >> > [ 64.650699] print_address_description+0x68/0x2c8
    >> > [ 64.660152] kasan_report+0x130/0x300
    >> > [ 64.667509] __asan_store4+0x84/0xa8
    >> > [ 64.674693] ext4_es_lookup_extent+0x130/0x980
    >> > [ 64.683623] ext4_map_blocks+0xe0/0x990
    >> > [ 64.691330] _ext4_get_block+0x130/0x2b8
    >> > [ 64.699211] ext4_get_block+0x40/0x50
    >> > [ 64.706571] generic_block_bmap+0x104/0x178
    >> > [ 64.714977] ext4_bmap+0xc4/0x198
    >> > [ 64.721636] bmap+0x54/0x70
    >> > [ 64.727250] jbd2_journal_init_inode+0x2c/0x208
    >> > [ 64.736355] ext4_fill_super+0x5080/0x5c90
    >> > [ 64.744587] mount_bdev+0x1e0/0x228
    >> > [ 64.751597] ext4_mount+0x44/0x58
    >> > [ 64.758255] mount_fs+0x58/0x1b8
    >> > [ 64.764740] vfs_kern_mount.part.2+0xc0/0x2a8
    >> > [ 64.773495] do_mount+0x7a8/0x13e8
    >> > [ 64.780327] ksys_mount+0x9c/0x110
    >> > [ 64.787160] __arm64_sys_mount+0x70/0x88
    >> > [ 64.795043] el0_svc_handler+0xac/0x150
    >> > [ 64.802749] el0_svc+0x8/0xc
    >> >
    >> > [ 64.811521] The buggy address belongs to the page:
    >> > [ 64.821149] page:ffff7e0088b607c0 count:0 mapcount:0 mapping:0000000000000000 index:0x0
    >> > [ 64.837249] flags: 0x1ffff00000000000()
    >> > [ 64.844959] raw: 1ffff00000000000 ffff7e0088b607c8 ffff7e0088b607c8 0000000000000000
    >> > [ 64.860527] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000
    >> > [ 64.876093] page dumped because: kasan: bad access detected
    >> >
    >> > [ 64.890278] Memory state around the buggy address:
    >> > [ 64.899907] ffff80222d81ef80: f2 f2 f2 f2 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
    >> > [ 64.914426] ffff80222d81f000: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
    >> > [ 64.928945] >ffff80222d81f080: f8 f8 f8 f8 f8 f8 f1 f1 f1 f1 f8 f8 f8 f8 00 f2
    >> > [ 64.943463] ^
    >> > [ 64.956759] ffff80222d81f100: f2 f2 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
    >> > [ 64.971278] ffff80222d81f180: f8 f8 f8 f8 f1 f1 f1 f1 00 00 00 f2 f8 f8 f8 f8
    >> > [ 64.985795] ==================================================================
    >> > [ 65.000312] Disabling lock debugging due to kernel taint
    >> > [ 65.037509] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
    >> >
    >> > I'm not seeing any issues like filesystem corruption or misbehaviour that could be related
    >> > the warning.
    >> >
    >> > Is this a false positive? Any thoughts?
    >>
    >>
    >> Hi Jan,
    >>
    >> What kernel commit are you using? Kernel config? Compiler?
    >> Please symbolize the report with scripts/decode_stacktrace.sh in
    >> kernel tree or https://github.com/google/sanitizers/blob/master/address-sanitizer/tools/kasan_symbolize.py
    >
    > Hi Dmitry,
    >
    > I can reproduce this since 4.16, the report above is from 4.19-rc6.
    > Kernel config:
    > https://paste.debian.net/1046031/
    >
    > Compiler is the stock gcc from Ubuntu 18.04.1:
    > gcc version 7.3.0 (Ubuntu/Linaro 7.3.0-27ubuntu1~18.04)
    >
    > Here is the decoded stacktrace:
    > [ 64.547333] ==================================================================
    > [ 64.561933] BUG: KASAN: use-after-scope in ext4_es_lookup_extent (fs/ext4/extents_status.c:795)
    > [ 64.576105] Write of size 4 at addr ffff80222d81f0ec by task exe/4075
    >
    > [ 64.592044] CPU: 102 PID: 4075 Comm: exe Not tainted 4.19.0-rc6-jang+ #29
    > [ 64.605690] Hardware name: To be filled by O.E.M. Saber/To be filled by O.E.M., BIOS 0ACKL018 03/30/2018
    > [ 64.624750] Call trace:
    > [ 64.629666] dump_backtrace (arch/arm64/kernel/traps.c:102)
    > [ 64.637024] show_stack (arch/arm64/kernel/traps.c:154)
    > [ 64.643687] dump_stack (lib/dump_stack.c:115)
    > [ 64.650699] print_address_description (mm/kasan/report.c:257)
    > [ 64.660152] kasan_report (mm/kasan/report.c:355 mm/kasan/report.c:412)
    > [ 64.667509] __asan_store4 (mm/kasan/kasan.c:699)
    > [ 64.674693] ext4_es_lookup_extent (fs/ext4/extents_status.c:795)
    > [ 64.683623] ext4_map_blocks (fs/ext4/inode.c:526)
    > [ 64.691330] _ext4_get_block (fs/ext4/inode.c:783 (discriminator 3))
    > [ 64.699211] ext4_get_block (fs/ext4/inode.c:802)
    > [ 64.706571] generic_block_bmap (fs/buffer.c:2969)
    > [ 64.714977] ext4_bmap (fs/ext4/inode.c:3314)
    > [ 64.721636] bmap (fs/inode.c:1595)
    > [ 64.727250] jbd2_journal_init_inode (fs/jbd2/journal.c:1257)
    > [ 64.736355] ext4_fill_super (fs/ext4/super.c:4615 fs/ext4/super.c:4770 fs/ext4/super.c:4157)
    > [ 64.744587] mount_bdev (fs/super.c:1158)
    > [ 64.751597] ext4_mount (fs/ext4/super.c:5869)
    > [ 64.758255] mount_fs (fs/super.c:1261)
    > [ 64.764740] vfs_kern_mount.part.2 (fs/namespace.c:961)
    > [ 64.773495] do_mount (fs/namespace.c:2454 fs/namespace.c:2457 fs/namespace.c:2787)
    > [ 64.780327] ksys_mount (fs/namespace.c:3003)
    > [ 64.787160] __arm64_sys_mount (fs/namespace.c:3014)
    > [ 64.795043] el0_svc_handler (arch/arm64/kernel/syscall.c:36 arch/arm64/kernel/syscall.c:48 arch/arm64/kernel/syscall.c:84 arch/arm64/kernel/syscall.c:130)
    > [ 64.802749] el0_svc (arch/arm64/kernel/entry.S:918)
    >
    > [ 64.811521] The buggy address belongs to the page:
    > [ 64.821149] page:ffff7e0088b607c0 count:0 mapcount:0 mapping:0000000000000000 index:0x0
    > [ 64.837249] flags: 0x1ffff00000000000()
    > [ 64.844959] raw: 1ffff00000000000 ffff7e0088b607c8 ffff7e0088b607c8 0000000000000000
    > [ 64.860527] raw: 0000000000000000 0000000000000000 00000000ffffffff 0000000000000000
    > [ 64.876093] page dumped because: kasan: bad access detected
    >
    > [ 64.890278] Memory state around the buggy address:
    > [ 64.899907] ffff80222d81ef80: f2 f2 f2 f2 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
    > [ 64.914426] ffff80222d81f000: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
    > [ 64.928945] >ffff80222d81f080: f8 f8 f8 f8 f8 f8 f1 f1 f1 f1 f8 f8 f8 f8 00 f2
    > [ 64.943463] ^
    > [ 64.956759] ffff80222d81f100: f2 f2 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8
    > [ 64.971278] ffff80222d81f180: f8 f8 f8 f8 f1 f1 f1 f1 00 00 00 f2 f8 f8 f8 f8
    > [ 64.985795] ==================================================================
    > [ 65.000312] Disabling lock debugging due to kernel taint
    > [ 65.037509] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)


    Thanks for the detailed info.

    I've looked at the code and I don't see how es variable can be subject
    to use-after-scope at that point. What happens with es is pretty
    straightforward, no loop, no pointer memorization for future use, etc.
    So this looks like a false positive.

    There was a know use-after-scope false positive related to STRUCTLEAK
    config, but it is not enabled here.

    I've looked at disasm of ext4_map_blocks and ext4_es_lookup_extent and
    do not see anything suspicious. I don't see use-after-scope (0xf8)
    poisoning happening before that point at all.

    "Memory state around the buggy address" looks somewhat suspicious. es
    variable is 5 words, and there is indeed 5 words between left and
    right (0xf1, 0xf2) redzones. However, only 4 words of the variable are
    poisoned as after-scope (0xf8). This generally should not happen:
    either the whole object is after-scope, or none of the object is
    after-scope.

    This all makes me think that somebody else has left these 0xf8 in
    shadow before ext4_map_blocks started executing.
    Unfortunately debugging garbage in stack shadow is not completely
    trivial and there is no common recipe. I don't have setup to run arm64
    kernel at the moment. I would try to locate that garbage in stack
    shadow earlier, e.g. calling another function before ext4_map_blocks,
    implementing that function in mm/kasan/kasan.c (non-instrumented
    itself) and then try to scan stack and verify presence of 0xf8
    garbage. If this works out, then try to catch garbage earlier and/or
    try to figure out what function left that garbage (that's possible by
    locating 0x41b58ab3 magic:
    https://bugzilla.kernel.org/show_bug.cgi?id=198435).

    \
     
     \ /
      Last update: 2018-10-05 17:33    [W:8.004 / U:0.128 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site