lkml.org 
[lkml]   [2020]   [Jan]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH 5.4 176/191] xfs: periodically yield scrub threads to the scheduler
    Date
    From: Darrick J. Wong <darrick.wong@oracle.com>

    [ Upstream commit 5d1116d4c6af3e580f1ed0382ca5a94bd65a34cf ]

    Christoph Hellwig complained about the following soft lockup warning
    when running scrub after generic/175 when preemption is disabled and
    slub debugging is enabled:

    watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [xfs_scrub:161]
    Modules linked in:
    irq event stamp: 41692326
    hardirqs last enabled at (41692325): [<ffffffff8232c3b7>] _raw_0
    hardirqs last disabled at (41692326): [<ffffffff81001c5a>] trace0
    softirqs last enabled at (41684994): [<ffffffff8260031f>] __do_e
    softirqs last disabled at (41684987): [<ffffffff81127d8c>] irq_e0
    CPU: 3 PID: 16189 Comm: xfs_scrub Not tainted 5.4.0-rc3+ #30
    Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.124
    RIP: 0010:_raw_spin_unlock_irqrestore+0x39/0x40
    Code: 89 f3 be 01 00 00 00 e8 d5 3a e5 fe 48 89 ef e8 ed 87 e5 f2
    RSP: 0018:ffffc9000233f970 EFLAGS: 00000286 ORIG_RAX: ffffffffff3
    RAX: ffff88813b398040 RBX: 0000000000000286 RCX: 0000000000000006
    RDX: 0000000000000006 RSI: ffff88813b3988c0 RDI: ffff88813b398040
    RBP: ffff888137958640 R08: 0000000000000001 R09: 0000000000000000
    R10: 0000000000000000 R11: 0000000000000000 R12: ffffea00042b0c00
    R13: 0000000000000001 R14: ffff88810ac32308 R15: ffff8881376fc040
    FS: 00007f6113dea700(0000) GS:ffff88813bb80000(0000) knlGS:00000
    CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: 00007f6113de8ff8 CR3: 000000012f290000 CR4: 00000000000006e0
    Call Trace:
    free_debug_processing+0x1dd/0x240
    __slab_free+0x231/0x410
    kmem_cache_free+0x30e/0x360
    xchk_ag_btcur_free+0x76/0xb0
    xchk_ag_free+0x10/0x80
    xchk_bmap_iextent_xref.isra.14+0xd9/0x120
    xchk_bmap_iextent+0x187/0x210
    xchk_bmap+0x2e0/0x3b0
    xfs_scrub_metadata+0x2e7/0x500
    xfs_ioc_scrub_metadata+0x4a/0xa0
    xfs_file_ioctl+0x58a/0xcd0
    do_vfs_ioctl+0xa0/0x6f0
    ksys_ioctl+0x5b/0x90
    __x64_sys_ioctl+0x11/0x20
    do_syscall_64+0x4b/0x1a0
    entry_SYSCALL_64_after_hwframe+0x49/0xbe

    If preemption is disabled, all metadata buffers needed to perform the
    scrub are already in memory, and there are a lot of records to check,
    it's possible that the scrub thread will run for an extended period of
    time without sleeping for IO or any other reason. Then the watchdog
    timer or the RCU stall timeout can trigger, producing the backtrace
    above.

    To fix this problem, call cond_resched() from the scrub thread so that
    we back out to the scheduler whenever necessary.

    Reported-by: Christoph Hellwig <hch@infradead.org>
    Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
    Reviewed-by: Christoph Hellwig <hch@lst.de>
    Signed-off-by: Sasha Levin <sashal@kernel.org>
    ---
    fs/xfs/scrub/common.h | 9 ++++++++-
    1 file changed, 8 insertions(+), 1 deletion(-)

    diff --git a/fs/xfs/scrub/common.h b/fs/xfs/scrub/common.h
    index 003a772cd26c..2e50d146105d 100644
    --- a/fs/xfs/scrub/common.h
    +++ b/fs/xfs/scrub/common.h
    @@ -14,8 +14,15 @@
    static inline bool
    xchk_should_terminate(
    struct xfs_scrub *sc,
    - int *error)
    + int *error)
    {
    + /*
    + * If preemption is disabled, we need to yield to the scheduler every
    + * few seconds so that we don't run afoul of the soft lockup watchdog
    + * or RCU stall detector.
    + */
    + cond_resched();
    +
    if (fatal_signal_pending(current)) {
    if (*error == 0)
    *error = -EAGAIN;
    --
    2.20.1


    \
     
     \ /
      Last update: 2020-01-07 22:40    [W:2.752 / U:0.056 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site