lkml.org 
[lkml]   [2021]   [Mar]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[tip: irq/core] tasklets: Replace spin wait in tasklet_unlock_wait()
    The following commit has been merged into the irq/core branch of tip:

    Commit-ID: da044747401fc16202e223c9da970ed4e84fd84d
    Gitweb: https://git.kernel.org/tip/da044747401fc16202e223c9da970ed4e84fd84d
    Author: Peter Zijlstra <peterz@infradead.org>
    AuthorDate: Tue, 09 Mar 2021 09:42:08 +01:00
    Committer: Thomas Gleixner <tglx@linutronix.de>
    CommitterDate: Wed, 17 Mar 2021 16:33:55 +01:00

    tasklets: Replace spin wait in tasklet_unlock_wait()

    tasklet_unlock_wait() spin waits for TASKLET_STATE_RUN to be cleared. This
    is wasting CPU cycles in a tight loop which is especially painful in a
    guest when the CPU running the tasklet is scheduled out.

    tasklet_unlock_wait() is invoked from tasklet_kill() which is used in
    teardown paths and not performance critical at all. Replace the spin wait
    with wait_var_event().

    There are no users of tasklet_unlock_wait() which are invoked from atomic
    contexts. The usage in tasklet_disable() has been replaced temporarily with
    the spin waiting variant until the atomic users are fixed up and will be
    converted to the sleep wait variant later.

    Signed-off-by: Peter Zijlstra <peterz@infradead.org>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    Link: https://lore.kernel.org/r/20210309084241.783936921@linutronix.de

    ---
    include/linux/interrupt.h | 13 ++-----------
    kernel/softirq.c | 18 ++++++++++++++++++
    2 files changed, 20 insertions(+), 11 deletions(-)

    diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
    index b7f0012..b50be4f 100644
    --- a/include/linux/interrupt.h
    +++ b/include/linux/interrupt.h
    @@ -664,17 +664,8 @@ static inline int tasklet_trylock(struct tasklet_struct *t)
    return !test_and_set_bit(TASKLET_STATE_RUN, &(t)->state);
    }

    -static inline void tasklet_unlock(struct tasklet_struct *t)
    -{
    - smp_mb__before_atomic();
    - clear_bit(TASKLET_STATE_RUN, &(t)->state);
    -}
    -
    -static inline void tasklet_unlock_wait(struct tasklet_struct *t)
    -{
    - while (test_bit(TASKLET_STATE_RUN, &t->state))
    - cpu_relax();
    -}
    +void tasklet_unlock(struct tasklet_struct *t);
    +void tasklet_unlock_wait(struct tasklet_struct *t);

    /*
    * Do not use in new code. Waiting for tasklets from atomic contexts is
    diff --git a/kernel/softirq.c b/kernel/softirq.c
    index 8d56bbf..ef6429a 100644
    --- a/kernel/softirq.c
    +++ b/kernel/softirq.c
    @@ -25,6 +25,7 @@
    #include <linux/smpboot.h>
    #include <linux/tick.h>
    #include <linux/irq.h>
    +#include <linux/wait_bit.h>

    #include <asm/softirq_stack.h>

    @@ -632,6 +633,23 @@ void tasklet_kill(struct tasklet_struct *t)
    }
    EXPORT_SYMBOL(tasklet_kill);

    +#ifdef CONFIG_SMP
    +void tasklet_unlock(struct tasklet_struct *t)
    +{
    + smp_mb__before_atomic();
    + clear_bit(TASKLET_STATE_RUN, &t->state);
    + smp_mb__after_atomic();
    + wake_up_var(&t->state);
    +}
    +EXPORT_SYMBOL_GPL(tasklet_unlock);
    +
    +void tasklet_unlock_wait(struct tasklet_struct *t)
    +{
    + wait_var_event(&t->state, !test_bit(TASKLET_STATE_RUN, &t->state));
    +}
    +EXPORT_SYMBOL_GPL(tasklet_unlock_wait);
    +#endif
    +
    void __init softirq_init(void)
    {
    int cpu;
    \
     
     \ /
      Last update: 2021-03-17 16:51    [W:4.709 / U:0.124 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site