lkml.org 
[lkml]   [2015]   [Mar]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH RFC] nohz,blk-mq: do not create blk-mq workqueues on nohz dedicated CPUs
    On Tue, Mar 31, 2015 at 10:27:26AM -0400, Rik van Riel wrote:
    > CPUs with nohz_full do not want disruption from timer interrupts,
    > or other random system things. This includes block mq work.
    >
    > There is another issue with block mq vs. realtime tasks that run
    > 100% of the time, which is not uncommon on systems that have CPUs
    > dedicated to real time use with isolcpus= and nohz_full=
    >
    > Specifically, on systems like that, a block work item may never
    > get to run, which could lead to filesystems getting stuck forever.
    >
    > We can avoid both issues by not scheduling blk-mq workqueues on
    > cpus in nohz_full mode.
    >
    > Question for Jens: should we try to spread out the load for
    > currently offline and nohz CPUs across the remaining CPUs in
    > the system, to get the full benefit of blk-mq in these situations?
    >
    > If so, do you have any preference on how I should implement that?
    >
    > Cc: Frederic Weisbecker <fweisbec@redhat.com>
    > Cc: Ingo Molnar <mingo@kernel.org>
    > Cc: Jens Axboe <axboe@kernel.org>
    > Signed-off-by: Rik van Riel <riel@redhat.com>
    > ---
    > block/blk-mq.c | 5 +++++
    > 1 file changed, 5 insertions(+)
    >
    > diff --git a/block/blk-mq.c b/block/blk-mq.c
    > index 4f4bea21052e..1004d6817fa4 100644
    > --- a/block/blk-mq.c
    > +++ b/block/blk-mq.c
    > @@ -21,6 +21,7 @@
    > #include <linux/sched/sysctl.h>
    > #include <linux/delay.h>
    > #include <linux/crash_dump.h>
    > +#include <linux/tick.h>
    >
    > #include <trace/events/block.h>
    >
    > @@ -1760,6 +1761,10 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
    > if (!cpu_online(i))
    > continue;
    >
    > + /* Do not schedule work on nohz full dedicated CPUs. */
    > + if (tick_nohz_full_cpu(i))
    > + continue;
    > +
    > hctx = q->mq_ops->map_queue(q, i);
    > cpumask_set_cpu(i, hctx->cpumask);
    > hctx->nr_ctx++;

    Rik,

    I suppose any bound workqueue queued on isolated CPUs should be moved at
    queue time to other CPUs (sacrifficing performance).

    So that by doing "queue_work" on an isolated CPU would move that
    work somewhere else.




    \
     
     \ /
      Last update: 2015-04-01 04:41    [W:3.845 / U:0.356 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site