lkml.org 
[lkml]   [2012]   [Mar]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC] AutoNUMA alpha6
    On Wed, Mar 21, 2012 at 08:53:49AM +0100, Ingo Molnar wrote:
    > My impression is that while threading is on the rise due to its
    > ease of use, many threaded HPC workloads still fall into the
    > second category.

    This is why after Peter's initial complains that a threaded
    application had to be handled perfectly by AutoNUMA even if it had
    more threads than CPU in a node, I had to take a break, and rewrite
    part of AutoNUMA to handle this scenario automatically, by introducing
    the numa hinting page faults. Before Peter complains I only had the
    pagetable scanner. So I appreciate his criticism for having convinced
    me that AutoNUMA had to have this working immediately.

    Perhaps somebody remembers what I told at KVMForum on stage about
    this, back then I was planning to automatically handle only processes
    that fit in a node. So the talk with Peter has been fundamental to add
    one more gear to the design or I wouldn't be able to compete with his
    syscalls.

    > In fact they are often explicitly *turned* into the second
    > category at the application level by duplicating shared global
    > data explicitly and turning it into per thread local data.

    per-thread local data is the best case of AutoNUMA. AutoNUMA already
    detects and reacts to false sharing putting all false-sharing threads
    in the same node statistically over time. It also cancels pending
    migration pages queued, and requires two more consecutive hits from
    threads in the same node before re-allowing migration. There's quite a
    bit of work I did to make false sharing handled properly. But the
    absolute best case is per-thread local storage (both numa01
    -DTHREAD_ALLOC and numa02, numa02 spans over the whole system with the
    same process, numa01 has two processes, where each fit in a node, with
    local thread storage).

    > And to default-enable any of this on stock kernels we'd need to
    > even more testing and widespread, feel-good speedups in almost
    > every key Linux workload... I don't see that happening though,
    > so the best we can get are probably some easy and flexible knobs
    > for HPC.

    This is a very good point. We can merge AutoNUMA in a disabled way. It
    won't ever do anything unless explicitly enabled, and even more
    important if you disable it (echo 0 >enabled) it will deactivate
    completely and everything will settle down like if has never run, it
    will leave zero signs in the VM and scheduler.

    There are three gears, if the pagetable scanner never runs (first
    gear), all other gears never activates and it is a complete bypass (noop).

    There are environments like virt that are quite memory static and
    predictable, so if demonstrated it would work for them, it would be
    real easy for virt admin to echo 1 >/sys/kernel/mm/autonuma/enabled .


    \
     
     \ /
      Last update: 2012-03-21 13:21    [W:4.161 / U:0.728 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site