lkml.org 
[lkml]   [2019]   [Sep]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH 0/2] pseries/hotplug: Change the default behaviour of cede_offline
    Michael Ellerman wrote:
    > "Gautham R. Shenoy" <ego@linux.vnet.ibm.com> writes:
    >> From: "Gautham R. Shenoy" <ego@linux.vnet.ibm.com>
    >>
    >> Currently on Pseries Linux Guests, the offlined CPU can be put to one
    >> of the following two states:
    >> - Long term processor cede (also called extended cede)
    >> - Returned to the Hypervisor via RTAS "stop-self" call.
    >>
    >> This is controlled by the kernel boot parameter "cede_offline=on/off".
    >>
    >> By default the offlined CPUs enter extended cede.
    >
    > Since commit 3aa565f53c39 ("powerpc/pseries: Add hooks to put the CPU into an appropriate offline state") (Nov 2009)
    >
    > Which you wrote :)
    >
    > Why was that wrong?
    >
    >> The PHYP hypervisor considers CPUs in extended cede to be "active"
    >> since the CPUs are still under the control fo the Linux Guests. Hence, when we change the
    >> SMT modes by offlining the secondary CPUs, the PURR and the RWMR SPRs
    >> will continue to count the values for offlined CPUs in extended cede
    >> as if they are online.
    >>
    >> One of the expectations with PURR is that the for an interval of time,
    >> the sum of the PURR increments across the online CPUs of a core should
    >> equal the number of timebase ticks for that interval.
    >>
    >> This is currently not the case.
    >
    > But why does that matter? It's just some accounting stuff, does it
    > actually break something meaningful?

    Yes, this broke lparstat at the very least (though its quite unfortunate
    we took so long to notice).

    With SMT disabled, and under load:
    $ sudo lparstat 1 10

    System Configuration
    type=Shared mode=Uncapped smt=Off lcpu=2 mem=7759616 kB cpus=6 ent=1.00

    %user %sys %wait %idle physc %entc lbusy vcsw phint
    ----- ----- ----- ----- ----- ----- ----- ----- -----
    100.00 0.00 0.00 0.00 1.10 110.00 100.00 128784460 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128784860 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128785260 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128785662 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128786062 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128786462 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128786862 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128787262 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128787664 0
    100.00 0.00 0.00 0.00 1.07 107.00 100.00 128788064 0


    With cede_offline=off:
    $ sudo lparstat 1 10

    System Configuration
    type=Shared mode=Uncapped smt=Off lcpu=2 mem=7759616 kB cpus=6 ent=1.00

    %user %sys %wait %idle physc %entc lbusy vcsw phint
    ----- ----- ----- ----- ----- ----- ----- ----- -----
    100.00 0.00 0.00 0.00 1.94 194.00 100.00 128961588 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128961988 0
    100.00 0.00 0.00 0.00 inf inf 100.00 128962392 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128962792 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128963192 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128963592 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128963992 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128964392 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128964792 0
    100.00 0.00 0.00 0.00 1.91 191.00 100.00 128965194 0

    [The 'inf' values there show a different bug]

    Also, since we expose [S]PURR through sysfs, any tools that make use of
    that directly are also affected due to this.


    - Naveen

    \
     
     \ /
      Last update: 2019-09-18 08:54    [W:3.274 / U:0.472 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site