lkml.org 
[lkml]   [2021]   [May]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRE: About add an A64FX cache control function into resctrl
    Date
    Hi Reinette,

    I’m sorry for the late reply.
    I think I could not explain A64FX’s sector cache function well in
    my first mail. While answering the question, I will also explain
    this function in more detail. Though maybe you have already learned
    more about this function by reading specification and manual,
    in order to better understand this function, some contents may have
    duplicate explanations.

    > >> The overview in section 12 was informative but very high level.
    > >
    > > I'm considering how to answer your questions from your email which I
    > > received before, when I check the email again, I am sorry that the
    > > information I provided before are insufficient.
    > >
    > > To understand the sector cache function of A64FX, could you please see
    > > A64FX_Microarchitecture_Manual - section 12. Sector Cache
    > >
    > https://github.com/fujitsu/A64FX/blob/master/doc/A64FX_Microarchitectu
    > > re_Manual_en_1.4.pdf
    > > and,
    > > A64FX_Specification_HPC_Extension ? section 1.2. Sector Cache
    > >
    > https://github.com/fujitsu/A64FX/blob/master/doc/A64FX_Specification_H
    > > PC_Extension_v1_EN.pdf
    >
    > Thank you for the direct links - I missed that there are two documents available.
    >
    > After reading the spec portion it does seem to me even more as though
    > "sectors" could be considered the same as the resctrl "classes of service". The
    > Fujitsu hardware supports four sectors that can be configured with different
    > number of ways using the registers you mention above. In resctrl this could be
    > considered as hardware that supports four classes of service and each class of
    > service can be allocated a different number of ways.

    Fujitsu hardware supports four sectors that can be configured with
    different number of ways by using "IMP_SCCR" registers, and when this
    function is added into resctrl, the maximum ways of each sector are
    indicated by bitmap.

    However, A64FX's L2 cache setting registers are shared among PEs
    (Processor Element) in NUMA. If two PEs in the same NUMA are assigned
    to different resource groups, changing one PE's L2 setting on one
    resource group, the other PE's L2 setting on other resource groups
    will be influenced. So, adding this function into resctrl, we will
    assign NUMA to the resource group. (On F64FX, each NUMA has 12 PEs,
    and each PE has L1 cache setting registers, but these registers are
    not shared.) There are 4 NUMAs on A64FX, 4 NUMAs could be considered
    as hardware that supports four classes of service at most, and each
    class of service has 4 sectors (4 L1 sectors& 4 L2 sectors),
    and each sector can be allocated a different number of ways.
    And, when a running task on resource group, the [56:57] bits of
    virtual address are used for sector selection (cache affinity).

    > The other part is how hardware knows which sector is being used at any
    > moment in time. In resctrl that is programmed by writing the active class of
    > service into needed register at the time the application is context switched
    > (resctrl_sched_in()). This seems different here since as you describe the
    > sector is chosen by bits in the address. Even so, which bits to set in the
    > address needs to be programmed also and I also understand that there is a
    > "default" sector that can be programmed via register. Could these be equivalent
    > to what is done currently in resctrl?

    Adding this function into resctrl, there is no need to write active
    class of service into needed register. When running a task, the sector
    id is decided by [56:57] bits of virtual address, and these bits are
    programed by users. When creating a resource group, the maximum number
    of ways of each sector are set by "IMP_SCCR" setting registers.
    As long as the task is running in a certain resource group, the sector
    and the maximum number of ways of sectors are used will not be changed.
    Therefore, we need not consider context switches on A64FX.

    > (Could you please also consider my original questions?)
    I will reply to the original questions mail.


    Best regards,
    Tan Shaopeng

    \
     
     \ /
      Last update: 2021-05-17 10:32    [W:4.359 / U:27.404 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site