Messages in this thread | | | Date | Wed, 8 Jun 2022 21:43:52 +0530 | Subject | Re: [PATCH v5 1/9] mm/demotion: Add support for explicit memory tiers | From | Aneesh Kumar K V <> |
| |
On 6/8/22 9:25 PM, Johannes Weiner wrote: > Hello, > > On Wed, Jun 08, 2022 at 10:11:31AM -0400, Johannes Weiner wrote: >> On Fri, Jun 03, 2022 at 07:12:29PM +0530, Aneesh Kumar K.V wrote: >>> @@ -0,0 +1,20 @@ >>> +/* SPDX-License-Identifier: GPL-2.0 */ >>> +#ifndef _LINUX_MEMORY_TIERS_H >>> +#define _LINUX_MEMORY_TIERS_H >>> + >>> +#ifdef CONFIG_TIERED_MEMORY >>> + >>> +#define MEMORY_TIER_HBM_GPU 0 >>> +#define MEMORY_TIER_DRAM 1 >>> +#define MEMORY_TIER_PMEM 2 >>> + >>> +#define MEMORY_RANK_HBM_GPU 300 >>> +#define MEMORY_RANK_DRAM 200 >>> +#define MEMORY_RANK_PMEM 100 >>> + >>> +#define DEFAULT_MEMORY_TIER MEMORY_TIER_DRAM >>> +#define MAX_MEMORY_TIERS 3 >> >> I understand the names are somewhat arbitrary, and the tier ID space >> can be expanded down the line by bumping MAX_MEMORY_TIERS. >> >> But starting out with a packed ID space can get quite awkward for >> users when new tiers - especially intermediate tiers - show up in >> existing configurations. I mentioned in the other email that DRAM != >> DRAM, so new tiers seem inevitable already. >> >> It could make sense to start with a bigger address space and spread >> out the list of kernel default tiers a bit within it: >> >> MEMORY_TIER_GPU 0 >> MEMORY_TIER_DRAM 10 >> MEMORY_TIER_PMEM 20 > > Forgive me if I'm asking a question that has been answered. I went > back to earlier threads and couldn't work it out - maybe there were > some off-list discussions? Anyway... > > Why is there a distinction between tier ID and rank? I undestand that > rank was added because tier IDs were too few. But if rank determines > ordering, what is the use of a separate tier ID? IOW, why not make the > tier ID space wider and have the kernel pick a few spread out defaults > based on known hardware, with plenty of headroom to be future proof. > > $ ls tiers > 100 # DEFAULT_TIER > $ cat tiers/100/nodelist > 0-1 # conventional numa nodes > > <pmem is onlined> > > $ grep . tiers/*/nodelist > tiers/100/nodelist:0-1 # conventional numa > tiers/200/nodelist:2 # pmem > > $ grep . nodes/*/tier > nodes/0/tier:100 > nodes/1/tier:100 > nodes/2/tier:200 > > <unknown device is online as node 3, defaults to 100> > > $ grep . tiers/*/nodelist > tiers/100/nodelist:0-1,3 > tiers/200/nodelist:2 > > $ echo 300 >nodes/3/tier > $ grep . tiers/*/nodelist > tiers/100/nodelist:0-1 > tiers/200/nodelist:2 > tiers/300/nodelist:3 > > $ echo 200 >nodes/3/tier > $ grep . tiers/*/nodelist > tiers/100/nodelist:0-1 > tiers/200/nodelist:2-3 > > etc.
tier ID is also used as device id memtier.dev.id. It was discussed that we would need the ability to change the rank value of a memory tier. If we make rank value same as tier ID or tier device id, we will not be able to support that.
-aneesh
| |